Ticket #2532944 (accepted enhancement)

Reporter


Satyam
Opened: 11/8/12
Last modified: 12/4/12
Status: accepted
Type: enhancement

Owner


Dav Glass
Target Release:
Priority: P3 (normal)
Summary: Provide a garbage collector
Description:

Due to memory leaks produced by Node instances left over in the cache or events whose target elements are gone, it would be great to have a garbage collector available. Even if most people would
never realize the GC is there, some would soon find out as their application starts eating up memory. Even though that is usually due to some sloppy programming, which should not be encouraged, the
fact is, some leaks really take the most disciplined effort to find.

The GC should be able to run asynchronously, doing a little batch at a time, say, going through Y.Node._instances a dozen entries at a time.

Eventually, if any of the browsers adds an idle event (not a bad idea) this GC could be called automatically, otherwise, perhaps we can identify situations that would presumably involve some idle
time, such as showing a modal panel, which inevitably requires user intervention to carry on, that would make a great opportunity for GC. Another might be getting the focus on the editing area of a
RTE, or the popping up of a DataTable cell editor.

If the DOM structure could be reflected in the Node cache, such as the ev.target or ev.currentTarget Nodes as descendants of a Node where event listeners are delegated, that might optimize the
process.

Some Node instances might not be worth including in a GC sweep, such as bounding and content boxes of Widgets since we know they are cleared. Y.Node.create might accept an extra argument to flag
this. Some Node instances cannot be GCed, if they have state information (data or plugins) in them. Inversely, others might be worth paying a lot of attention to, such as those resulting on a call
to Y.all

Would timestamping be of any help? I don't know. I can imagine that stale Node instances with no state information in them would be prime candidates. Perhaps Node instances without state
information might not be cached at all.

Y.Node might have a pool of blank instances available to attach to any node. They would be moved to the cache when state information is set on them but otherwise would be returned to the pool when
available. An array could be built of pooled instances pressed into service which would be the first to get a GC sweep.

Type: enhancement Observed in Version: development master
Component: YUI Global Object Severity: S3 (normal)
Assigned To: Dav Glass Target Release:
Location: Library Code Priority: P3 (normal)
Tags: Relates To:
Browsers: N/A
URL:
Test Information:

Change History

Marco Asbreuk

YUI Contributor

Posted: 12/4/12

What is the status of this ticket? I think it's quite trivial.

Marco.

Dav Glass

Posted: 12/4/12
  • status changed from new to accepted

It's about the most untrivial thing there is to add to a library. Not only do you have to have a scheduler to run the collector at times that it needs to be run (when nothing else is happening), you also have to make sure that the things you are collecting are exposed. You need an API into the private internals of other objects & that's just bad. The opposite of this is that Objects would need an API to use the Collector to clean up. Now we've stepped into a refactor that requires other modules to register themselves with the Collector so that it can collect and clean itself up.

Cleaning Node instances is trivial and yes very simple. But providing a solid, usable and fast Garbage Collector is far from trivial.

Marco Asbreuk

YUI Contributor

Posted: 12/4/12

Hey Dav,

Well I'm, not sure how to response this.
All browsers do have their own garbagecollection and accepted the drawback. I just don't see YUI's "Industrial Strength" when I create long-lived app's that run out of memory. I'm sure it's me to blame on that.

So, even not trivial, I'm glad you accepted the ticket :)

Marco.

Satyam

YUI Contributor

  • Username: satyam
  • GitHub: Satyam
Posted: 12/4/12

I agree that trivial it is not. However, it would be important to have an object with a clear interface in place so that developers can plan on using it. I don't think browsers can go for long without having an 'idle' event and a 'gc' method, and having the interfaces on YUI for applications to register objects to be collected or methods that could do the collection would be great, even if they do not much initially. And if there are no plans on any browser to have an 'idle' event or 'gc' method, well, it is about time they do, and I'm sure you know the people who can make this happen.

Satyam

YUI Contributor

  • Username: satyam
  • GitHub: Satyam
Posted: 12/4/12

When I was about to submit the previous, when I looked at the button I read "Submit Challenge" instead of "Submit Changes". This is a challenge, I'm sure.

Marco Asbreuk

YUI Contributor

Posted: 12/4/12

I'm a bit disapointed here, that two of the persons I see as gothfathers of YUI, see this as non-trivial�
Let me be a greene, but I see it as most trivial.

As you both know, there is a lot discussion going on in topic http://yuilibrary.com/forum/viewtopic.php?f=92&t=10943 and http://yuilibrary.com/forum/viewtopic.php?f=18&t=10914, but let give two horror-examples:

1) What if a developer creates a containernode (functioning as a body-container), places a lot of stuff within (some widgets) and lets the content of this node updated through a seperate menu (remaining at the top of the page). The developer will use myNode.load() and the enduser will be loading all of the pages all over again. http://yuilibrary.com/yui/quick-start/#ajax explains how easy one can start using this. But wait: when does all the suff cleanup up?

2) I'm developing Mojito at the moment. Great way to use Mojits where you can bind event using a "binder-function". I can also refresh a Mojit using mojitProxy.refreshView(); But what happens here? At what stage does everything cleaned up? I might be calling mojitProxy.refreshView(); a dozen of times (and rebind everything). Now, I have post this at the Mojitoforum and they seem to be surprised about these sideeffects as well. I've got the idea they even didn't think about it before.

So what I'm trying to say is: there is no programmer that can programm without failures straight ahead. When it comes to the memory-pitfalls, I think 98-99% will deliver a longlived application that is consuming memory. You cannot lay the full repsonsibility with the programmer. And if you do, then please remove the "industrial strenght" sentency on the YUI-library landingspage, for at the moment it is not.

Marco Asbreuk

YUI Contributor

Posted: 12/4/12

Correction: I'm developing using Mojito, not Mojito itself..

Dav Glass

Posted: 12/4/12

Marco --

You are looking at this from a narrow point of view. Yes, adding a Node instance GC could be very trivial and is already in the system as a feature for Node: http://yuilibrary.com/projects/yui3/ticket/2532270

But adding a Library wide way to do this is non-trivial and requires an API design, several use cases and an action plan. Mojito is a platform built on YUI and is constantly pushing the boundaries of it's use. It's all new to them and us and we are both learning together as we go since no one has ever done this before. We are currently working with them very closely to merge Mojito and YUI even more so that these things are handled in a better way.

Also comments like this "And if you do, then please remove the "industrial strenght" sentency on the YUI-library landingspage, for at the moment it is not." are disrespectful and uncalled for. Please take your time when posting so you don't insult those that read this later and remember we are all in this together and working towards the same goals.

Marco Asbreuk

YUI Contributor

Posted: 12/4/12

Hey Dav,

Sorry, really didn't want to insult.
You probably know -reading my other posts- that I like YUI very much and take a positive approach all the time.

The matter is just a big burden to me that I can't manage well. Writing the last sentence, I wanted to make a point.
So again, sorry for that. I can't edit the post otherwise I would have change it.

Marco.

Dav Glass

Posted: 12/4/12

Just letting you know that it "could" come across that way and in these discussion we have to weigh our words carefully across many different languages ;)

No worries here, just stating that comment.

If the node cache issue is biting you that much, please look at the above ticket as it has code attached. Try it out and comment on it to get it moved up in the queue. Once we have a valid way that we like to handle GC, we can attempt to build the API and use cases required to fully bake this out.

Marco Asbreuk

YUI Contributor

Posted: 12/4/12

Dav, thank you.

I hope Node CG does what I expect. It's more that I'm not sure: I feel I don't master memory-management. And I dislike not-mastering it for I mostly write long-lived apps... I've had a couple of times now, that I realized: hey this also might leads to memory-leaks. In forum http://yuilibrary.com/forum/viewtopic.php?f=18&t=10956&start=10 Satyam and me looked at another example where I didn't know the side-effects.

I'll take a look at Node CG. I hope everything I found up till now is Node-related.

Satyen Desai

YUI Developer

  • Username: sdesai
  • GitHub: sdesai
Posted: 12/4/12

0. As mentioned in the forum thread, I'm skeptical of getting a robust enough GC solution, and even if it is robust, we introduce ad-hoc performance hits when we run it, so I think the GC solution is probably the furthest thing down the road and has very low probability of success.

1. We should assign this ticket to Matt. He's done some legwork on this problem already (as I listed in the forum post) and can fill in with his ideas/roadblocks which don't involve a manual "GC". We can maybe flush one of those out, even if for a subset of browsers (e.g. solutions which may depend on property getters/setters)

2. We can consider switching Y.Widget, Y.View, and Mojito binders (as things which "own" nodes), to deep destroy nodes on "destroy" as a stop gap - which should take care of many long-lived-page node instances. IMO, that's the has the best short term value add.