The only way to even have a chance to find this is to have a procedure that is
1) short
2) causes memory to go up every time,
3) is simple enough to be pruned to isolate the source.

I've looked into memory allocation multiple times only rarely with success.

I realize that...memory trouble-shooting is a bitch and a half. I ran a set of tests 
today, info/data is below. They're not 100% scientific (e.g. I didn't uninstall any 
add-ons and didn't touch plug-ins, etc.) but I think it definitely shows that while 
FF itself seems to have it's own 300-500k memory leak, with FB enabled, the 
difference is more like 2-4MB. 

One thing to note: I usually waited about 30 seconds to over a minute for memory to 
settle each time. I'm assuming FF uses GC and if so has a rough time limit on unused 
objects getting collected but I don't know what the limit is and so on.

You tell me if there's anything that you can think of that could affect this (for 
instance, if I have the FB Console showing versus minimized does that factor into 
memory usage?) that I can try. I'm willing to help out as much as I can but I don't 
think I'll have the time (or the patience) to uninstall plugins and add-ons. If 
disabling is just as good then we're fine since disabling is so easy to do.

Let me know.


kn

--------------------------------

FF3.5.3 / Win XP SP 3 / Quad Core 2.4Ghz / 2GB RAM / Tons of HD space

Active: 
Aviary 0.8.6
Battlefield Heroes Updater 4.0.21.0
Java Quick Starter 1.0
Pixel Perfect 1.1.0
Real Player Record Plugin 1.0
Web Developer 1.1.8


Disabled:
AVG Safe Search 8.5
Firebug 1.5a26
Firecookie 0.9.1
FoxyTunes 3.5.9
Microsoft .Net Framework Assistant 1.1
Move Media Player 7
Xdebug Helper 0.3.1

Startup page:
http://www.penny-arcade.com/comic/

Memory:

Startup (note "active" add-ons above are enabled): 4:23pm

Usage: 58,721k
Peak: 74,782k

Idle until: 5:48pm

Usage: 85,656k
Peak: 89,012k

Restart: 5:49pm
Activated Firebug only.
Usage: 64,320k
Peak: 80,412k

Pulled up site/Ajax heavy (console active) (this is a local dev site running under 
Visual Studio 2005's built-in Web server):
Usage: 76,44k
Peak: 83,176k

Refreshed page:
Usage: 106,120k
Peak: 119,132k

Refreshed page again:
Usage: 109,784k
Peak: 135,936k

Disabled FireBug again (restart): 5:53pm.
Usage: 55,520k
Peak: 60,684k

Refreshed ajax page (already loaded):
Usage: 59,020k
Peak: 63,440k

Clicked into error-heavy ajax page (this is a different site but running under VS 
2k5's built-in server - in this case it's a site setup with Windows Authentication--
no anonymous access--so every ajax hit results in two errors (401 unauthorized) 
before a successful 200 response):
Usage: 59,040k
Peak: 63,544k

Refresh: 
Usage: 56,520k
Peak: 63,544k

Enabled FireBug (restart): 5:57pm.
Usage: 73,516k
Peak: 101,232k

Refresh:
Usage: 79,748k
Peak: 112,744k

Refresh:
usage: 81,872k
Peak: 117,716k

Refresh:
Usage: 84,592k
Peak: 119,236k

Disabled all add-ons (restart): 6:00pm
Usage: 46,428k
Peak: 50,208k

Refresh (still on error-heavy ajax page):
Usage: 46,932k
Peak: 50,924k

Refresh Again:
Usage: 47,292k
Peak: 50,924k

Refresh again: 6:04pm.
Usage: 47,708k
Peak: 50,940k

Enabled FireBug only (restart): 6:05pm.
Usage: 68,264k
Peak: 95,472k

Refresh: 
Usage: 73,916k
Peak: 108,928k

Refresh again:
Usage: 75,924k
Peak: 110,352k

Refresh again:
Usage: 79,096k
Peak: 111,544k

If the memory goes up after refresh every time, then recovers when you close the
page, this would point to problems in Firebug's metadata (context).

You can try to isolate issues by disabling Console, Net, or Script panel to see if
any memory increases are related to those features.

It doesn't recover 100%. If it did, I would've just assumed GC was taking some time
and I would expect to NOT see quite as much memory usage over time...short term,
sure, I'd expect some dogpiling but if it was releasing all that memory I wouldn't
see 200MB of memory usage (not peak) with a single window/single tab open.

I'll try disabling the different panels and see what happens. Stay tuned.

 Issue 2384  has been merged into this issue.

I have put together a simple test-page that executes XHRs and also logs into the
Console panel (repeatedly on timeout).

http://www.janodvarko.cz/firebug/tests/2372/issue2372.html

I have tested with Firebug SVN HEAD and Firefox 3.6 and 3.7, and used Task Manager in
Windows Vista to see how much memory Firefox gets (not the most precise tool).

The scenario was:
1) Open the test page
2) Enable/Disable Script, Net, Console (various configurations)
3) Run the test
4) Watch the memory consumption (for 5 min)

When the Console and Net panel were activated (the console needs Script panel, so it
was activated too, but I don't think the Script panel is the culprit here, at least
not in this test) - the memory consumption was growing all the time the test was running.

When I refreshed the page (and so, also stopped the test), the memory dropped to the
same state as it was before the test (more or less). This would indicate a problem
with the Firebug context (the object created for every page and destroyed with the
page unload).

My feeling is that the limit, which is implemented for the Console and the Net panel
is somehow not working properly. Even if the oldest logs (TR elements) are evidently
removed, when the limit is reached, it looks like objects (JS objects and DOM
elements?) are not released from the memory (If only Net or Console is enabled, I
think the memory grows slower).

If I am correct here, the page refresh not only releases the context object but also
reloads the panel document (i.e. panel.html that represents content of all panels).
Perhaps there a reference to this document holds all the removed TR elements and
associated JS objects till it's reloaded...

What do you think?

Honza

Forgot to note that it would be useful if somebody else can try my test page and
verify that it's producing leaks (and so, it isn't only a weird behavior on my machine)

Thanks!
Honza

I'll see if I can reproduce this. I might also be able to get a little more
information with a debug build.

We use Firebug a lot during development and the memory problem is real. 
Our pages also use xmlhttprequest calls. To indicate : yesterday I did a lot of
development and refreshing ; I left firefox running overnight with only one tab ;
this morning it had 500+ MB of memory usage. (sadly I did not check how much it had
when I left for home yesterday)
 
I have tried the testpage presented by Honza. Firebug console active, Net tab not
active. Here's my results.

Initial memory usage before the test : 110 MB
Opened Honza's testpage : still 110 MB
Started the test : usage steadily increased from 110 MB to 140 MB in only 1 minute
Reload the page, in effect stopping the test : memory drops to 125 MB
Close the tab containing the test page : memory drops to 123 MB

There seem to be two problems ; the first being the ever and fast increasing memory
consumption, secondly after closing the page not all memory is released (went from
110 before the test to 123 afterwards)

To indicate this indeed is a firebug problem ; opened up the page again, firebug
disabled. Start test ... no memory increase at all.

Ubuntu 9.10 ; Firefox 3.5.4 ; Firebug 1.4.5

The results in comment 9 are not useful, please retest with Firebug 1.5b3,
http://getfirebug.com/releases. Also we don't know how much memory is needed, so 30MB
is not a very big number.

I tried the test on my WinXP/FF 3.6b2/Firebug 1.5b3 and saw insignificant memory effects.

Boris can you please read comment 6 ? I recall you know of some specific ways web
developers often mistakenly hold on to memory (Firebug panels are HTML in a 'browser'
element, with event listeners added via JS running in browser.xul).

The simplest way to effectively leak memory for page lifetime is to have
function-valued properties (e.g. event listeners) that entrain various objects by
closing over things.  Of course you still have to have some part of the ball of
string reachable (or the cycle collector needs to be buggy)...

I have committed and experimental patch for this (1.5, R4823)

In my test case this patch quite helps for me, but I would definitely
want to hear that somebody else is also seeing a progress.

The patch implements clearDomplate method that removes all created properties
on nodes (e.g. _repObject, _stackTrace, etc.) before these nodes are
removed from the Firebug UI DOM tree.

Should be in Firebug 1.5b4

Honza

There must be some kind of OS dependence here, like we had with the activation bug. 

Should we put the clearDomplate on an option and see if we figure out why some people
see this problem and some do not? 

> Should we put the clearDomplate on an option and see if we figure out why some people
> see this problem and some do not? 
I have created: extensions.firebug.clearDomplate preference. It's false by default
so, developers who are experiencing the problem should set it to true and verify if
it helps.

Honza

Hey, finally had a chance to test again. Sorry for the delay. In the end this testing 
doesn't take a whole lot of time (about 20 mins) but it's tedious and I'm pretty much 
always working. :)

I'll attach the numbers rather than post them since it's kind of long. I started 
without anything enabled then enabled all tabs and went through combinations of tabs 
(Console, Script and Net) and each one on it's own. One caveat was that I couldn't 
disable the Script tab on startup. It would enable no matter what. So, the non-Script 
tab numbers might be skewed a bit (higher) as a result. The tests were against a dev 
site on my local machine that generates a lot of 401 errors because I'm using Windows 
Auth and I don't know why. 

I'm on Windows XP SP3, FF 3.5.3, and FB 1.5x.0b4. Let me know if there is anything 
else you want me to try. And, yes, I still get the full on memory leak (even when 
using a dev site with minimal or no errors in the Console). Because of the various 
memory hogs I'm running (SQL Server, Visual Studio, FF, etc) things get bad on my 
machine with its 2GB of RAM when FF + FB is taking up 300+MB of RAM.

kn

FF3.5_FB1.50b4_Test_091120.txt 2.8 KB   Download

First I want everyone reading here to know that if we can find a way to reproduce a
problem we can fix it. But I just do not ever see the effect here so I cannot help.


You need to help me understand your conclusion here. 

With Firebug, Chromebug, Chromebug's trace window, and Firebug test window my Firefox
has 83M. That is similar to your numbers in the attachment.  300 - 83 = 217. If you
see 4MB / page reload then you must be doing 50 page reloads before you get to 300MB.
Is that the use model? You are doing 50 page loads and getting to 300MB? 

Please us about:config and set extensions.firebug.clearDomplate true and repeat your
tests. Any difference?

I understand completely. What you're saying is what I tell testers and customers 
every day.

In answer to your first set of questions, yes and no. I am reloading a lot more than 
the average user (Web dev after all). But, I've also had situations where I'd leave 
it at around 200MB in use and come back to find it up closer to 300MB after a couple 
of hours. I'll try to pay attention to memory and usage before I shut down the 
browser (I have to usually restart because the excessive memory usage slows down FF 
and affects my development/unit testing).

I'll also try the config setting to see if it changes things...

Stay tuned.

kn

Alright, I left FF running with just FB alone with all tabs active. Loaded up the one 
test page immediately and left it running for 1.75 hours. Memory did not change at 
all. So, it seems to be tied to reloads of the page rather than a standing memory 
leak. This  makes sense though in that it seems like something isn't getting released 
more than something is running in the background churning up memory. 

I'm going to enable that one config setting and do my usual. I'll let you know if I 
see a difference.

kn

removing blocking. Not a new bug. Would like a better testcase and some ideas for
leak analysis.

Linux 64bit, FB 1.5b5, FF 3.6 beta
The following page causes FF to use more and more memory on each page reload. I stopped 
once it hit over 350MB. With FB turned off memory sits at about 120MB (as reported by 
ps).
http://www.extjs.com/deploy/dev/examples/form/anchoring.html

For the case in comment 21 I get
112K
117K
125K
128K
cycle thru the panels
154K
reload
187K
delete the tab 
140K
So it does seem to increase on each reload. Unfortunately these extjs pages are very
nasty to work with because they have so many errors. Of course that may be were the
memory is going ;-)

What errors does the page in comment #21 have?

Regarding comment 23.
The Firefox Error Console does not seem to allow copy-all so here is the Firebug
trace of errors.

firebug-tracing-logs.txt 168 KB   Download

OK, I was looking at the firebug console. I opened up the Firefox error console and
see a bunch of stuff. But those are all CSS warnings. I guess they leak into the
tracing console... this is a pitch for Issue 2338. ;)

<tangent>
I hope css warnings (errors really) are the cause here, since there is a lot of that
out there. Just for opacity, devs write:

.show-50 { -moz-opacity:.50; filter:alpha(opacity=50); opacity:.50; }

So for that simple thing, -moz-opacity and filter both generate errors. 67% error
rate. And when you try rounded corners, with all the -moz-*, -webkit-*, -o-*, -ms-*,
etc., then they add up.
</tangent>

Firebug relies on Firefox GC to cleanup memory, correct?

Not like web apps that have to worry about memory allocation patterns in IE6, and
thus can sometimes have elaborate tear-down of their own data structures.

> Firebug relies on Firefox GC to cleanup memory, correct?

Yes, but GC can only GC things that actually aren't being referenced.  If you stick
something in an expando property on a node and hold on to the node, say, the expando
is going to stick around too.

Using the example from comment 21, I ran with Net, Script and Console disabled. No
clear memory increase. Then Net only. Same: no clear increase. Then with Script only.
I can see memory grow each reload.

So this narrows down the problem to debugger.js and firebug-service.js. Which is too
bad in a way. I'm working on code that would replace a lot of that code and thus I'm
not very motivated to work on this problem.

Well, what's the ETD on the rewrite/replace you're doing? If it's relatively soon, it's 
worth getting it up there and seeing if the problem persists. If it's still 6-12 months 
out...well...it's your call, I guess.

Ok the other day I noticed one of my tracing output listing the un-cleared top level
scripts in firebug-service component. This is global list of jsdIScripts that firebug
could not categorize, but thinks are top-level scripts. This debug output was there
to remind me that there is a data structure in firebug-service that could grow
indefinitely: if we don't know how to categorize these we don't know when to delete them.

Using our newly added support for turning off jsd, I cleared the top-level script
list when jsd is turned off.

To get jsd to turn off you need Firebug to be active on zero pages. The easy way is
to have one tab open for debugging. Every time you reload that tab, the number of
active pages goes to zero briefly and jsd gets turned off. 

Now I get the following results for Mem Usage in WinXP in the firefox process:

One tab only using the page from comment 21, Script panel active, Net and Console
disabled.
each line is a reload (and wait for GC)
92M
89M
92
90
91
Now open a second tab same url
100
109
113
121
123
Now close the second tab and reload the first one:
98
92
92
95
92
93

So maybe this will help. Of course it would be better to not have this funky array at
all, but I don't know how to avoid it until we get bugzilla 449464

R5397 on branches/firebug1.5, for 1.5b9