Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

top shows much less memory(500mb) but tmalloc shows cumulative memory as 2000mb for my process #651

Closed
alk opened this issue Aug 23, 2015 · 6 comments

Comments

@alk
Copy link
Contributor

alk commented Aug 23, 2015

Originally reported on Google Code with ID 648

What steps will reproduce the problem?
1. Trying to understand the difference between what is reported by tmalloc and top
output on linux
2.
3.

What is the expected output? What do you see instead?
i thought there should be at least some correlation between the memory usage reported
by top and tmalloc or they should be pretty close


What version of the product are you using? On what operating system?
tmalloc from gperftools-2.0

Please provide any additional information below.


Reported by krmayank on 2014-09-12 19:00:10

@alk
Copy link
Contributor Author

alk commented Aug 23, 2015

Please attach output of MallocExtension::GetStats. I need to know what exact metric
2000mb is.

Reported by alkondratenko on 2014-09-12 20:13:01

@alk
Copy link
Contributor Author

alk commented Aug 23, 2015

i dont link to this library but LD_PRELOAD it. I am talking about the following output
when heap profiling is on:-
Dumping heap profile to /tmp/abc.0043.heap (2050 MB allocated cumulatively, 167 MB
currently in use). Here 2050 MB allocated cumlatively is way off then what i see in
output of top command which shows VIRT, RES and SHR as 500m, 326m and 50m. Also eventually
probably because of a leak in my program kernel oom kills my process with the following
output:-

%KERN-1-SYSTEM_MSG: [49326.103001] Proc xyz (15511) with Total_VM 594236 KB Resident_Mem
87991 KB Anon_Resident_Mem 87615 KB being killed due to lack of memory  - kernel

when tcmalloc in the very end reported the following:-
Dumping heap profile to /tmp/abc.0055.heap (2623 MB allocated cumulatively, 231 MB
currently in use)

So why this different, where is 2623 mb allocation value coming from ?

Reported by krmayank on 2014-09-12 20:59:52

@alk
Copy link
Contributor Author

alk commented Aug 23, 2015

Misunderstanding appears to be due to not paying attention to "cumulative" in this message.

It really means cumulative. I.e. it is sizes of all memory allocations ever made added
together.

Reported by alkondratenko on 2014-09-14 20:07:14

  • Status changed: NotABug

@alk
Copy link
Contributor Author

alk commented Aug 23, 2015

Great thanks for your help. I am just beginning to use this great tool. Few question,
i am guessing you can help answer:-
-So cumulative means all memory ever allocated by the process(using free/new) including
the memory which is already free. Please confirm ?
-231mb currently in use. This is allocated using malloc/new but not yet freed ? Please
confirm

-i am trying to find memory leaks using the following command,
pprof) Linux(debug)# /tmp/pprof /isan/bin/abc --base abc.0080.heap abc.0090.heap. and
it reports the following:-
(pprof) top
Total: 3.3 MB
     0.6  19.7%  19.7%      0.6  19.7% _ZN7adaptor18EthPortStatsHistMo7factoryEN2mo11FactoryTypeE
     0.5  13.7%  33.3%      0.5  13.7% _ZN7adaptor23EthPortMcastStatsHistMo7factoryEN2mo11FactoryTypeE

A couple of questions about this:-
a: Is this a good way to detect leaks?
b: is the Total reported here the extra malloc's done from the base which were not
freed yet
c: i get the above output which  doesnt display the proper stack. is there a way i
can get the complete stack where the extra malloc happened ?

PS: if you think there is a better forum i should go please let me know.

Reported by krmayank on 2014-09-15 18:38:54

@alk
Copy link
Contributor Author

alk commented Aug 23, 2015

>> -So cumulative means all memory ever allocated by the process(using free/new) including
the memory which is already free. Please confirm ?

Yes.

>> -231mb currently in use. This is allocated using malloc/new but not yet freed ?
Please confirm

Yes

Reported by alkondratenko on 2014-09-20 23:20:10

@alk
Copy link
Contributor Author

alk commented Aug 23, 2015

>> a: Is this a good way to detect leaks?

Yes. As pointed out in the docs.

>> b: is the Total reported here the extra malloc's done from the base which were not
freed yet

Yes.


>> c: i get the above output which  doesnt display the proper stack. is there a way
i can get the complete stack where the extra malloc happened ?

--web may give you that information

Reported by alkondratenko on 2014-09-20 23:29:12

@alk alk closed this as completed Aug 23, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant