| Issue 5: | btree RAM usage is excessive | |
| 1 person starred this issue and may be notified of changes. | Back to list |
Sign in to add a comment
|
At the moment, we assume that RAM is cheap, and the snapshot exception btree takes up (size of snapshot store / snapshot blocksize) * 16 bytes, or 512MB for a 5OOGB snapshot store with 16KB snapshot blocks. This is excessive for some customers, for instance, popular low cost Linux-based NAS appliances tend to ship with only 128MB of RAM. Even if they increase the amount of RAM they ship with, it won't be a huge amount, and people would rather use that precious RAM for caching data rather than fluffy metadata. Dan P. estimates it would take about a month to switch to an extent-based tree which would reduce RAM consumption significantly. |
||||||||||
,
Dec 10, 2007
Actually, in the example above, 512MB is the on-disk size of the btree index, not the memory cache footprint. Given a non-random access disk access pattern (the expected case) the memory required for efficiently caching the btree index in the example will be much less than 512MB. It would be nice to measure this caching behavior, and find out just where the knee of the curve lies. |
|||||||||||
,
Dec 10, 2007
(No comment was entered for this change.)
Labels: Milestone-Release0.6
|
|||||||||||
,
Dec 10, 2007
(No comment was entered for this change.)
Labels: -Milestone-Release0.6 Milestone-Release6
|
|||||||||||
,
Dec 21, 2007
A --cachesize option was added to zumastor define volume, which at least lets users tune the size of this cache. Doesn't solve the excessive RAM problem, but still worth noting. |
|||||||||||
,
Dec 21, 2007
While we might make some changes so small volumes can use a more efficient format in release 0.6, the big extent-based change probably won't come until later. Deferring.
Labels: -Milestone-Release6 Milestone-Release7
|
|||||||||||
,
Feb 13, 2008
(No comment was entered for this change.)
Labels: -Milestone-Release7
|
|||||||||||
,
Apr 09, 2008
With a large volume size and after a lot of copyouts, btree can become very large and ddsnap create/delete/status may take tens of seconds to finish.We need to add some performance monitoring code and do more tests to better understand this. We may want to separate the btree code to ease the performance testing and optimization.
Labels: Milestone-Release10
|
|||||||||||
,
May 15, 2008
See also issue 140 (implement the more efficient format mentioned in comment 5). |
|||||||||||
 Sign in to add a comment
|
|||||||||||
