New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
runtime: garbage collection confused by static tables on 386 #1925
Labels
Milestone
Comments
This is not a Windows port issue; it's a GOARCH=386 memory management issue. This issue is likely a duplicate of issue #909. |
Yes. fmt.Println -> println windows/386 0 5070800 8887420 110 1 5107664 14564476 111 2 5107664 14564476 111 3 5107664 14564476 111 4 5107664 14564476 111 5 5107664 14564476 111 6 5107664 14564476 111 7 5107664 14564476 111 8 5107664 14564476 111 9 5107664 14564476 111 0 20144144 37125244 113 1 20181008 59686012 114 2 20181008 59686012 114 3 20181008 59686012 114 4 20181008 59686012 114 5 20181008 59686012 114 6 20181008 59686012 114 7 20181008 59686012 114 8 20181008 59686012 114 9 20181008 59686012 114 linux/386 0 5095152 9018492 52 1 5132016 14695548 53 2 5132016 14695548 53 3 5132016 14695548 53 4 5132016 14695548 53 5 5132016 14695548 53 6 5132016 14695548 53 7 5132016 14695548 53 8 5132016 14695548 53 9 5132016 14695548 53 0 20168496 37256316 55 1 20205360 59817084 56 2 20205360 59817084 56 3 20205360 59817084 56 4 20205360 59817084 56 5 20205360 59817084 56 6 20205360 59817084 56 7 20205360 59817084 56 8 20205360 59817084 56 9 20205360 59817084 56 |
fmt imports unicode, which has some tables. They should all be in static memory and thus invisible to the garbage collector (and so not compounding issue #909), but apparently they are not. I have been meaning to fix this for months. Owner changed to @rsc. Status changed to 386GCconfusedbystatictables. |
Seeing it also on Linux/386 (golang tip revision), when using index/suffixarray on quite large data (e.g. 4MB text). Qsufsort internally creates a temporarily inverted array `inv' of the same size as the suffix array (http://golang.org/src/pkg/index/suffixarray/qsufsort.go?h=inv#L37). It appears not to be freed after runtime.GC(), which -- in case of multiple search indices -- can sum up to a quite large amount of data. I created a minimal version of the program, which allocates temporary slices in a loop. The effect only shows up if "fmt" is imported, and slices >= 4MB are allocated. It vanishes if "println" is used instead (as in #3), or smaller slices are allocated. There is also an effect of "saturation" visible: Nothing is freed up to an amount of 165MB, then the sum of allocated memory remains stable, i.e. on each GC run 4MB are freed: gc1: 0+0+0 ms 0 -> 0 MB 104 -> 102 (104-2) objects 40 pointer lookups (29 size, 11 addr) gc2: 0+0+0 ms 4 -> 4 MB 273 -> 233 (275-42) objects 242 pointer lookups (159 size, 83 addr) gc3: 0+0+0 ms 4 -> 4 MB 233 -> 233 (275-42) objects 243 pointer lookups (159 size, 84 addr) gc4: 0+0+0 ms 8 -> 8 MB 235 -> 235 (277-42) objects 247 pointer lookups (160 size, 87 addr) gc5: 0+0+0 ms 12 -> 12 MB 237 -> 237 (279-42) objects 260 pointer lookups (161 size, 99 addr) ... gc41: 8+1+0 ms 157 -> 157 MB 309 -> 309 (351-42) objects 1130 pointer lookups (197 size, 933 addr) gc42: 13+0+0 ms 161 -> 161 MB 311 -> 311 (353-42) objects 1132 pointer lookups (198 size, 934 addr) gc43: 5+0+0 ms 165 -> 165 MB 313 -> 313 (355-42) objects 1133 pointer lookups (199 size, 934 addr) gc44: 18+0+0 ms 169 -> 165 MB 315 -> 314 (357-43) objects 1134 pointer lookups (200 size, 934 addr) gc45: 11+0+0 ms 169 -> 165 MB 315 -> 314 (358-44) objects 1134 pointer lookups (200 size, 934 addr) ... gc81: 20+0+0 ms 169 -> 165 MB 315 -> 314 (394-80) objects 1134 pointer lookups (200 size, 934 addr) gc82: 16+0+0 ms 169 -> 165 MB 315 -> 314 (395-81) objects 1134 pointer lookups (200 size, 934 addr) In use: 173596448 337231104 By applying some debug output to runtime/mgc0.c I noticed that it detected 41 pointers (41*4MB = 164MB) to the beginning of the slice buffers while "consulting span table". Attachments:
|
Owner changed to builder@golang.org. |
Status changed to Duplicate. Merged into issue #909. |
This issue was closed.
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
by peter.polakovic:
The text was updated successfully, but these errors were encountered: