Export to GitHub

weed-fs - issue #45

[Compact issue] Offset overflow


Posted on Sep 19, 2013 by Swift Rhino

You are using uint32(Maximum 4Gb) to store needle offset(Maximum 32Gb) when compacting. Currently It is ok if the volume size is < 4gb Change variable "offset" in ScanVolumeFile function to uint64 to fix the issue.

Comment #1

Posted on Sep 19, 2013 by Grumpy Cat

This is by design. There is a up-to-8-bytes padding to use uint32 to represent 32GB offset.

Comment #2

Posted on Sep 19, 2013 by Swift Rhino

Yes, I know that. You are using uint32(* 8 bytes padding = 32Gb maximum) to represent the needle offset in memory. When compacting, you read needles from dat file, the "offset" variable in ScanVolumeFile function is not divided by 8 --> so it should be uint64 not uint32

Please read the code carefully

Comment #3

Posted on Sep 19, 2013 by Swift Rhino

Read the code carefully or you can test by Println offset in ScanVolumeFile function when compacting volume(size > 4gb) :)

Comment #4

Posted on Sep 19, 2013 by Grumpy Cat

(No comment was entered for this change.)

Comment #5

Posted on Sep 19, 2013 by Grumpy Cat

Thanks for your detailed suggestions! Fix now is in v0.42

Status: Fixed

Labels:
Type-Defect Priority-Medium