What steps will reproduce the problem? 1. launch an empty volume server alone on a master 2. concurrently write many files, then try to read the created files

What is the expected output? What do you see instead?

Sometimes - the frequency of the event depends on the amount of write concurrency - the first file written to a volume is missing: the volume server is issuing a 404 NOT FOUND error.

In debug mode the volume server logs: I1029 16:18:28 30121 weed.go:225] [volume 35 reading 0xc21000fe60] I1029 16:18:28 30121 weed.go:225] [read bytes -1 error Not Found] I1029 16:18:28 30121 weed.go:225] [read error: Not Found /35,289ebbe3630a]

What version of the product are you using? On what operating system?

.43 linux

This seems a tough bug to reproduce.

If you try to get the same file again later, will it be successful?

No it always returns a 404 even after restarting the volume server.

Is this related to #51? Maybe volume 35 is assigned to other server previously and this server does not really have it?

No I don't think it is related to #51.

The volume 35 exists, the writing of 35,289ebbe3630a does not return any error but the file does not exist. Subsequent writes to the volume 35 it are working as expected, every files are readable. I will provide you some sample code tomorrow to reproduce the issue.

Here is some sample code to reproduce the issue: https://github.com/zenria/weed-fs/commit/9ea74e4a5670530dd921d8bd1eadfce761f18ee2

Issue 46 has been merged into this issue.

Thanks a lot for the test case! It helped tremendously!

The problem was in the compact map implementation, where the CompactSection needs to be kept sorted. This is fixed in version 0.44.

This issue still exists in 0.45.

I1206 17:06:33 03375 weed.go:226] [volume 3 reading 0x1a5540c0] I1206 17:06:33 03375 weed.go:226] [read bytes -1 error Not Found] I1206 17:06:33 03375 weed.go:226] [read error: Not Found /3,0572a4789ab9f1_6.jpg]

Is it possible to send me the data files? Specifically 3.dat and 3.idx.

This seems a different issue. The file should be written around 400MB.

The expected file is not in the 3.dat file you sent me. Do you have some error when saving the file?

Chris, I've sent you an email with more details.

If the files are not correctly written, there will be errors returned. From the data files you sent, I did not find the files. Can you make sure that you check the file writing response?