Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Need an option to tell LevelDB to perform fsync periodly #158

Closed
cmumford opened this issue Sep 9, 2014 · 1 comment
Closed

Need an option to tell LevelDB to perform fsync periodly #158

cmumford opened this issue Sep 9, 2014 · 1 comment
Assignees

Comments

@cmumford
Copy link
Contributor

cmumford commented Sep 9, 2014

Original issue 152 created by wuzuyang on 2013-03-22T05:30:49.000Z:

What steps will reproduce the problem?

  1. Open a database
  2. ls -l in the database directory
  3. Perform several Put operation
  4. Wait for 10 seconds
  5. ls -l in the same directory

What is the expected output? What do you see instead?

The size of *.log should change. But it is not, all files remain unmodified.

What version of the product are you using? On what operating system?

1.8.0, Linux

Please provide any additional information below.

  • Feature required

Provide an option(sync, etc) when Open a database, if it is set to 1(one second), a background thread will perform fsync on opened files every second.

As Linux kernel actually commits writes after 30 seconds, this is a long time, may cause a lot data loss on machine failure.

@cmumford
Copy link
Contributor Author

cmumford commented Sep 9, 2014

Comment #1 originally posted by sanjay@google.com on 2013-03-25T23:22:30.000Z:

First, leveldb does something unusual that might be worth knowing about: the size of the *.log file is not expected to change very fast since the log file is pre-enlarged and then mmapped and written to directly using memcpy.

About your feature request: there are already ways to achieve this.
(a) Kernel tuning flags should allow tweaking the 30 second delay downwards (I think the parameter you are looking for is /proc/sys/vm/dirty_expire_centisecs).

(b) If you want to msync() just the leveldb data, you can achieve this by having a background thread of your own that just does a sync=true innocuous write to the database. Deleting a non-existent key or writing an empty string to a key you do not use should suffice.

Given that there are existing mechanisms to achieve what you want, any feature we add would just be a convenience. So I am not inclined to add the complexity of such a feature to leveldb.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant