My favorites | Sign in
Project Home Downloads
New issue   Search
  Advanced search   Search tips   Subscriptions
Issue 253: Read Repair has trouble when dealing with multiple concurrent versions on nodes
1 person starred this issue and may be notified of changes. Back to list
Status:  Started

Sign in to add a comment
Project Member Reported by, Jun 15, 2010
The Read Repairer generates entries that need read-repaired under the following scenarios:
- When a node contains multiple concurrent versions;
- When a node contains both current and obsolete versions.

Under these scenarios, the Read Repairer can generate many entries to be read-repaired, even though the node is up-to-date.

The problem has to do with how the "existing" structure is initialized and where.
Jun 17, 2010
Project Member #1
Hey Mark,

Do you have an update on this? You can just paste the patch here and I'll commit it ASAP.

- Alex
Cc: feinberg
Jun 21, 2010
Project Member #2
Mark submitted this patch by email:

diff --git a/src/java/voldemort/store/routed/ b/src/java/voldemort/store/routed/
index 45667e4..a43c54a 100644
--- a/src/java/voldemort/store/routed/
+++ b/src/java/voldemort/store/routed/
@@ -79,6 +79,7 @@ public class ReadRepairer<K, V> {
         // A list of obsolete nodes that need to be repaired
         Set<Integer> obsolete = new HashSet<Integer>(3);
+        Set<NodeValue<K, V>> existing = new HashSet<NodeValue<K, V>>(nodeValues);
         // A Map of Version=>NodeValues that contains the current best estimate
         // of the set of current versions
@@ -133,7 +134,10 @@ public class ReadRepairer<K, V> {
                 NodeValue<K, V> repair = new NodeValue<K, V>(id,
-                repairs.add(repair);
+                if(!existing.contains(repair)) {
+                    repairs.add(repair);
+                    existing.add(repair);
+                }
@@ -141,15 +145,16 @@ public class ReadRepairer<K, V> {
             // if there are more then one concurrent versions on different
             // nodes,
             // we should repair so all have the same set of values
-            Set<NodeValue<K, V>> existing = new HashSet<NodeValue<K, V>>(repairs);
             for(NodeValue<K, V> entry1: concurrents.values()) {
                 for(NodeValue<K, V> entry2: concurrents.values()) {
                     if(!entry1.getVersion().equals(entry2.getVersion())) {
                         NodeValue<K, V> repair = new NodeValue<K, V>(entry1.getNodeId(),
-                        if(!existing.contains(repair))
+                        if(!existing.contains(repair)) {
+                            existing.add(repair);
+                        }

Jun 21, 2010
Project Member #3 ismaelj
It would be nice to add a test for this.
Nov 18, 2010
Project Member #4
The changes were fixed in and unit tests were added in the read repair branch:
Status: Fixed
Nov 23, 2010
Project Member #5
As requested, I am leaving this issue as open until the changes have been merged to master.
Status: Accepted
Dec 14, 2010
Project Member #6
(No comment was entered for this change.)
Status: Started
Sign in to add a comment

Powered by Google Project Hosting