My favorites | Sign in
Project Home Downloads
New issue   Search
for
  Advanced search   Search tips   Subscriptions
Issue 253: Read Repair has trouble when dealing with multiple concurrent versions on nodes
1 person starred this issue and may be notified of changes. Back to list
Status:  Started
Owner:  mramb...@gmail.com
Cc:  feinb...@gmail.com


Sign in to add a comment
 
Project Member Reported by mramb...@gmail.com, Jun 15, 2010
The Read Repairer generates entries that need read-repaired under the following scenarios:
- When a node contains multiple concurrent versions;
- When a node contains both current and obsolete versions.

Under these scenarios, the Read Repairer can generate many entries to be read-repaired, even though the node is up-to-date.

The problem has to do with how the "existing" structure is initialized and where.
Jun 17, 2010
Project Member #1 feinb...@gmail.com
Hey Mark,

Do you have an update on this? You can just paste the patch here and I'll commit it ASAP.

Thanks,
- Alex
Cc: feinberg
Jun 21, 2010
Project Member #2 feinb...@gmail.com
Mark submitted this patch by email:

diff --git a/src/java/voldemort/store/routed/ReadRepairer.java b/src/java/voldemort/store/routed/ReadRepairer.java
index 45667e4..a43c54a 100644
--- a/src/java/voldemort/store/routed/ReadRepairer.java
+++ b/src/java/voldemort/store/routed/ReadRepairer.java
@@ -79,6 +79,7 @@ public class ReadRepairer<K, V> {
 
         // A list of obsolete nodes that need to be repaired
         Set<Integer> obsolete = new HashSet<Integer>(3);
+        Set<NodeValue<K, V>> existing = new HashSet<NodeValue<K, V>>(nodeValues);
 
         // A Map of Version=>NodeValues that contains the current best estimate
         // of the set of current versions
@@ -133,7 +134,10 @@ public class ReadRepairer<K, V> {
                 NodeValue<K, V> repair = new NodeValue<K, V>(id,
                                                              concurrent.getKey(),
                                                              concurrent.getVersioned());
-                repairs.add(repair);
+                if(!existing.contains(repair)) {
+                    repairs.add(repair);
+                    existing.add(repair);
+                }
             }
         }
 
@@ -141,15 +145,16 @@ public class ReadRepairer<K, V> {
             // if there are more then one concurrent versions on different
             // nodes,
             // we should repair so all have the same set of values
-            Set<NodeValue<K, V>> existing = new HashSet<NodeValue<K, V>>(repairs);
             for(NodeValue<K, V> entry1: concurrents.values()) {
                 for(NodeValue<K, V> entry2: concurrents.values()) {
                     if(!entry1.getVersion().equals(entry2.getVersion())) {
                         NodeValue<K, V> repair = new NodeValue<K, V>(entry1.getNodeId(),
                                                                      entry2.getKey(),
                                                                      entry2.getVersioned());
-                        if(!existing.contains(repair))
+                        if(!existing.contains(repair)) {
                             repairs.add(repair);
+                            existing.add(repair);
+                        }
                     }
                 }
             }

Jun 21, 2010
Project Member #3 ismaelj
It would be nice to add a test for this.
Nov 18, 2010
Project Member #4 mramb...@gmail.com
The changes were fixed in and unit tests were added in the read repair branch:

https://github.com/mrambacher/voldemort/tree/read_repair/test/unit/voldemort/store/routed/
Status: Fixed
Nov 23, 2010
Project Member #5 mramb...@gmail.com
As requested, I am leaving this issue as open until the changes have been merged to master.
Status: Accepted
Dec 14, 2010
Project Member #6 mramb...@gmail.com
(No comment was entered for this change.)
Status: Started
Sign in to add a comment

Powered by Google Project Hosting