My favorites | Sign in
Project Home Downloads Wiki Issues Source
New issue   Search
for
  Advanced search   Search tips   Subscriptions
Issue 3322: Unusual/poor handling of incomplete push via replication
2 people starred this issue and may be notified of changes. Back to list
Status:  New
Owner:  ----


Sign in to add a comment
 
Project Member Reported by dougk....@gmail.com, Apr 21, 2015
Affected Version: 2.9.1

What steps will reproduce the problem?
1. Unknown what led to this situation, other than the site has a very unreliable network connection, and may not have been dropped or server's power lost mid-replication
2. Replication is set up using git protocol over SSH tunneling

What is the expected output? What do you see instead?

Gerrit log:
[2015-04-21 12:49:50,827] ERROR com.googlesource.gerrit.plugins.replication.ReplicationQueue : Cannot replicate to git://localhost:9426/repository.git
org.eclipse.jgit.errors.TransportException: git://localhost:9426/repository.git: Broken pipe
	at org.eclipse.jgit.transport.BasePackPushConnection.doPush(BasePackPushConnection.java:217)
	at org.eclipse.jgit.transport.BasePackPushConnection.push(BasePackPushConnection.java:152)
	at org.eclipse.jgit.transport.PushProcess.execute(PushProcess.java:165)
	at org.eclipse.jgit.transport.Transport.push(Transport.java:1173)
	at org.eclipse.jgit.transport.Transport.push(Transport.java:1219)
	at com.googlesource.gerrit.plugins.replication.PushOne.pushVia(PushOne.java:394)
	at com.googlesource.gerrit.plugins.replication.PushOne.runImpl(PushOne.java:366)
	at com.googlesource.gerrit.plugins.replication.PushOne.runPushOperation(PushOne.java:271)
	at com.googlesource.gerrit.plugins.replication.PushOne.access$000(PushOne.java:78)
	at com.googlesource.gerrit.plugins.replication.PushOne$1.call(PushOne.java:244)
	at com.googlesource.gerrit.plugins.replication.PushOne$1.call(PushOne.java:241)
	at com.google.gerrit.server.util.RequestScopePropagator$5.call(RequestScopePropagator.java:222)
	at com.google.gerrit.server.util.RequestScopePropagator$4.call(RequestScopePropagator.java:201)
	at com.google.gerrit.server.git.PerThreadRequestScope$Propagator$1.call(PerThreadRequestScope.java:75)
	at com.googlesource.gerrit.plugins.replication.PushOne.run(PushOne.java:241)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
	at java.util.concurrent.FutureTask.run(FutureTask.java:166)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
	at com.google.gerrit.server.git.WorkQueue$Task.run(WorkQueue.java:364)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:724)
Caused by: java.net.SocketException: Broken pipe
	at java.net.SocketOutputStream.socketWrite0(Native Method)
	at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
	at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
	at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
	at java.io.BufferedOutputStream.write(BufferedOutputStream.java:126)
	at org.eclipse.jgit.internal.storage.pack.PackOutputStream.write(PackOutputStream.java:126)
	at org.eclipse.jgit.internal.storage.file.ByteArrayWindow.write(ByteArrayWindow.java:90)
	at org.eclipse.jgit.internal.storage.file.PackFile.copyAsIs2(PackFile.java:502)
	at org.eclipse.jgit.internal.storage.file.PackFile.copyAsIs(PackFile.java:357)
	at org.eclipse.jgit.internal.storage.file.WindowCursor.copyObjectAsIs(WindowCursor.java:190)
	at org.eclipse.jgit.internal.storage.pack.PackWriter.writeObjectImpl(PackWriter.java:1514)
	at org.eclipse.jgit.internal.storage.pack.PackWriter.writeObject(PackWriter.java:1491)
	at org.eclipse.jgit.internal.storage.pack.PackOutputStream.writeObject(PackOutputStream.java:164)
	at org.eclipse.jgit.internal.storage.file.WindowCursor.writeObjects(WindowCursor.java:196)
	at org.eclipse.jgit.internal.storage.pack.PackWriter.writeObjects(PackWriter.java:1479)
	at org.eclipse.jgit.internal.storage.pack.PackWriter.writeObjects(PackWriter.java:1466)
	at org.eclipse.jgit.internal.storage.pack.PackWriter.writePack(PackWriter.java:1036)
	at org.eclipse.jgit.transport.BasePackPushConnection.writePack(BasePackPushConnection.java:305)
	at org.eclipse.jgit.transport.BasePackPushConnection.doPush(BasePackPushConnection.java:197)
	... 23 more

From the replication mirror, running git fsck:
$ git fsck
error: object file ./objects/15/0ab62090a4bca701b49d3437b2768aec46445c is empty
error: unable to find 150ab62090a4bca701b49d3437b2768aec46445c
error: object file ./objects/15/0ab62090a4bca701b49d3437b2768aec46445c is empty
fatal: loose object 150ab62090a4bca701b49d3437b2768aec46445c (stored in ./objects/15/0ab62090a4bca701b49d3437b2768aec46445c) is corrupt

Manually pushing master branch from the master to the mirror:
$ git push git://localhost:9426/repository.git refs/heads/master:refs/heads/master
Counting objects: 5151, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (2677/2677), done.
remote: error: object file ./objects/92/1a676fb2126f71e440b9b0dd85c3b25c1325cc is empty
remote: error: unable to find 921a676fb2126f71e440b9b0dd85c3b25c1325cc
remote: fatal: SHA1 COLLISION FOUND WITH 921a676fb2126f71e440b9b0dd85c3b25c1325cc !
error: pack-objects died of signal 13
error: pack-objects died with strange error
error: failed to push some refs to 'git://localhost:9426/repository.git'

I ran a "git gc" manually, and git fsck cleared the corrupt (and unreferenced) objects, which allowed the push/replication to succeed.  Basically, more than anything else, I just wish there was a slightly cleaner error message from the replication plugin backtrace.
Sign in to add a comment

Powered by Google Project Hosting