Issue 2060: Change marked as merged, but isn't merged.
Status:  New
Owner: ----
Reported by docw...@gmail.com, Aug 16, 2013
Affected Version: 2.6.1

What steps will reproduce the problem?

1. A user "Submit"ted a change.
2. Gerrit marked the change as MERGED
3. However, a git clone of the repository shows no trace of the commit in question.

What is the expected output? What do you see instead?

I expected either: a) the commit to be marked as not merged or b) the commit to be merged.

Please provide any additional information below.

In the traceback, it says the DB went away.  Possibly the network hiccuped or something, because the DB was shown as up the whole time in our monitoring system.

I would think that gerrit would be able to recover from a DB problem anyway.

As a work-around I manually pushed the commit directly to master.  It is bad if we cannot trust Gerrit's UI to show us the proper state of the repository.

Here's the traceback:

[2013-08-15 11:46:18,469] ERROR com.google.gerrit.server.git.ChangeMergeQueue : Merge attempt for my_project,refs/heads/master failed
com.google.gerrit.server.git.MergeException: Cannot query the database
	at com.google.gerrit.server.git.MergeOp.merge(MergeOp.java:321)
	at com.google.gerrit.server.git.ChangeMergeQueue$2.call(ChangeMergeQueue.java:207)
	at com.google.gerrit.server.git.ChangeMergeQueue$2.call(ChangeMergeQueue.java:204)
	at com.google.gerrit.server.util.RequestScopePropagator$5.call(RequestScopePropagator.java:222)
	at com.google.gerrit.server.util.RequestScopePropagator$4.call(RequestScopePropagator.java:201)
	at com.google.gerrit.server.git.PerThreadRequestScope$Propagator$1.call(PerThreadRequestScope.java:75)
	at com.google.gerrit.server.git.ChangeMergeQueue.mergeImpl(ChangeMergeQueue.java:204)
	at com.google.gerrit.server.git.ChangeMergeQueue.merge(ChangeMergeQueue.java:124)
	at com.google.gerrit.server.change.Submit.apply(Submit.java:111)
	at com.google.gerrit.server.change.Submit.apply(Submit.java:53)
	at com.google.gerrit.httpd.restapi.RestApiServlet.service(RestApiServlet.java:288)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
	at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
	at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178)
	at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62)
	at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
	at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
	at com.google.gwtexpui.server.CacheControlFilter.doFilter(CacheControlFilter.java:70)
	at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
	at com.google.gerrit.httpd.RequireSslFilter.doFilter(RequireSslFilter.java:68)
	at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
	at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
	at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
	at com.google.gerrit.httpd.AllRequestFilter$FilterProxy$1.doFilter(AllRequestFilter.java:64)
	at com.google.gerrit.httpd.AllRequestFilter$FilterProxy.doFilter(AllRequestFilter.java:57)
	at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
	at com.google.gerrit.httpd.RequestContextFilter.doFilter(RequestContextFilter.java:75)
	at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
	at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
	at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
	at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1307)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:453)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1072)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:382)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1006)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
	at org.eclipse.jetty.server.Server.handle(Server.java:365)
	at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:485)
	at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:937)
	at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:998)
	at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:856)
	at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
	at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
	at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:627)
	at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:51)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
	at java.lang.Thread.run(Thread.java:780)
Caused by: com.google.gwtorm.server.OrmException: fetch failure on changes
	at com.google.gwtorm.schema.sql.SqlDialect.convertError(SqlDialect.java:151)
	at com.google.gwtorm.jdbc.JdbcAccess.convertError(JdbcAccess.java:448)
	at com.google.gwtorm.jdbc.JdbcAccess.queryList(JdbcAccess.java:146)
	at com.google.gerrit.reviewdb.client.Change_Access_changes_GwtOrm$$13.submitted(Unknown Source)
	at com.google.gerrit.server.git.MergeOp.merge(MergeOp.java:273)
	... 61 more
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet successfully received from the server was 683,009 milliseconds ago.  The last packet sent successfully to the server was 0 milliseconds ago.
	at sun.reflect.GeneratedConstructorAccessor107.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:528)
	at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
	at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1117)
	at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3603)
	at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3492)
	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4043)
	at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2503)
	at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2664)
	at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2794)
	at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2155)
	at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:2322)
	at org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:96)
	at org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:96)
	at com.google.gwtorm.jdbc.JdbcAccess.queryList(JdbcAccess.java:134)
	... 63 more
Caused by: java.io.EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost.
	at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:3052)
	at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3503)
	... 73 more

Aug 16, 2013
#1 docw...@gmail.com
More information....

I had a total of 10 of these over a 24 hour period (obviously, something was up that day).  I manually confirmed that the other 9 failed merges all merged correctly into git.

I picked one of the successful failures at random and it is identical to the one above except for the project name, timestamp and the "last packet successfully received from the server" milliseconds value.

Hope that helps. :-/
Nov 19, 2013
#2 barry...@gmail.com
this problem can be resolved.