Issue 29: Database error during push to refs/for/gerrit2 doesn't log actual failure
Status:  Duplicate
Owner: ----
Closed:  Sep 2009

Blocked on:
issue 31
Reported by code-rev...@gtempaccount.com, Sep 24, 2009
Reported by Shawn Pearce <sop@google.com> on Fri Jan 16 19:06:35 PST 2009
Source: JIRA GERRIT-29
Affected Version: 2.0

While trying to push I encountered a database error.  The server log has no
additional information.  The client has no data.  There's no way to debug what
went wrong.

$ git push ssh://gerrit2.source.android.com:29418/tools/gerrit.git
devwork:refs/for/gerrit2
Counting objects: 131, done.
Delta compression using 4 threads.
Compressing objects: 100% (72/72), done.
Writing objects: 100% (88/88), 18.02 KiB, done.
Total 88 (delta 61), reused 0 (delta 0)
To ssh://gerrit2.source.android.com:29418/tools/gerrit.git
 ! [remote rejected] devwork -> refs/for/gerrit2 (database error)
error: failed to push some refs to 'ssh://gerrit2.source.android.com:29418/
tools/gerrit.git'
Sep 24, 2009
#1 code-rev...@gtempaccount.com
Comment by Shawn Pearce <sop@google.com> on Fri Jan 16 19:16:11 PST 2009

The cause of the database error has been fixed, but I'm keeping this issue
open to deal with logging in the Receive task
Sep 24, 2009
#2 code-rev...@gtempaccount.com
Update by Shawn Pearce <sop@google.com> on Wed Jan 21 12:43:44 PST 2009
Blockedon: 31
Sep 24, 2009
#3 code-rev...@gtempaccount.com
Comment by Shawn Pearce <sop@google.com> on Wed Jan 21 12:43:44 PST 2009

Basically the same issue as GERRIT-31 at this point.
Sep 24, 2009
#4 code-rev...@gtempaccount.com
Update by Shawn Pearce <sop@google.com> on Wed Jan 21 12:44:01 PST 2009
Status: Duplicate
Sep 29, 2009
#5 arun.sri...@gmail.com
Hi Shawn,
    we have a user who ran into the same remote rejected (database error) as the one 
you have pasted above. What is the cause of this kind of error? his changes are up-
to-date.

thanks,
-Arun
Sep 29, 2009
#6 sop+code@google.com
Anything in the server log?  I thought I had put enough logging into the server
code around this area that we would at least have stack traces to help debug a
case like this again.

I have no idea what I meant in comment #1 about the database error being fixed,
it may have been a database connectivity error or something that was specific to
the gerrit2 host at that time.  Too bad I didn't describe it further, but I must
have thought it was unrelated to the code but was tied to the machine environment,
and thus not worthy of describing further.
Owner: ---
Sep 29, 2009
#7 arun.sri...@gmail.com
hopefully I copied the relevant portion in here... 

Caused by: java.sql.BatchUpdateException: Batch entry 30 INSERT INTO patches
(change_type,patch_type,nbr_comments,source_file_name,change_id,patch_set_id,file_nam
e)VALUES('A','U','0',NULL,'74','1','include/linux/GlobalTypes.h') was aborted.  Call 
getNextException to see the cause.
        at org.postgresql.jdbc2.AbstractJdbc2Statement$BatchResultHandler.handleError
(AbstractJdbc2Statement.java:2569)
        at org.postgresql.core.v3.QueryExecutorImpl.processResults
(QueryExecutorImpl.java:1796)
        at org.postgresql.core.v3.QueryExecutorImpl.execute
(QueryExecutorImpl.java:407)
        at org.postgresql.jdbc2.AbstractJdbc2Statement.executeBatch
(AbstractJdbc2Statement.java:2708)
        at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeBatch
(NewProxyPreparedStatement.java:1723)
        at com.google.gwtorm.jdbc.JdbcAccess.execute(JdbcAccess.java:228)
        at com.google.gwtorm.jdbc.JdbcAccess.doInsert(JdbcAccess.java:167)
        ... 16 more
Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value violates 
unique constraint "patches_pkey"
        at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse
(QueryExecutorImpl.java:2062)
        at org.postgresql.core.v3.QueryExecutorImpl.processResults
(QueryExecutorImpl.java:1795)
        ... 21 more
2009-09-29 12:19:43,026::ERROR: com.google.gerrit.server.ssh.commands.Receive  - 
Error creating change for commit b2eebe6c54e21a8a306b9a83fa8b092fed201c43
com.google.gwtorm.client.OrmDuplicateKeyException: patches
        at com.google.gwtorm.schema.sql.DialectPostgreSQL.convertError
(DialectPostgreSQL.java:47)

Sep 29, 2009
#8 sop+code@google.com
Interesting bug; there are two records for the file 'include/linux/GlobalTypes.h'.
This should not have happened, and is why the database is choking.

I have not seen this happen before, but it must be on a pre 2.0.21 version of Gerrit
because that particular database table was removed in 2.0.21.

So two things:

  1) This issue has nothing to do with  issue 29 , and should be in its own
     unique issue, because I know its an unrelated bug.

  2) I think 2.0.21 works around this by removing the code which is having
     a problem.  But I don't know what .21 would do if there were duplicate
     records for the same file, it might fail with a different exception.
Sep 29, 2009
#9 arun.sri...@gmail.com
it is 2.0.19 version

I don't understand what it means to say that there are duplicate records for the 
same file. from the user's perspective, what needs to be checked to ensure that this 
can be fixed?

thanks 
-Arun

Sep 29, 2009
#10 sop+code@google.com
Somehow when Gerrit was scanning the output of `git diff-tree -M -p ID^1 ID` it came
up with the notion that there were two different records for the same file path.

My initial guess is, there is a delete and add pair for the same file path which
somehow got broken up in diffcore to perform rename detection, but never got put
back together again?  I'm honestly just guessing at things now, I don't really
have a great idea as to how we would get two records for the same path.
Sep 29, 2009
#11 arun.sri...@gmail.com
ok... so I got more info from the user.

It appears that this file is actually a symlink pointing to a file in another 
directory.. could it be possible that the file existed there and a symlink was also 
created on it later that caused the confusion? 
Oct 1, 2009
#12 sop+code@google.com
Possible.  Gerrit treats symlinks like text files though; the link target path
is the sole line in the text file, but the link name itself is a file name.

If a file was deleted and replaced by a symlink, that could cause a failure,
we would have two records for the same path.