| Issue 3412: | Create Project is blocking on replication plugin | |
| 1 person starred this issue and may be notified of changes. | Back to list |
Affected Version: 2.10.2
What steps will reproduce the problem?
1. Have a large replication queue to any Gerrit slave
2. Attempt to create a project over the REST API
What is the expected output? What do you see instead?
Expect project to create normally, but instead, the request times out at the reverse proxy.
Please provide any additional information below.
I captured the following jstack output:
"HTTP-1366923" #1366923 prio=5 os_prio=0 tid=0x00007fbd82e40000 nid=0x42ea waiting for monitor entry [0x00007fbd4bffc000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.eclipse.jgit.transport.JschConfigSessionFactory.getSession(JschConfigSessionFactory.java:93)
- waiting to lock <0x00007fc1d5e45300> (a com.google.gerrit.server.git.LocalDiskRepositoryManager$Lifecycle$1)
at com.googlesource.gerrit.plugins.replication.ReplicationQueue.connect(ReplicationQueue.java:416)
at com.googlesource.gerrit.plugins.replication.ReplicationQueue.executeRemoteSsh(ReplicationQueue.java:396)
at com.googlesource.gerrit.plugins.replication.ReplicationQueue.createRemoteSsh(ReplicationQueue.java:277)
at com.googlesource.gerrit.plugins.replication.ReplicationQueue.createProject(ReplicationQueue.java:238)
at com.googlesource.gerrit.plugins.replication.ReplicationQueue.onNewProjectCreated(ReplicationQueue.java:151)
at com.google.gerrit.server.project.PerformCreateProject.createProject(PerformCreateProject.java:130)
at com.google.gerrit.server.project.CreateProject.apply(CreateProject.java:142)
at com.google.gerrit.server.project.CreateProject.apply(CreateProject.java:51)
at com.google.gerrit.httpd.restapi.RestApiServlet.service(RestApiServlet.java:304)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
at com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:279)
at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:269)
at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:180)
at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85)
at com.google.gerrit.httpd.RequireIdentifiedUserFilter.doFilter(RequireIdentifiedUserFilter.java:55)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gwtexpui.server.CacheControlFilter.doFilter(CacheControlFilter.java:70)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.RunAsFilter.doFilter(RunAsFilter.java:113)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.RequireSslFilter.doFilter(RequireSslFilter.java:68)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.ProjectDigestFilter.doFilter(ProjectDigestFilter.java:107)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.AllRequestFilter$FilterProxy$1.doFilter(AllRequestFilter.java:64)
at com.google.gerrit.httpd.AllRequestFilter$FilterProxy.doFilter(AllRequestFilter.java:57)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.gerrit.httpd.RequestContextFilter.doFilter(RequestContextFilter.java:75)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:119)
at com.google.inject.servlet.GuiceFilter$1.call(GuiceFilter.java:133)
at com.google.inject.servlet.GuiceFilter$1.call(GuiceFilter.java:130)
at com.google.inject.servlet.GuiceFilter$Context.call(GuiceFilter.java:203)
- locked <0x00007fc2db8a9718> (a com.google.inject.servlet.GuiceFilter$Context)
at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:130)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1636)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:564)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:219)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1111)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:498)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:183)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1045)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:98)
at org.eclipse.jetty.server.Server.handle(Server.java:461)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:284)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:534)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
at java.lang.Thread.run(Thread.java:745)
Since there's a long replication queue ahead of the creation, the creation times out. This would better be handled if projects were created asynchronously/queued for replication on the mirrors, instead of requiring synchronous creation.
Jun 5, 2015
Project Member
#1
dougk....@gmail.com
Oct 21, 2015
(No comment was entered for this change.)
Status:
Duplicate
Mergedinto: 3625
Oct 21, 2015
Picking 3625, since it has slightly more detail on this issue. :) |
|
| ► Sign in to add a comment |