************************************************************
***** NOTE: THIS BUG TRACKER IS FOR GERRIT CODE REVIEW *****
***** DO NOT SUBMIT BUGS FOR CHROME, ANDROID, INTERNAL *****
***** ISSUES WITH YOUR COMPANY'S GERRIT SETUP, ETC. *****
***** THOSE ISSUE BELONG IN DIFFERENT ISSUE TRACKERS! *****
************************************************************
Affected Version: 2.5
What steps will reproduce the problem?
1. Add a sufficiently large replication delay to replication.config so that it's possible to have a replication job in Gerrit's queue before it's processed by Gerrit ( replicationDelay = 35 )
2. Create a change and upload it to Gerrit
3. Remove the replication plugin while the job is queued by Gerrit waiting to run
What is the expected output? What do you see instead?
The replication plugin is never removed by Gerrit. It's necessary to restart Gerrit but before restarting, the replication.jar.disabled file needs to be renamed to replication.jar. If not, the plugin does not start on a Gerrit restart.
Please provide any additional information below.
Replication config:
[remote "slave"]
url = git://slave:9419/${name}.git
adminUrl = slaveuser@slave:/git/repositories/${name}.git
push = +refs/*:refs/*
replicationDelay = 35
threads = 10
mirror = true
Confirm queue is empty:
Task State Command
------------------------------------------------------------------------------
36fdcf6f 12:11:10.675 Log File Compressor
------------------------------------------------------------------------------
1 tasks
Plugin filename:
ls plugins/
replication.jar
Plugin is enabled:
ssh -p 29418 master gerrit plugin ls
Name Version Status
-------------------------------------------------------------------------------
replication 1.0 ENABLED
Push a change to Gerrit
git push origin HEAD:refs/for/master
Task State Command
------------------------------------------------------------------------------
d7db1a63 11:20:58.880 push git://slave:9419/myproject
36fdcf6f 12:11:10.676 Log File Compressor
------------------------------------------------------------------------------
2 tasks
Remove replication plugin:
ssh -p 29418 master gerrit plugin remove replication
ls plugins/
replication.jar.disabled
Gerrit's LOG while trying to remove plugin
[2013-03-31 11:20:44,621] INFO com.google.gerrit.server.plugins.PluginLoader : Disabling plugin replication
[2013-03-31 11:20:44,621] WARN com.googlesource.gerrit.plugins.replication.ReplicationQueue : Cancelled 1 replication events during shutdown
[2013-03-31 11:21:48,589] INFO com.google.gerrit.server.plugins.PluginLoader : Cleaned plugin plugin_replication_130331_1120_4820055441694090468.jar
[2013-03-31 11:21:48,590] INFO com.google.gerrit.server.plugins.PluginLoader : Cleaned plugin plugin_replication_130331_1111_3769375487048786266.jar
[2013-03-31 11:21:48,590] WARN com.google.gerrit.server.plugins.PluginLoader : 1 plugins still waiting to be reclaimed after 1 minutes
[2013-03-31 11:22:26,100] INFO com.google.gerrit.server.plugins.PluginLoader : Cleaned plugin plugin_replication_130331_1121_1666127951484955268.jar
[2013-03-31 11:22:26,101] WARN com.google.gerrit.server.plugins.PluginLoader : 1 plugins still waiting to be reclaimed after 1 minutes
[2013-03-31 11:25:36,219] INFO com.google.gerrit.server.plugins.PluginLoader : Cleaned plugin plugin_replication_130331_1122_1424127429921268974.jar
[2013-03-31 11:25:36,220] INFO com.google.gerrit.server.plugins.PluginLoader : Cleaned plugin plugin_replication_130331_1124_253066937408235522.jar
[2013-03-31 11:25:36,220] INFO com.google.gerrit.server.plugins.PluginLoader : Cleaned plugin plugin_replication_130331_1123_4188205777389701287.jar
[2013-03-31 11:25:36,220] WARN com.google.gerrit.server.plugins.PluginLoader : 1 plugins still waiting to be reclaimed after 4 minutes
[2013-03-31 11:29:50,172] INFO com.google.gerrit.server.plugins.PluginLoader : Cleaned plugin plugin_replication_130331_1128_1462254843060851799.jar
[2013-03-31 11:29:50,172] INFO com.google.gerrit.server.plugins.PluginLoader : Cleaned plugin plugin_replication_130331_1127_8797027389254032869.jar
[2013-03-31 11:29:50,172] INFO com.google.gerrit.server.plugins.PluginLoader : Cleaned plugin plugin_replication_130331_1125_209452049010666226.jar
[2013-03-31 11:29:50,172] INFO com.google.gerrit.server.plugins.PluginLoader : Cleaned plugin plugin_replication_130331_1126_6693391358698355539.jar
[2013-03-31 11:29:50,172] WARN com.google.gerrit.server.plugins.PluginLoader : 1 plugins still waiting to be reclaimed after 9 minutes
The number of Plugin Cleaner jobs increases until it reaches a limit, then starts over.
Task State Command
------------------------------------------------------------------------------
57196a36 11:21:44.623 Plugin Cleaner (waiting for 2 plugins)
36fdcf6f 12:11:10.676 Log File Compressor
------------------------------------------------------------------------------
2 tasks
Task State Command
------------------------------------------------------------------------------
57196a36 11:21:44.622 Plugin Cleaner (waiting for 3 plugins)
36fdcf6f 12:11:10.675 Log File Compressor
------------------------------------------------------------------------------
2 tasks
Task State Command
------------------------------------------------------------------------------
7724aea1 11:22:18.589 Plugin Cleaner (waiting for 1 plugins)
36fdcf6f 12:11:10.675 Log File Compressor
------------------------------------------------------------------------------
2 tasks
Task State Command
------------------------------------------------------------------------------
7724aea1 11:22:18.589 Plugin Cleaner (waiting for 2 plugins)
36fdcf6f 12:11:10.675 Log File Compressor
------------------------------------------------------------------------------
2 tasks
STOP GERRIT, then RESTART
ls plugins/
replication.jar.disabled
ssh -p 29418 master gerrit plugin ls
Name Version Status
-------------------------------------------------------------------------------
Task State Command
------------------------------------------------------------------------------
37845e2c 11:33:48.957 Plugin Cleaner (waiting for 1 plugins)
d7862234 12:32:48.950 Log File Compressor
------------------------------------------------------------------------------
2 tasks
Task State Command
------------------------------------------------------------------------------
5730f2d1 11:37:20.295 Plugin Cleaner (waiting for 3 plugins)
d7862234 12:32:48.951 Log File Compressor
------------------------------------------------------------------------------
2 tasks
[2013-03-31 11:33:49,421] INFO com.google.gerrit.server.plugins.PluginLoader : Cleaned plugin plugin_replication_130331_1132_5939560954018629418.jar
[2013-03-31 11:33:49,421] WARN com.google.gerrit.server.plugins.PluginLoader : 1 plugins still waiting to be reclaimed after 1 minutes
[2013-03-31 11:34:20,296] WARN com.google.gerrit.server.plugins.PluginLoader : 1 plugins still waiting to be reclaimed after 1 minutes
[2013-03-31 11:37:21,572] INFO com.google.gerrit.server.plugins.PluginLoader : Cleaned plugin plugin_replication_130331_1133_5538450256879605994.jar
[2013-03-31 11:37:21,572] INFO com.google.gerrit.server.plugins.PluginLoader : Cleaned plugin plugin_replication_130331_1135_1955457217262328990.jar
[2013-03-31 11:37:21,572] INFO com.google.gerrit.server.plugins.PluginLoader : Cleaned plugin plugin_replication_130331_1134_2102375682649291206.jar
[2013-03-31 11:37:21,572] WARN com.google.gerrit.server.plugins.PluginLoader : 1 plugins still waiting to be reclaimed after 4 minutes