My favorites | Sign in
Project Home Downloads Wiki Issues Source
New issue   Search
for
  Advanced search   Search tips   Subscriptions
Issue 3365: Redirection is broken when Apache mod_rewrite is used for Proxying
3 people starred this issue and may be notified of changes. Back to list
Status:  Accepted
Owner:  ----
Cc:  David.Os...@gmail.com, huga...@gmail.com, marco.mm...@gmail.com


Sign in to add a comment
 
Project Member Reported by zaro0508, May 13, 2015

Affected Version: 2.10

What steps will reproduce the problem?

We setup our Gerrit server (ver 2.8.4) to use openid  ('openIdSsoUrl = https://login.launchpad.net/+openid') which worked fine.  When you login it redirects you to 'https://review.openstack.org/'  but after we upgraded to Gerrit ver 2.10.2 the post login redirect changed to 'https://review.openstack.org//', the extra forward slash in the URL causes links to the change IDs and commit SHAs to return 404. The link to a change id is actually a query 'https://review.openstack.org//#q,I91303ba1883f5eac9a5245ea67904a24f9bcb74c,n,z' 

That query does work with only one forward slash at the end of the hostname so i'm guessing the root cause of the problem is that extra forward slash. 

Please provide any additional information below.

This bug only occurred after we moved updated Gerrit to ver 2.10.2.  We tried to update the setting 'canonicalWebUrl = https://review-dev.openstack.org' but that did not fix this problem.

May 13, 2015
Project Member #1 zaro0508
correction, i meant setting 'canonicalWebUrl = https://review.openstack.org' in previous sentence.
May 13, 2015
Project Member #2 David.Os...@gmail.com
> This bug only occurred after we moved updated Gerrit to ver 2.10.2

Just wondering why you CC /me on this bug? Have you identified that my recent work on OAuth & Hybrid OpenID/OAuth stuff may be related?
Status: AwaitingInformation
May 13, 2015
Project Member #3 zaro0508
I was not able to repro this on my local machine using Gerrit master or the 2.10.2 version we used on  review.openstack.org.  Maybe it's some configuration on that machine that's setting the extra slash.  I'm still investigating.
Nov 4, 2015
Project Member #4 David.Os...@gmail.com
Can you point me to proxy Reverse Proxy on review-dev.openstack.org? LibreOffice Gerrit gerrit.libreoffice.org is using 2.10.7. We are not seeing this issue. When I'm trying to use review-dev.o.o I'm seeing double slashes: [1].

* [1] https://review-dev.openstack.org//#/dashboard/self
Nov 4, 2015
Project Member #6 David.Os...@gmail.com
Prosy settings from gerrit.libreoffice.org is here: [1].

[1] http://fpaste.org/287127/70879214
Nov 5, 2015
Project Member #7 huga...@gmail.com
I tested openid on my local machine and I could not reproduce the issue.

I tested all use case I could think of:
-redirection to gerrit home page
-redirection to gerrit page without parameter
-redirection to gerrit page with parameter

I did previous test casea with canonicalWebUrl ending with a slash and without.
Nov 5, 2015
Project Member #8 zaro0508
I tried changing my proxy settings to something similar to libreoffice but was not able to remove the extra forward slash.  I then tried to revert change https://gerrit-review.googlesource.com/#/c/57800 and re-deployed gerrit.war to review-dev.o.o which _did_ fix this issue.  So it's definitely that change that introduced this problem for us.
Nov 5, 2015
Project Member #9 huga...@gmail.com
I don't doubt that this change introduced the issue for you, I just need to find a way to reproduce it before attempting to fix it.

I will setup a reverse proxy similar to yours and try to reproduce the issue (We do not have this extra slash issue here at Ericsson, running gerrit 2.11.3).
Nov 5, 2015
Project Member #10 zaro0508
Thanks for testing.  I have proposed change https://gerrit-review.googlesource.com/#/c/72105  that fixes it for us. 
Nov 5, 2015
Project Member #11 david.pu...@sonymobile.com
https://gerrit-review.googlesource.com/#/c/72130/
Status: ChangeUnderReview
Nov 10, 2015
Project Member #12 zaro0508
Something changed between Gerrit 2.8 and 2.10 which cause our reverse proxy configuration to expose this issue.  We fixed this by changing the setting to the  recommended Gerrit proxy configs[1].  I suspect that our rewrite rule[2] was the culprit, we noticed that the double slash occurred when only when this rule was is in our proxy config.  We run gerrit on Ubuntu Trusty with Apache2 ver 2.4

[1] https://gerrit-review.googlesource.com/Documentation/config-reverseproxy.html#_apache_2_configuration
[2] RewriteRule ^/(.*)$ http://localhost:8081/$1 [NE,P]

Status: Invalid
Nov 19, 2015
Project Member #13 zaro0508
Abandoned change 72130 because it's not the correct fix, moving the conversation from that change to this issue.

The change I made to make this problem go away for use was to remove the our existing apache rewrite rule however I didn't realize that we really need that rule.  The purpose of the rule is to offload serving of the Gerrit 'http://host/p/' (note the '/p/') to apache instead of directly from gerrit.

I incorrectly evaluated this problem as Gerrit simply adding on an extra '/' character in the URL.  However I believe the real issue is actually that the redirect no longer works in Gerrit 2.11 with our SSO configuration.  I'm still not sure why it's not working though.  What I do know is that it worked in Gerrit 2.8 and does not work in Gerrit 2.11.  There has been no changes to our Apache configuration so something must have changed in Gerrit to cause this problem.   All signs seem to point to change 57800 as being the culprit.  

I'm trying to investigate that now but I'm finding it difficult to configure Gerrit with the exact same SSO auth on my local env as on review-dev.o.o.  I've attempted to setup the exact same auth config but Gerrit would not start on my local env.  

Status: AwaitingInformation
Cc: huga...@gmail.com
Nov 19, 2015
Project Member #14 huga...@gmail.com
As described in the commit message, change 57800 is fixing redirection after login to a page with parameters (e.g https://host/#/admin/groups/?filter=someFilter Before that change, Gerrit would have redirected to https://host/#/admin/groups/ instead of https://host/#/admin/groups/?filter=someFilter

I would prefer to not revert this change otherwise it will break the redirection with parameters and maybe more(IIRC, other changes depended on that one).

I am willing to help fixing the issue but I need your help to reproduce it.
Cc: marco.mm...@gmail.com
Nov 20, 2015
Project Member #15 zaro0508
Still not able to reproduce locally yet,  however I did notice the following:

This is what is the generated URL when you click 'sign in': 
  https://review-dev.openstack.org/login/%23%2Fq%2Fstatus%3Aopen

The single encoded token causes redirect to not work and adds the '//' at the end.

Conversely if you send this URL:
  https://review-dev.openstack.org/login/%2523%252Fq%252Fstatus%253Aopen

The redirect works fine and no '//' either.  I think there's some extra decoding going in either the gerrit client or in gerrit-httpd however i'm having a hard time locating where it's at.





 

Nov 22, 2015
Project Member #16 David.Os...@gmail.com
It seems to be related to mod_rewrite vs. mod_proxy.

When Gerrit site relies on rewrite rules, like:

  RewriteEngine on
  RewriteCond %{REQUEST_URI} !^/p/
  RewriteCond %{REQUEST_URI} !^/fakestore$
  RewriteCond %{REQUEST_URI} !^/robots.txt$
  RewriteRule ^/(.*)$ http://localhost:8081/$1 [NE,P]

redirection is broken with 57800. Reverting 57800 fixed all issues.
Apache version: [1]. Gerrit virtual host configuration: [2].

Interestingly, removing NE|noescape flag: [3] from the RewriteRule,

   RewriteRule ^/(.*)$ http://localhost:8081/$1 [P]

fixed the redirection, but broke side-by-side screen: slashes issue.

Replacing RewriteRule with ProxyPass fixed all issues:

   #RewriteRule ^/(.*)$ http://localhost:8081/$1 [NE,P]
   ProxyPass / http://127.0.0.1:8081/ nocanon

It worth noting, that current reverse proxy configuration was changed in: [4].
But at least the Troubleshooting section still mentions, that mod_rewrite
is supposed to work:

"alternatively a 'mod_rewrite' configuration with 'AllowEncodedSlashes NoDecode' set"

However it seems to be inherently broken since 57800.

[1] http://paste.openstack.org/show/479690
[2] https://github.com/openstack-infra/puppet-gerrit/blob/master/templates/gerrit.vhost.erb
[3] https://httpd.apache.org/docs/2.4/rewrite/flags.html#flag_ne
[4] https://gerrit-review.googlesource.com/46471
Summary: Redirection is broken when Apache mod_rewrite is used for Proxying (was: Redirect from openid contains an extra forward slash in the url)
Nov 22, 2015
Project Member #17 David.Os...@gmail.com
(No comment was entered for this change.)
Labels: -Priority-Minor Priority-Major
Nov 24, 2015
Project Member #18 zaro0508
This is a revert change 57800 to get back to gerrit redirect working with apache rewrite rules:   https://gerrit-review.googlesource.com/72720

fwiw, it looks like Bjorn had a similar issue with the URL encoding in this change: https://gerrit-review.googlesource.com/#/c/61230/2/gerrit-gwtui/src/main/java/com/google/gerrit/client/Gerrit.java

Status: Accepted
Dec 2, 2015
#19 icee...@googlemail.com
I am running it with an unpatched version again, using mod_rewrite proxying.
If I remember correctly, setting 
 AllowEncodedSlashes NoDecode 
on the host level in apache config solved the problem.
Dec 2, 2015
Project Member #20 zaro0508
@ieceelch, We did have AllowEncodedSlashes set to 'NoDecode' and still saw this issue.

We were finally able to find a ProxyPass config[1] that does the same thing as our existing RewriteRules so we are going to switch to using ProxyPass which works as described by DavidO above.  Support for mod_rewrite is still broken though so this is a valid issue and should be fixed.  

 [1] https://review.openstack.org/#/c/252054/

Sign in to add a comment

Powered by Google Project Hosting