#4507 kojipkgs redirects from https to http for directories with missing trailing slashes
Closed: Fixed 2 years ago Opened 4 years ago by till.

This can be easily seen with curl:
$ curl -sI https://kojipkgs.fedoraproject.org/packages/libHX | grep Location
Location: http://kojipkgs.fedoraproject.org/packages/libHX/

It is easyfix if someone figures out where this host this configured I guess, it looks like it is in puppet.

I'd like to try and fix this, however as an apprentice I can't log in to kojipkgs02.phx2 itself - could I get read-only access there to take a look at the deployed configuration? (going to try and reproduce it in my own virtual machine before submitting the fix)


I've added you to the sysadmin and sysadmin-noc groups. That should get you access there.

Note that we are heading into freeze tomorrow, so after that we will need to get a freeze break to commit a fix.

Replying to [comment:3 kevin]:

I've added you to the sysadmin and sysadmin-noc groups. That should get you access there.

Note that we are heading into freeze tomorrow, so after that we will need to get a freeze break to commit a fix.

Thanks Kevin. I have retrieved the Squid and Apache configuration files from kojipkgs02 and placed them in my test VM, and oddly could not replicate the bug yet.. given the freeze, if I find a solution it could probably wait until after F21 is out before we apply it.

Anyone knows if there's another service running on that server apart from httpd and squid, that could be affecting how Apache is serving files? On the server, browsing to https://kojipkgs.fedoraproject.org/packages/ shows

Please browse to: \

while on my machine I just get a directory listing. Perhaps the odd redirection is (mis-)configured there as well.

There shouldn't be any other services running there. ;(

Behavior replicated after the move to the new kojipkgs01 configured using Ansible. Will try and take a look.

Yeah, hopefully it will be easier to debug now?

Happy to help out if you come up with a proposed fix. :)

Per http://www.gossamer-threads.com/lists/apache/users/347885 -- which is a similar but not identical problem -- this might be because Apache sends back a redirect with the trailing slash, and this is then fixed up by Squid so the port number is changed from 8080 to 80 (since Squid is serving on both 80 and 443).

We could try using mod_headers to rewrite the header on the way out, but since Apache doesn't actually know which port the request originally comes from... hmm.

I see two solutions:
- get Squid to be rewrite the outgoing redirection from Apache with the correct port depending on which port was originally used
- set up Apache to serve on both port 8080 and 8081, redirect Squid on port 80 to 8080 and Squid on port 443 to 8081, and use Apache's mod_headers to rewrite redirections with the correct port (8080->http:80, 8081->https:443)


Well, we have been talking about making it redirect http to https anyhow. I'm not sure if squid can do that, but if we can get it to do so, then would that solve things? Just make port 80 requests redirect to 443 and have that send the real reply.

I suppose perhaps we could even do that in apache?

Yes, once you only need to support HTTPS (with requests over HTTP being redirected) the configuration becomes much simpler.

That leaves one final decision:
- do you still want Squid running on both ports 80 and 443, or just on 443, with Apache serving on port 80 (it's only doing redirects after all)

IMHO it would also be fine to just not server port 80 at all, since everything links to https.

Actually not everything. The builders still use http I am pretty sure. This would require a change in koji.

Hi I'm new in fi-apprentice and I want to start with this easyfix but I don't know how, but I think maybe getting access to the server I will got a clue.

Unfortunately, that server is pretty high security as it's used by all our build machines to build Fedora packages.

However, the config should be in ansible so you could duplicate the setup locally. Look in:


the files there has the squid and apache configs.

After good day of reading Squid, mod_headers and mod_dir documentation I only came up with this (see patch attached).
The change in roles/kojipkgs/files/kojipkgs.conf can be applied server-wide, if we need this behaviour for all URLs.
I have testing environment handy, so can test any other config changes quickly.



If no one is currently working on this, I found an apache module '''mod_dir''' that looks promising. The
module has a directive '''DirectoryCheckHandler''' - when set to '''on''' it claims to ''determine whether mod_dir should check for directory indexes or add trailing slashes when some other handler has been configured for the current URL. Handlers can be set by directives such as SetHandler or by other modules, such as mod_rewrite during per-directory substitutions.''


There are a couple other directives we can look at as well.

Regards - Zach

I am not having success trying to recreate this issue. I am using a CentOS 7 VM with apache 2.4.6
Release : 31.el7.centos.1 and Squid Version : 3.3.8 Release : 12.el7_0. The version of apache matches, but the C7 version of Squid is behind what I believe to be on kojipkgs Squid Version : 3.3.8 Release : 13.el7. Here is a link to a diff of my squid.conf with the one from ansible inventory [ https://paste.fedoraproject.org/288619/ ] and kojipkgs.conf [ https://paste.fedoraproject.org/288621/ ] (I used a pretty standard httpd.conf). I will go ahead a test the DirectoryCheckHandler with my version of squid to see if I get any breakage and will also see about maybe using Fedora Infra Cloud to set up the same version of squid.

added Rewrite rules for http->https redirect to kojipkgs.conf

The problem with the patch that @aikidouke wrote is that httpd here has no idea about https at all - all of the TLS decryption is handled by squid.

I think that what is happening here is that the redirect (that comes from httpd) is not being munged properly by squid:

[root@kojipkgs01 httpd][PROD]# tail -f access_log | fgrep kernel - - [06/Nov/2016:00:49:57 +0000] "GET /packages/kernel HTTP/1.1" 301 258 "-" "Wget/1.18 (linux-gnu)" - - [06/Nov/2016:00:49:57 +0000] "GET /packages/kernel/ HTTP/1.1" 200 58550 "-" "Wget/1.18 (linux-gnu)"

Now I'm not sure if this is an actual problem or not anymore due to HSTS:

[jstanley@dhcp-108 ~]$ wget https://kojipkgs.fedoraproject.org/packages/kernel
--2016-11-05 20:49:56-- https://kojipkgs.fedoraproject.org/packages/kernel
Resolving kojipkgs.fedoraproject.org (kojipkgs.fedoraproject.org)...
Connecting to kojipkgs.fedoraproject.org (kojipkgs.fedoraproject.org)||:443... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: http://kojipkgs.fedoraproject.org/packages/kernel/ [following]
URL transformed to HTTPS due to an HSTS policy
--2016-11-05 20:49:57-- https://kojipkgs.fedoraproject.org/packages/kernel/
Reusing existing connection to kojipkgs.fedoraproject.org:443.
HTTP request sent, awaiting response... 200 OK

Notice how wget transformed the request to https anyhow, due to the HSTS policy. I'd obviously still like to figure out why the redirect is wrong :)

Looking at it again, I am pretty sure this is apache trying to be nice and redirecting a directory without a trailing / to one with a trailing /

See: http://httpd.apache.org/docs/current/mod/mod_dir.html#directoryslash

And the fix is to set: ServerName https://kojipkgs.fedoraproject.org so it knows that the server scheme is https and it should use that for directory slash redirects. So, now it still does the redirect, but it's from https to https.


@kevin changed the status to Closed

2 years ago

Login to comment on this ticket.