#2888 Implement HTTP Strict Transport Security

Created 6 years ago by till
Modified 6 months ago

= problem =

People are accessing Fedora Websites with login pages without SSL prior to login. This allows active man in the middle attacks.

= analysis =

There is a new HTTP-header making it less likely that people do this. This helps to ensure that people use HTTPS when accessing Fedora websites where they enter their FAS password: http://en.wikipedia.org/wiki/HTTP_Strict_Transport_Security

= enhancement recommendation =
Add this header to relevant hosts such as
* admin.fedoraproject.org
* fedorahosted.org
* fedoraproject.org (for the wiki)

I'm not sure this matters much for admin.fedoraproject.org or the wiki...

admin redirects all http requests to https, so it shouldn't be possible to get a http page there before logging in.
The wiki redirects to https for anytime you login. (You can't use http with a logged in user). http is also cached, while https is not.

There may be cases where fedorahosted does the wrong thing tho, so I could see adding it there. Thoughts? Counterexamples?

Replying to [comment:1 kevin]:

I'm not sure this matters much for admin.fedoraproject.org or the wiki...

admin redirects all http requests to https, so it shouldn't be possible to get a http page there before logging in.

An active man in the middle attacker can change the response to the first http request to make sure the victim does not get redirected to https. This attack will be mitigated by this new header, because the browser will use https by default when only the hostname is entered in the address bar. With the header an attack will only be possible for the first unencrypted access to the website with the browser.

I cant understand how does server-side implementation work; after fist client request (http or https) server responses: "client! just use https!", and in case of an attack, attacker will cut this from server response, then client will send the passwords, and attacker will win! Besides, the wikipedia page says "Strict-Transport-Security headers must be sent via HTTPS responses only"! Nevertheless, it can be used to prevent those clients who started https some point, to go back http.
I think it can only be implemented in clients.

Replying to [comment:3 mahrud]:

I cant understand how does server-side implementation work; after fist client request (http or https) server responses: "client! just use https!",

Actually the header says "client! just use https for the next XYZ seconds" and XYZ can be high enough to cover several months or years.

and in case of an attack, attacker will cut this from server response, then client will send the passwords, and attacker will win! Besides, the wikipedia page says "Strict-Transport-Security headers must be sent via HTTPS responses only"! Nevertheless, it can be used to prevent those clients who started https some point, to go back http.
I think it can only be implemented in clients.

As long as a client received the header once, it will only use https for the amount of time specified in the header, i.e. if a users once accessed a respective site via https, they will do it until they did not access the site for a long time. If the attacker does not manage to manipulate the first request to the site, he won't be able to do so later.

This is done now for https://id.fedoraproject.org - but there are still hosts to go. IMHO this is a good easy fix ticket with two subtasks:

  1. Identify which services/host names should be protected (at least admin.fedoraproject.org and apps.fedoraproject.org come to my mind)
  2. Find the necessary config files in puppet and/or ansible and submit patches to adjust them.

Here is a patch for fedorapeople, which is also an appropriate candidate. It can also be added to https://hstspreload.appspot.com/ once the patch is applied.

fedorahosted can use HSTS as well.

Other candidates:

  • badges.fedoraproject.org
  • kojipkgs.fedoraproject.org
  • admin.fedoraproject.org
  • apps.fedoraproject.org

I added changes for copr.fpo to ansible, they need to be applied to enable HSTS there.

I've pushed those 2 patches (hosted and people) and run the playbook on copr.

Replying to [comment:10 kevin]:

I've pushed those 2 patches (hosted and people) and run the playbook on copr.

Thank you very much. There is a bug in the hosted setup which makes apache not send the HSTS header on the initial redirect:

{{{
$ curl -I https://fedorahosted.org/
HTTP/1.1 302 Found
Date: Fri, 26 Sep 2014 17:35:30 GMT
Server: Apache/2.2.15 (Red Hat)
Location: https://fedorahosted.org/web/
Content-Type: text/html; charset=iso-8859-1
}}}

I did not yet take a look but noticed it.

I added a patch for puppet(0001-Improve-HSTS-header.patch) that improves the HSTS headers and should take care of the missing header on redirections. I added a similar patch already to ansible, here someone needs to run ansible. Also I added an ansible patch here (0001-Add-HSTS-header-to-bodhi-elections-fas-pkgdb.patch) which needs more review. From what I understood this is necessary to get most admin.fpo services set a HSTS header. Otherwise it needs to be done somehow in varnish, since varnish terminates SSL afaics.

I've run ansible with that change.

I don't think varnish is in the picture here... apache terminates the https connections, then talks to varnish locally which in turn talks to haproxy or directly via http over vpn.

So, IMHO, that patch should be fine, but I'd love some more input on that first.

so I noticed that the config snippets for Bodhi, pkgdb, etc. for the reverse proxy are all evaluated at the same level but I do not yet understand if/how the main config files for admin.fpo and apps.fpo are created, which would be better places for the HSTS header lines. But is this probably a lack of time. Or is it possible that there are still files on the proxy servers that are not in ansible/puppet anymore? But it might also be that there are generic templates that make identifying the relevant source files harder.

Those are the main config files for admin/apps.

Those templates are used on the proxies.

request (https)-> proxy server -> (local) varnish and/or haproxy -> (http) internal application server.

The files you have been modifying at the ones that generate the apache config on the proxies, so we are doing this in the right place.

I'm not sure if there's more to be done here now, or if we can close this.

There are still more candidates:

  • kojipkgs.fedoraproject.org (need to figure how where to configure this best)
  • ask.fedoraproject.org (HSTS header is missing when it is served by ask02.phx2.fedoraproject.org

There are also some blockers to get *.fedoraproject.org completely HSTS enabled. The biggest offender is planet.fedorarproject.org and torrent.fedoraproject.org is not reachable via HTTPS iirc.

Maybe the missing HSTS header for ask was related to something not being fully updated (e.g. via puppet or ansible), because it seems to be ok now.

Replying to [comment:16 till]:

There are still more candidates:

  • kojipkgs.fedoraproject.org (need to figure how where to configure this best)

It would have to be in squid. Thats the thing listening on port 443.

There are also some blockers to get *.fedoraproject.org completely HSTS enabled. The biggest offender is planet.fedorarproject.org and torrent.fedoraproject.org is not reachable via HTTPS iirc.

planet is a bit of a difficult case. It is indeed reachable via https, but not all the blogs on it use https, so browsers may complain that the page isn't fully encrypted, and I see no way around that.

torrent I need to move to ansible soon, so I can look at adding https there when I do that.

Current list of hosts making HSTS for *.fedoraproject.org impossible:

Hosts with proper HTTPS but no HSTS header/redirecct:

  • paste.fedoraproject.org (name based v-host with only proper cert for paste.fpo, therefore sslonly cannot be used in ansible config)
  • https://keys.fedoraproject.org/
  • some more

Hosts with HSTS but not yet in the HSTS preload list:

Might be a problem:

  • The wiki, due to load on release days?
Edited 10 months ago by till

Replying to [comment:20 till]:
...snip...

Might be a problem:
* The wiki, due to load on release days?

We should ask Patrick to check the varnish config here.
Right now, I know it's caching http access to the wiki, but not sure about https. (and the wiki only allows login over https).

torrent.fedoraproject.org should now be fixed to have and require https.

So where are we now?

jenkins and koschei are moved/done.

Not sure what to do about status... we could use some other domain for it perhaps.

Replying to [comment:23 kevin]:

So where are we now?

jenkins and koschei are moved/done.

jenkins still has a jenkins.cloud.fedoraproject.org hostname that is a DNS alias, so if there rae any references to this name, they need to be updated.

Not sure what to do about status... we could use some other domain for it perhaps.

Not sure if it applies here as well. AFAIK it runs on openshift. Can it maybe updated to a new plan like it needs to be done for flocktofedora so it can use HTTPS?

With taiga a new cloud service emerged that does not support https but is in the fedoraproject domain.

For jenkins we are planning to deploy apache as a proxy and then ssl enable it.

For status, yeah, I suppose we could get it a real cert and enable it.

We should move tiaga to fedorainfracloud domain.

FYI: A new non-https website was recently created:
http://communityblog.fedoraproject.org/

Yeah, we want to fix that to go via proxies and use https... but at this point might have to wait until after freeze. ;(

communityblog is now available via https, a ticket for taiga is here:
https://fedorahosted.org/fedora-infrastructure/ticket/4950

Replying to [comment:25 kevin]:

For status, yeah, I suppose we could get it a real cert and enable it.
Note that status is an OpenShift Online hosted application (which is done to make sure it stays up when "everything" else goes down), and that the project it's located under does not support SSL certificates as far as I know.

I've been thinking about this more, but maybe we should move this to fedoraproject-status.org or something like that, because that means we won't have to add SSL to it, and also that it could be entirely separate from our infrastructure (currently its availability depends on our DNS servers, which almost never goes down, but they might).

Coming in late to the conversation, but been helping startups (including my own) do this very https:// migration.

@Puiterwijk not sure what you mean by your comment about Openshift not needing to be on the infra at all, as from where I see it if it is supposed to be the doomsday instance if you will why would you not want it to be https:// regardless of its location on infra or off it?

Replying to [comment:30 corey84]:

@Puiterwijk not sure what you mean by your comment about Openshift not needing to be on the infra at all, as from where I see it if it is supposed to be the doomsday instance if you will why would you not want it to be https:// regardless of its location on infra or off it?

status.fpo is hosted on OpenShift to be less dependent on Fedora. However the OpenShift account used for hosting it does not support HTTPS.
But it would make more sense to move status.fpo outside of the fedoraproject.org domain to make it also not rely on the fedoraproject.org name servers. Then it does not need to be HTTPS to complete the fedoraproject.org HTTPS migration.

Just an update here. I think the following remains to be done:

  • Move status.fedoraproject.org to some other domain.

  • Finish moving alternative arch kojis behind proxies.

And then we are hopefully done. ;)

Move status.fedoraproject.org to some other domain.

fedorastatus.org seems available :)

Here is a new update. There is still some other hosts except status.fpo that might be a problem:

Current list of hosts making HSTS for *.fedoraproject.org a problem:

Hosts with proper HTTPS but no HSTS header/redirecct:

  • paste.fedoraproject.org (name based v-host with only proper cert for paste.fpo, therefore sslonly cannot be used in ansible config)
  • https://keys.fedoraproject.org/
  • some more

Hosts with HSTS but not yet in the HSTS preload list:

Might be a problem:
* The wiki, due to load on release days?

Here is a new update. There is still some other hosts except status.fpo that might be a problem:
Current list of hosts making HSTS for *.fedoraproject.org a problem:

status.fedoraproject.org - uses certificate for rhcloud, might be moved to fedorainfrastatus.org - #5057

fedorastatus.org is live now.

jenkins.cloud.fedoraproject.org - no https -> should be accessed via jenkins.fedorainfracloud.org - related #4907

jekins is using https and a letsencrypt cert.

http://torrent.fedoraproject.org:6969/stats - no https, not clear, if this URL is accessed in browsers via plain http

It should never be. Only torrent clients.

http://directory.fedoraproject.org/
http://pki.fedoraproject.org/

These are redirects to those new projects. They moved long ago tho, so I think the redirects can now be dropped.

pkgs.fedoraproject.org (uses custom CA, src.fedoraproject.org should be used instead)

yep. No one should need to/want to use pkgs anymore.

Hosts with proper HTTPS but no HSTS header/redirecct:

paste.fedoraproject.org (name based v-host with only proper cert for paste.fpo, therefore sslonly cannot be used in ansible config)

Now fixed with the move to modernpaste.

https://keys.fedoraproject.org/

Still not fixed, but there's a ticket to move it behind proxies, which would fix it.

some more

Hosts with HSTS but not yet in the HSTS preload list:

torrent.fedoraproject.org
spins.fedoraproject.org
http://meetbot.fedoraproject.org/
bodhi.fedoraproject.org
http://communityblog.fedoraproject.org/

Might be a problem:
* The wiki, due to load on release days?

Naw, it's cached via memcached and varnish, so I am not worried.

So, we just need to fix keys and drop those 2 redirects and we can finally done?

Currentls pkgs.fedoraproject.org still has the old certificate from the old Fedora CA. Since the wildcard certificate should not be added there my idea is to add a iptables redirection for TCP 443 to src.fedoraproject.org and add a virtual host on the proxies for pkgs.fedoraproject.org that just redirects everything to src.fedoraproject.org. Since the proxies use a wildcard certificate, there should not be any TLS errors and everyone gets redirected to the right host.

Hum, actually, plans are that we will have a pagure frontend in front of pkgs, so I guess when that happens pkgs/pagure-frontend would need to have a valid cert and could just take over pkgs?

Login to comment on this ticket.