Using https does not help, because the certificate won't be signed by a well known CA. For reference see ticket:2336.

Replying to [comment:2 till]:

Using https does not help, because the certificate won't be signed by a well known CA. For reference see ticket:2336.

That ticket was wontfix-ed because https://pkgs.fedoraproject.org currently is not intended to provide integrity-protected access to end users. If the infrastructure team decides to implement this request via https://pkgs.fedoraproject.org, I presume they will revisit the issue of the certificate.

A Fedora-signed certificate is still usable because users can verify it manually against the Fedora CA certificate, which is [https://admin.fedoraproject.org/accounts/fedora-server-ca.cert available for download] integrity-protected by a public CA. But a certificate that just works is preferable.

My current workaround to fetch one branch (for example, f13/master of libsoup) is:

1. Fetch the objects without setting the ref. Git will not accept corrupt objects, so no harm can come of this.
   {{{
   git fetch git://pkgs.fedoraproject.org/libsoup.git refs/heads/f13/master:
   }}}
2. Obtain the ref value from gitweb at [https://pkgs.fedoraproject.org/gitweb/?p=libsoup.git;a=shortlog;h=refs/heads/f13/master] and set it manually:
   {{{
   git update-ref refs/remotes/origin/f13/master cba883b515118f24a5af54a9b3fa92ac699c7263
   }}}

Possible solution:

• use the standard wildcard certificate for pkgs.fedoraproject.org
• add a second IP/hostname (if SNI is supported by every client that needs to be supported) like koji-pkgs.fedoraproject.org for the custom CA Fedora currently uses for pkgs.fedoraproject.org

For the migration add first the new koji-pkgs.fpo hostname and change everything to use it (fedpkg, koji, ...?), then monitor whether the old hostname is still used and adjust the missing pieces. Then change the certificate for pkgs.fpo.

Replying to [comment:5 till]:

Possible solution:

• use the standard wildcard certificate for pkgs.fedoraproject.org
• add a second IP/hostname (if SNI is supported by every client that needs to be supported) like koji-pkgs.fedoraproject.org for the custom CA Fedora currently uses for pkgs.fedoraproject.org

For the migration add first the new koji-pkgs.fpo hostname and change everything to use it (fedpkg, koji, ...?), then monitor whether the old hostname is still used and adjust the missing pieces. Then change the certificate for pkgs.fpo.

I think the second approach would be much easier to implement. We'd only have to patch fedpkg to use that for anonymous clones. Git is already compatible with https://pkgs.fedoraproject.org/ (verified with GIT_SSL_NO_VERIFY=1 git pull), so this should be really straightforward.

Oh, and I checked that git pull does indeed send the right SNI, so no new IP address is needed.

So here is what needs to be done in fedpkg, the only official tool that usually talks directly to pkgs.fpo.

For the file upload with the new-sources command, the certificate is not checked:

src/fedpkg/init.py:
{{{
270 def _do_curl(self, file_hash, file):
271 """Use curl manually to upload a file"""
272
273 # This is overloaded to add in the fedora user's cert
274 cmd = ['curl', '-k', '--cert', self.cert_file, '--fail', '-o',
275 '/dev/null', '--show-error', '--progress-bar', '-F',
276 'name=%s' % self.module_name, '-F', 'md5sum=%s' % file_hash,
277 '-F', 'file=@%s' % file]
}}}

-k disables the certificate check. The ca certificate (~/.fedora-server-ca.cert} is loaded by the pycurl object, that is used to

1) check if a file is already uploaded
2) upload a file directly (not via the new-sources command)

The fedpkg sources command uses plain HTTP, therefore koji would not be affected by this. The code for this is here:

{{{
212 # Set the Fedora CA certificate:
213 if os.path.exists(self.ca_cert):
214 curl.setopt(pycurl.CAINFO, self.ca_cert)
215 else:
216 self.log.warn("Missing certificate: %s" % self.ca_cert)
}}}

So to move this forward, I propose:

1) to update fedpkg to temporarily include a copy of the Fedora CA cert and the CA cert for the Fedora wildcard certificate for example in /usr/share/fedpkg/fedora-ca.crt and use this file as ca_cert as used above

2) Push this update into stable and wait at least 3 days

3) Update pkgs.fpo to use a proper certificate

4) Update fedpkg to use /etc/ssl/certs/ca-bundle.crt as ca_cert in above code and remove the bundled certificate

5) Update fedpkg to use https://pkgs.fpo for cloning

6) Fix fedpkg to check certificates when uploading files and to use https for the sources command (the latter is a setting in /etc/rpkg/fedpkg.conf)

Replying to [comment:8 till]:

So here is what needs to be done in fedpkg, the only official tool that usually talks directly to pkgs.fpo.

I can't comment on specific implementation details (I'm not familiar with fedpkg internals), but the overall approach looks fine to me.

This was deferred to after the Fedora 22 release:
https://fedorahosted.org/rel-eng/ticket/6111

I cloned a Fedora package repository for the first time in several months and checked on this issue.

Replying to [comment:6 fweimer]:

Git is already compatible with https://pkgs.fedoraproject.org/ (verified with GIT_SSL_NO_VERIFY=1 git pull)

Great! Furthermore, I can use git's url.insteadOf option to automatically redirect fedpkg anonymous clones to this URL. I've added a copy-and-pasteable command recipe to https://fedoraproject.org/wiki/Package_maintenance_guide#Using_fedpkg_anonymously. I'm updating this ticket to refer to the work still to be done.

I have a new proposal that would allow to get this easier/faster addressed.

1) Add a new virtual hostname src.fedoraproject.org for cgit and make it available using the wildcard certificate
2) Redirect http://pkgs.fedoraproject.org to https://src.fedoraproject.org but keep https://pkgs.fedoraproject.org unchanged
3) Update fedpkg to use https://src.fedoraproject.org for anonymous GIT cloning but keep it using https://pkgs.fedoraproject.org for the lookaside cache

This should allow a seemless migration without breaking fedpkg for people who did not update it.

I'm ok with that plan. Would you be willing to work on the needed playbooks/ansible changes?

Replying to [comment:13 kevin]:

I'm ok with that plan. Would you be willing to work on the needed playbooks/ansible changes?

Yes, but probably after Devconf.cz

kevin already added the hostnames to DNS (src.fpo and src.stg.fpo). The attached patch adds initial support (tests pending).

TODOs:
- add http->https redirect for src.fpo
- redirect from http://pkgs.fpo to https://src.fpo
- Update http URL in cgit
- consider disabling git protocol
- Update URLs to pkgs.fpo everywhere else (ansible, releng, ...)
- Continue with fedpkg changes
- disable smart http for http://pkgs.fpo to find everything that is not using https://src.fpo

attachment
0001-Add-src.fedoraproject.org-alias-to-dist-git.patch

Instead of using a second name on the pkgs servers (since that would require putting the wildcard cert on servers that lots of people can SSH in to, albeit in a limited environment but still), I have now set up a reverse proxy on our proxies.

This is at this moment live in staging, and after Infra Freeze we can enable this for production.

Example: git clone https://src.stg.fedoraproject.org/git/rpms/0ad.git
(Updated the example to the https url, which you would have been redirected to anyway: src(.stg).fp.o are only available over TLS, and send strict-transport-security headers).

Thank you very much.

I believe git does not known about HSTS. Since src.fpo is not known yet, it might be valid to make it https only and start not providing plain http access from the beginning, but I guess this is not possible with the current setup, since then all proxies need to be https only. But I guess we could disable git http access by matching on URLs used for smart http and block them via plain http.

Also I noticed that the connection from the proxy to pkgs is http instead of https. AFAIU it could be made https if the Fedora CA is specified using the SSLProxyCACertificateFile Directive ( https://httpd.apache.org/docs/2.4/mod/mod_ssl.html#sslproxycacertificatefile )

We probably need the HTTPS backend connection to be able to redirect from http://pkgs.fpo to https://src.fpo without creating a redirect loop.

attachment
0001-Use-https-for-src-to-pkgs-proxy-in-staging.patch

attachment
0002-Use-https-for-src-to-pkgs-proxy-in-production.patch

attachment
0003-Use-src.fedoraproject.org.patch

I just added three more patches that show a possible way forward to make the reverse proxies use https to talk to pkgs.fpo and to change all references in ansible to use src.fpo. This should already add many redirects from http://pkgs.fpo to https://src.fpo. When using the internal/VPN hostname of pkgs.fpo, then the https cannot be easily used to talk to it because the certificate will not match the hostname.

ok, after the releng flag day today I think we can finally close this.

https://src.fedoraproject.org/ has a well known cert and allows users to clone and download source.

Thanks for hanging in there, and sorry this too so very long.

:fireworks:

"fedpkg clone --anonymous" is still using an insecure URL:

https://pagure.io/fedpkg/blob/6d863f5cb78aee3d29043d041425fa0827f44899/f/conf/etc/rpkg/fedpkg.conf#_6

What URL should it be using? Shall I reopen this issue or file a new issue somewhere?

@mattmccutchen There is already a fedpkg bug for the URL change: https://bugzilla.redhat.com/show_bug.cgi?id=1188634

Thanks @fweimer for the reference. Then we're indeed done here.