So, as part of this I would really really love to drop the old docs from our proxies. :( It takes up about 20GB and it's a pain because we have to hardlink it to avoid filling up disk space on proxies.

The process currently is a messy set of rsyncs that combine the old docs, the new docs and docs-redirects into one tree of files that we serve.

So, hopefully we can just drop that tree from the process, but I want to make sure we do that. ;)

I'll edit that in to step 4.

Here's the old docs converted (very quickly with a script) into asciidoc: https://fedorapeople.org/~mattdm/misc/old-release-notes-adoc.tar.xz

These need to be put into branches of https://pagure.io/fedora-docs/release-notes (with the corresponding framework), and also I guess https://pagure.io/fedora-docs/release-docs-home updated?

Also those converted docs should have a "This is old material for reference only" bit added at the top. I didn't do that.

Can someone other than me volunteer to drive the next steps of this? I have too many ideas right now and not enough follow-through. :)

A couple questions here:

1. Where are the old docs stored? Are they just put on the docs.fp.org server and site there (i.e. not a part of the antora setup at all)
2. All old docs other than release notes are going to be dropped?

1. Not sure where they are physically stored -- but they're not part of the antora setup as I understand it.

2. That's the idea, yeah. If there's anything useful in there that hasn't been updated yet, time to do it. If it turns out someone was using something, we can see what we can do to get it up to date.

The old docs are in https://pagure.io/fedora-docs-web.git all 4.3GB of them.

We clone that out on sundries01, then rsync it to proxies.

It's a kind of anoying dance there, what we do is:

1. rsync the old docs (4.3GB)
2. rsync docs-redirects and old docs to a 'docs combined'
3. rsync new docs and docs-redirects and old docs into 'docs combined'

(this allows us to hardlink things so the old docs + docs combined doesn't take up 8.6GB + new docs)

ok, i'm going to start having a crack at this -- starting at step 1.

Going to Do the following steps for Fedora 25 release notes:

1. Branch (from rawhide/master) into a new branch f25 in the release notes repo https://pagure.io/fedora-docs/release-notes/
2. Convert the html-single from the publican output (aka the old docs) into asciidoc output, and put it in the f25 branch.
3. Create a f25 branch of https://pagure.io/fedora-docs/release-docs-home/ and update the index.adoc to only talk about the release notes.
4. add the two f25 branches created in 2. and 3. to the proper locations in https://pagure.io/fedora-docs/docs-fp-o/blob/prod/f/site.yml

@ryanlerch Awesome, thanks! See https://fedorapeople.org/~mattdm/misc/old-release-notes-adoc.tar.xz for my stab ad converting to asciidoc.

@ryanlerch Any progress?

I realize I'm a couple months late here, but assuming @ryanlerch hasn't put a significant amount of work into this yet:

• Issue no. 1 is keeping these old docs out of search results. I see the repo has a robots.txt that excludes some old IPA docs that people were complaining about earlier. I know those are excluded from searches - is it based on this file or is that handled somewhere else? If it does, why don't we just add en-US/Fedora/26}/* etc. and a few more lines for the other URLs, and fix that problem immediately?

• As for reducing space taken up by old docs - each doc we published using publican back in the day has a PDF version. How about we just take all those PDFs, put them into one big tarball, put that in a normal repo as an attachment, and link to it for people who want to download the archive? Actually it doesn't even need to be one big tarball, we could just point them to a pagure repo where they could grab anything they wanted. That way we wouldn't need to worry about asciidoc conversions, and Kevin should be able to get rid of the whole rsync process since everything would be in the current system.

• Issue no. 1 is keeping these old docs out of search results. I see the repo has a robots.txt that excludes some old IPA docs that people were complaining about earlier. I know those are excluded from searches - is it based on this file or is that handled somewhere else? If it does, why don't we just add en-US/Fedora/26}/* etc. and a few more lines for the other URLs, and fix that problem immediately?

That makes sense to me.

• As for reducing space taken up by old docs - each doc we published using publican back in the day has a PDF version. How about we just take all those PDFs, put them into one big tarball, put that in a normal repo as an attachment, and link to it for people who want to download the archive? Actually it doesn't even need to be one big tarball, we could just point them to a pagure repo where they could grab anything they wanted. That way we wouldn't need to worry about asciidoc conversions, and Kevin should be able to get rid of the whole rsync process since everything would be in the current system.

Yeah, I'd even go as far as saying "let's throw them on the Internet archive and provide links to that for anyone who needs the old docs".

We agreed today that we like the approach of dropping the old docs from the server and pointing people to the old fedora-docs-web repo when they need historical content. Before we do this, we'll wait a week for comment on the Discussion thread.

Just ran across this issue again while doing some housekeeping. In my understanding this is fixed now, isn't it? The old docs are gone and if anyone wants them they can find them in https://pagure.io/fedora-docs-web - so we can finally close this, right? :)

sure

Yes, I think so... although there's something fishy on the infrastructure side still. The new docs and redirects combined are under 1GB, but then combined dir is 13GB... so some old content didn't get removed somewhere along the line. However, we can track that on the infra side.

Alright. Ping me if there's anything I can do.