Tool to manage the Fedora Archive yum repository
sudo tee /etc/yum.repos.d/fedora-archive.repo <<'EOF' [archive] name=Fedora $releasever - $basearch - Archive baseurl=https://dustymabe-archive-repo-poc.s3.amazonaws.com/$basearch/ #baseurl=http://download.example/pub/fedora/linux/releases/$releasever/Everything/$basearch/os/ enabled=1 metadata_expire=7d repo_gpgcheck=0 type=rpm gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-$releasever-$basearch skip_if_unavailable=True cost=10000 #default is 1000 EOF dnf install foo
export S3BUCKET=dustymabe-archive-repo-poc export AWSACCESSKEYID= export AWSSECRETACCESSKEY= podman build -t archive-repo-manager . podman run -it --rm \ -e AWSACCESSKEYID \ -e AWSSECRETACCESSKEY \ -e S3BUCKET \ --device /dev/fuse \ --name archive-repo-manager \ archive-repo-manager
If you'd like you can add --entrypoint=/bin/bash
. Then you can do
the s3fs mount and run /usr/local/lib/archive_repo_manager.py directly.
Set up credentials. One way is to use the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables.
Then create the bucket:
aws s3 mb s3://myarchivebucket
Set the bucket to be completely public (no private things are getting stored here).
POLICY='{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": "*", "Action":["s3:GetObject","s3:GetObjectVersion"], "Resource": "arn:aws:s3:::myarchivebucket/*" } ] }' aws s3api put-bucket-policy --bucket myarchivebucket --policy "$POLICY"
Create an IAM policy to allow for access to the bucket.
POLICY='{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:*" ], "Resource": [ "arn:aws:s3:::myarchivebucket" ] }, { "Effect": "Allow", "Action": [ "s3:*" ], "Resource": [ "arn:aws:s3:::myarchivebucket/*" ] } ] }' aws iam create-policy --policy-name read-write-archive-repo-s3-bucket --policy-document "$POLICY"
Create a user and attach the policy to it.
# Optional: create new user first. aws iam create-user --user-name userforarchiverepoaccess # Attach the policy to a user aws iam attach-user-policy --user-name userforarchiverepoaccess --policy-arn arn:aws:iam::011111111111:user/userforarchiverepoaccess
Then use the credentials for the user to manage the repo. The credentials that are used by the automation script for uploading to the s3 bucket should really be limited to access to that bucket and nothing else.
The directory structure we're adopting for now looks like
fedora/${release}/${arch}
. For Fedora 33 this would look like:
/fedora/33/aarch64/
/fedora/33/armhfp/
/fedora/33/ppc64le/
/fedora/33/s390x/
/fedora/33/x86_64/
To create the structure, mount up the newly created bucket using s3fs:
export AWSACCESSKEYID=xxx export AWSSECRETACCESSKEY=xxx mkdir /tmp/bucket/ s3fs -o uid=$(id -u),gid=$(id -g) $S3BUCKET /tmp/bucket
Then create the directory structure needed:
pushd /tmp/bucket/ mkdir fedora && pushd fedora for release in 31 32 33; do for arch in aarch64 armhfp ppc64le s390x x86_64; do mkdir -p "${release}/${arch}" done done popd; popd