From 91ee96ab531660780294ebbff2d30ffb40f5da50 Mon Sep 17 00:00:00 2001 From: Jonathan Steffan Date: Sep 05 2017 06:23:55 +0000 Subject: [PATCH 1/3] Update README - Remove md README - Update README to be prescritive - Set +x on referenced Scripts files --- diff --git a/README b/README index c1f7a26..b59c392 100644 --- a/README +++ b/README @@ -1,93 +1,49 @@ -## Fedora Respins SIG HowTo Guides and Files Page. -================================================== ---- RESPINS SIG -- Updated Lives Repo ------------ -================================================== -Repo License: GPL v3+ [[ https://www.gnu.org/licenses/gpl.html ]] +Fedora Respins [SIG](https://fedoraproject.org/wiki/SIGs) SOPs +-------------------------------------------------------------- -Welcome to the Fedora Respins SIG Repo, Here you will find the final work that is used by the SIG Members to provide Community Supported Updated Live ISOs at the regular interval of every even point kernel release (i.e. 4.9.10 > 4.9.12 > 4.9.14 ... etc) +Repo License: [GPL v3+](https://www.gnu.org/licenses/gpl.html) -All Directories have a directory specific README file, please take the few seconds needed to read them. +Welcome to the Fedora Respins SIG repo. Here you will find the automation that is used by the SIG members to provide Community supported updated live ISOs regularly. -This Repo is a full suite of the scripts/files you will need to run your own single Updated/Custom ISO Builder/Seeder or a Personal Infrastruture which is using the same scripts and tools as the Respins SIG and Release Engineering Team. +Please take the time to `README` for everything in this repo. -Some baseline must haves: +Getting Started +--------------- -# Packages -========== +Install base requirements: -* fedora-kickstarts +``` +dnf install fedora-kickstarts pungi mock lorax-lmc-novirt lorax-lmc-virt lorax-templates-generic git l10n-kickstarts pykickstart +``` -* pungi +Compose Storage Requirements +---------------------------- +It is recommended that you have at least 50GB of space available to do a full compose. This includes storage for building, final compose and sources. If you're expecting to mirror the SIG, dedicate at least 100GB. -* mock -* lorax-lmc-novirt +SELinux +------- +To compose, you must have SELinux supported in the kernel. However, it needs to be in permissive mode to build. Ensure SELinux support and move to permissive. -* lorax-lmc-virt +``` +setenforce 0 +``` -# Hard Drive Space -================== +Composing +--------- +Install ISOs and Live images are build using different tools. -~15gb of space for the final ISOs, DOES NOT included needed space for building, merely seeding or web hosting. +Install ISO Compose +------------------- +Use the `pungirun` script to create install images. -Full set creation WITHOUT source ISO and related pungi bits is ~ 30Gb of space +``` +./Scripts/pungirun.sh 26 `date +%Y-%m-%d-%R` +``` -Full set WITH source is roughly 42gb of space, if you are hosting several (for example 3) releases like the SIG members do allocate roughly 100gb or more as you will need 42gb for the first set and nearly 15gb for each set of final ISOs therafter. - - -# Optional Packages -=================== - -* git (for using the git repo for fedora-kickstarts --defaults to pulling the present rawhide version (as of 201703323 this is 27) (if not using the packaged fedora-kickstarts) or if you are usign git packaged stuff for your custom ISOs /remix. - -* l10n-kickstarts -- for localization ready kickstart files - -* pykickstart -- provides ksvalidator (cli and gui versions available as of F25) sample invocation: - - [ user@local-builder /srv/Livecds/ ] $ ksvalidator /usr/share/spin-kickstarts/flat-my_custom_kicksstart.ks - - -Note: These packages are for more underlying functionality that is NOT needed for a vanilla set of ISOs but for AMI/virtsh uses is needed. - -* lorax-lmc-templates - -* lorax-lmc-examples - -## Server-Side tracker/seeder ## -================================ - -If you wish to make this a full stop solution, complete with the seeder and whitelisted tracker you will also need the following: - -* opentracker-ipv{4,6} - -* transmission-daemon - -* transmission-remote-cli - -* transmission - - -### Now that we have the minimal packages needed installed, you'll need SELinux set to permissive to run the scripts. ### -------------------------------------------------------------------------------------------------------------------------- - -[user@local-builder ~] /srv/Livecds/ $ setenforce 0 - -ALL scripts WILL need you to be root or be run from an account within the wheel group (can use sudo). - -The Scripts directory is for those that have their system setup but merely lack the scripts or are updating their scripts. - -ALL generated ISOs using these scripts are libvirt [[ dnf install @Virtualization ]], Virtualbox [[ sudo dnf install http://download.virtualbox.org/virtualbox/5.1.18/VirtualBox-5.1-5.1.18_114002_fedora25-1.x86_64.rpm ]] friendly and are tested using both with at least 1gb RAM and a single dual core cpu. - -The scripts assume /srv/fedora-repo/ and /srv/Livecds/ for torrents and ISO generation respectively. If these directories do not exist on your infrastructure or are not where you would like to host / build your ISOs/Torrents edit the respective buildmedia.sh or build-torrents.sh prior to running them. - - -***** SAMPLE INVOCATIONS: ***** - -user@local-mirror /srv/Livecds/ $ sudo sh ./pungirun.sh 25 20170301 - -user@local-mirror /srv/Livecds/ $ sudo sh ./buildmedia.sh 25 20170301 - -user@local-mirror /srv/fedora-repo | /srv/Livecds/ $ sudo ./buildtorrents.sh 25 20170301 +LiveCD ISO Compose +------------------ +./Scripts/buildmedia.sh 26 `date +%Y-%m-%d-%R` ** Notes on specific scripts: @@ -107,3 +63,35 @@ Using livemedia-creator this script makes the hybrid ISOs that are becoming the This script is normally on the seeder host not the builder but can be on other. Seeing as the SIG hosts the ISOs on that host in /srv/fedora-repo/F25-Respins this is non standard in regard to the others but is easily changed. The script runs a for loop through that directory looking for .iso files matching the spin variables (CINN,KDE,LXDE,MATE,XFCE,SOAS,XFCE) and runs transmission-create on them. For those running opentracker this script also outputs the contents of buildfile (default output file for the torrent hashes) to opentracker's whitelist file, all that remains is a manual restart of opentracker. If you are not running opentracker this will exit with a non fatal error on the attempted move, or you can comment out the last 5 lines of the script. We have provided a complete opentracker.conf that is ready to accept teh geoipblocks and whitelisted IPs ( ours have been sanitized in this file for dummy values, so please make sure to edit with valid IPs for your uses/environment. + + +Testing +------- +We utilize KVM and VirtualBox to test the ISOs. To ensure these tools are installed: + +KVM: +``` +dnf install @Virtualization +``` + +VirtualBox via [RPMFusion](https://rpmfusion.org/Configuration): +``` +dnf install akmod-VirtualBox +``` + +Building Torrents +----------------- +Building the torrents. + +``` +./Scripts/buildtorrents.sh 26 `date +%Y-%m-%d-%R` +``` + +Server-Side tracker/seeder +-------------------------- + +If you are joining the torrent, please also install the following: + +``` +dnf install opentracker-ipv{4,6} transmission-daemon transmission-remote-cli transmission +``` diff --git a/README.md b/README.md deleted file mode 100644 index f53c6f6..0000000 --- a/README.md +++ /dev/null @@ -1,97 +0,0 @@ -## Fedora Respins SIG HowTo Guides and Files Page. - -Repo License: GPL v3 [[ https://www.gnu.org/licenses/gpl.html ]] - -Welcome to the Fedora Respins SIG Repo, Here you will find the final work that is used by the SIG Members to provide Community Supported Updated Live ISOs at the regular interval of every even point kernel release (i.e. 4.9.10 > 4.9.12 > 4.9.14 ... etc) - -All Directories have a directory specific README files, please take the few seconds needed to read them. - -This Repo is a full suite of the scripts/files you will need to run your own single Updated/Custom ISO or a Personal Infrastruture which is using the same scripts and tools as the Respins SIG and Release Engineering Team. - -Some baseline must haves: - -* Packages -fedora-kickstarts - -pungi - -mock - -lorax-lmc-novirt - -* Hard Drive Space -~10gb of space for the final ISOs - -full set creation WITH source ISO and related pungi bits is ~ 15-20Gb of space - -full set WITH source is roughly 42gb of space, if you are hosting several (for example 3) releases like the SIG members do allocate 100gb or more as you will need 42gb for the first set and nearly 10gb for each set of final ISOs therafter. - - -* Optional Packages -git (for using the git repo for fedora-kickstarts --defaults to pulling the present rawhide version (as of 201703323 this is 27) (if not using the packaged fedora-kickstarts) or if you are usign git packaged stuff for your custom ISOs /remix. - -l10n-kickstarts -- for localization ready kickstart files - -pykickstart -- provides ksvalidator (cli and gui versions available as of F25) sample invocation: - - user@local-mirror /srv/Livecds/ $ ksvalidator /usr/share/spin-kickstarts/flat-my_custom_kicksstart.ks - - -Note: These packages are for more underlying functionality that is NOT needed for a vanilla set of ISOs but for AMI/virtsh uses is needed. -lorax-lmc-templates - -lorax-lmc-examples - -lorax-lmc-virt - - -****** Server-Side tracker/seeder ****** - -If you wish to make this a full suite complete with the seeder and whitelisted tracker you will also need the following: - -opentracker-ipv{4,6} - -transmission-daemon - -transmission - - -* Now that we have the minimal packages needed installed, you'll need SELinux set to permissive to run the scripts. - -user@local-mirror /srv/Livecds/ $ setenforce 0 - -ALL scripts WILL need you to be root or be run from an account within the wheel group (can use sudo). - -The Scripts directory is for those that have their system setup but merely lack or are updating their scripts. - -ALL generated ISOs using these scripts are libvirt [[ dnf install @Virtualization ]], Virtualbox sudo dnf install [[ http://download.virtualbox.org/virtualbox/5.1.18/VirtualBox-5.1-5.1.18_114002_fedora25-1.x86_64.rpm ]] friendly and are tested using both with at least 1gb RAM and 1 cpu. - -The scripts assume /srv/fedora-repo/ and /srv/Livecds/ for torrents and ISO generation respectively. If these directories do not exist on your infrastructure or are not where you would like to host / build your ISOs/Torrents edit the respective buildmedia.sh or build-torrents.sh prior to running them. - - -***** SAMPLE INVOCATIONS: ***** - -user@local-mirror /srv/Livecds/ $ sudo sh ./pungirun.sh 25 20170301 - -user@local-mirror /srv/Livecds/ $ sudo sh ./buildmedia.sh 25 20170301 - -user@local-mirror /srv/fedora-repo | /srv/Livecds/ $ sudo ./buildtorrents.sh 25 20170301 - -** Notes on specific scripts: - - ***** pungirun.sh ***** - -will create a Fedora-$1-source-$2.iso which will reside at /srv/Livecds/Pungi/25/.../Fedora-$1-source-$2.iso (assuming no changes to script) - -pungirun.sh will also generate a sha256sum checksum which we later convert to a sha512sum by re-running sha512sum on ALL the ISOs in the set once they are all generated. - - ***** buildmedia.sh ***** - -Using livemedia-creator this script makes the hybrid ISOs that are becoming the common standard. Livemedia-creator IS ABLE to make ami and virtinstall images however the script would need editing of the switches used to do this. sha512sum is used here for all ISOs at this phase. Generally at this point the source iso is MANUALLY moved to /srv/Livecds/25/$2/.../ where the other ISOs reside and tehn a final sha512sum is run on the entire directory (including all 7 Updated ISOs and the source ISO. - - - ***** build-torrents.sh ***** - -This script is normally on the seeder host not the builder but can be on other. Seeing as the SIG hosts the ISOs on that host in /srv/fedora-repo/F25-Respins this is non standard in regard to the others but is easily changed. The script runs a for loop through that directory looking for .iso files matching the spin variables (CINN,KDE,LXDE,MATE,XFCE,SOAS,XFCE) and runs transmission-create on them. For those running opentracker this script also outputs the contents of buildfile (default output file for the torrent hashes) to opentracker's whitelist file, all that remains is a manual restart of opentracker. If you are not running opentracker this will exit with a non fatal error on the attempted move, or you can comment out the last 5 lines of the script. - -We have provided a complete opentracker.conf that is ready to accept teh geoipblocks and whitelisted IPs ( ours have been sanitized in this file for dummy values, so please make sure to edit with valid IPs for your uses/environment. diff --git a/Scripts/buildmedia.sh b/Scripts/buildmedia.sh old mode 100644 new mode 100755 diff --git a/Scripts/pungirun.sh b/Scripts/pungirun.sh old mode 100644 new mode 100755 From a767c3c8fb9c584c84e9893711f07ae4554759f5 Mon Sep 17 00:00:00 2001 From: Jonathan Steffan Date: Sep 05 2017 07:56:24 +0000 Subject: [PATCH 2/3] Update README to MD location. --- diff --git a/README b/README deleted file mode 100644 index b59c392..0000000 --- a/README +++ /dev/null @@ -1,97 +0,0 @@ -Fedora Respins [SIG](https://fedoraproject.org/wiki/SIGs) SOPs --------------------------------------------------------------- - -Repo License: [GPL v3+](https://www.gnu.org/licenses/gpl.html) - -Welcome to the Fedora Respins SIG repo. Here you will find the automation that is used by the SIG members to provide Community supported updated live ISOs regularly. - -Please take the time to `README` for everything in this repo. - -Getting Started ---------------- - -Install base requirements: - -``` -dnf install fedora-kickstarts pungi mock lorax-lmc-novirt lorax-lmc-virt lorax-templates-generic git l10n-kickstarts pykickstart -``` - -Compose Storage Requirements ----------------------------- -It is recommended that you have at least 50GB of space available to do a full compose. This includes storage for building, final compose and sources. If you're expecting to mirror the SIG, dedicate at least 100GB. - - -SELinux -------- -To compose, you must have SELinux supported in the kernel. However, it needs to be in permissive mode to build. Ensure SELinux support and move to permissive. - -``` -setenforce 0 -``` - -Composing ---------- -Install ISOs and Live images are build using different tools. - -Install ISO Compose -------------------- -Use the `pungirun` script to create install images. - -``` -./Scripts/pungirun.sh 26 `date +%Y-%m-%d-%R` -``` - -LiveCD ISO Compose ------------------- -./Scripts/buildmedia.sh 26 `date +%Y-%m-%d-%R` - -** Notes on specific scripts: - - ***** pungirun.sh ***** - -will create a Fedora-$1-source-$2.iso which will reside at /srv/Livecds/Pungi/25/.../Fedora-$1-source-$2.iso (assuming no changes to script) - -pungirun.sh will also generate a sha256sum checksum which we later convert to a sha512sum by re-running sha512sum on ALL the ISOs in the set once they are all generated. - - ***** buildmedia.sh ***** - -Using livemedia-creator this script makes the hybrid ISOs that are becoming the common standard. Livemedia-creator IS ABLE to make ami and virtinstall images however the script would need editing of the switches used to do this. sha512sum is used here for all ISOs at this phase. Generally at this point the source iso is MANUALLY moved to /srv/Livecds/25/$2/.../ where the other ISOs reside and tehn a final sha512sum is run on the entire directory (including all 7 Updated ISOs and the source ISO. - - - ***** build-torrents.sh ***** - -This script is normally on the seeder host not the builder but can be on other. Seeing as the SIG hosts the ISOs on that host in /srv/fedora-repo/F25-Respins this is non standard in regard to the others but is easily changed. The script runs a for loop through that directory looking for .iso files matching the spin variables (CINN,KDE,LXDE,MATE,XFCE,SOAS,XFCE) and runs transmission-create on them. For those running opentracker this script also outputs the contents of buildfile (default output file for the torrent hashes) to opentracker's whitelist file, all that remains is a manual restart of opentracker. If you are not running opentracker this will exit with a non fatal error on the attempted move, or you can comment out the last 5 lines of the script. - -We have provided a complete opentracker.conf that is ready to accept teh geoipblocks and whitelisted IPs ( ours have been sanitized in this file for dummy values, so please make sure to edit with valid IPs for your uses/environment. - - -Testing -------- -We utilize KVM and VirtualBox to test the ISOs. To ensure these tools are installed: - -KVM: -``` -dnf install @Virtualization -``` - -VirtualBox via [RPMFusion](https://rpmfusion.org/Configuration): -``` -dnf install akmod-VirtualBox -``` - -Building Torrents ------------------ -Building the torrents. - -``` -./Scripts/buildtorrents.sh 26 `date +%Y-%m-%d-%R` -``` - -Server-Side tracker/seeder --------------------------- - -If you are joining the torrent, please also install the following: - -``` -dnf install opentracker-ipv{4,6} transmission-daemon transmission-remote-cli transmission -``` diff --git a/README.md b/README.md new file mode 100644 index 0000000..b59c392 --- /dev/null +++ b/README.md @@ -0,0 +1,97 @@ +Fedora Respins [SIG](https://fedoraproject.org/wiki/SIGs) SOPs +-------------------------------------------------------------- + +Repo License: [GPL v3+](https://www.gnu.org/licenses/gpl.html) + +Welcome to the Fedora Respins SIG repo. Here you will find the automation that is used by the SIG members to provide Community supported updated live ISOs regularly. + +Please take the time to `README` for everything in this repo. + +Getting Started +--------------- + +Install base requirements: + +``` +dnf install fedora-kickstarts pungi mock lorax-lmc-novirt lorax-lmc-virt lorax-templates-generic git l10n-kickstarts pykickstart +``` + +Compose Storage Requirements +---------------------------- +It is recommended that you have at least 50GB of space available to do a full compose. This includes storage for building, final compose and sources. If you're expecting to mirror the SIG, dedicate at least 100GB. + + +SELinux +------- +To compose, you must have SELinux supported in the kernel. However, it needs to be in permissive mode to build. Ensure SELinux support and move to permissive. + +``` +setenforce 0 +``` + +Composing +--------- +Install ISOs and Live images are build using different tools. + +Install ISO Compose +------------------- +Use the `pungirun` script to create install images. + +``` +./Scripts/pungirun.sh 26 `date +%Y-%m-%d-%R` +``` + +LiveCD ISO Compose +------------------ +./Scripts/buildmedia.sh 26 `date +%Y-%m-%d-%R` + +** Notes on specific scripts: + + ***** pungirun.sh ***** + +will create a Fedora-$1-source-$2.iso which will reside at /srv/Livecds/Pungi/25/.../Fedora-$1-source-$2.iso (assuming no changes to script) + +pungirun.sh will also generate a sha256sum checksum which we later convert to a sha512sum by re-running sha512sum on ALL the ISOs in the set once they are all generated. + + ***** buildmedia.sh ***** + +Using livemedia-creator this script makes the hybrid ISOs that are becoming the common standard. Livemedia-creator IS ABLE to make ami and virtinstall images however the script would need editing of the switches used to do this. sha512sum is used here for all ISOs at this phase. Generally at this point the source iso is MANUALLY moved to /srv/Livecds/25/$2/.../ where the other ISOs reside and tehn a final sha512sum is run on the entire directory (including all 7 Updated ISOs and the source ISO. + + + ***** build-torrents.sh ***** + +This script is normally on the seeder host not the builder but can be on other. Seeing as the SIG hosts the ISOs on that host in /srv/fedora-repo/F25-Respins this is non standard in regard to the others but is easily changed. The script runs a for loop through that directory looking for .iso files matching the spin variables (CINN,KDE,LXDE,MATE,XFCE,SOAS,XFCE) and runs transmission-create on them. For those running opentracker this script also outputs the contents of buildfile (default output file for the torrent hashes) to opentracker's whitelist file, all that remains is a manual restart of opentracker. If you are not running opentracker this will exit with a non fatal error on the attempted move, or you can comment out the last 5 lines of the script. + +We have provided a complete opentracker.conf that is ready to accept teh geoipblocks and whitelisted IPs ( ours have been sanitized in this file for dummy values, so please make sure to edit with valid IPs for your uses/environment. + + +Testing +------- +We utilize KVM and VirtualBox to test the ISOs. To ensure these tools are installed: + +KVM: +``` +dnf install @Virtualization +``` + +VirtualBox via [RPMFusion](https://rpmfusion.org/Configuration): +``` +dnf install akmod-VirtualBox +``` + +Building Torrents +----------------- +Building the torrents. + +``` +./Scripts/buildtorrents.sh 26 `date +%Y-%m-%d-%R` +``` + +Server-Side tracker/seeder +-------------------------- + +If you are joining the torrent, please also install the following: + +``` +dnf install opentracker-ipv{4,6} transmission-daemon transmission-remote-cli transmission +``` From e0f181549fd978aa68cb54f8cfb91042c4c3210c Mon Sep 17 00:00:00 2001 From: Jonathan Steffan Date: Sep 05 2017 07:57:59 +0000 Subject: [PATCH 3/3] Fix LiveCD ISO Compose syntax. --- diff --git a/README.md b/README.md index b59c392..8b1214d 100644 --- a/README.md +++ b/README.md @@ -43,7 +43,9 @@ Use the `pungirun` script to create install images. LiveCD ISO Compose ------------------ +``` ./Scripts/buildmedia.sh 26 `date +%Y-%m-%d-%R` +``` ** Notes on specific scripts: