#73 Support creating instances by fedora:XX string
Merged 3 years ago by frantisekz. Opened 3 years ago by frantisekz.

file modified
+36 -36
@@ -13,13 +13,13 @@ 

  To use **testcloud** on a production system:

  

  1. Install the **testcloud**.

-    

+ 

         ```

         $ sudo dnf install testcloud

         ```

  

  2. Add yourself to the `testcloud group`.

-    

+ 

        ```

        $ sudo usermod -a -G testcloud $USER

        ```
@@ -27,7 +27,7 @@ 

  3. Restart your user session to update the group privileges, or use

     `su -` to get a login shell for that particular user where the group

     settings will be updated.

-    

+ 

        ```

        $ su -i $USER

        ```
@@ -46,7 +46,7 @@ 

  To create a new instance with the cloud image, run:

  

  ```

- $ testcloud instance create <instance name> -u <url for qcow2 image>

+ $ testcloud instance create <instance name> -u <url for qcow2 image> or <fedora:XX>

  ```

  

  **testcloud** will download the *qcow2* image and save it in the
@@ -66,7 +66,7 @@ 

  To see all instances, use:

  

  ```

- $ testcloud instance list --all 

+ $ testcloud instance list --all

  ```

  

  Alternatively, the instances can also be viewed and manipulated using
@@ -78,25 +78,25 @@ 

  the **testcloud**, too:

  

  1. List all instances to see the correct name of the instance:

-    

+ 

         ```

         $ testcloud instance list --all

         ```

  

  2. Start the instance:

-    

+ 

         ```

         $ testcloud instance start <instance-name>

         ```

  

  3. Stop the instance:

-    

+ 

         ```

         $ testcloud instance stop <instance-name>

         ```

  

  4. Remove the instance:

-    

+ 

         ```

         $ testcloud instance remove <instance-name>

         ```
@@ -108,13 +108,13 @@ 

  ### Other instance operations

  

  1. Reboot the instance:

-    

+ 

         ```

         $ testcloud instance reboot <instance-name>

         ```

  

  2. Remove non-existing libvirt VMs from testcloud:

-    

+ 

         ```

         $ testcloud instance clean

         ```
@@ -185,15 +185,15 @@ 

  following to the `~/.config/testcloud/settings.py`:

  

  ```yaml

- USER_DATA = """#cloud-config 

- users: 

-     - default 

-     - name: root 

-       password: %s 

-       chpasswd: { expire: False } 

-       ssh-authorized-keys: 

-       - <my ssh pub key> 

- """ 

+ USER_DATA = """#cloud-config

+ users:

+     - default

+     - name: root

+       password: %s

+       chpasswd: { expire: False }

+       ssh-authorized-keys:

+       - <my ssh pub key>

+ """

  ```

  

  ## Testcloud Development
@@ -207,25 +207,25 @@ 

  ### Prerequisites

  

  1. Install the dependencies for **testcloud**.

-    

+ 

         ```

         $ sudo dnf install libvirt python3-libvirt libguestfs libguestfs-tools python3-requests python3-jinja2

         ```

  

  2. Start **libvirtd**.

-    

+ 

         ```

         $ sudo systemctl start libvirtd

         ```

  

  3. Add the `testcloud` group to the system.

-    

+ 

         ```

         $ sudo groupadd testcloud

         ```

  

  4. Add a user into the `testcloud` group.

-    

+ 

         ```

         $ sudo usermod -a -G testcloud $USER

         ```
@@ -233,7 +233,7 @@ 

  5. Log out of the system and log in again to update the group

     information on your user or use a login shell on a different

     terminal.

-    

+ 

         ```

         $ su - $USER

         ```
@@ -241,28 +241,28 @@ 

  ### Installation

  

  1. Clone the **testcloud** repository.

-    

+ 

         ```

         $ git clone https://pagure.io/testcloud.git

         ```

  

  2. Create the application directories.

-    

+ 

         ```

         $ sudo mkdir -p -m 775 /var/lib/testcloud

         ```

  

         ```

-        $ sudo mkdir -p -m 775 /var/lib/testcloud/instances 

+        $ sudo mkdir -p -m 775 /var/lib/testcloud/instances

         ```

  

         ```

         $ sudo mkdir -p -m 775 /var/lib/testcloud/backingstores

         ```

-     

+ 

  3. Change ownership on these directories to enable their use with

     **testcloud**.

-    

+ 

         ```

         $ sudo chown qemu:testcloud /var/lib/testcloud

         ```
@@ -274,9 +274,9 @@ 

         ```

         $ sudo chown qemu:testcloud /var/lib/testcloud/backingstores

         ```

-     

+ 

  4. Copy the `.rules` file to the **polkit** rules.

-    

+ 

         ```

         $ sudo cp conf/99-testcloud-nonroot-libvirt-access.rules /etc/polkit-1/rules.d/

         ```
@@ -284,21 +284,21 @@ 

  ### Running testcloud

  

  1. Navigate to your **testcloud** git repository.

-    

+ 

         ```

         $ cd testcloud

         ```

  

  2. Execute the `run_testcloud.py` script to run the **testcloud**. You

     can use any options as with the regular installation, for example:

-        

+ 

         ```

             $ ./run_testcloud.py instance create ...

         ```

-     

+ 

  3. Alternatively, you can use **pip** to install **testcloud** onto the

     system and then use it like it has been installed normally.

-    

+ 

         ```

         $ pip3 install -e . --user

         ```

file modified
+9 -8
@@ -7,11 +7,12 @@ 

  \fBtestcloud\fR [-h, --help] [COMMAND]... [OPTION]... [SUBCOMMAND]... [OPTION]...

  .SH DESCRIPTION

  .LP

- \fBtestcloud\fR is a small helper script to download and boot cloud images on your machine locally. 

+ \fBtestcloud\fR is a small helper script to download and boot cloud images on your machine locally.

  Currently, only Fedora \fBqcow2\fR images are supported.

  .PP

- To run \fBtestcloud\fR, you need to provide the \fBURL\fR of a cloud image to the script which

- will be used for booting. If you do not have an image location of your own, you can use one

+ To run \fBtestcloud\fR, you need to provide the \fBURL\fR of a cloud image or string in fedora:XX format

+ (where XX is desired Fedora version or 'latest') to the script which will be used for booting.

+ If you do not have an image location of your own, you can use one

  of the images from the \fBFedora Cloud\fR download pages at \fIhttps://alt.fedoraproject.org/cloud/\fR.

  .PP

  Then, \fBtestcloud\fR will download the image and save it in the \fI/var/lib/testcloud/backingstores/\fR.
@@ -20,7 +21,7 @@ 

  download it again and it will use the current image to create the instance.

  .PP

  When the new instance is created, \fBtestcloud\fR will show its IP address that you can use to

- access the running instance via \fIssh\fR. The login name is \fBfedora\fR and the password is 

+ access the running instance via \fIssh\fR. The login name is \fBfedora\fR and the password is

  \fBpassw0rd\fR. The IP address of an instance is shown when you list the instance (see COMMANDS).

  .PP

  The instance can also be manipulated and controlled with \fBvirt-manager\fR.
@@ -32,9 +33,9 @@ 

  .PP

  The configuration file must be placed in one of the following locations in order to be

  recognized.

- .IP 

+ .IP

  \fIconf/settings.py\fR in the git checkout

- .IP 

+ .IP

  \fI~/.config/testcloud/settings.py\fR

  .IP

  \fI/etc/testcloud/settings.py\fR
@@ -60,7 +61,7 @@ 

  

  \fBstart\fR

          Start an existing instance.

-  

+ 

  \fBstop\fR

          Stop a running instance.

  
@@ -72,7 +73,7 @@ 

  

  \fBdestroy\fR

          Remove an instance or image. Deprecated, do not use.

-  

+ 

  \fBclean\fR

          Remove non-existing or unsynced libvirt VMs from testcloud.

  

file modified
+40 -2
@@ -12,10 +12,13 @@ 

  import os

  import sys

  import libvirt

+ import requests

+ import re

  from . import config

  from . import image

  from . import instance

  from .exceptions import TestcloudPermissionsError, TestcloudInstanceError

+ from json import JSONDecodeError

  

  config_data = config.get_config()

  
@@ -61,6 +64,33 @@ 

  

      print("")

  

+ def __get_image_url(release_string):

+     version = release_string.groups()[0]

+ 

+     if version == "latest":

+         try:

+             latest_release = requests.get('https://packager.fedorainfracloud.org:5000/api/v1/releases').json()

+         except (JSONDecodeError, ConnectionError):

+             print("Couldn't fetch the latest Fedora release...")

+             print("Expected format is 'fedora:XX' where XX is version number or 'latest'.")

+             return None

+         version = str(latest_release["fedora"]["stable"])

+ 

+     try:

+         releases = requests.get('https://getfedora.org/releases.json').json()

+     except (JSONDecodeError, ConnectionError):

+         print("Couldn't fetch releases list...")

+         return None

+ 

+     url = None

+     for release in releases:

+         if release["version"] == version and release["variant"] == "Cloud" and release["link"].endswith(".qcow2"):

+             # Currently, we support just the x86_64

+             if release["arch"] == "x86_64":

+                 url = release["link"]

+                 break

+ 

+     return url

  

  def _create_instance(args):

      """Handler for 'instance create' command. Expects the following elements in args:
@@ -71,7 +101,15 @@ 

  

      log.debug("create instance")

  

-     tc_image = image.Image(args.url)

+     image_by_name = re.match(r'fedora:(.*)', args.url)

+     if image_by_name and "http" not in args.url and "file" not in args.url:

+         url = __get_image_url(image_by_name)

+         if not url:

+             print("Couldn't find the desired image...")

+             sys.exit(1)

+         tc_image = image.Image(url)

+     else:

+         tc_image = image.Image(args.url)

      try:

          tc_image.prepare()

      except TestcloudPermissionsError as error:
@@ -322,7 +360,7 @@ 

      # this might work better as a second, required positional arg

      instarg_create.add_argument("-u",

                                  "--url",

-                                 help="URL to qcow2 image is required.",

+                                 help="URL to qcow2 image or fedora:XX string is required.",

                                  type=str)

      instarg_create.add_argument("--timeout",

                                  help="Time (in seconds) to wait for boot to "

no initial comment

Might be worth doing before accessing the "slow" online resource, or maybe even change the regexp image_by_name = re.match(r'fedora:(.*)', args.url) to allow only for numbers in the capturing group. You decide!

Matching only numbers can be limiting in the future, it would be nice to be able to run something like testcloud instance create testinstance -u fedora:34final1.1

Is the goal here to get the "last" release in the list that matches the criteria? If not, then a break here might be worth it

Matching only numbers can be limiting in the future, it would be nice to be able to run something like testcloud instance create testinstance -u fedora:34final1.1

Fair point. If that's something we want to pursue, then let's add a comment explaining that plan to the code. Thus the execution sequence is "validated" and won't be accidentally refactored out of the code.

1 new commit added

  • Support fedora:latest
3 years ago

Feedback should be addressed.

The code now supports "fedora:latest".

"fedora:nominated" will come later, I need to figure out a reliable way to get currently nominated compose url.

rebased onto c77db08

3 years ago

Pull-Request has been merged by frantisekz

3 years ago