#222 Add testcase to test the Package_install_remove testcase.
Merged 2 years ago by adamwill. Opened 3 years ago by lruzicka.

file modified
+25
@@ -603,6 +603,31 @@ 

                  "USER_LOGIN": "false"

              }

          },

+         "base_package_install_remove": {

+             "profiles": {

+                 "fedora-Cloud_Base-qcow2-qcow2-aarch64-*-aarch64": 40,

+                 "fedora-Cloud_Base-qcow2-qcow2-ppc64le-*-ppc64le": 40,

+                 "fedora-Cloud_Base-qcow2-qcow2-x86_64-*-64bit": 40,

+                 "fedora-KDE-live-iso-x86_64-*-64bit": 42,

+                 "fedora-Minimal-raw_xz-raw.xz-arm-*-ARM": 42,

+                 "fedora-Minimal-raw_xz-raw.xz-aarch64-*-aarch64": 42,

+                 "fedora-Server-dvd-iso-aarch64-*-aarch64": 40,

+                 "fedora-Server-dvd-iso-ppc64le-*-ppc64le": 40,

+                 "fedora-Server-dvd-iso-x86_64-*-64bit": 40,

+                 "fedora-Server-raw_xz-raw.xz-aarch64-*-aarch64": 42,

+                 "fedora-Workstation-live-iso-ppc64le-*-ppc64le": 40,

+                 "fedora-Workstation-live-iso-x86_64-*-64bit": 40,

+                 "fedora-Workstation-raw_xz-raw.xz-aarch64-*-aarch64": 42

+             },

+             "settings": {

+                 "BOOTFROM": "c",

+                 "HDD_1": "disk_%FLAVOR%_%MACHINE%.qcow2",

+                 "POSTINSTALL": "base_package_install_remove",

+                 "ROOT_PASSWORD": "weakpassword",

+                 "START_AFTER_TEST": "%DEPLOY_UPLOAD_TEST%",

+                 "USER_LOGIN": "false"

+             }

+         },

          "base_services_start": {

              "profiles": {

                  "fedora-Cloud_Base-qcow2-qcow2-aarch64-*-aarch64": 40,

@@ -0,0 +1,57 @@ 

+ use base "installedtest";

+ use strict;

+ use testapi;

+ use utils;

+ 

+ sub run {

+     my $self = shift;

+     # switch to TTY3 for both, graphical and console tests

+     $self->root_console(tty=>3);

+ 

+     # This test case tests that packages can be correctly installed and removed.

+     # We will test by installing two packages - ftp and mc.

+     #

+     # Install the FTP package.

+     assert_script_run("dnf install -y ftp", timeout => 240);

+     # Check the main packages are installed.

+     # Confirm that dnf lists the package

+     assert_script_run("dnf list ftp");

+     # Confirm that RPM lists the packages

+     assert_script_run("rpm -q ftp");

+     # Verify the installations using rpm --verify

+     assert_script_run("rpm --verify ftp");

+ 

+     # Install the MC package.

+     assert_script_run("dnf install -y mc", timeout => 240);

+     # Check the main packages are installed.

+     # Confirm that dnf lists the package

+     assert_script_run("dnf list mc");

+     # Confirm that RPM lists the packages

+     assert_script_run("rpm -q mc");

+     # Verify the installations using rpm --verify

+     assert_script_run("rpm --verify mc");

+ 

+     # Now we will uninstall the packages again and we will check that they have been uninstalled.

+     # We will not check that all of the dependencies have been uninstalled, too, because the

+     # dependencies might have been on the system already to satisfy some other packages' needs,

+     # which we believe is the normal user approach.

+     #

+     # Uninstall the packages.

+     assert_script_run("dnf remove -y ftp mc");

+     # Reports by the DNF

+     assert_script_run("!dnf list ftp");

+     assert_script_run("!dnf list mc");

+     # Reports by the RPM

+     assert_script_run("!rpm -q ftp");

+     assert_script_run("!rpm -q mc");

+ 

+ }

+ 

+ 

+ sub test_flags {

+     return { fatal => 1 };

+ }

+ 

+ 1;

+ 

+ # vim: set sw=4 et:

I am proposing to automate the Package_install_remove test case.

Build succeeded.

rebased onto e7a66fc146902cd1c6439456190ec169f271748f

3 years ago

Build succeeded.

Thanks for this!

My first concern here is that we're sort of lacking an 'independent check' on DNF behaviour. We're using DNF to decide what dependencies should be installed, then using DNF to install them. What if a bug is simply that DNF doesn't calculate dependencies right? In that case dnf deplist might agree with what dnf install does, but they might both be wrong.

I'm not sure if there's a good way to avoid this, but it's worth thinking about. Perhaps also/instead checking the dependencies with rpm directly might be interesting?

Thanks for this!

My first concern here is that we're sort of lacking an 'independent check' on DNF behaviour. We're using DNF to decide what dependencies should be installed, then using DNF to install them. What if a bug is simply that DNF doesn't calculate dependencies right? In that case dnf deplist might agree with what dnf install does, but they might both be wrong.

I'm not sure if there's a good way to avoid this, but it's worth thinking about. Perhaps also/instead checking the dependencies with rpm directly might be interesting?

Yeah, that sounds interesting. I will think about it.

So, after discussing this problem with @lbrabec, this seems to be solvable by the following procedure:

  1. Use rpm -qR <installed_application> to get the list of needed dependencies, for example mc deps look like:
/usr/bin/perl
/usr/bin/sh
config(mc) = 1:4.8.26-3.fc34
libc.so.6()(64bit)
libc.so.6(GLIBC_2.14)(64bit)
libc.so.6(GLIBC_2.15)(64bit)
libc.so.6(GLIBC_2.2.5)(64bit)
libc.so.6(GLIBC_2.3)(64bit)
libc.so.6(GLIBC_2.3.4)(64bit)
libc.so.6(GLIBC_2.33)(64bit)
libc.so.6(GLIBC_2.4)(64bit)
libc.so.6(GLIBC_2.6)(64bit)
libc.so.6(GLIBC_2.7)(64bit)
libe2p.so.2()(64bit)
libglib-2.0.so.0()(64bit)
libgpm.so.2()(64bit)
libslang.so.2()(64bit)
libslang.so.2(SLANG2)(64bit)
libutil.so.1()(64bit)
libutil.so.1(GLIBC_2.2.5)(64bit)
perl(File::Basename)
perl(File::Temp)
perl(POSIX)
perl(bytes)
perl(strict)
rpmlib(CompressedFileNames) <= 3.0.4-1
rpmlib(FileDigests) <= 4.6.0-1
rpmlib(PayloadFilesHavePrefix) <= 4.0-1
rpmlib(PayloadIsZstd) <= 5.4.18-1
rtld(GNU_HASH)
  1. Use rpm -q --whatprovides $dependency to get the name of the original package.
  2. Use rpm -qa $package to see if it is installed.

What do you think?

Or would rpm --verify $package suffice?

My first concern here is that we're sort of lacking an 'independent check' on DNF behaviour. We're using DNF to decide what dependencies should be installed, then using DNF to install them. What if a bug is simply that DNF doesn't calculate dependencies right? In that case dnf deplist might agree with what dnf install does, but they might both be wrong.

I think this is going too far in scope. An alternative package manager would be needed for this (including an alternative depsolving library rather then libsolv). Then they might not agree which dependency to pick (when there are multiple options), yet both can be correct.

There is a difference between "does the right thing" and "does it right". I believe we can only check the latter, at least in this case.

I'm not sure if there's a good way to avoid this, but it's worth thinking about. Perhaps also/instead checking the dependencies with rpm directly might be interesting?

This can be tested. rpm --verify checks dependencies. You don't need to mess up with rpm -qR or dnf --repoquery --requires --resolve. Simply run rpm --verify on that primary package that got installed, and you'll know if it has everything it needs (and also whether the files were written correctly):

$ sudo dnf install bzflag
$ sudo rpm -e opengl-games-utils
error: Failed dependencies:
    opengl-games-utils is needed by (installed) bzflag-2.4.22-1.fc33.x86_64
$ sudo rpm -V bzflag
$ sudo rpm -e opengl-games-utils --nodeps
$ sudo rpm -V bzflag
Unsatisfied dependencies for bzflag-2.4.22-1.fc33.x86_64:
    opengl-games-utils is needed by (installed) bzflag-2.4.22-1.fc33.x86_64

Note however, that rpm -V often complains about changed files in /etc (not by you, but even by post-install scripts), so it might require some tuning (additional --no* options or some grep magic). This is less of a problem if you intend to run it on the same set of packages every time, but would be more complicated if you ran it on random packages in every run.

I like the approach of using rpm -V but ignoring non-dependency-related errors the best, I guess.

I like the approach of using rpm -V but ignoring non-dependency-related errors the best, I guess.

Ok, I will rewrite the test case so that it uses rpm --verify for verification of the installations.
Thanks.

rebased onto 859d3fdb74e38cd5d212659716647693bdd11ece

2 years ago

Build succeeded.

rebased onto 1b19c958f9854114a7b92473e0d7a301a3240db8

2 years ago

Build succeeded.

Uh, I see one thing - this is scheduled to run on CoreOS. That's not right, is it? CoreOS is an ostree build...

rebased onto 94a929627cdcee225d9c0a88853f6c6884e6de79

2 years ago

rebased onto 477d918

2 years ago

Build succeeded.

OK, so I think this should be good to go now.
We might want to run this test on updates too, I guess...

Pull-Request has been merged by adamwill

2 years ago