#16 Create Installer repo test(s)
Closed: Fixed None Opened 14 years ago by jlaska.

The current [https://fedoraproject.org/wiki/QA:Fedora_11_RC_Install_Test_Results Fedora test case matrix] does not cover the yum repo dialogs. We'll need to improve coverage for:
* Add/Update/Edit Repo's
* Adding a network repo for a non-network install
* Does the repo location matter (http, ftp, nfs, local?)?

= Potential Steps to verify =
* A repo with a comps.xml file should display group information in package screen
* Packages from the requested repo are added
* The repo is accessible on the installed system (/etc/yum.repos.d/foo.repo)


Sorry Liam, this ticket got lost post F11. Do you think it makes sense to update the existing test matrix with these suggested yum repo configuration UI cases?

Changing to F13 milestone ... let's get these cases defined and added to the plan.

Sorry, I did not receive this ticket before.Perhaps some settings were wrong for me on this website before.I think you are right, I also remember some people reflected the repo issues in mailing list before the GA release. We indeed have some gaps in repo configuration.If you like, I can try to add some.

Dropping ownership so this is available for others to grab.

I added some cases to:

https://fedoraproject.org/wiki/Category:Repository

I will search bugzilla later to see whether I can cover more. Please feel free to give comments.

Thanks

jlaska:what do you mean "The repo is accessible on the installed system (/etc/yum.repos.d/foo.repo) "? If added a repository during install, after complete install, the installed system will have the added repo in /etc/yum.repos.d ?

Replying to [comment:9 liam]:

jlaska:what do you mean "The repo is accessible on the installed system (/etc/yum.repos.d/foo.repo) "? If added a repository during install, after complete install, the installed system will have the added repo in /etc/yum.repos.d ?

I guess I'm not sure what expectations there are for how the customized install repos are reflected on the installed system. So as you mention, if I add a new repo during the install screen, does that repo exist in ''/etc/yum.repos.d/anaconda.repo'' after install?

In general the tests look great, thanks for taking initiative to make these suggested improvements :)

I think you've captured the different target repos (CD/DVD, ftp, http, nfs) allowed by anaconda nicely. Some general thoughts noted below, let me know if any of these are off base, or don't apply:

Since yum repos can include additional metadata on package groups (aka comps.xml), perhaps we should consider:
* modifying the tests so that they also verify whether the expected packages or package groups provided by the custom yum repos appear on the subsequent package detail screen.
* also, updating the tests to confirm that you can select a package provided by the added yum repo and ensure it's selected and installed as expected
* I think each test might also add the following instructions to confirm all create/modify/delete operations work on the added repos.
1. Add a new yum repo using the recommended method (CD/DVD, http, nfs etc...)
1. Remove the created repo
1. Add it again ... then modify the repo (changing the name)
1. Enable and disable the created repository

To accomplish the first 2 points, we may need to create several sample yum repos (hosted on one of our fedorapeople.org pages) that provides the needed conditions to satisfy the tests. The command {{{createrepo -g comps.xml /path/to/packages}}} can be used to create a yum repo with package groups. The comps.xml file used with FEdora is included in the repodata directory (see [http://download.fedora.redhat.com/pub/fedora/linux/releases/12/Fedora/x86_64/os/repodata/6a72ae27742d1b4ac04f2eec0e5ffb0b7c909d58b9c6e346a7ee5cfb627832d4-Fedora-12-comps.xml comps.xml]).

Replying to [comment:10 jlaska]:

I guess I'm not sure what expectations there are for how the customized install repos are reflected on the installed system. So as you mention, if I add a new repo during the install screen, does that repo exist in ''/etc/yum.repos.d/anaconda.repo'' after install?

After tested,there is no /etc/yum.repos.d/anaconda.repo in installed system when added a repo during install.

In general the tests look great, thanks for taking initiative to make these suggested improvements :)

I think you've captured the different target repos (CD/DVD, ftp, http, nfs) allowed by anaconda nicely. Some general thoughts noted below, let me know if any of these are off base, or don't apply:

Since yum repos can include additional metadata on package groups (aka comps.xml), perhaps we should consider:
* modifying the tests so that they also verify whether the expected packages or package groups provided by the custom yum repos appear on the subsequent package detail screen.
* also, updating the tests to confirm that you can select a package provided by the added yum repo and ensure it's selected and installed as expected

Very good suggestion, I will modify cases to cover that

  • I think each test might also add the following instructions to confirm all create/modify/delete operations work on the added repos.
    1. Add a new yum repo using the recommended method (CD/DVD, http, nfs etc...)
    1. Remove the created repo
    1. Add it again ... then modify the repo (changing the name)
    1. Enable and disable the created repository

I considered that before,if put all the specific step in cases, the cases will become more. Just like the partition cases,if the New/Edit/Delete/Reset all write as a case,there will be so many.But what you said is also important,I will try to add these operations to cases.

To accomplish the first 2 points, we may need to create several sample yum repos (hosted on one of our fedorapeople.org pages) that provides the needed conditions to satisfy the tests. The command {{{createrepo -g comps.xml /path/to/packages}}} can be used to create a yum repo with package groups. The comps.xml file used with FEdora is included in the repodata directory (see [http://download.fedora.redhat.com/pub/fedora/linux/releases/12/Fedora/x86_64/os/repodata/6a72ae27742d1b4ac04f2eec0e5ffb0b7c909d58b9c6e346a7ee5cfb627832d4-Fedora-12-comps.xml comps.xml]).

If we uncheck the current installation repo and provide a full repo to complete installation. I mean we do not use the current default repo,but use provided repo to finish installation,does this make sense to complete your first 2 points? I am not sure whether the testers can distinguish added repo packages from default repo

Liam, thanks for the initial test cases. With your focus on the dvd install test case, I'm going to reassign this to Hurry.

Replying to [comment:10 jlaska]:

<skip>
* modifying the tests so that they also verify whether the expected packages or package groups provided by the custom yum repos appear on the subsequent package detail screen.
* also, updating the tests to confirm that you can select a package provided by the added yum repo and ensure it's selected and installed as expected

I've modified the test cases for covering these 2 points.

  • I think each test might also add the following instructions to confirm all create/modify/delete operations work on the added repos.
    1. Add a new yum repo using the recommended method (CD/DVD, http, nfs etc...)
    1. Remove the created repo
    1. Add it again ... then modify the repo (changing the name)
    1. Enable and disable the created repository

I can see that you want to make the test cases cover more conditions. But here I agree with Liam that if we do so, the test cases may have too much possible conditions and look very wired. I think we design general cases to serve our need and that's ok.

So I improved especially mirrorlist repo case and now we have covered the following items:

  1. Check the existing repos listed in the screen.
  2. Provide new repo address by Add/modify buttion to have additional one.
  3. The method includes HTTP/FTP, CD/DVD, NFS.
  4. Verify them in next screen and after reboot.

To accomplish the first 2 points, we may need to create several sample yum repos (hosted on one of our fedorapeople.org pages) that provides the needed conditions to satisfy the tests. The command {{{createrepo -g comps.xml /path/to/packages}}} can be used to create a yum repo with package groups. The comps.xml file used with FEdora is included in the repodata directory (see [http://download.fedora.redhat.com/pub/fedora/linux/releases/12/Fedora/x86_64/os/repodata/6a72ae27742d1b4ac04f2eec0e5ffb0b7c909d58b9c6e346a7ee5cfb627832d4-Fedora-12-comps.xml comps.xml]).

I've made some experiments on my pc by setting up a local apache to test it. It works but I don't know how to host them on fedorapeople.org pages for everyone to use. Is it very tough? :)

Replying to [comment:13 rhe]:

Replying to [comment:10 jlaska]:

  • modifying the tests so that they also verify whether the expected packages or package groups provided by the custom yum repos appear on the subsequent package detail screen.
  • also, updating the tests to confirm that you can select a package provided by the added yum repo and ensure it's selected and installed as expected

I've modified the test cases for covering these 2 points.

Nice work, that helps make the tests more explicit.

  • I think each test might also add the following instructions to confirm all create/modify/delete operations work on the added repos.
    1. Add a new yum repo using the recommended method (CD/DVD, http, nfs etc...)
    1. Remove the created repo
    1. Add it again ... then modify the repo (changing the name)
    1. Enable and disable the created repository

I can see that you want to make the test cases cover more conditions. But here I agree with Liam that if we do so, the test cases may have too much possible conditions and look very wired. I think we design general cases to serve our need and that's ok.

Yeah, you two make a good point. Always a balance trying to figure out how much to stuff into a single test. Besides, the list of steps above are really corner case type issues.

So I improved especially mirrorlist repo case and now we have covered the following items:

  1. Check the existing repos listed in the screen.
  2. Provide new repo address by Add/modify buttion to have additional one.
  3. The method includes HTTP/FTP, CD/DVD, NFS.
  4. Verify them in next screen and after reboot.

    To accomplish the first 2 points, we may need to create several sample yum repos (hosted on one of our fedorapeople.org pages) that provides the needed conditions to satisfy the tests. The command {{{createrepo -g comps.xml /path/to/packages}}} can be used to create a yum repo with package groups. The comps.xml file used with FEdora is included in the repodata directory (see [http://download.fedora.redhat.com/pub/fedora/linux/releases/12/Fedora/x86_64/os/repodata/6a72ae27742d1b4ac04f2eec0e5ffb0b7c909d58b9c6e346a7ee5cfb627832d4-Fedora-12-comps.xml comps.xml]).

I've made some experiments on my pc by setting up a local apache to test it. It works but I don't know how to host them on fedorapeople.org pages for everyone to use. Is it very tough? :)

Yeah, you should have access to create a fedorapeople.org page. Take a look at http://fedoraproject.org/wiki/Infrastructure/fedorapeople.org for access information.

If you're able to create a small sample repo, with some dummy packages that can be used for producing consistent test results ... it might not be a bad thing to try. Possibly overkill, and I don't know if this might be helpful for your repo needs, but checkout https://fedorahosted.org/rpmfluff/ for a quick way to build packages that meet specific criteria (name, version, release).

Replying to [comment:15 jlaska]:

<skip>
Yeah, you should have access to create a fedorapeople.org page. Take a look at http://fedoraproject.org/wiki/Infrastructure/fedorapeople.org for access information.

If you're able to create a small sample repo, with some dummy packages that can be used for producing consistent test results ... it might not be a bad thing to try. Possibly overkill, and I don't know if this might be helpful for your repo needs, but checkout https://fedorahosted.org/rpmfluff/ for a quick way to build packages that meet specific criteria (name, version, release).

Thanks for your helpful info, James. WoW I've learned a lot since modifing these cases..The [http://rhe.fedorapeople.org/ testing repo] has been set up. I wrote the mini.xml so that this repo contains unique strategy(Repo Test), group(Mini Repo) and package(kong-0.1.1.noarch). This package generated by rpmfluff has been verified to install easily through system installation.

Therefore, I updated
[https://fedoraproject.org/wiki/QA:Testcase_Additional_Http_Repository http_repository] case. Any comment/improvement welcomed. I love overkill, thanks. :)

Replying to [comment:16 rhe]:

Therefore, I updated
[https://fedoraproject.org/wiki/QA:Testcase_Additional_Http_Repository http_repository] case. Any comment/improvement welcomed. I love overkill, thanks. :)

Awesome, I'm glad that worked out for you. I've made a few minor wording changes to the test cases. But feel free to revert if it changes things.

One suggestion I might offer is to relocate the test repository to a subdirectory underneath your main ~/public_html. Just so you have room to add other content without cluttering up the data used for this test.

Another thought, definitely above and beyond the original call for this ticket, what are your thoughts on providing several packages in your sample repo? I'm wondering if it would make sense to include several different packages each tagged differently in your XML file. For example ...
<packagereq type="default">kong</packagereq>
<packagereq type="optional">wife-of-kong</packagereq>
<packagereq type="mandatory">mother-of-kong</packagereq>
<packagereq type="conditional" requires="wife-of-kong">daughter-of-kong</packagereq>

Okay, I'm having fun with the names, but what do you think?

Replying to [comment:17 jlaska]:

Replying to [comment:16 rhe]:
<skip>
One suggestion I might offer is to relocate the test repository to a subdirectory underneath your main ~/public_html. Just so you have room to add other content without cluttering up the data used for this test.

Yeah, thanks for reminding me.

Another thought, definitely above and beyond the original call for this ticket, what are your thoughts on providing several packages in your sample repo? I'm wondering if it would make sense to include several different packages each tagged differently in your XML file. For example ...
<packagereq type="default">kong</packagereq>
<packagereq type="optional">wife-of-kong</packagereq>
<packagereq type="mandatory">mother-of-kong</packagereq>
<packagereq type="conditional" requires="wife-of-kong">daughter-of-kong</packagereq>

Okay, I'm having fun with the names, but what do you think?

There's no bad side about this, so I added them. But I don't think we need to check the packages with different types too carefully in new repo. since the existing repo can also check these. If the new repo packages can be listed and installed correctly, that's ok. [https://fedoraproject.org/wiki/QA:Testcase_Additional_Http_Repository http_repository] has been updated regarding this. Do you have any other concerns? Thanks.

Replying to [comment:18 rhe]:

There's no bad side about this, so I added them. But I don't think we need to check the packages with different types too carefully in new repo. since the existing repo can also check these. If the new repo packages can be listed and installed correctly, that's ok. [https://fedoraproject.org/wiki/QA:Testcase_Additional_Http_Repository http_repository] has been updated regarding this. Do you have any other concerns? Thanks.

No other concerns, the tests look great. Thanks to Liam for kicking things off, and to you for bringing things to a close.

Replying to [comment:19 jlaska]:

<skip>
No other concerns, the tests look great. Thanks to Liam for kicking things off, and to you for bringing things to a close.

I updated the other cases again according to your changes on http one and now I'm happy with the current [https://fedoraproject.org/wiki/Category:Repository repo cases]. Many thanks for your guidance and Liam's help.

I have one more concern, which pre-release should we test these cases on? Beta? I read the [https://fedoraproject.org/wiki/Fedora_Release_Criteria Release Criteria], but it doesn't indicate much about repo.

Replying to [comment:20 rhe]:

I have one more concern, which pre-release should we test these cases on? Beta? I read the [https://fedoraproject.org/wiki/Fedora_Release_Criteria Release Criteria], but it doesn't indicate much about repo.

Oh good question! I think we can prioritize these tests using the existing criteria regarding package install source. How do others feel?

For example, the '''Alpha''' indicates ...
* The installer must be able to use at least one of the HTTP or FTP remote package source options
* The installer must be able to use the DVD local package source options

The '''Beta''' lists ...
* The installer must be able to use the HTTP, FTP and NFS remote package source options

The '''Final''' lists ...
* The installer must be able to use all supported local and remote package source options

How do others feel?

Replying to [comment:21 jlaska]:

Replying to [comment:20 rhe]:

I have one more concern, which pre-release should we test these cases on? Beta? I read the [https://fedoraproject.org/wiki/Fedora_Release_Criteria Release Criteria], but it doesn't indicate much about repo.

Oh good question! I think we can prioritize these tests using the existing criteria regarding package install source. How do others feel?

For example, the '''Alpha''' indicates ...
* The installer must be able to use at least one of the HTTP or FTP remote package source options
* The installer must be able to use the DVD local package source options

The '''Beta''' lists ...
* The installer must be able to use the HTTP, FTP and NFS remote package source options

The '''Final''' lists ...
* The installer must be able to use all supported local and remote package source options

How do others feel?

Yes, we agree this priority.

I have added the cases to [https://fedoraproject.org/wiki/QA:Fedora_13_Install_Results_Template#Test_Matrix Matrix]. If no more comments about this ticket, I will close it tomorrow. Thanks all.

Replying to [comment:23 rhe]:

I have added the cases to [https://fedoraproject.org/wiki/QA:Fedora_13_Install_Results_Template#Test_Matrix Matrix]. If no more comments about this ticket, I will close it tomorrow. Thanks all.

They look great, thanks!

Metadata Update from @adamwill:
- Issue untagged with: test review
- Issue tagged with: test cases

6 years ago

Login to comment on this ticket.

Metadata