Test day will be organized and prepared by the IPA team. We have experience in holding such events. The suggested features to test can be found on the feature page but will create a much more detailed writeup.

We would prefer to have it in February. If this date is not possible we might look at March but this would not leave us any time to address the issues found.

sounds awesome, thanks for the proposal! The test day schedule is here:

https://fedoraproject.org/wiki/QA/Fedora_15_test_days

February 10th is currently available.

Yes this is why I asked for the day. I was not sure who is supposed to update the table. It seems that the request needs to be approved first before it gets into the schedule. But I do not know by whom.

nope, as the SOP says, you can just add your day into the schedule. there's no approval. go ahead and do it.

I spoke to dpal on IRC earlier today. Some bugs have surfaced and he would like to reschedule the test day. A likely candidate may be next Tuesday, Feb 15. Stay tuned...

Due to some internal issues and bugs we decided to move test day to Feb 15.

I tried to edit the page and see that I do not have permission to do so. Please move the freeIPA to Feb 15th.
Also please do an announcement about it.

You only need to log into the Wiki using your FAS account to edit it.

Feedback about the test day.

The test day was not a success. It was actually a bit of a shame at least for me. We put a lot of preparation into it but definitely that was not enough.

What was done:
1. Announcements on the project wiki and mailing lists.
2. Test cases and procedures were prepared and put on the Fedora test day wiki page. [[BR]]https://fedoraproject.org/wiki/Test_Day:2011-02-15_FreeIPAv2

Issues:
1. We were too late in our development cycle to provide the stable build for F15 early and actually try it and have time to resolve the issues.
2. F15 was not stable. systemd was at blame but other issues too. As a result of the F15 instability people spent most of the time setting up Fedora and not IPA.
3. Dogtag did not work at all (worked fine on F14) and thus did not make a test day
4. SSSD was outdated
5. We did not have a good guidance that if something fails there are other test cases that people can execute.

So there is a mixture of reasons why the test day can be counted as failed.

No the question is: what can we do about it and what should we do about it?

May be we can have another test day? Do you think it makes sense to have one?
Here is an open slot: 2011-04-21

I am open for suggestions. I am not sure that deserve another test day if there other projects that need it. But if it is empty may be we should go for it and have better tester alignment in advance?

Sorry you had a bad experience :(. I think definitely coming early in the cycle to a release with some major changes didn't help.

The 04-21 slot is open; the only issue with that slot is that it's very close to the final change deadline, so it would be hard to get many changes into the release as a consequence of the test day. More likely any fixes would have to go out as post-release updates.

It's definitely key to get preparation done for the test day in time and to do a 'dry run' with the prepared materials to make sure the testing can actually be achieved; do you think I should explain that more clearly in the SOP?

I am not sure it is something that needs to be explained. It was obvious for me without explanation but we did not have enough time for a "dry run" and this is the main problem. I think having test day late in the cycle should be OK. We will decide if the fixes should go into the IPA 2.0 updates or actually into a 2.1 release. Either case it should not be a problem for F15.

Replying to [comment:10 dpal]:

I am open for suggestions. I am not sure that deserve another test day if there other projects that need it. But if it is empty may be we should go for it and have better tester alignment in advance?

Another thought worth considering, who is the target audience for the freeIPA test day? Is it a novice tester who may be unfamiliar with what freeIPA is and how to set it up, or is it the average freeIPA user who is already familiar with setup? One reason I ask is the test cases, while very well written, are somewhat lengthy/complex/involved (see https://fedoraproject.org/wiki/QA:Testcase_freeipav2_ui). If the tests/event is focused on novice users who haven't used freeIPA before, perhaps the level of detail in these tests is a bit too daunting for a first-time tester.

Also, is there an active community of freeIPA users, developers and testers already? Or are we also trying to build/develop interest in freeIPA with the test day? I think that question also helps frame the scope and audience for the event.

The hope was that the IPA community would use the Fedora test day as a way to test IPA. We were hoping that those people would constitute most part of the testing group. I do not think that a novice would be able to do anything. IPA is an identity management product so to be able to test it you need to be fully exposed to the problems of identity management and authentication.
But the hope is to reach out to a broader IT community via the Fedora lists and announcements that actually is exposed and understands what IPA tries to solve.
May be the expectation is wrong?

Just following up on what I proposed on QA meeting that we host seperated test days and test each component individually as 389 Directory Server. MIT Kerberos, NTP, Bind DNS and the Dogtag Certificate Server and finally end up hooking up each those servers to a final FreeIPA test day.

We probably should also involve servers SIG in help hosting those test days and reaching out to the admins that possess the necessary skill and experience in setting test event like this up.

Replying to [comment:15 johannbg]:

Just following up on what I proposed on QA meeting that we host seperated test days and test each component individually as 389 Directory Server. MIT Kerberos, NTP, Bind DNS and the Dogtag Certificate Server and finally end up hooking up each those servers to a final FreeIPA test day.

I disagree with this approach. Decomposing it into the separate components looses the whole point of IPA as product. The whole point of IPA is to have one thing to configure and install than five. Also the components are configured in a specific way and it is not possible in some cases to test them the way we need if they are installed by itself.

You do realize that I mentioned in the end that we would test FreeIPA in the end right?

Anyway we need to reach out to the correct audience ( Server SIG ) that has the necessary skills experience and infrastructure to setup and test deployment and correct usage of FreeIPA and all of the previously mentioned components..

Replying to [comment:17 johannbg]:

You do realize that I mentioned in the end that we would test FreeIPA in the end right?

Yes but it won't work as you propose.

Anyway we need to reach out to the correct audience ( Server SIG ) that has the necessary skills experience and infrastructure to setup and test deployment and correct usage of FreeIPA and all of the previously mentioned components..

I agree with this 100%.

Am I getting you right that you are unable to hook already existing components that freeipa uses that may or may not be running on different *nix to it?

Basically you cant implement it in an already existing infrastructure environment?

Replying to [comment:19 johannbg]:

Am I getting you right that you are unable to hook already existing components that freeipa uses that may or may not be running on different *nix to it?

Basically you cant implement it in an already existing infrastructure environment?

This is correct. IPA is one stop shop replacement for those stand alone solutions. It is installed on one box (but you can have many replicas) and has all parts tightly integrated. KDC + DS + CS + NDS. You can avoid using CS or DNS but that means you are using stand alone CS or DNS and not testing it in the configuration in which it is embedded into IPA so there is no value for IPA from such testing.

Completed, closing ticket.