#594 What about a QA category on discussion.f.o.?
Opened a year ago by alciregi. Modified a year ago

What do you think if we create a category related to QA on discussion.f.o.?

Audience: newcomers. Organizing onboarding calls is a big effort. Newcomers could ask questions on irc or on the mailing list. But IMO thesle channels can be a little intimidating: "gasp, all these experts here, all this difficult conversations, if I ask a rookie thing I can look pretty bad" :-)

Goal: offer another method, mainly for newcomers to ask questions, clarifications, suggestions.


@adamwill @alciregi It will be great idea to get started and I believe it will be very helpful for the QA teams presence.
I will take this on me and will talk to @alciregi and @bcotton and see how this can be worked out

Metadata Update from @sumantrom:
- Issue assigned to sumantrom

a year ago

Metadata Update from @sumantrom:
- Assignee reset

a year ago

If you do this, I suggest retiring the mailing list. Otherwise you end up with conversations in two different places.

Edit: Or at least make very clear which conversations should happen on the mailing list and which should happen on discussion.

@bcotton my proposal was to address the lack of onboarding calls for newcomers. Sometimes people is happy to help testing things, but as far as I see they don't know (me too! :smile:) what they are supposed to do, or in what way they should contribute, or what is important to test and to report.
My proposal is not aimed to retire the mailing list. The mailing list should be stay in place as usual (if the QA team doesn't agree on use Discourse exclusively of course, but I think this is not the case) to manage the daily work.

So to summarise: the target of discussion.f.o. should be the newcomers when someone need assistance in their first steps, and without disturbing the work of the team with questions already asked in the past but which are difficult to search in a mailing list, and above all, from the point of view of a newcomer, where somebody is pretty sure that this is a confortable place where to ask rookie questions.

@alciregi okay, but then how does a person know when it's a rookie question that should go to discussion.fp.o and when it's daily work that should go to the mailing list? And is the QA team now obligated to monitor both places? (And if not, who will answer the questions?)

without disturbing the work of the team with questions already asked in the past but which are difficult to search in a mailing list

When you say this, I think maybe the answer is better documentation (perhaps with an FAQ) rather than adding another communication channel.

On a related note, is this the right place to have the conversation? 20 people are subscribed to this issue, but 43 people have contributed to the QA mailing list in the past 30 days according to Hyperkitty.

@alciregi okay, but then how does a person know when it's a rookie question that should go to discussion.fp.o and when it's daily work that should go to the mailing list? And is the QA team now obligated to monitor both places? (And if not, who will answer the questions?)

This is true. :pensive:

When you say this, I think maybe the answer is better documentation (perhaps with an FAQ) rather than adding another communication channel.

Right.
But you know, I can read tons of documentation. But sometime I need to discuss with people expressing doubts, feelings, ideas. And sometime people needs guidance. I know that a lot of people in the community have a ton of messages to read and to reply to. But it would be nice if the newcomers could animate the discussion around the activity of testing Fedora.
If you look at the mailing list, as far as I can see, there are various introduction mails. After the introduction, where such people goes? Are they all at work testing things? I'm afraid not. Maybe I've not the tools to look and measure that, @sumantrom what do you think?

However this was only an idea 😊

@alciregi okay, but then how does a person know when it's a rookie question that should go to discussion.fp.o and when it's daily work that should go to the mailing list? And is the QA team now obligated to monitor both places? (And if not, who will answer the questions?)

without disturbing the work of the team with questions already asked in the past but which are difficult to search in a mailing list

When you say this, I think maybe the answer is better documentation (perhaps with an FAQ) rather than adding another communication channel.

I am working on setting up a qa-docs for docs.fp.o in some time. which can have a FAQ if required.
I am although wondering, would it really harm to have a QA presence on discourse which we can monitor from time to time? I mean I can take some time answering questions. Although I ought not to be doing anything with the current test or any other qa ML that the team follows.

But you know, I can read tons of documentation. But sometime I need to discuss with people expressing doubts, feelings, ideas. And sometime people needs guidance. I know that a lot of people in the community have a ton of messages to read and to reply to. But it would be nice if the newcomers could animate the discussion around the activity of testing Fedora.
If you look at the mailing list, as far as I can see, there are various introduction mails. After the introduction, where such people goes? Are they all at work testing things? I'm afraid not.

Is this a technology problem or a process problem? What about discourse would fix this that the mailing list won't solve? And if there's a reason that discourse would be better, why keep the day-to-day work on the mailing list?

For what it's worth, I see a lot of benefit in moving to Discourse (once some of the automated tooling is ported to post to Discourse instead of sending emails), but I don't see value in splitting the communication across both the mailing list and discourse.

would it really harm to have a QA presence on discourse which we can monitor from time to time?

It would harm to split the conversations. Apart from having to monitor both places, it separates the newcomers and the experienced contributors, which could potentially make it more difficult to convert newcomers into experienced contributors.

Is this a technology problem or a process problem? What about discourse would fix this that the mailing list won't solve? And if there's a reason that discourse would be better, why keep the day-to-day work on the mailing list?

Good questions.
My point is still how to facilitate onboarding. How to make newcomers feel in a comfortable place. And I don't want to say that people is not polite, not nice or so on. Far from it! There are always a lot of interesting things, many new things to learn in the team discussions. What I spotted, and I could be wrong, is that many newcomers stay silent. Why?

Maybe, another idea could be something like a monthly or bi weekly IRC (and telegram) office hour, in order to make up the difficulty to organize onboarding calls.

When I decided to participate (for what I am able to do) in the qa team, I had the luck to attend an onboarding call hosted by @sumantrom. I was impressed. Very interesting, nice people, many doubts gone away. Such experience contributed to let me stay in the qa team and to better explore the Fedora community.

For what it's worth, I see a lot of benefit in moving to Discourse (once some of the automated tooling is ported to post to Discourse instead of sending emails), but I don't see value in splitting the communication across both the mailing list and discourse.

This should be actually asked in the mailing list, as you have observed before, many people are not here on pagure.
But migrating to Discourse was not the central point of my proposal :-)

It would harm to split the conversations. Apart from having to monitor both places, it separates the newcomers and the experienced contributors, which could potentially make it more difficult to convert newcomers into experienced contributors.

You're right.

Here you can find some data: https://alciregi.fedorapeople.org/qa594/intro.ods
Disclaimer: I'm not a data scientist :sweat_smile:

Data is referred to this year, from 2019/01/01 until 9/21.
These data are referred to people that sent a mail to the QA list with the subject containing “intro”.

Total newcomers are 41

Last seen in September: 11
Last seen in August: 9
Last seen before August: 21

5 did not apply to the qa fas group
4 are still unapproved

Total messages to the QA mailing list since January: 2028
1064 sent by rawhide@fedoraproject.org or updates@fedoraproject.org

Participation to the mailing list after the introduction mail is equal to 0 on average. 12 people sent one mail after the introduction, 5 more tan one, 2 people more than ten, and the rest, 22 people, 0 mails.

If we don’t count people with a last_seen value on FAS corresponding to this month and to August: they are 21 people.
They presumably left Fedora (no more FAS logins). After 70 days on average (5 users have FAS last_seen value corresponding to the same day they sent the introduction mail, 5 after a couple of months, 2 after nearly 6 months).
At the same time, activity of these newcomers is 0 even in the rest of the mailing lists of the Fedora Project.
Only 4 of these people have performed some activities on Bodhi, Kerneltest, and on the wiki (very low in this case). The rest, 16 people, did not performed any activity.

Also for people considered still active, activities in other mailing lists of the project is near to 0, while there are some bodhi, on the wiki, and kerneltests.

Only two people are part of an additional FAS group in addition to the QA one (not counting cla_done and cla_fpca).

Columns in the file:

username: FAS account
first_intro: date of first mail to QA mailing list containing the string "intro" (ignorecase) in the subject
last_seen: last login to any Fedora application
privacy: privacy flag on FAS
followups: additional mails sent to the QA mailing list after the introductory one
count_additional_groups: additional FAS groups the user is part of
qa_group_status: QA FAS group status (approved, unapproved, na: the user didn't ask to join the group)
bodhi_activity: number of activities from datagrepper in "bodhi" category (from introduction mail date until 9/21)
kerneltest_activity: number of activities from datagrepper in "kerneltest" category (from introduction mail date until 9/21)
bugzilla_count: number of activities from datagrepper in "bugzilla" category (from introduction mail date until 9/21)
wiki_activity: number of activities from datagrepper with "wiki" category (from introduction mail date until 9/21)
mailman_count: number of activities from datagrepper with "mailman" category (number of mails sent to all the fedora mailing lists) (from introduction mail date until 9/21)

I would like to thank @bt0dotninja for some advices about querying FAS and datagrepper.

those are interesting numbers, thanks!

A couple of things to check:

  1. Bugzilla email address does not always match list email address - did you check these?
  2. Did you consider Test Day activity fully? Test Days are one of the most common ways for people to get involved, but may not result in activity countable by Bugzilla, wiki, or mailing list involvement. Most Test Days use the blockerbugs webapp for result submission; we eventually transfer the results from there to the wiki, but that will show up as an edit by @sumantrom or me or some other admin-type person, not multiple separate edits by the people who filed the results.

Bugzilla email address does not always match list email address - did you check these?

These are the parameters used to query datagrepper:
'contains': user, 'category': "bugzilla"

So, yes the results could be inaccurate.

Did you consider Test Day activity fully?

No, you are right.
I didn't check that. But now I can do it with this tool: https://pagure.io/testdayscraper :-)

blockerbugs webapp for result submission; we eventually transfer the results from there to the wiki, but that will show up as an edit by @sumantrom or me or some other admin-type person, not multiple separate edits by the people who filed the results.

Yes, but I suppose that wiki activities also contains updates to the Test_Results pages, right? So this should reflect activities on testing composes, right?

Ok. Looking at test days of this year (from http://testdays.fedorainfracloud.org/events/56 to /71), supposing that such newcomers inserted their FAS username in the related field, these are the results.

People with last_seen value < August:
1 person took part in 4 test days,
2 people in one test day,
18 people didn't take part in any test day.

People last seen in August:
4 people took part in 1 test day
5 people didn't take part in any test day.

People last seen this month, excluding the ones that asked to join the team this same month:
1 person took part in 5 test days
3 people took part in 1 test day
3 people didn't take part in any test day

"Yes, but I suppose that wiki activities also contains updates to the Test_Results pages, right? So this should reflect activities on testing composes, right?"

Yeah, any release validation test activity should show up as wiki edits by the person in question. You can use wikitcms to query this quite easily too, btw.

Login to comment on this ticket.

Metadata