#327 Let's run a yearly Fedora survey
Opened 9 months ago by bookwar. Modified 4 months ago

The goal of the survey is to know our community, and to make decisions in Fedora based on data, rather than a gut feeling.

I've collected some ideas for questions in https://etherpad.gnome.org/p/fedora-survey-ideas


The Fedora D&I Team tried launching a contributor survey two years ago and was unsuccessful (mostly because of questions with Personally-Identifiable Information). It would be great to include non-risky survey questions from the D&I Team in a new survey (see fedora-diversity#12, fedora-diversity#62, fedora-diversity#130). These questions also got through some degree of RH Legal review, but it stalled out somewhere.

This is a good opportunity to collaborate with the D&I Team to launch a single survey (versus creating and launching multiple surveys and risking survey fatigue). We haven't done "global" surveys of the Fedora Community before, which I hypothesize is an advantage for launching a first survey.

It's worth noting that Matthew has been working with folks at Red Hat who do Net Promoter Score (NPS) surveys to run one for Fedora. This was in The Before Times, though, so I don't know if any progress has happened recently.

I'm unsure if I agree with Justin about single survey. We definitely do want to avoid survey fatigue, but if we space them out, I don't think two surveys is too much. I see Aleksandra's proposal focusing more on the technology and D&I's focusing more on the community. That's a good thing, IMO, we want to know both. My concern with a single survey is that it might get to be too long and we'll lose respondents.

I know D&I has been working on this for a long time, but it might make sense to run Aleksandra's proposal first (if we do them separately) since it's less likely to have anything construed as PII. We can get it started sooner and take the lessons learned from that to improve how we conduct D&I's survey.

In any case, there are still unanswered questions about who the audience for this survey is (contributors? users? people who have heard of Fedora?), how we'll reach that audience, etc. Definitely room for discussion still. (And if the two surveys have different answers, particularly to the audience question, that would be an argument to conduct them separately)

I support @bcotton's point here.

I think that analyzing DE choices with respect to the person's role in Fedora (i.e maintainer vs packager) is ok. While getting user base split based on gender for example, and claiming that "women prefer emacs over vim", would get us in a grey area. Even though it is not a Personal Info and not exactly regulated by law, I'd rather not dive into these kind of conversations.

Thus I wouldn't build links explicitly between demographic questions and tooling choices.

And also I don't think having two surveys is a problem, i'd rather have two then one which is twice longer.

I concur with @bcotton and @bookwar. I apologize, I don't think I communicated clearly and don't want to add stop energy for creatively collecting feedback from the Fedora Community.

I mean to say, there are useful questions about Fedora and contributing to Fedora that would be useful to "borrow" from the D&I Survey draft because deliberate thinking went into designing those questions. I don't think the P.I.I. concerns will be resolved anytime soon, from past precedent.

Anything with P.I.I. should be in a future survey when we have capacity and clearance to responsibly manage that data.

Screenshot of questions about Fedora and using Fedora and open source for an early draft of a community survey

Metadata Update from @bcotton:
- Issue assigned to bookwar
- Issue priority set to: Waiting on Assignee (was: Needs Review)

8 months ago

In today's Council meeting, @bookwar agreed to prepare a finalized proposal by 5 November.

A HackMD doc is in the works, with feedback requested by eventually.

I've regrouped questions and added more variants for answers, based on feedback from early testers.

There is a technical question regarding the Other field. Ideally every time someone chooses Other, as a single answer or part of the multi-choice answer, we would like to have a free-from text field where the answer can be typed directly. Is it possible with Lime Survey platform?

@jflory7 On the "do you regularly use proprietary software", I think the "only web based" is not super helpful because... it's basically impossible to avoid so goes without saying. (Someone who only visits completely open source websites would be a very rare exception!).

There is an option I would add, though: "yes, but only games". That's where I'm at. I love open source games, but I also feel like proprietary games are a different thing from other software... just like I wouldn't demand all of the books I read to be CC-BY-SA.

What is the goal

  1. We need to understand the usage pattern in various Fedora flavours and tooling
  2. We should check if we see clusters created in the community based on particular tools, or media or role.

Do we have a non-overlaping mailing list and Discord bubbles, or is it more randomly, evenly distributed?

By follow-up surveys we can track whether we improve the situation and connect these clusters better or we making it worse driving different kinds of contributors further apart. We can also check if we should provide a dedicated Spin or Flavour to a certain large cluster of our userbase.

What we plan to do with the results

  1. I'd like to publish the open dataset (anonimized and cleaned up) for others to play
  2. I will create charts for the distribution of votes for each particular question
  3. I'd like also to create multiple two-dimensional views by pairing the answers from the "About you" section and answers on Tools and Apps parts.

Some of these views may show us certain correlations between user personas and their set of tools. I'd like to verify if we can confirm some theories, like correlation between being a beginner and using nano, or we actually are driven by stereotypes.

Audience

Fedora users and contributors

@riecatnor
Do you think we need to handle also "probable future users" in this survey? From my point of view I'd rather not include them. I want to focus on the current Fedora users and contributors and their current choices. Keeping the topics of the outreach for some another time.

Do you think we need to handle also "probable future users" in this survey? From my point of view I'd rather not include them. I want to focus on the current Fedora users and contributors and their current choices. Keeping the topics of the outreach for some another time.

I agree let's leave "probably future users" out for now. I'm a little worried about combining users and contributors. It might be better to pick one of those and focus to start with. (I'd suggest contributors since we have more reliable ways of getting the survey in front of them)

@bcotton

I didn't mean any additional reaching out effort attached to this survey, so I assumed we already have it limited to our closer community.

I am just afraid that if we say "contributors" people may read it as "maintainers". And I want it to be more accessible for people who for example run a local community or support other users on Telegram, or write articles for the documentation or file bugreports.

Maybe participants ?

This is possible

I've regrouped questions and added more variants for answers, based on feedback from early testers.

There is a technical question regarding the Other field. Ideally every time someone chooses Other, as a single answer or part of the multi-choice answer, we would like to have a free-from text field where the answer can be typed directly. Is it possible with Lime Survey platform?

@bcotton

I didn't mean any additional reaching out effort attached to this survey, so I assumed we already have it limited to our closer community.

I am just afraid that if we say "contributors" people may read it as "maintainers". And I want it to be more accessible for people who for example run a local community or support other users on Telegram, or write articles for the documentation or file bugreports.

Maybe participants ?

I think we can focus the survey to contributors to the Fedora Project. I see your point on accessibility... I edited the intro to the survey, see below, and feel free to edit :)

This survey is targeted to Fedora Contributors and Community Members. Fedora contributors and community members are those that participate in the project in any way, be it coding, bug testing, docs, graphic design, translations, outreach, etc.

Hey folks! The Mindshare committee met today and we discussed this ticket: https://pagure.io/mindshare/issue/235

Because of survey fatigue etc, we thought we could potentially add in one question to this council survey instead of creating an entirely new survey.

Goal:
Gain further insight into how much the Fedora community understands about the Mindshare Committee and it's processes.

What we plan to do with the data:
Use the data we gain to improve the Mindshare Committees visibility and make our processes more useable to the entirety of the Fedora community.
(for example, is there a correlation between folks who know about Mindshare and their area of contribution?)

Proposed question:
Who would you first try to contact if you wanted to run a Fedora event/needed swag?
- Fedora Council
- FCAIC
- Mindshare Committee
- Regional Fedora Ambassador group

Login to comment on this ticket.

Metadata