#12 Survey - Numbers are important
Closed: Complete 6 years ago Opened 8 years ago by amsharma.

We need to conduct survey for diversity statistics in Fedora.
Here is the rough draft from @tatica ::
*** Survey : I might like to start kicking a small survey to know how diverse our community is, it's important to me to understand the reach of our contributors, their experiences, needs and culture prior to start any project. Since we might have some people answering it just for fun instead of the right reasons, it could be a good idea to make it only-FAS for now; if at least 30% of the Fedora environment answer it, it would be a success. Here is a list of the things I would like to know, some might go off some might be included:
I'm interested into people knowing that Fedora has an insane cultural background and maybe in a near future, this will open the eyes of those who think that everything in Fedora is plan blue!

Questions: https://fedoraproject.org/wiki/Diversity/Survey

We may add one more question as any blockers and motivational factors, if you face/observe any in Fedora.


I've gathered the questions into a wiki page for easier contributions. Please refer to https://fedoraproject.org/wiki/Diversity/Survey

So, somewhat recently, I stumbled across this giant survey of open source contributors (seems like roughly 2000 people). You can find the questionnaire they used along with the anonymized, raw dataset from responses (for importing into either R or a LimeSurvey instance).

Might be useful for us to refer to later on. :smile:

I expanded on the survey topic and tried to offer justification for it in the FAD page. Proof me and make sure I got the vision right when you have a moment. :grinning:

This has long been one of the founding goals of the Diversity Team, but it's helpful to have some background context. This item has been filed as tickets in the Fedora Council Trac for over two years. See both tickets here.

Discussed during the first ever Fedora Diversity FAD.

To say that this was heavily discussed would be an understatement! As a note, it would be impossible for me to cover the level of detail that we took when discussing elements of the survey during the FAD. But for the benefit of anyone who was not in attendance, I am confidently able to assure that the strategies, questions, and level of thought that have gone into these are significant and have been given great consideration. Here goes!

Creating / identifying questions

As a way for us to reference how we did this in the past, the process for how we created the questions is as follows:

  1. @marinaz reviewed, edited, and expanded on the questions originally identified by @tatica by cross-referencing other surveys such as the FLOSS 2013 survey and Apache Contributor User Survey
  2. Reviewed questions for content, made cuts to questions we believed to be irrelevant
  3. With final set of questions, grouped all questions into categories (five 1s, five 2s, five 3s, five 4s, five 5s, five 6s, and two 7s) and identified the strongest set of ~20 questions
  4. Further discussed and narrowed down answer options from final subset of questions

This is a very, very short breakdown of what was two days of discussion, debate, and critical thinking / analysis.

Questions

The final questions we have along with answer choices will be posted towards the end of the FAD once we finalize the list. I invite @amsharma, @bex, @bee2502, @jonatoni, @rhea, @marinaz, and @tatica to add anything else to this ticket comment that I may have overlooked or missed, as far as details for strategy, planning, or other misc. details go.

Discussed during Diversity FAD and 2017-02-07 meeting.

This comment is the "big one" to summarize and pull in everything we discussed in person in Brno. I'll revisit our objectives and goals we wanted to have for the questions as well as post a non-final list of the questions converted from the Etherpad.

Survey objectives

The primary objectives we identified for the demographic survey were…

  1. Gather baseline demographics about the contributor community
    • Why is this objective important?
      • Better allows us to focus on groups who are underrepresented and try to improve the user experience for new contributors in the community
    • Examples of target criteria:
      • Numbers for underrepresented groups (race / gender / sexuality / etc.)
      • Age of contributors
      • Gender
      • Race/Cultural background
      • Location
      • Disability
      • Family / kids / Motherhood ?
      • Non-English speaking community
  2. Determine contributor knowledge about project components that ease contribution
    • Join SIG
    • Mentoring Programs
    • Onboarding programs
    • Classrooms?
    • Code of conduct / diversity team

Other comments and thoughts

Other questions, comments, and ideas from our raw notes…

  • @bee2502's musings - some questions
    • How were you introduced to Fedora?
      1. By another Fedora contributor
      2. At a Fedora event (e.g. FAD / workshop / meetup / release party)
      3. Conference
      4. Hackerspace
      5. Red Hat
      6. Web / online
    • What makes it difficult contributing to Fedora for you?

We also wanted to identify how the data will be used and clearly communicate how the data will be used (ideally, this outline or information being available for participants in the survey to review).

Survey questions

Below are the "mk. 1" version of the survey questions. Our last plan for these was to submit them to Fedora Legal for feedback, which I believe @bex and @marinaz were going to work on. These questions will also be updated in the Survey wiki page.

Fedora and free and open source software usage and contributions

  • How long have you been a Fedora user?
    • I have never used Fedora
    • Less than 1 year
    • 1 to 3 years
    • 3 to 6 years
    • 6 to 10 years
    • Over 10 years
    • Prefer not to say
  • How long have you been a Fedora contributor?
    • I have never contributed to Fedora
    • Less than 1 year
    • 1 to 3 years
    • 3 to 6 years
    • 6 to 10 years
    • Over 10 years
    • Prefer not to say
  • Please select types of contributions that you make to Fedora:
    • Community (community leadership, new contributor outreach and onboarding, diversity and inclusion work, marketing, event organization)
    • Technical (packaging, programming, infrastructure, operations, release engineering, testing / QA, language-specific SIGs)
    • Other non-technical (design, documentation, localization, translation)
    • Prefer not to say
    • Other (please specify)
  • Do you contribute to Fedora as a volunteer or as an employee of a company? [select all that apply]
    • Volunteer (student)
    • Volunteer (unemployed)
    • Volunteer (retired)
    • Volunteer (outside regular job)
    • Employee (work-related contributions)
    • Prefer not to say
    • Other (please specify)
  • On average, how many hours a week did you spend on Fedora contributions in the last three months?
    • 0, I have not contributed in the last three months
    • Less than 2
    • 2 - 5
    • 6 - 10
    • 11 - 20
    • 21 - 40
    • More than 40
    • Prefer not to say
  • What are the top reasons you contribute to Fedora? Please check a maximum of 4 answers.
    • To participate in a global cooperative project
    • To participate in a free and open source software project
    • To improve a free and open source software project
    • Because I think that software should not be proprietary
    • To solve a problem that could not be done by proprietary software
    • To get help in realizing a good idea for a software project
    • To learn and develop new skills
    • To share my knowledge and skills
    • To improve my job opportunities
    • To get a reputation in the free and open source software scene
    • To meet other people in the free and open source software community
    • Because I enjoy working with other people in the community
    • To get funding to travel internationally
    • To earn a living
    • Prefer not to say
    • Other (please specify)
  • Think of the time you made your first contribution to Fedora that was incorporated. What best describes your experience?
    • My co-worker, who was paid to work on Fedora, helped me prepare the contribution
    • A person I already knew socially helped me prepare the contribution
    • A person I did not know before helped me prepare the contribution at an in-person event
    • A person I did not know before helped me prepare the contribution via electronic communications
    • A mentor for a third-party program (e.g. Google Summer of Code, Outreachy, etc.) helped me prepare the contribution via electornic communications
    • A mentor identified by the Fedora Project helped me prepare the contribution via electronic communications
    • A person whom I identified as a mentor helped me prepare the contribution via electronic communications
    • I submitted my contribution electronically for review and it was reviewed by my co-worker, who was paid to work on Fedora
    • I submitted my contribution electronically for review and it was reviewed by someone I already knew socially
    • I submitted my contribution electronically for review and it was reviewed by someone I didn’t know before
    • Prefer not to say
    • Other (please specify)
  • Do you routinely use software that is not free and open source on Fedora?
    • Yes, but only web-based applications like Google Docs
    • Yes, but only locally installed applications (including hardware drivers)
    • Yes, both web-based and locally installed applications
    • No
    • Prefer not to say
    • Other (please specify)

Demographics

  • What is your gender?
    • Male
    • Female
    • Non-binary
    • Prefer not to say
    • Other (please specify)
  • Do you identify as lesbian, gay, bisexual, transgender or queer?
    • Yes
    • No
    • Prefer not to say
  • What is your age range?
    • 17 or younger
    • 18 to 24
    • 25 to 34
    • 35 to 44
    • 45 to 54
    • 55 to 64
    • 65 to 74
    • Over 75
    • Prefer not to say
  • Do you have any disability, including, but not limited to any of the ones listed here?
    • Yes
    • No
    • Prefer not to say
  • Do you have a partner?
    • No, I am a single
    • Yes, although we do not live together
    • Yes, I live together with my partner
    • Yes, I am married
    • Yes, I am married, but we do not live together
    • Prefer not to say
    • Other (please specify)
  • How old is your youngest child?
    • I don’t have children
    • Less than 2 years old
    • Between 2 years and 5 years old
    • Between 6 years and 10 years old
    • Between 11 years and 17 years old
    • 18 years or older
    • Prefer not to say
  • What region of the world do you reside in?
    • Asia and Pacific Countries (APAC)
    • Europe and the Middle East (EMEA)
    • Latin and South America (LATAM)
    • North America (NA)
    • Prefer not to say
  • What is your country of residence? [FAS Country List - Divided by Region]
  • What is the highest level of education you have completed?
    • Less than secondary (High) school
    • Secondary (High) school graduate or equivalent
    • Vocational / trade program or apprenticeship
    • Some college (working towards a first level university degree)
    • Some college (not pursuing a degree anymore)
    • Associate degree
    • Bachelor’s degree
    • Master’s degree
    • Doctorate (Ph.D.) or other advanced degree (e..g M.D., J.D.)
    • Prefer not to say
    • Other (please specify)
  • If you are employed, what organization do you work for?
    • I am not employed
    • Employed (please specify organization name)
    • Employed, but prefer not to specify the organization
    • Prefer not to say
  • If you are employed, what do you work on? [select all that apply]
    • I am not employed
    • Software development
    • System administration
    • Testing / QA / QE
    • Technical writing
    • UX/UI design
    • Marketing /communications
    • Community and event management
    • Prefer not to say
    • Other (please specify)

Perception of, experience with, and attitudes towards diversity and inclusion

Should we use headers / categories for questions, we will want something easier to read for a "production" version of this section.

The following questions can be presented as a table, with the same set of answer options. Questions about diverse perspectives, comfort with contributing ideas, valuing best ideas, and feeling respected were adapted from Red Hat Associate Survey.

  • Please indicate how you feel about the following statements.
    • Diverse perspectives are valued in the Fedora community.
    • I feel comfortable contributing my ideas (in meetings or online forums), even when my views differ from other contributors.
    • The people I collaborate with value the best ideas regardless of their source.
    • I feel respected when I share my ideas and opinions in the Fedora community.
    • Having greater diversity of contributors will positively impact the Fedora Project.
    • I’m appreciated for the contributions I make.
  • Information about Fedora processes, policies or guidelines is easy to find.
    • Strongly agree
    • Agree
    • Slightly agree
    • Neither agree nor disagree
    • Slightly disagree
    • Disagree
    • Strongly disagree

Please do not put any identifying information about yourself or others in either text box.

  • Is there anything you believe would improve the inclusivity of the Fedora community?
  • Do you have any feedback on this survey and are there any additional questions you believe it should have asked?

@bex / @marinaz Whenever either of you have a chance to submit these questions to Fedora Legal, it would be great if you could leave an update in the ticket. :smiley: I'm not sure how long it will take for Legal to review these, but it would be helpful to know when they've been submitted for our own planning purposes.

@bex Can you update this ticket about the current situation and also, some timeline till when we can hear back from Legal team? It would help us greatly in planning our tasks for the upcoming year.

Metadata Update from @bee2502:
- Issue priority set to: happening now (was: urgent (1-2 weeks))
- Issue tagged with: blocked, needs feedback

7 years ago

Meaningwhile by the time @bex revert on this, can we decide on how we are going to ship this survey to the end users and how we are going to socialize about it. Also testing is something we would like to do before shipping,
AFAIR Bex gonna help us to review the questions, not the platform we are going to use.
Do we know which platform we are planning to use? Can we test that with few questions before we hear back from Bex?

I don't think we are blocked here, we still have actions to do?

For platform, does hosting it on limesurvey or surveymonkey seem appropriate? Limesurvey is open source but Surveymonkey is easier to use and gives some auto generated statistics too.
Personally, I am +1 for Surveymonkey and 0 for Limesurvey.

About promoting the survey, I can think CommBlog and social media channels but this is something I want to leave out for later as the more important part here is identifying the timeline of these posts which we cannot work on till we have updates.

re: testing, What points will we test the survey for? As far as I know, the testing was mainly for not making the survey too long but I think we addressed it well in FAD. If we are testing the platform i.e. surveymonkey vs limesurvey for UX, Diversity team members can look at sample surveys on surveymonkey. The FLOSS 2013 survey was on limesurvey. Surveymonkey definitely doesnt have any platform related bugs and possibly, limesurvey too since both are quite famous worldwide.

I'm +1 for Limesurvey because it's open source. Sorry but I would like avoiding platforms that are not open source, even that it's easier to use them.

Metadata Update from @bee2502:
- Issue set to the milestone: None (was: Fedora 26)

7 years ago

Metadata Update from @jflory7:
- Issue priority set to: normal (3-4 weeks) (was: happening now)
- Issue set to the milestone: Fedora 27 (to Nov. 2017)

7 years ago

I'm +1 for Limesurvey because it's open source. Sorry but I would like avoiding platforms that are not open source, even that it's easier to use them.

@jonatoni I also think if there is a viable open source platform available, we should use it.

Any updates on this? @jflory7 @bex @amsharma

As far as I know, this is still waiting for Fedora Legal approval. I know @marinaz was also helping with this but she's on maternity leave for the next month or two.

This got sidelined by Flock. I will begin working on it again as soon as Flock is squared away. :)

One blocker before going to legal is the infrastructure. We need to find a third-party that can guarantee we don't see data that we shouldn't. We do not want to be able to identify, even accidentally, specific respondents. This research is taking time.

One blocker before going to legal is the infrastructure. We need to find a third-party that can guarantee we don't see data that we shouldn't. We do not want to be able to identify, even accidentally, specific respondents. This research is taking time.

This is fine. Please let us know if there's any way that we can help with these efforts. For now, I'm going to assign this ticket to you and you can update us once there's progress after Flock.

Metadata Update from @jflory7:
- Issue untagged with: needs feedback, outreach
- Issue assigned to bex (was: bee2502)
- Issue priority set to: no deadline (was: normal (3-4 weeks))
- Issue set to the milestone: Future releases (was: Fedora 27 (to Nov. 2017))
- Issue tagged with: internal operations

7 years ago

@bex @marinaz @jflory7 do we have any update on this one. TIA

@amsharma Last update I heard was preparing to send off to or awaiting feedback from RH Legal.

That is the current status.

Ticket moved

This ticket focused on implementation and strategic advancement of the demographic survey. Since the legwork in creating the survey scope and focus is done, I created a new ticket, #62, that more accurately explains the progress on publishing this survey.

Closing ticket as complete, since the implementation and strategic focus was completed at our 2017 FAD.

Metadata Update from @jflory7:
- Issue untagged with: blocked
- Issue close_status updated to: Complete
- Issue set to the milestone: Fedora 26 (to June 2017) (was: Future releases)
- Issue status updated to: Closed (was: Open)

6 years ago

Log in to comment on this ticket.

Metadata