queryfedbus.py
queryfedbus2.py
They get the same data: a list of messages sent to the specified mailing list, in this format:
sender email, mail subject, timestamp, datetime, datagrepper uuid
parsecsv.py
It parses the resulting CSV from one of the previous script.
For each line:
- it checks if the sender is not in exclude option (i.e. automated messages from some bot like updates@fedoraproject.org)
- it get the mail address of the sender
- it queries FAS using fas.people_query and the mail as the constraint in order to get the FAS username
- if the query fails (the mail used to send the message to the mailing list is not the same used in FAS)
- it strip the @domain part of the mail and as a best effort it assumes the username is this part of the mail
- using fas.person_by_username it gets these information:
- last_seen
- group_roles (excluding the ones specified in excludegroups, like cla_done, cla_fpca)
- then it parse again the CSV file containing the mailing list stuff in order to look for subsequent messages after the introduction one
- it queries datagrepper in order to grep some activities in the time period following the introduction e-mail; for QA:
- total number of Bodhi activities ('user': user, 'category': 'bodhi')
- total number of Wiki edits ('user': user, 'category': 'wiki')
- total numebr of Bugzilla activities ('user': user, 'category': 'bugzilla')
- total kernel tests activities ('user': user, 'category': 'kerneltest')
- total number of mails to any other Fedora mailing list (excluding the one in question)
- the result is a CSV file containing:
- username: FAS username
- first_intro: date of the introductory mail sent to the mailing list
- last_seen: date of the last login to FAS
- privacy: privacy setting status on FAS
- followups: number of mails sent to the mailing list following the introductory one
- count_additional_groups: total number of FAS groups the user is part of (in addition to the ones specified in the excludegroups config option)
- qa_group_status: status of the sponsoring
- bodhi_activity: number of activities in Bodhi
- kerneltest_activity: number of performed kernel tests
- bugzilla_count: number of bugzilla activities
- wiki_activity: number of wiki activities
- mailman_count: number of mails sent to any mailing list
python3 ./queryfedbus.py <START_DATE> <END_DATE> <mailing_list_name>
python3 ./queryfedbus2.py <START_DATE> <END_DATE> <mailing_list_name>
START_DATE
and END_DATE
should have this format YYYY-MM_DD
mailing_list_name
shoud be the part before @lists.fedoraproject.org