queryfedbus.py
queryfedbus2.py
They get the same data: a list of messages sent to the specified mailing list, in this format:
sender email, mail subject, timestamp, datetime, datagrepper uuid
parsecsv.py
It parses the resulting CSV from one of the previous script.
For each line: * it checks if the sender is not in exclude option (i.e. automated messages from some bot like updates@fedoraproject.org) * it get the mail address of the sender * it queries FAS using fas.people_query and the mail as the constraint in order to get the FAS username * if the query fails (the mail used to send the message to the mailing list is not the same used in FAS) * it strip the @domain part of the mail and as a best effort it assumes the username is this part of the mail * using fas.person_by_username it gets these information: * last_seen * group_roles (excluding the ones specified in excludegroups, like cla_done, cla_fpca) * then it parse again the CSV file containing the mailing list stuff in order to look for subsequent messages after the introduction one * it queries datagrepper in order to grep some activities in the time period following the introduction e-mail; for QA: * total number of Bodhi activities ('user': user, 'category': 'bodhi') * total number of Wiki edits ('user': user, 'category': 'wiki') * total numebr of Bugzilla activities ('user': user, 'category': 'bugzilla') * total kernel tests activities ('user': user, 'category': 'kerneltest') * total number of mails to any other Fedora mailing list (excluding the one in question) * the result is a CSV file containing: * username: FAS username * first_intro: date of the introductory mail sent to the mailing list * last_seen: date of the last login to FAS * privacy: privacy setting status on FAS * followups: number of mails sent to the mailing list following the introductory one * count_additional_groups: total number of FAS groups the user is part of (in addition to the ones specified in the excludegroups config option) * qa_group_status: status of the sponsoring * bodhi_activity: number of activities in Bodhi * kerneltest_activity: number of performed kernel tests * bugzilla_count: number of bugzilla activities * wiki_activity: number of wiki activities * mailman_count: number of mails sent to any mailing list
python3 ./queryfedbus.py <START_DATE> <END_DATE> <mailing_list_name>
python3 ./queryfedbus2.py <START_DATE> <END_DATE> <mailing_list_name>
START_DATE
and END_DATE
should have this format YYYY-MM_DD
mailing_list_name
shoud be the part before @lists.fedoraproject.org
python3 parsecsv.py
It will parse the CSV defined as filetoparse in the configuration file.