#2997 Create a script to check whether GeoIP updates remove used country codes
Closed: Fixed None Opened 8 years ago by toshio.

Sometimes ISO removes country codes (usually due to country renames). When this happens, there could be fas records where the old country code is still in fas. This can break things like the ambassador's verification script if there's people with the old country_codes but the current GeoIP doesn't contain the code anymore. We need a little script to import the country_codes from GeoIP and check that they don't exist in FAS.

Once the script is written, need to add running it to a SOP on doing package updates on the servers.

the GeoIP module is used roughly like this:
{{{
import GeoIP
print GeoIP.country_codes
}}}

We'll need to compare that to the country_code field for fas people probably using this function:
https://fedorahosted.org/releases/p/y/python-fedora/doc/existing.html#fedora.client.AccountSystem.people_by_key

Note that when we run it for real, it will need to use one of the system accounts that ignores the privacy flag (since country_code is protexted by privacy).

When accounts are found that have a removed country code we'll need to figure out how to merge the country code. This wikipedia page may help explain why the country code went away and where to move people to:

http://en.wikipedia.org/wiki/ISO_3166-3

Then we can change the country codes in the FAS db.


I've attached a Python script to find FAS accounts with invalid country codes. I suggest running the script with a system account and then e-mailing the users that are found. Not all country code changes can be accurately determined. Ex. if a user had lived in Sudan, how would you now determine if their proper country code is North or South Sudan?

I've added an option to the script to e-mail the users that are found.

Of course we can't auto-fix most of the country code change situations, but we need to report such situations as there is no relational database reference between GeoIP and FAS. Otherwise the ambassadors script is failing. Can we also send one e-mail with all FAS users that have no longer existing country codes and belong to the ambassadors group to Fedora Ambassadors Membership Administration (fama@...)? That helps FAMA to kick people if needed... ;-)

Changed to read fas information from config file (also use kitchen instead of yum.i18n b/c I needed kitchen's exception_to_bytes)
find_invalid_country_codes.3.py

You can test that latest version by specifying a config file like this:

find_invalid_country_codes.3.py --config fas.conf

A minimal fas.conf that will work with the script would be:

{{{
[global]
; url - Location to fas server
url = http://localhost/accounts/

; login - username to contact fas (must be in fas-system group - TODO: README for all of this)
login = admin

; password - password for login name
password = admin

}}}

Substitute a valid fas username and password in your config file.

The script has been placed in /usr/local/bin on bapp01 and schedule to run every Sunday at 3:01 A.M.

Login to comment on this ticket.

Metadata