PR#69: Fix #68 'list' object has no attribute 'username' on pgimport push - pagure-importer

pagure-importer

#69 Fix #68 'list' object has no attribute 'username' on pgimport push

Merged 7 years ago by cverna. Opened 7 years ago by clime.

clime/pagure-importer master into master

Fix #68 'list' object has no attribute 'username' on pgimport push

clime • 7 years ago

01c105c

file modified

+9 -29

		`@@ -123,44 +123,24 @@`
		`github_password,`
		`github_project_name):`
		`''' Will create a json file containing details of all the user`
		`- who have commented on any issue in the given project`
		`+ who have commented on or filed any issue in the given project`
		`'''`

		`github_obj = Github(github_username, github_password)`
		`otp_auth = get_auth_token(github_obj)`
		`github_obj = Github(otp_auth)`
		`project = github_obj.get_repo(github_project_name)`
		`- issue_comment_url = project.issue_comment_url.replace('{/number}', '')`
		`-`
		`- page = 0`
		`issue_commentors = []`
		`- while True:`
		`- page += 1`
		`- payload = {'page': page}`
		`- data_ = json.loads(requests.get(`
		`- issue_comment_url, params=payload,`
		`- auth=HTTPBasicAuth(github_username, github_password)).text)`
		`-`
		`- if not data_:`
		`- break`

		`- for data in data_:`
		`- try:`
		`- commentor = data['user']['login']`
		`- except TypeError:`
		`- click.echo('Maybe one of the issue commentors have been\`
		`- dropped because of lack of details')`
		`- continue`
		`+ for issue in project.get_issues(state='all'):`
cverna commented 7 years ago :+1: for using the python api
		`+ if not issue.user.login in issue_commentors:`
		`+ issue_commentors.append(issue.user.login)`
		`+ click.echo('commentor added: ' + issue.user.login)`

		`- present = False`
		`- for i in issue_commentors:`
		`- if i == commentor:`
		`- present = True`
		`- break`
		`-`
		`- if not present:`
		`- click.echo('commentor added: ' + commentor)`
		`- issue_commentors.append(commentor)`
		`+ for comment in project.get_issues_comments():`
		`+ if not comment.user.login in issue_commentors:`
		`+ issue_commentors.append(comment.user.login)`
		`+ click.echo('commentor added: ' + comment.user.login)`

		`with open('issue_commentors.json', 'w') as f:`
		`f.write(json.dumps(issue_commentors))`

pagure_importer/utils/importer_github.py

file modified

+2 -1

		`@@ -74,7 +74,8 @@`
		`pagure_issue_user = models.User(`
		`name=github_issue.user.login,`
		`fullname=github_issue.user.name,`
		`- emails=[github_issue.user.email])`
		`+ emails=[github_issue.user.email] if github_issue.user.email`
		`+ else [github_get_commentor_email(github_issue.user.login)])`

		`pagure_issue = models.Issue(`
		`id=None,`

clime commented 7 years ago

The problem is that current codebase does not dump issue/pr owners into data files (assembled_commentors.csv, issue_commentors.json), only 'commenters' (people who just commented after the issue or PR was created). But issue/PR owners are also needed for a valid issue creation. This the output of pgimport github before the commit:

 clime@coprbox ~/progress $ pgimport github
 Enter your Github Username: clime
 Password:
 Enter github project name like pypingou/pagure: verigak/progress
 Do you want to generate jsons for project's contributers and issue commentors? [y/N]: y
 contributor added: web-flow
 contributor added: aduriseti
 contributor added: verigak
 contributor added: msabramo
 contributor added: sindreij
 contributor added: nournia
 contributor added: verigak
 Maybe one of the contributors is dropped because                            of lack of details
 commentor added: verigak
 commentor added: prologic
 commentor added: lugtigheid
 commentor added: coyotebush
 commentor added: victorhooi
 commentor added: neithere
 commentor added: frewsxcv
 commentor added: simminni
 commentor added: msabramo
 commentor added: praiskup
 commentor added: saxtouri
 commentor added: farchy
 commentor added: aduriseti
 commentor added: albertferras
 commentor added: bmbouter
 commentor added: mat2py
 commentor added: clime

And this the output after the commit (all needed users are present now):

 clime@coprbox ~/progress $ pgimport github
 Enter your Github Username: clime
 Password:
 Enter github project name like pypingou/pagure: verigak/progress
 Do you want to generate jsons for project's contributers and issue commentors? [y/N]: y
 contributor added: web-flow
 contributor added: aduriseti
 contributor added: verigak
 contributor added: msabramo
 contributor added: sindreij
 contributor added: nournia
 contributor added: verigak
 Maybe one of the contributors is dropped because                            of lack of details
 commentor added: clime
 commentor added: mat2py
 commentor added: aduriseti
 commentor added: bmbouter
 commentor added: TobiX
 commentor added: gaopengpian
 commentor added: farchy
 commentor added: praiskup
 commentor added: jlec
 commentor added: Drekin
 commentor added: pjdelport
 commentor added: msabramo
 commentor added: sindreij
 commentor added: simminni
 commentor added: bkabrda
 commentor added: tjstum
 commentor added: nournia
 commentor added: paparomeo
 commentor added: neithere
 commentor added: prologic
 commentor added: skalkoto
 commentor added: saxtouri
 commentor added: glenbot
 commentor added: shawnsi
 commentor added: verigak
 commentor added: lugtigheid
 commentor added: coyotebush
 commentor added: victorhooi
 commentor added: frewsxcv
 commentor added: albertferras

Also note that I refrained from doing direct http API requests and used python API instead. These two ways should be equivalent.

cverna commented on line 34 of pagure_importer/utils/__init__.py 7 years ago

:+1: for using the python api

cverna commented 7 years ago

Looks good to me :)

Thanks