#801 Git lookaside_upload script returns 500 server error
Closed: Fixed 2 years ago by arrfab. Opened 2 years ago by anoopcs.

Since the recent migration lookaside_upload (and probably lookaside_upload_sig) is returning server error 500 as seen below:

$ ../../centos-git-common.git/lookaside_upload -f SOURCES/samba-4.16.2.tar.xz -n samba -b c8s-sig-storage-samba-416
[+] CentOS Lookaside upload tool -> Checking if file already uploaded
[+] CentOS Lookaside upload tool -> Initialing new upload to lookaside
[+] CentOS Lookaside upload tool -> URL : https://git.centos.org
[+] CentOS Lookaside upload tool -> Source to upload : SOURCES/samba-4.16.2.tar.xz 
[+] CentOS Lookaside upload tool -> Package name: samba
[+] CentOS Lookaside upload tool -> sha1sum: 7ee5776f92bbca2508da3d06d2d03a8e5ff5ed67
[+] CentOS Lookaside upload tool -> Remote branch: c8s-sig-storage-samba-416
[+] CentOS Lookaside upload tool ->  ====== Trying to upload =======

######################################################################################################################################################## 100.0%
curl: (22) The requested URL returned error: 500

[+] CentOS Lookaside upload tool -> [ERROR] Something didn't work to push to https://git.centos.org/sources/samba/c8s-sig-storage-samba-416/7ee5776f92bbca2508da3d06d2d03a8e5ff5ed67
[+] CentOS Lookaside upload tool -> [ERROR] Verify at the server side

Confirmed and sees kerberos related, so investigating

[Mon Jun 13 12:17:32.394762 2022] [cgid:error] [pid 260890:tid 140417622329088] [client 109.89.109.151:59732] Traceback (most recent call last):: /var/www/cgi-bin/upload.cgi
[Mon Jun 13 12:17:32.394802 2022] [cgid:error] [pid 260890:tid 140417622329088] [client 109.89.109.151:59732]   File "/usr/lib/python3.6/site-packages/requests_gssapi/gssapi_.py", line 152, in generate_request_header: /var/www/cgi-bin/upload.cgi
[Mon Jun 13 12:17:32.394817 2022] [cgid:error] [pid 260890:tid 140417622329088] [client 109.89.109.151:59732]     gss_response = self.context[host].step(): /var/www/cgi-bin/upload.cgi
[Mon Jun 13 12:17:32.394842 2022] [cgid:error] [pid 260890:tid 140417622329088] [client 109.89.109.151:59732]   File "<decorator-gen-15>", line 2, in step: /var/www/cgi-bin/upload.cgi
[Mon Jun 13 12:17:32.394867 2022] [cgid:error] [pid 260890:tid 140417622329088] [client 109.89.109.151:59732]   File "/usr/lib64/python3.6/site-packages/gssapi/_utils.py", line 167, in check_last_err: /var/www/cgi-bin/upload.cgi
[Mon Jun 13 12:17:32.394879 2022] [cgid:error] [pid 260890:tid 140417622329088] [client 109.89.109.151:59732]     return func(self, *args, **kwargs): /var/www/cgi-bin/upload.cgi
[Mon Jun 13 12:17:32.394892 2022] [cgid:error] [pid 260890:tid 140417622329088] [client 109.89.109.151:59732]   File "<decorator-gen-5>", line 2, in step: /var/www/cgi-bin/upload.cgi
[Mon Jun 13 12:17:32.394918 2022] [cgid:error] [pid 260890:tid 140417622329088] [client 109.89.109.151:59732]   File "/usr/lib64/python3.6/site-packages/gssapi/_utils.py", line 127, in catch_and_return_token: /var/www/cgi-bin/upload.cgi
[Mon Jun 13 12:17:32.394930 2022] [cgid:error] [pid 260890:tid 140417622329088] [client 109.89.109.151:59732]     return func(self, *args, **kwargs): /var/www/cgi-bin/upload.cgi
[Mon Jun 13 12:17:32.394953 2022] [cgid:error] [pid 260890:tid 140417622329088] [client 109.89.109.151:59732]   File "/usr/lib64/python3.6/site-packages/gssapi/sec_contexts.py", line 521, in step: /var/www/cgi-bin/upload.cgi
[Mon Jun 13 12:17:32.394967 2022] [cgid:error] [pid 260890:tid 140417622329088] [client 109.89.109.151:59732]     return self._initiator_step(token=token): /var/www/cgi-bin/upload.cgi
[Mon Jun 13 12:17:32.394993 2022] [cgid:error] [pid 260890:tid 140417622329088] [client 109.89.109.151:59732]   File "/usr/lib64/python3.6/site-packages/gssapi/sec_contexts.py", line 542, in _initiator_step: /var/www/cgi-bin/upload.cgi
[Mon Jun 13 12:17:32.394998 2022] [cgid:error] [pid 260890:tid 140417622329088] [client 109.89.109.151:59732]     token): /var/www/cgi-bin/upload.cgi
[Mon Jun 13 12:17:32.395022 2022] [cgid:error] [pid 260890:tid 140417622329088] [client 109.89.109.151:59732]   File "gssapi/raw/sec_contexts.pyx", line 244, in gssapi.raw.sec_contexts.init_sec_context: /var/www/cgi-bin/upload.cgi
[Mon Jun 13 12:17:32.395070 2022] [cgid:error] [pid 260890:tid 140417622329088] [client 109.89.109.151:59732] gssapi.raw.misc.GSSError: Major (851968): Unspecified GSS failure.  Minor code may provide more information, Minor (2529639053): No Kerberos credentials available (default cache: KCM:): /var/www/cgi-bin/upload.cgi

Metadata Update from @arrfab:
- Issue assigned to arrfab

2 years ago

Metadata Update from @arrfab:
- Issue tagged with: centos-common-infra, high-gain, medium-trouble

2 years ago

While I had a look at the git.dev.centos.org, where it was tested, I realized that we had to bump selinux policy after it initially got kerberos ticket (and set to permissive for initial testing).
I then bumped again after having looked at audit.log and I was able to upload to git.stg and then also to git.centos.org :

lookaside_upload -f ara-test -n ara -b c8-sig-configmanagement
[+] CentOS Lookaside upload tool -> Checking if file already uploaded
[+] CentOS Lookaside upload tool -> Initialing new upload to lookaside
[+] CentOS Lookaside upload tool -> URL : https://git.centos.org
[+] CentOS Lookaside upload tool -> Source to upload : ara-test 
[+] CentOS Lookaside upload tool -> Package name: ara
[+] CentOS Lookaside upload tool -> sha1sum: 1cbead659086803c9d757c3d971825543f1575e0
[+] CentOS Lookaside upload tool -> Remote branch: c8-sig-configmanagement
[+] CentOS Lookaside upload tool ->  ====== Trying to upload =======

############################################################################################################### 100.0%
File ara-test stored OK (checksum 1cbead659086803c9d757c3d971825543f1575e0 , size 27)
[+] CentOS Lookaside upload tool -> Validating that source was correctly uploaded ....
[+] CentOS Lookaside upload tool -> [SUCCESS] Source should be available at https://git.centos.org/sources/ara/c8-sig-configmanagement/1cbead659086803c9d757c3d971825543f1575e0

Asking for feedback that it's now working for you too

Metadata Update from @arrfab:
- Issue priority set to: Waiting on Reporter (was: Needs Review)

2 years ago

Upload script works now, thanks

thanks for feedback. closing

Metadata Update from @arrfab:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

2 years ago

Login to comment on this ticket.

Metadata
Boards 1