#721 Random JSONDecodeError when building static site
Closed: Won't fix / Can't fix 6 years ago Opened 6 years ago by sijis.

When running the syncStatic script we ocassionally see the errors below. We believe it may be partially infrastructure related, however, the stacktrace given should providea little more details (like json url it tried to retrieve).

getfedora.org build failed
====================================
>>> POs pulled
python /srv/web/fedora-websites/getfedora.org/../build.d/build.py -o out -s static -b /
msgfmt -o po/af.mo po/af.po
python /srv/web/fedora-websites/getfedora.org/../build.d/construct-translations.py /srv/web/fedora-websites/getfedora.org/../build.d/translations.ini po/LINGUAS data/templates/translations.html
python /srv/web/fedora-websites/getfedora.org/../build.d/build.py -o out -i data/content -l af -p po -b /
INFO:atomic_vars:Looking for latest atomic release for 26
INFO:atomic_vars:    Found composedate: 20170821.0
INFO:fedimg_vars:Looking for AMIs for Fedora-Atomic-26-20170821.0.x86_64
Traceback (most recent call last):
  File "/srv/web/fedora-websites/getfedora.org/../build.d/build.py", line 308, in <module>
    main()
  File "/srv/web/fedora-websites/getfedora.org/../build.d/build.py", line 305, in main
    process(args)
  File "/srv/web/fedora-websites/getfedora.org/../build.d/build.py", line 118, in process
    process_dir(dirpath, filenames)
  File "/srv/web/fedora-websites/getfedora.org/../build.d/build.py", line 230, in process_dir
    collected_fedimg_vars = collect_fedimg_vars(globalvar.release)
  File "/usr/lib/python2.7/site-packages/dogpile/cache/region.py", line 1053, in decorate
    should_cache_fn)
  File "/usr/lib/python2.7/site-packages/dogpile/cache/region.py", line 657, in get_or_create
    async_creator) as value:
  File "/usr/lib/python2.7/site-packages/dogpile/core/dogpile.py", line 158, in __enter__
    return self._enter()
  File "/usr/lib/python2.7/site-packages/dogpile/core/dogpile.py", line 98, in _enter
    generated = self._enter_create(createdtime)
  File "/usr/lib/python2.7/site-packages/dogpile/core/dogpile.py", line 149, in _enter_create
    created = self.creator()
  File "/usr/lib/python2.7/site-packages/dogpile/cache/region.py", line 625, in gen_value
    created_value = creator()
  File "/usr/lib/python2.7/site-packages/dogpile/cache/region.py", line 1049, in creator
    return fn(*arg, **kw)
  File "build/fedimg_vars.py", line 107, in collect
    for message in messages:
  File "build/fedimg_vars.py", line 55, in get_messages
    data = get_page(1, 'unknown', target)
  File "build/fedimg_vars.py", line 48, in get_page
    return response.json()
  File "/usr/lib/python2.7/site-packages/requests/models.py", line 802, in json
    return json.loads(self.text, **kwargs)
  File "/usr/lib64/python2.7/site-packages/simplejson/__init__.py", line 516, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python2.7/site-packages/simplejson/decoder.py", line 374, in decode
    obj, end = self.raw_decode(s)
  File "/usr/lib64/python2.7/site-packages/simplejson/decoder.py", line 404, in raw_decode
    return self.scan_once(s, idx=_w(s, idx).end())
simplejson.scanner.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
make: *** [af] Error 1

It seems to be datagrepper related, because when datagrepper is slow then we get this error and it persists often for more than 1-2 days.
Optimizing the db normally helps, but it seems we cannot do anything more than telling the script to retry to listen to datagrepper if the connections times out the first time.

I am closing this ticket, because we probably found out the reason with Infra. A fix is coming soon after we are out of freeze. Let's keep an eye on it, anyway, but the issue is not on our side.

Metadata Update from @robyduck:
- Issue close_status updated to: Won't fix / Can't fix
- Issue status updated to: Closed (was: Open)

6 years ago

Login to comment on this ticket.

Metadata