#3617 clean up broken planet feeds
Closed: Fixed None Opened 7 years ago by skvidal.

These are errors outputted when the planet runs. Most of them are broken .planet files and/or broken feeds/servers.

Hunt them down and kill them.

Problem parsing /home/fedora/magjogui/.planet - File contains no section headers.
file: /home/fedora/magjogui/.planet, line: 1
'http://magjogui.blogspot.com/search/label/%23fedora\n'
Problem parsing /home/fedora/xigatec/.planet - File contains no section headers.
file: /home/fedora/xigatec/.planet, line: 1
'[http://fedora-uy.org/category/Planet/feed/\n'
Problem parsing /home/fedora/stevej/.planet - File contains no section headers.
file: /home/fedora/stevej/.planet, line: 1
'name = Steve Jenkins\n'
/usr/lib/python2.6/site-packages/planet/reconstitute.py:16: DeprecationWarning: the md5 module is deprecated; use hashlib instead
import re, time, md5, sgmllib
/usr/lib/python2.6/site-packages/planet/vendor/httplib2/init.py:41: DeprecationWarning: the sha module is deprecated; use the hashlib module instead
import sha
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /blog/feed/ via 4
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /category/planetfedora/feed via 7
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /wp/mips/?feed=rss2 via 15
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /atom.xml via 2
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /feed/ via 14
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /wp/fifthpillar/feed/ via 4
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /one-soap-box/fedora/index.atom via 8
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /blog/index.rss via 19
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /category/fedora/feed via 4
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /wp/sgordon/feed/ via 0
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /?tag=fedora via 16
ERROR:planet.runner:HTTP Error: [Errno 111] Connection refused in thread-16
ERROR:planet.runner:Error processing http://http://blog.mavjs.org//feeds/posts/default/-/Fedora/?alt=rss
ERROR:planet.runner:InvalidURL: nonnumeric port: ''
ERROR:planet.runner: File "/usr/lib/python2.6/site-packages/planet/spider.py", line 312, in httpThread
(resp, content) = h.request(idna, 'GET', headers=headers)
ERROR:planet.runner: File "/usr/lib/python2.6/site-packages/planet/vendor/httplib2/init.py", line 780, in request
conn = self.connections[scheme+":"+authority] = connection_type(authority)
ERROR:planet.runner: File "/usr/lib64/python2.6/httplib.py", line 661, in init
self._set_hostport(host, port)
ERROR:planet.runner: File "/usr/lib64/python2.6/httplib.py", line 686, in _set_hostport
raise InvalidURL("nonnumeric port: '%s'" % host[i+1:])
ERROR:planet.runner:Error 500 while updating feed http://www.dahyabhai.net/nalin/blog/blosxom.cgi/development/index.atom
ERROR:planet.runner:Error 500 while updating feed http//people.byte-code.com/fcrippa/feed/
ERROR:planet.runner:Error 500 while updating feed http://www.lbotti.net/blog/feed/
ERROR:planet.runner:HTTP Error: [Errno -2] Name or service not known in thread-0
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /wp/elapi/feed via 0
ERROR:planet.runner:Error 404 while updating feed http://www.silassewell.com/blog/tag/fedora/rss2.xml
ERROR:planet.runner:Error 404 while updating feed http://sayamindu.randomink.org/ramblings/feed
ERROR:planet.runner:Error 404 while updating feed http://screwyouenterpriseedition.blogspot.com/feeds/posts/default/-/fedora
ERROR:planet.runner:Error 500 while updating feed http://noopenblockers.com/category/planetfedora/feed
ERROR:planet.runner:Error 403 while updating feed http://affix.me/category/fedora/feed/
ERROR:planet.runner:Error 500 while updating feed http://blogs.fedoraproject.org/wp/mips/?feed=rss2
ERROR:planet.runner:Error 500 while updating feed http://www.matahariproject.org/atom.xml
ERROR:planet.runner:Error 500 while updating feed http://www.fedora.org.gt/feed/
ERROR:planet.runner:HTTP Error: [Errno 113] No route to host in thread-2
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /wp/inode0/feed/ via 1
ERROR:planet.runner:Error 404 while updating feed http://michaeldehaan.net/category/linux/feed/
ERROR:planet.runner:Error 404 while updating feed http://www.binarypenguins.com/journal/rss.xml
ERROR:planet.runner:Error 404 while updating feed http://blog.gbraad.nl/feeds/posts/default
ERROR:planet.runner:Error 500 while updating feed http://blogs.fedoraproject.org/wp/fifthpillar/feed/
ERROR:planet.runner:Error 404 while updating feed http://proyectofedora.org/mexico/feed/
ERROR:planet.runner:Error 404 while updating feed http://spindazzle.org/greenblog/index.php?/feeds/index.rss2
ERROR:planet.runner:Error 500 while updating feed http://blipper.dev.benboeckel.net/one-soap-box/fedora/index.atom
ERROR:planet.runner:Error 500 while updating feed http://christopher.aillon.org/blog/index.rss
ERROR:planet.runner:Error 404 while updating feed http://ankurs.com/category/fedora/feed/
ERROR:planet.runner:Error 500 while updating feed [http:/linuxnewbies.wordpress.com/tag/fedora/feed/
ERROR:planet.runner:Error 404 while updating feed http://www.marcanoonline.com/plog/rss/1/11
ERROR:planet.runner:Error 404 while updating feed http://www.vaidikkapoor.info/tag/fedora/feed/
ERROR:planet.runner:Error 404 while updating feed http://paulmellors.blogspot.com/feeds/posts/default/-/Fedora/?alt=rss
ERROR:planet.runner:Error 404 while updating feed http://proyectofedora.org/argentina/?feed=rss2
ERROR:planet.runner:Error 500 while updating feed http://blog.arkezis.com/tag/planet-fedora/feed
ERROR:planet.runner:Error 404 while updating feed http://www.stevemilner.org/blog/feeds/category/Fedora/
ERROR:planet.runner:Error 404 while updating feed http://blog.oktud.com/tag/fedora/feed/
ERROR:planet.runner:Error 500 while updating feed http://blog.sysdroid.com/category/fedora/feed
ERROR:planet.runner:Error 500 while updating feed http://blogs.fedoraproject.org/wp/sgordon/feed/
ERROR:planet.runner:Error 500 while updating feed http://anish-patil.blogspot.in//feeds/posts/default
ERROR:planet.runner:Error 500 while updating feed http://blog.dafrito.com/?tag=fedora
ERROR:planet.runner:Error 500 while updating feed http://blog.newnewyork.nl/feed/
ERROR:planet.runner:Error 500 while updating feed http://http://blog.mavjs.org//feeds/posts/default/-/Fedora/?alt=rss
ERROR:planet.runner:Error 401 while updating feed http://fedora-redhat.blogspot.com/feeds/posts/default?alt=rss
ERROR:planet.runner:Error 404 while updating feed http://luke.asia/tag/fedora/feed/
ERROR:planet.runner:Error 502 while updating feed http://hekafs.org/index.php/feed/
ERROR:planet.runner:Error 404 while updating feed http://bacharakis.com/feed/
ERROR:planet.runner:Error 500 while updating feed http://blogs.fedoraproject.org/wp/escuela/feed/
ERROR:planet.runner:Error 500 while updating feed http://blogs.fedoraproject.org/wp/elapi/feed
ERROR:planet.runner:Error 404 while updating feed http://kishangoyal.co.cc/tagged/planet/rss
ERROR:planet.runner:Error 403 while updating feed http://www.atmayogi.com/category/planet-fedora/feed/
ERROR:planet.runner:Error 500 while updating feed http://www.translate.org.za/blogs/dwayne/en/taxonomy/term/5/0/feed
ERROR:planet.runner:Error 500 while updating feed http://blogs.fedoraproject.org/wp/inode0/feed/
ERROR:planet.runner:Error 404 while updating feed http://zach.oglesby.co/blog/categories/fedora/atom.xml
ERROR:planet.runner:Error 404 while updating feed http://feeds.feedburner.com/nixknack

= bug description =

= bug analysis =

= fix recommendation =


I went though the planet ini file and was able to find all the broken links and broken feeds. I created a new ini file but I imagine the .planet files in the users home directories need to be fixed. I have a list of all their FAS user names except for Fedora Tunisia so they can be contacted fairly easily I imagine. There we 56 links that were broken, however many of them were simple needing a extra space or change in the way the url was written.

for example:

{{{

http://anish-patil.blogspot.in//feeds/posts/default = http://anish-patil.blogspot.com/feeds/posts/default
http://bacharakis.com/feed/ = http://www.bacharakis.com/?feed=rss2
}}}

planet planet configuration with broken links and invalid feeds removed.
planet.diff

ok, for those that have simple space changes or whatever, could we just whip up a script to fix them?

Then we could mail the remaining folks?

I have done some attempts to fix the addresses, as many of the addresses are still valid just url changes caused the links to break. Manual intervention is required for many of them. The only issue I could see from this is that they may not want their new site to be on the planet. Emailing them should be easy enough though since we have the usernames for the most part. I'll try to make a script to get the users and or emails.

How would the emails be sent out?

If you can generate a list of users, we can script sending out from admin@fedoraproject.org to them asking them to look at and fix their planet feed. ;)

Here's a new list of the current output:

{{{
Problem parsing /home/fedora/magjogui/.planet - File contains no section headers.
file: /home/fedora/magjogui/.planet, line: 1
'http://magjogui.blogspot.com/search/label/%23fedora\n'
Problem parsing /home/fedora/xigatec/.planet - File contains no section headers.
file: /home/fedora/xigatec/.planet, line: 1
'[http://fedora-uy.org/category/Planet/feed/\n'
Problem parsing /home/fedora/stevej/.planet - File contains no section headers.
file: /home/fedora/stevej/.planet, line: 1
'name = Steve Jenkins\n'
/usr/lib/python2.6/site-packages/planet/reconstitute.py:16: DeprecationWarning: the md5 module is deprecated; use hashlib instead
import re, time, md5, sgmllib
/usr/lib/python2.6/site-packages/planet/vendor/httplib2/init.py:41: DeprecationWarning: the sha module is deprecated; use the hashlib module instead
import sha
ERROR:planet.runner:Error processing http://http://littledaemons.tumblr.com/rss
ERROR:planet.runner:InvalidURL: nonnumeric port: ''
ERROR:planet.runner: File "/usr/lib/python2.6/site-packages/planet/spider.py", line 328, in httpThread
(resp, content) = h.request(idna, 'GET', headers=headers)
ERROR:planet.runner: File "/usr/lib/python2.6/site-packages/planet/vendor/httplib2/init.py", line 780, in request
conn = self.connections[scheme+":"+authority] = connection_type(authority)
ERROR:planet.runner: File "/usr/lib64/python2.6/httplib.py", line 661, in init
self._set_hostport(host, port)
ERROR:planet.runner: File "/usr/lib64/python2.6/httplib.py", line 686, in _set_hostport
raise InvalidURL("nonnumeric port: '%s'" % host[i+1:])
ERROR:planet.runner:HTTP Error: [Errno 111] Connection refused in thread-5
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /blog/feed/ via 0
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /wp/mips/?feed=rss2 via 3
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /feed/ via 10
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /atom/fedora-planet via 14
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /wp/fifthpillar/feed/ via 11
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /one-soap-box/fedora/index.atom via 2
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /till/tag/fedora/feed/ via 19
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /?tag=fedora via 11
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /blog/index.rss via 2
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /category/fedora/feed via 4
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /wp/sgordon/feed/ via 4
ERROR:planet.runner:Error processing http://http://blog.mavjs.org//feeds/posts/default/-/Fedora/?alt=rss
ERROR:planet.runner:InvalidURL: nonnumeric port: ''
ERROR:planet.runner: File "/usr/lib/python2.6/site-packages/planet/spider.py", line 328, in httpThread
(resp, content) = h.request(idna, 'GET', headers=headers)
ERROR:planet.runner: File "/usr/lib/python2.6/site-packages/planet/vendor/httplib2/init.py", line 780, in request
conn = self.connections[scheme+":"+authority] = connection_type(authority)
ERROR:planet.runner: File "/usr/lib64/python2.6/httplib.py", line 661, in init
self._set_hostport(host, port)
ERROR:planet.runner: File "/usr/lib64/python2.6/httplib.py", line 686, in _set_hostport
raise InvalidURL("nonnumeric port: '%s'" % host[i+1:])
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /feed/ via 16
ERROR:planet.runner:HTTP Error: [Errno 113] No route to host in thread-6
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /weblog/category/foss/feed/ via 4
ERROR:planet.runner:HttpLib2Error: Only absolute URIs are allowed. uri = http:ppppp/////////fedoraboost.blogspot.com/feeds/posts/default?alt=rss via 2
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /wp/escuela/feed/ via 16
ERROR:planet.runner:HTTP Error: [Errno -2] Name or service not known in thread-16
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /wp/inode0/feed/ via 0
ERROR:planet.runner:Error 404 while updating feed http://abr4xas.org/tag/fedora/feed/
ERROR:planet.runner:Error 500 while updating feed http://http://littledaemons.tumblr.com/rss
ERROR:planet.runner:Error 500 while updating feed http//people.byte-code.com/fcrippa/feed/
ERROR:planet.runner:Error 500 while updating feed http://fetzig.org/category/linux/fedora-planet/feed
ERROR:planet.runner:Error 406 while updating feed http://ankursinha.in/blog/category/fedora/feed/
ERROR:planet.runner:Error 500 while updating feed http://www.lbotti.net/blog/feed/
ERROR:planet.runner:Error 404 while updating feed http://www.silassewell.com/blog/tag/fedora/rss2.xml
ERROR:planet.runner:Error 404 while updating feed http://sayamindu.randomink.org/ramblings/feed
ERROR:planet.runner:Error 500 while updating feed http://blogs.fedoraproject.org/wp/mips/?feed=rss2
ERROR:planet.runner:Error 500 while updating feed http://www.fedora.org.gt/feed/
ERROR:planet.runner:Error 500 while updating feed http://blog.sergiodj.net/atom/fedora-planet
ERROR:planet.runner:Error 404 while updating feed http://michaeldehaan.net/category/linux/feed/
ERROR:planet.runner:Error 404 while updating feed http://www.binarypenguins.com/journal/rss.xml
ERROR:planet.runner:Error 404 while updating feed http://blog.gbraad.nl/feeds/posts/default
ERROR:planet.runner:HTTP Error: [Errno 113] No route to host in thread-8
ERROR:planet.runner:Error 500 while updating feed http://blogs.fedoraproject.org/wp/fifthpillar/feed/
ERROR:planet.runner:Error 404 while updating feed http://proyectofedora.org/mexico/feed/
ERROR:planet.runner:Error 403 while updating feed http://www.marcusmoeller.ch/tag/fedora/feed
ERROR:planet.runner:Error 404 while updating feed http://spindazzle.org/greenblog/index.php?/feeds/index.rss2
ERROR:planet.runner:Error 500 while updating feed http://blipper.dev.benboeckel.net/one-soap-box/fedora/index.atom
ERROR:planet.runner:Error 404 while updating feed http://asrob.eu/taxonomy/term/3/feed
ERROR:planet.runner:Error 404 while updating feed http://blog.mustu.info/category/gnu-linux-foss/redhat-fedora/feed/
ERROR:planet.runner:Error 404 while updating feed http://blog.cleitonlima.com.br/tagged/fedora/rss
ERROR:planet.runner:Error 404 while updating feed http://www.marcanoonline.com/plog/rss/1/11
ERROR:planet.runner:Error 404 while updating feed http://ankurs.com/category/fedora/feed/
ERROR:planet.runner:Error 403 while updating feed http://www.gadgetwisdom.com/category/featured/feed/
ERROR:planet.runner:Error 500 while updating feed http://blogs.23.nu/till/tag/fedora/feed
ERROR:planet.runner:Error 404 while updating feed http://www.vaidikkapoor.info/tag/fedora/feed/
ERROR:planet.runner:Error 404 while updating feed http://sadin.ifndev.com/?q=rss.xml
ERROR:planet.runner:Error 500 while updating feed [http:/linuxnewbies.wordpress.com/tag/fedora/feed/
ERROR:planet.runner:Error 500 while updating feed http://blog.dafrito.com/?tag=fedora
ERROR:planet.runner:Error 404 while updating feed http://proyectofedora.org/argentina/?feed=rss2
ERROR:planet.runner:Error 404 while updating feed http://www.stevemilner.org/blog/feeds/category/Fedora/
ERROR:planet.runner:Error 404 while updating feed http://www.azneita.org/category/fedora/feed/
ERROR:planet.runner:Error 404 while updating feed http://blog.oktud.com/tag/fedora/feed/
ERROR:planet.runner:Error 500 while updating feed http://christopher.aillon.org/blog/index.rss
ERROR:planet.runner:Error 404 while updating feed http://www.livejournal.com/users/skeggsb/data/rss/
ERROR:planet.runner:Error 500 while updating feed http://blog.sysdroid.com/category/fedora/feed
ERROR:planet.runner:Error 500 while updating feed http://blogs.fedoraproject.org/wp/sgordon/feed/
ERROR:planet.runner:Error 500 while updating feed http://http://blog.mavjs.org//feeds/posts/default/-/Fedora/?alt=rss
ERROR:planet.runner:Error 500 while updating feed http://blog.newnewyork.nl/feed/
ERROR:planet.runner:Error 401 while updating feed http://fedora-redhat.blogspot.com/feeds/posts/default?alt=rss
ERROR:planet.runner:Error 500 while updating feed http://www.stahnkage.com/serendipity/index.php?/feeds/categories/7-Fedora.rss
ERROR:planet.runner:Error 510 while updating feed http://fabian-affolter.ch/blog/category/fedora/feed/
ERROR:planet.runner:Error 403 while updating feed http://www.braincache.de/wp/category/fedora-design/feed
ERROR:planet.runner:Error 404 while updating feed http://thempixels.tumblr.com/tagged/fedora/rss
ERROR:planet.runner:Error 500 while updating feed http://dimitris.glezos.com/weblog/category/foss/feed/
ERROR:planet.runner:Error 404 while updating feed http://luke.asia/tag/fedora/feed/
ERROR:planet.runner:Error 500 while updating feed http://blog.arkezis.com/tag/planet-fedora/feed
ERROR:planet.runner:Error 404 while updating feed http://www.projetofedora.org/rss.xml
ERROR:planet.runner:Error 500 while updating feed http://anish-patil.blogspot.in//feeds/posts/default
ERROR:planet.runner:Error 404 while updating feed http://bacharakis.com/feed/
ERROR:planet.runner:Error 500 while updating feed http:ppppp/////////fedoraboost.blogspot.com/feeds/posts/default?alt=rss
ERROR:planet.runner:Error 500 while updating feed http://blogs.fedoraproject.org/wp/escuela/feed/
ERROR:planet.runner:Error 500 while updating feed http://blogs.fedoraproject.org/wp/elapi/feed
ERROR:planet.runner:Error 404 while updating feed http://www.adamantio.net/wordpress/tag/fedora/
ERROR:planet.runner:Error 404 while updating feed http://kishangoyal.co.cc/tagged/planet/rss
ERROR:planet.runner:Error 500 while updating feed http://www.j5live.com/feed
ERROR:planet.runner:Error 500 while updating feed http://blogs.fedoraproject.org/wp/inode0/feed/
ERROR:planet.runner:Error 404 while updating feed http://thomasj-fedora.livejournal.com/data/rss
ERROR:planet.runner:Error 404 while updating feed https://geleim.com/category/fedora-planet/feed/
ERROR:planet.runner:Error 404 while updating feed http://zach.oglesby.co/blog/categories/fedora/atom.xml
ERROR:planet.runner:Error 404 while updating feed http://nicubunu.blogspot.com/feeds/posts/default?alt=rss
ERROR:planet.runner:Error 404 while updating feed http://blog.warpgeek.co.uk/feeds/posts/default/-/Fedora/?alt=rss
ERROR:planet.runner:Error 404 while updating feed http://loviniltra.com/category/fedoraplanet/
ERROR:planet.runner:Error 404 while updating feed http://feeds.feedburner.com/nixknack
ERROR:planet.runner:Error 500 while updating feed http://www.translate.org.za/blogs/dwayne/en/taxonomy/term/5/0/feed
ERROR:planet.runner:Error 404 while updating feed http://vinzv.de/category/fedora/feed/
}}}

planet file needs to be channged under this user: /home/fedora/magjogui

location:/home/fedora/magjogui

http://magjogui.blogspot.com/search/label/%23fedora
name = magjogui
face = https://fedoraproject.org/w/uploads/3/3d/Little-Me.jpg

this is what the file needs to be changed to:

[http://magjogui.blogspot.com/search/label/%23fedora]
name = magjogui
face = https://fedoraproject.org/w/uploads/3/3d/Little-Me.jpg


planet file needs to be channged under this user: /home/fedora/xigatec

location/home/fedora/xigatec

[http://fedora-uy.org/category/Planet/feed/
name = Fedora Uruguay
face = http://fedora-uy.org/wp-content/uploads/2012/10/furuguay.png

this needs to be changed to :

[http://fedora-uy.org/category/Planet/feed/]
name = Fedora Uruguay
face = http://fedora-uy.org/wp-content/uploads/2012/10/furuguay.png


this should fix the first two errors. simple changes made.

The 406 errors have been fixed. The default httplib2 user-agent was causing this. Threebean changed it and the planet feeds on my blog just fine now. Thanks :)

Moving all currently open easyfix tickets to the HANDYWAVY-FUTURE milestone.

I'm clearing the assigned status on all easyfix tickets.

If you are an apprentice actively working on this ticket, feel free to reassign to yourself. Otherwise let a new apprentice have a look.

Hi,

I would like to work on this ticket. There is something else that can be made here?

Sure. Here's the current output:

{{{
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /taxonomy/term/5/feed via 4
ERROR:planet.runner:HTTP Error: [Errno 111] Connection refused in thread-17
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /feed/ via 5
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /journal/rss.xml via 4
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /category/gnu-linux-foss/redhat-fedora/feed/ via 15
ERROR:planet.runner:HTTP Error: [Errno 1] _ssl.c:492: error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure in thread-9
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /atom.xml via 14
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /blog/index.rss via 15
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /aggregator/fedora-planet/feed/ via 8
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /?q=rss.xml via 5
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /?tag=fedora via 1
ERROR:planet.runner:HTTP Error: [Errno 111] Connection refused in thread-3
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /index.php/feed/ via 14
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /blog/4267/feed/ via 11
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /rss2 via 2
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /blog/tag/fedora/rss2 via 19
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /feed/ via 15
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /weblog/category/foss/feed/ via 13
ERROR:planet.runner:HTTP Error: [Errno 111] Connection refused in thread-19
ERROR:planet.runner:HTTP Error: [Errno 1] _ssl.c:492: error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol in thread-12
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /Blog_Feed via 11
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /?cat=3&feed=rss2 via 2
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /?cat=5&feed=rss2 via 7
ERROR:planet.runner:Error 403 while updating feed http://chitlesh.ch/wordpress/chitlesh/category/free-electronic-lab/feed/
ERROR:planet.runner:HttpLib2Error: Unable to find the server at /tags/fedora/feed/ via 14
ERROR:planet.runner:Error 404 while updating feed http://www.fedoreame.com/index.php/category/fedora/feed/
ERROR:planet.runner:Error 500 while updating feed http://www.dahyabhai.net/nalin/blog/blosxom.cgi/development/index.atom
ERROR:planet.runner:Error 500 while updating feed http//people.byte-code.com/fcrippa/feed/
ERROR:planet.runner:Error 404 while updating feed http://abr4xas.org/tag/fedora/feed/
ERROR:planet.runner:Error 500 while updating feed http://asrob.eu/taxonomy/term/5/feed
ERROR:planet.runner:Error 500 while updating feed http://dcr226.co.uk/blog/?cat=4&feed=rss2
ERROR:planet.runner:Error 500 while updating feed http://fetzig.org/category/linux/fedora-planet/feed
ERROR:planet.runner:Error 404 while updating feed http://wm161.net/browse/linux/fedora-linux/feed/
ERROR:planet.runner:Error 503 while updating feed http://misnotasdelinux.wordpress.com/tag/planet/feed/
ERROR:planet.runner:Error 500 while updating feed http://www.fedora.org.gt/feed/
ERROR:planet.runner:Error 500 while updating feed http://blog.loviniltra.com/category/fedoraplanet/
ERROR:planet.runner:Error 404 while updating feed http://michaeldehaan.net/category/linux/feed/
ERROR:planet.runner:Error 500 while updating feed http://dennis-blog.appelon.net/?cat=43&feed=rss2
ERROR:planet.runner:Error 500 while updating feed http://www.binarypenguins.com/journal/rss.xml
ERROR:planet.runner:Error 404 while updating feed http://spindazzle.org/greenblog/index.php?/feeds/index.rss2
ERROR:planet.runner:Error 500 while updating feed http://blog.mustu.info/category/gnu-linux-foss/redhat-fedora/feed/
ERROR:planet.runner:Error 500 while updating feed https://www.brianlane.com/feeds/Fedora.rss.xml
ERROR:planet.runner:Error 404 while updating feed http://uditsharma.in/category/blog/fedora/feed
ERROR:planet.runner:Error 404 while updating feed http://ankurs.com/category/fedora/feed/
ERROR:planet.runner:Error 500 while updating feed http://www.matahariproject.org/atom.xml
ERROR:planet.runner:Error 500 while updating feed http://christopher.aillon.org/blog/index.rss
ERROR:planet.runner:Error 500 while updating feed http://blog.ianweller.org/aggregator/fedora-planet/feed/
ERROR:planet.runner:Error 500 while updating feed http://sadin.ifndev.com/?q=rss.xml
ERROR:planet.runner:Error 500 while updating feed http://blog.dafrito.com/?tag=fedora
ERROR:planet.runner:Error 404 while updating feed http://blog.grdryn.me/category/technology/linux/fedora/feed/
ERROR:planet.runner:Error 500 while updating feed http://blog.arkezis.com/tag/planet-fedora/feed
ERROR:planet.runner:Error 404 while updating feed http://www.vaidikkapoor.info/tag/fedora/feed/
ERROR:planet.runner:Error 500 while updating feed https://web.elastic.org/~fche/blog3/index.php?feed=rss&c=tech
ERROR:planet.runner:Error 404 while updating feed http://www.stevemilner.org/blog/feeds/category/Fedora/
ERROR:planet.runner:Error 503 while updating feed http://blogs.fedoraunity.org/bobjensen/atom.xml
ERROR:planet.runner:Error 503 while updating feed http://blogs.fedoraunity.org/damaestro/rss.xml
ERROR:planet.runner:Error 404 while updating feed http://blog.oktud.com/tag/fedora/feed/
ERROR:planet.runner:Error 404 while updating feed http://oddshocks.com/rss
ERROR:planet.runner:Error 500 while updating feed http://hekafs.org/index.php/feed/
ERROR:planet.runner:Error 500 while updating feed http://www.kdedevelopers.org/blog/4267/feed/
ERROR:planet.runner:Error 500 while updating feed http://www.llaumgui.com/feed/rss2
ERROR:planet.runner:Error 500 while updating feed http://mahut.sk/blog/tag/fedora/rss2
ERROR:planet.runner:Error 404 while updating feed http://qalthos.github.io/blog/feeds/tag/fedora.atom.xml
ERROR:planet.runner:Error 500 while updating feed http://blog.newnewyork.nl/feed/
ERROR:planet.runner:Error 401 while updating feed http://fedora-redhat.blogspot.com/feeds/posts/default?alt=rss
ERROR:planet.runner:Error 404 while updating feed http://stick.gk2.sk/blog/tag/fedora/feed/
ERROR:planet.runner:Error 404 while updating feed http://thempixels.tumblr.com/tagged/fedora/rss
ERROR:planet.runner:Error 500 while updating feed http://dimitris.glezos.com/weblog/category/foss/feed/
ERROR:planet.runner:Error 500 while updating feed http://luke.asia/tag/fedora/feed/
ERROR:planet.runner:Error 404 while updating feed http://cicku.me/feed
ERROR:planet.runner:Error 404 while updating feed http://pycured.com/tag/fedora/feed/
ERROR:planet.runner:Error 404 while updating feed http://www.projetofedora.org/rss.xml
ERROR:planet.runner:Error 404 while updating feed http://bacharakis.com/feed/
ERROR:planet.runner:Error 404 while updating feed http://www.adamantio.net/wordpress/tag/fedora/
ERROR:planet.runner:Error 500 while updating feed http://www.translate.org.za/blogs/dwayne/en/taxonomy/term/5/0/feed
ERROR:planet.runner:Error 503 while updating feed http://fedora.my/blog/aggregator/RSS
ERROR:planet.runner:Error 500 while updating feed http://kishangoyal.co.cc/tagged/planet/rss
ERROR:planet.runner:Error 500 while updating feed http://www.j5live.com/feed
ERROR:planet.runner:Error 404 while updating feed http://blog.gbraad.nl/feeds/posts/default
ERROR:planet.runner:Error 404 while updating feed http://blog.fedora-fr.org/pingoomax/feed/rss2
ERROR:planet.runner:Error 404 while updating feed http://en.eladalfassa.com/?category_name=fedora&feed=rss2
ERROR:planet.runner:Error 500 while updating feed http://tech.cleitonlima.com.br/Blog_Feed
ERROR:planet.runner:Error 500 while updating feed https://bmartins.eu/?cat=3&feed=rss2
ERROR:planet.runner:Error 500 while updating feed http://fedora.uy/?cat=5&feed=rss2
ERROR:planet.runner:Error 500 while updating feed http://fabriciojr.com/tags/fedora/feed/
ERROR:planet.runner:Error 404 while updating feed http://blog.jebpages.com/archives/tag/fedora/feed/
ERROR:planet.runner:Error 404 while updating feed http://benl.co.uk/category/topic/linux/feed
ERROR:planet.runner:Error 404 while updating feed http://feeds.feedburner.com/nixknack
ERROR:planet.runner:Error 404 while updating feed http://pascasio.org/category/fedora/feed/rss/
ERROR:planet.runner:Error 404 while updating feed http://vinzv.de/category/fedora/feed/
}}}

We need to track down what .planet feeds are causing these and notify the owners to fix, fix ourselves (if it's easy to do so), or disable them.

I made a script to find the urls of the users, and check if it works.
I already have a list of the urls that failed on the test with the usernames.

Should I send a message to each user describing the error and asking to fix?

List: http://dbruno.fedorapeople.org/brokenfeeds.txt

Script: https://github.com/danielbruno/labs/blob/master/fedora/checkplanet.py

Awesome work. ;)

Yes, a email to each person would be great asking them to fix things. Then we can wait a few weeks and just disable those if they didn't fix them. If you want to send that feel free, or if you prefer we can send it from a more official looking email address?

Also, it would be great to get the checkplanet.py setup on cron on people03 and send a weekly report, then we can nag those people closer to when they add entries (assuming we fix all the current ones).

People03 is in puppet. You can login to lockbox01 and git clone /git/puppet to look. There should be lots of other examples of scripts running there, we just need to add this one. :)

Thanks again for working on this.

Hello,

I made some adjusts on the script that check URLs, to generate a report on my public_html/reports with this format "brokenplanetreportdd_mm_aaaa.txt", and a json file with this report content to use with planetnotification.py that read this json and send a notification to the users with the URL and the problem.

The script planetnotification.py should be run manually because I used the fas authentication to get the information of the users like Real Name and E-mail address. In the script have a default message, but it is only for test purpose, I would like to know if exist a default message or someone to review the message because I'm not a native English speaker and there is a high risk to have a grammar errors.

Scripts: https://github.com/danielbruno/labs/tree/master/fedora

I accessed the server lockbox01 and cloned the puppet repo, when we have a default message, I'll configure the check on the cron. Do you have any suggestions of the periodicity to run the script?

Thank you for your support :)

Overall looks great. ;)

I can see some wording changes, and if we want to run it from inside infrastructure we could just send it as from 'admin@fedoraproject.org'. I can make those changes after it's in puppet, but before we run it?

I'd say we should run it once a week. More often than that will be anoying to people I suspect.

Hello,

First, I'm sorry on the delay to update the ticket, I had some hard days on my job that took all my useful time.

Should I add the scripts on puppet repo, and you'll make the changes and add on the cron?

Thank you

Sure, but could you attach the patch here? I don't think you have commit to the puppet repo, but I can apply your patch(es).

Script to check broken planets configs.
checkplanet.py

Script to notify the users that have broken planet configs.
planetnotification.py

Hello,

I attached the scripts.

The checkplanet.py to run on the cron. The planetnotification.py, I can run manually because need to input the password to get the users informations to send the notifications.

ok. I have checked these into puppet with a few changes. (path output change and wording on the notice).

I've not added the check to cron yet, I wanted to run it a few times manually and make sure all is well before we run it weekly.

I've run it once now, can you look at the output, and if all looks ok, I (or you) can run the notification?

I found 2 bugs on the script that check the planets, on the error handling that resulted in a wrong json output. Another bug was related on the real name, because not everyone registers they real name on FAS, in these cases, I use the username instead the real name.

The patchs are attached.

I ran the scripts patched on fedorapeople, first on debug mode just printing the records and the message, and after sending the notification. In both cases worked well.

I tried to push the changes on puppet repo, but I do not have persmision.

changes pushed, thanks.

So, notification did go out?

Yes, the notifications were sent.

Excellent.

Lets see how many get fixed in the next week or so, we can then set a time to disable outstanding ones.

After all those are disabled, we can set the check in cron to tell us anytime a new broken one appears so we can notify on it. ;)

Thank you.

While we wait for the next reports, I'll make a script to analyze who do not fix the config, and then we have this info more easily to disable these users, ok?

Sure. Sounds great. ;)

I ran again the script, but with my user, and I noticed a good difference between the results. So, I did some comparisons and I saw that I didn't checked the users owned by root, when I ran the script with my user.

I did a search for these users owned by root, and noticed that almost every user is inactive on FAS, not all.

Do you think that is relevant to list the inactive FAS users and remove their confs?

Yeah, normally inactive users are moved to a different directory... were all these under a /home/fedora.sav or something? I think the planet script doesn't look anywhere except under /home/fedora/ was your checker script looking further?

The script list all the homedirs on /home/fedora/ and look for .planet config.

There is 5184 homedirs and 1871 are owned but root. I checked some accounts of these 1871, and most are inactive accounts on FAS.

And when I run the script with user, I was unable to check these users owned by root, and becuase of that I would like to ask to you to run again the script with root privileges to check these users, and generate a second list to compare with the first one, to know who fixed or not. :)

ok, ran the script, output in:

/srv/planet/output/reports

on people03.

Hello,

First of all, I want to justify the delay to update this ticket, I was moving to another city, as I'd scheduled on Fedora Calendar.

So, I made a script a script to analyze the previous reports and I noticed that we have:

100 still broken
8 fixed
18 new broken confs

The list of the accounts is available at: http://dbruno.fedorapeople.org/reportbrokenplanet.txt

What you recommend to do?

Best Regards

How about we do another round of mailing people with broken feeds, then in a week or so remove them.

I guess another category would be inactive users with /home/fedora/user/.planet files. These we should just remove now.

I'm a new fi-apprentice, and this ticket is right up my alley!

Is @dbruno's repo on github (https://github.com/danielbruno/labs/tree/master/fedora) up to date with the latest bits?

Thanks :)

So, the status here is that we have pushed dbruno's excellent scripts in place on people03, but we haven't done the mailing people and removing broken ones or set it up to run from cron.

people is sadly still in puppet (although we plan to migrate it soon to ansible).

I just ran the script and the output is in:
/srv/planet/output/reports
on people03.fedoraproject.org

Next step would be to look at that output, mail people about broken feeds again, wait a week and then make a list of still broken feeds to just remove.

I've found broken links https://pklos.fedorapeople.org/brokenplanetreport26_07_2015.txt
Notifications was sent also. After a week I will make a diff and we'll think about next steps with links which won't be fixed.

Only 3 problems with sending notification:
Failed to send the notification to: mgelei@outlook.com
Failed to send the notification to: fedora@gbraad.nl
Failed to send the notification to: guillermo.gomez@gmail.com

I made a diff of actual and last week report:

{{{
6a7,8

{"url": "http://www.mostafadaneshvar.com/tag/linux/feed/", "user": "lashar", "error": "[Errno -2] Name or service not known"}
{"url": "http://fetzig.org/category/linux/fedora-planet/feed", "user": "heffer", "error": "[Errno 111] Connection refused"}
10d11
< {"url": "http://linuxtoy.org/tag/fedora/feed/", "user": "lovenemesis", "error": "404 Not Found"}
20d20
< {"url": "http://blogs.gnome.org/uraeus/category/fedora/feed/", "user": "uraeus", "error": "[Errno 110] Connection timed out"}
32a33,34
{"url": "http://blog.delouw.ch/category/linux/fedora/feed/", "user": "delouw", "error": "Internal Server Error"}
{"url": "http://dnf.baseurl.org/feed/", "user": "akozumpl", "error": "[Errno 110] Connection timed out"}
35a38
{"url": "http://blog.hiperborea.com.bo/tag/fedora/rss", "user": "lorddemon", "error": "[Errno 110] Connection timed out"}
38d40
< {"url": "http://gofedora.com/feed", "user": "kulbirsaini", "error": "Internal Server Error"}
42d43
< {"url": "http://jonarcher.info/tag/fedora/feed", "user": "jfarcher", "error": "Internal Server Error"}
49c50
< {"url": "https://bmartins.eu/?cat=3&feed=rss2", "user": "skorzen", "error": "[Errno 111] Connection refused"}


{"url": "https://bmartins.eu/?cat=3&feed=rss2", "user": "skorzen", "error": "[Errno 104] Connection reset by peer"}
57a59
{"url": "http://asingh.com.np/tag/planetfedora/feed/", "user": "aks", "error": "Internal Server Error"}
63c65,66
< {"url": "http://bacharakis.com/feed/", "user": "cmpahar", "error": "404 Not Found"}


{"url": "http://bacharakis.com/feed/", "user": "cmpahar", "error": "Bad Gateway"}
{"url": "http://blog.michel-slm.name/rss", "user": "salimma", "error": "[Errno -2] Name or service not known"}
67,68c70
< {"url": "http://blog-ezq.rhcloud.com/feed.rss", "user": "ezq", "error": "[Errno -2] Name or service not known"}
< {"url": "http://www.matahariproject.org/atom.xml", "user": "astokes", "error": "404 Not Found"}


{"url": "http://www.matahariproject.org/atom.xml", "user": "astokes", "error": "[Errno -2] Name or service not known"}
75d76
< {"url": "http://www.binarypenguins.com/journal/rss.xml", "user": "tuxbrewr", "error": "[Errno -2] Name or service not known"}
}}}

Because there are a few new positions I desided to send one more portion of notifications.

Thanks for working on this. ;)

Have you had much if any response back from notifications?

No problem, it's my duty ;)

I've sent notifications as admin@fedoraproject.org as mentioned before in this ticket.
Is anybody reading mails sent to admin@?

I didn't know you had sent them with that address... and yes we read all emails there, but there have not been any planet related ones. :(

So, perhaps some people are fixing things silently or these are all out of date.

pklos: any news here?

Shall we disable any still broken now?

kevin: I made a few iteration and get some info to disable links. I think we can disable also any still broken.

Nevertheless I recommend to schedule future iterations of checking/notifying as one of our normal tasks.

ok, so we should run another check now and disable all still broken, correct?

yes, I agree we should run the checker from cron and mail sysadmin-members about it so as new broken ones appear we can fix them/notify. Would you like to add a ansible patch to create such a cron job?

Here we have actual broken links report: https://pklos.fedorapeople.org/brokenplanetreport27_08_2015.txt and in json https://pklos.fedorapeople.org/brokenplanetreport27_08_2015.json

I think it's time to just disable broken ones. Could sb with sudo do that?

Done. All those are disabled (except 2 where I pinged people on irc to fix theirs).

So, now all we need is a cron to run and send us the report say once a week on mondays.

Tkanks Kevin. Do you want me to make that once a week checks or should I make some script (and ansible?) for some host to automate that?

Please do make a patch for it. Just a cron job in ansible to run it once a week and mail the output out.

Hi,

The last interaction was over 6 weeks, I would like to continue. Is it ok for you guys?

Absolutely.

I removed all the broken feeds a while back, but we still need to add the check to ansible to run as a cron job every week or so... and there may be some broken ones again now.

A merge request with the cron configuration and maybe some improvements (I'm thinking about what could be enhanced) in the script, on Ansible repo would be a good approach?

ansible/roles/people/tasks/main.yml
main.patch

ansible/roles/people/files/check-broken-planet.con
check-broken-planet.cron

ansible/roles/people/files/check-broken-planet.py
check-broken-planet.py

I've add the path for the file ansible/roles/people/tasks/main.yml, and a new version with just a few modifications, and the crontab conf.

The command wasn't quite right in the cron. ;)

I tweaked that and commited it.

Thanks so much for working on this!

I'm going to close it now and we can check back on sunday and confirm it's working as expected (and if not, open a new ticket to fix anything)

Login to comment on this ticket.

Metadata