I will try to fix this asap.

Here is what I have managed to do by now. Can someone have a look and tell me what is ok and what is not. Because I am a little bit lost here.

Did some further investigation. I have no idea what to add for ''paste'' and ''fedmsg''. And I am not certain if the option line for ''fas-openid'' and ''askbot'' is correct.
Could somebody have a look and give some feedback, advice etc.

For paste I think / should be ok. Ideally we would hit a paste so we can be sure the db is working right, but pastes expire, so we can't depend on anything there. ;(

Not sure for fedmsg. Adding Ralph for comment.

The datagrepper check looks fine to me. :)

Is it possible to somehow test my changes?
What about fedmsg? Any idea?

Yeah, we are in freeze now, but we could test in staging. ;)

There's a seperate ./modules/haproxy/files/haproxy.cfg.stg file for staging thats very similar to the production one. If you can whip up a patch for that one we can apply it and test out staging stuff with it.

Check this out. I didn't know what to add for gallery.

/ should be ok for gallery (it's not really even deployed yet).

I'll push this into staging and we can see what breaks. ;)

ok, a few things broke. ;)

We need to figure out another way to check the totpcgiprovision. It needs a username/password, so the check never gets a 2xx or 3xx reply from it, so it marks it down. ;(

ask is showing down, but it really actually is down.
tagger is showing down, but it really actually is down. :)
gallery is showing down, but thats because the default / page is not a 2xx or 3xx response, so thats fine.

I will try to figure out something for totpcgiprovision. Fingers crossed.

Hopefully totpcgiprovision will now work.

Changed totpcgiprovision
haproxy_patch

Fixed totpcgiprovisioning
haproxy_stg_patch

I fixed tagger in our puppet repo by changing the check from /tagger/_heartbeat (which doesn't exist) to just /tagger/.

Thats not going to work for totpcgiprovisioning sadly.

/ (everything) is set to require a password. So, we need to either not check it at all, or figure some way for haproxy to accept a "forbidden" as "ok" for the check.
:(

This directive has to be used:

http-check expect [!]

Link to documentation:

[http://cbonte.github.io/haproxy-dconv/configuration-1.4.html#4.2-http-check%20expect]

I think the easiest way to use this directive is with status code. For example:

http-check expect status 200

The problem is that I don't know what status code is returned if you hit /index.cgi for totpcgiprovisioning. So someone who can get status code will need to help me.

Added directive for totpciprovisioning
haproxy_patch_1

{{{
$ nc fas01.stg 8444
GET /index.cgi

Authorization Required

This server could not verify that you are authorized to access the document requested. Either you supplied the wrong credentials (e.g., bad password), or your browser doesn't understand how to supply the credentials required.

}}}

So, it looks like a 401?

Added status code
haproxy_patch_2

I hope this now works as it should.

Yep. That does the trick on totpcgi :)

Thanks. So, based on this, can you generate a new production patch that we can push after the freeze is over?

Hopefully final production patch.
haproxy_patch_3

I can't seem to get this to apply. ;(

Can you check it against current puppet and rebase?

Please try again with the latest attachment.

attachment
haproxy_patch_4

ok, that applied fine. ;)

However, ask and openid didn't work. :(

{{{
[kevin@proxy09 ~]$ nc ask01 80
GET /upfiles/

Forbidden

You don't have permission to access /upfiles/ on this server.

[kevin@proxy09 ~]$ nc openid01 80
GET /static/

Forbidden

You don't have permission to access /static/ on this server.

}}}

So, we may need to adjust those for production somehow?

The ask version in stg is newer, but not sure why openid would be different.

First I will check again Apache config again. If /upfiles/ and /static/ is okey. Otherwise we can use the same trick as for totpciprovisioning, that is to use http-check expect 403.

Kevin try this fix. I think it should work.

attachment
haproxy_patch_5

I am not sure but I think askbot should work with /upfiles. But it doesn't work with /upfiles/

The patch 5 above looks fine. Hitting /questions/ on ask will generate a fair bit of traffic, but it should be fine. :)

So, I think we are all set here.

Thanks a bunch for working on this!