anubis provides metircs at http://localhost:9090/ on hosts it runs on, would be nice to get those into zabbix.
Metadata Update from @james: - Issue assigned to gwmngilfen - Issue priority set to: Waiting on Assignee (was: Needs Review) - Issue tagged with: low-gain, medium-trouble
Issue tagged with: sprint-0
OK, I've made a start. Take a look at proxy02.stg on Zabbix
This works by getting a master item from the Prometheus-style metrics every 60s, and then calculating a bunch of other things from that: - Total requests & validations (but this is since the process started, so it's monotonic, thus ...) - Rate-of-change of total requests & validations (more useful, can detect spikes) - Discovery of backend hosts (via anubis_proxied_requests_total{host = ???} - Total proxied requests to each backend
anubis_proxied_requests_total{host = ???}
I've not done rate-of-change to each backend (not sure if we need that), nor have I set up any triggers (because I'm not sure if we want that, or are just interested in the data), but as a proof-of-concept, this should be easy to adapt/extend/iterate on.
I'll do a PR for after freeze that rolls out the required changes to the proxies - it's another good example of "put the monitoring in the application role"
Issue status updated to: Closed (was: Open) Issue close_status updated to: Fixed
This has been rolled out with the anomaly-detection stuff as the main trigger - I've not seen any alerts yet, so it's either great or broken :P
Log in to comment on this ticket.