From 41c2ce9498428f5f12707a2adc09f4eb9e4f3f59 Mon Sep 17 00:00:00 2001
From: Clement Verna <clems.verna@gmail.com>
Date: Dec 03 2019 10:57:23 +0000
Subject: Merge #169 `Add resources justification for copr-fe`


---

diff --git a/docs/sysadmin-guide/sops/copr.rst b/docs/sysadmin-guide/sops/copr.rst
index b39502d..559caa8 100644
--- a/docs/sysadmin-guide/sops/copr.rst
+++ b/docs/sysadmin-guide/sops/copr.rst
@@ -364,3 +364,90 @@ Keygen:
 
 
 .. _`number of not-yet-processed actions`_:: https://copr.fedorainfracloud.org/backend/pending-action-count/
+
+
+Resources justification
+=======================
+
+Copr currently uses the following resources.
+
+Frontend
+--------
+
+- RAM: 2G (out of 4G) and some swap
+- CPU: 2 cores (3400mhz) with load 0.92, 0.68, 0.65
+
+Most of the memory is eaten by PostgreSQL, followed by Apache.
+The CPU usage is also mainly used for those two services but
+in the reversed order.
+
+I don't think we can settle down with any instance that provides
+less than (2G RAM, obviously), but ideally, we need 3G+. 2-core
+CPU is good enough.
+
+
+- Disk space: 17G for system and 8G for `pgsqldb` directory
+
+If needed, we are able to clean-up the database directory of
+old dumps and backups and get down to around 4G disk space.
+
+
+Backend
+-------
+
+- RAM: 5G (out of 16G)
+- CPU: 8 cores (3400MHz) with load 4.09, 4.55, 4.24
+
+
+Backend takes care of spinning-up builders and running ansible playbooks
+on them, running `createrepo_c` (on big repositories) and so on. Copr
+utilizes two queues, one for builds, which are delegated to OpenStack
+builders, and action queue. Actions, however, are processed directly
+by the backend, so it can spike our load up. We would ideally like to
+have the same computing power that we have now. Maybe we can go lower
+than 16G RAM, possibly down to 12G RAM.
+
+
+- Disk space: 30G for the system, 5.6T (out of 6.8T) for build results
+
+Currently, we have 1.3T of backup data, that is going to be
+deleted soon, but nevertheless, we cannot go any lower on storage.
+Disk space is a long-term issue for us and we need to do a lot of
+compromises and settling down just to survive our daily increase
+(which is around 10G of new data). Many features are blocked by not
+having enough storage. We cannot go any lower and also we cannot
+go much longer with the current storage.
+
+
+Distgit
+-------
+
+- RAM: ~270M (out of 4G), but climbs to ~1G when busy
+- CPU: 2 cores (3400MHz) with load 1.35, 1.00, 0.53
+
+Personally, I wouldn't downgrade the machine too much.
+Possibly we can live with 3G ram, but I wouldn't go any lower.
+
+
+- Disk space: 7G for system, 1.3T dist-git data
+
+We currently employ a lot of aggressive cleaning strategies
+on our distgit data, so we can't go any lower than what we have.
+
+
+Keygen
+------
+
+- RAM: ~150M (out of 2G)
+- CPU: 1 core (3400MHz) with load 0.10, 0.31, 0.25
+
+We are basically running just `signd` and `httpd` here, both with
+minimal resource requirements. The memory usage is topped by
+`systemd-journald`.
+
+
+- Disk space: 7G for system and ~500M (out of ~700M) for GPG keys
+
+We are slowly pushing the GPG keys storage to its limit, so in the
+case of migrating copr-keygen somewhere, we would like to scale-up
+it to at least 1G.