2018-02-10 12:52:24	-->	axk4545 (~axk4545@rit/ritlug/member/axk4545) has joined #fedora-buildsys
2018-02-10 12:52:24	--	Topic for #fedora-buildsys is "Fedora COPR and build tools discussion channel | #fedora-releng for koji / fedora specific items | #fedora-devel for general queries"
2018-02-10 12:52:24	--	Topic set by nirik (~nirik-fre@scrye.com) on Fri, 10 Apr 2015 10:13:32
2018-02-10 12:52:24	--	Channel #fedora-buildsys: 81 nicks (1 op, 1 voice, 79 normals)
2018-02-10 12:52:42	--	Channel created on Sat, 28 Jul 2012 18:44:35
2018-02-10 12:52:47	axk4545	clime: yay! you're back.
2018-02-10 12:52:56	clime	axk4545: hello!
2018-02-10 12:53:21	clime	axk4545: is there anything I can help with?
2018-02-10 12:56:02	axk4545	clime: hello. yes actually. I was wondering if it is possible to deploy a single node COPR where all pieces reside on a single VM/host/server (IE not using openshift builders). I also remember you mentioning something about it supporting multiple packages in one repo...
2018-02-10 12:57:07	clime	well, we have docker stack that will spawn everything in containers (like every single component)
2018-02-10 12:57:17	<--	imcleod (~imcleod@2601:249:8200:1ba1:5ad1:e2f1:c074:7b53) has quit (Ping timeout: 255 seconds)
2018-02-10 12:57:28	clime	but it just has only a single builder, it's for testing mainly
2018-02-10 12:58:06	axk4545	clime: that is fine. are there docs on how to modify that for prod? 
2018-02-10 12:58:08	clime	(and it's not written in ansible, which I would like)
2018-02-10 12:58:44	clime	axk4545: no, not really, I would actually have that stack written in Ansible
2018-02-10 12:58:53	clime	well atm, it is just experimental
2018-02-10 13:00:27	clime	axk4545: I guess you can take a look into the docker_compose.yml file
2018-02-10 13:00:44	axk4545	That sounds like exactly what I am looking for actually. a way to run an entire instance on a single box. the single builder would work fine for my use case as long as it runs. I have about 5 packages now and it will likely max out at 50. they are all either x86_64 or noarch
2018-02-10 13:00:57	axk4545	clime: in the pagure repo right?
2018-02-10 13:01:04	clime	axk4545: yup
2018-02-10 13:03:13	axk4545	clime: this is the one you meant that is as close to single node all on one VM/host/box/server COPR as possible atm right? https://pagure.io/copr/copr/blob/master/f/docker-compose.yaml
2018-02-10 13:03:29	clime	axk4545: yes.
2018-02-10 13:04:16	axk4545	clime: ok. will see what I can do with it and ping you with questions. do you prefer IRC or email?
2018-02-10 13:04:33	clime	axk4545: :), anything will do
2018-02-10 13:04:48	clime	happy to be of help
2018-02-10 13:06:10	axk4545	clime: dumb question. how do I use the compose file? :P
2018-02-10 13:06:24	clime	axk4545: install docker-compose
2018-02-10 13:06:42	clime	and then just run docker-compose up
2018-02-10 13:06:59	clime	mmt. there is a blog post about the stack...
2018-02-10 13:07:26	clime	axk4545: http://frostyx.cz/posts/copr-stack-dockerized
2018-02-10 13:07:29	axk4545	huh. deceptively easy
2018-02-10 13:08:12	clime	axk4545: better I guess is to run it with the -d as the blog post says
2018-02-10 13:08:23	clime	it will not block your shell
2018-02-10 13:08:29	axk4545	right
2018-02-10 13:10:56	axk4545	clime: so the way to get a public instance of this is just to reconfigure the network to be accessible outside localhost right? nothing fancy?
2018-02-10 13:11:28	clime	axk4545: yes, probably just bind the frontend to port 80 (or 443) instead of 5000
2018-02-10 13:11:54	clime	axk4545: well you will probably also want the backend instance to be accessible
2018-02-10 13:12:05	axk4545	clime: open 5000 in the firewall?
2018-02-10 13:12:17	clime	axk4545: you can do it as well
2018-02-10 13:12:54	clime	if you are ok with running it on those "non-standard" ports
2018-02-10 13:13:02	clime	then that's the easiest
2018-02-10 13:13:56	axk4545	clime: I imagine there are security implications there though... what ports would a normal COPR use? and do I need to be aware of anything since it is in docker that might break frontend <-> backend communication?
2018-02-10 13:13:57	clime	I guess you could also configure nginx/httpd to route the traffic to those ports
2018-02-10 13:14:42	axk4545	clime: I am cool with odd ports as long as it doesn't cause security problems
2018-02-10 13:15:17	clime	axk4545: ye, well I guess can also keep the stack as is and then route the traffic there.
2018-02-10 13:15:22	-->	imcleod (~imcleod@2601:249:8200:1ba1:5ad1:e2f1:c074:7b53) has joined #fedora-buildsys
2018-02-10 13:15:56	clime	axk4545: but if you open those ports, I don't think there is any security issue with it (except for frontend running on http)
2018-02-10 13:16:19	clime	if you know what services are there and you trust them, then it should be ok
2018-02-10 13:16:45	clime	axk4545: well, you might want to take a look if the containers run in privileged mode or not
2018-02-10 13:16:51	axk4545	clime: so I should change the frontend to use the standard 80 or 443 most likely.
2018-02-10 13:17:19	clime	axk4545: if you want https, then yes
2018-02-10 13:18:13	axk4545	clime: ok. so the builder container runs privileged but none of the others do.
2018-02-10 13:18:26	clime	axk4545: ye, that's a bit unpleasant
2018-02-10 13:18:51	clime	axk4545: the builder should not run privileged
2018-02-10 13:19:19	axk4545	clime: ok. I am gonna see what happens if I take that out then...
2018-02-10 13:21:42	axk4545	clime: oh. the backend also runs privileged...
2018-02-10 13:22:02	clime	axk4545: hmmp, I don't think that should be needed
2018-02-10 13:22:09	clime	axk4545: I can understand the builder
2018-02-10 13:22:17	axk4545	clime: didn't you write it lol?
2018-02-10 13:22:30	clime	axk4545: no, my colleague did
2018-02-10 13:22:56	axk4545	clime: are they in here? maybe we can pick their brain...
2018-02-10 13:23:00	clime	axk4545: the builder needs it because mock used to have (and probably still has) problems running in unpriv containers
2018-02-10 13:23:15	clime	axk4545: ping FrostyX
2018-02-10 13:23:26	clime	he might be around now or next week
2018-02-10 13:23:40	clime	axk4545: anyway for backend, you probably can remove it
2018-02-10 13:23:50	clime	but with builder it might be a problem due to mock
2018-02-10 13:23:52	axk4545	clime: yup. I ran in to the mock/ docker issue before
2018-02-10 13:24:32	clime	axk4545: ye well, this is difficult to solve
2018-02-10 13:24:47	clime	axk4545: if you want production, then builder should be a separate machine
2018-02-10 13:24:56	 *	FrostyX is here
2018-02-10 13:25:02	clime	I don't think we are able to fully isolate it atm
2018-02-10 13:25:48	clime	FrostyX: hello, we are discussing possibility of deploying the docker stack in production
2018-02-10 13:25:58	FrostyX	at first the builder was a part of the backend, so backend needed to be run as privileged. Then I've separated it to it's own container, but I probably forgot to remove privileged from backend
2018-02-10 13:26:51	axk4545	clime, FrostyX: ok. I honestly only need the building portion. the part that talks to git and makes the tarballs and srpms and rpms. Don't really need the frontend as long as I can access the results without it via ssh or the like
2018-02-10 13:27:09	clime	axk4545: it depends whom you allow access
2018-02-10 13:27:40	clime	axk4545: if you make it protected so that only you can access frontend, then you should be ok
2018-02-10 13:28:08	clime	frontend is needed to launch the builds.
2018-02-10 13:30:22	axk4545	clime: oh... hrm... that is a challenge. part of what I mean is that I won't be using the frontend to provide a public facing repo. we will have a separate box with a normal yum/dnf/RPM repo. I just need the part to bridge betwwen git and making the binaries...
2018-02-10 13:30:45	axk4545	FrostyX: ^
2018-02-10 13:32:37	FrostyX	axk4545: how are you going to submit the builds?
2018-02-10 13:33:06	axk4545	FrostyX: git push to the repo on GitHub
2018-02-10 13:36:06	axk4545	FrostyX: basically just need to part that can watch a git repo and when a change is pushed, assuming it has the right structure, build an RPM and an SRPM from the contents of the repo and put them in a place I can access as a user over ssh to the box running the COPR components 
2018-02-10 13:36:23	axk4545	clime: ^ 
2018-02-10 13:36:42	clime	axk4545: yes, cool
2018-02-10 13:37:02	clime	well, with Github wehooks you can specify "secret"
2018-02-10 13:37:23	clime	which will generate hashsum and signature.
2018-02-10 13:37:54	clime	Maybe that could be used to allow only those signed requests. But we would need to code support for it
2018-02-10 13:38:09	axk4545	clime: to what end?
2018-02-10 13:38:17	clime	to frontend
2018-02-10 13:38:29	axk4545	clime: I meant why...
2018-02-10 13:38:46	clime	frontend normally receives the webhooks.
2018-02-10 13:39:15	clime	axk4545: well, If you would want only accept only those requests and not any others
2018-02-10 13:39:32	clime	this would probably be a way to make things safe.
2018-02-10 13:39:45	clime	on a single machine.
2018-02-10 13:39:55	axk4545	clime: ok. is it possible to ssh in to the box running it and interact with the builder to trigger a task manually without talking to the frontend?
2018-02-10 13:40:45	clime	axk4545: not currently
2018-02-10 13:41:11	axk4545	FrostyX: also the docker compose for backend is choking on enabling the updates repo...
2018-02-10 13:41:47	clime	axk4545: we plan to implement it into copr-rpmbuild to be able to do just copr-rpmbuild scm --clone-url https://github.com/example. But you would lose automation like that.
2018-02-10 13:42:29	axk4545	clime: not if you run that command with a CI system. 
2018-02-10 13:42:38	clime	axk4545: that's true :)
2018-02-10 13:42:53	clime	axk4545: so yeah, if you have that system then that could be the way
2018-02-10 13:43:20	axk4545	clime: but that isn't currently in copr-rpmbuild right?
2018-02-10 13:43:39	axk4545	and it would use Fedora copr anyway
2018-02-10 13:43:39	clime	axk4545: no, not currently but we can make it happen quickly
2018-02-10 13:44:01	clime	basically in a few days if would like the feature
2018-02-10 13:44:11	axk4545	clime: ok. where is it? I would like to help if possible.
2018-02-10 13:44:22	clime	axk4545: no, it would be completely separated from Fedora COPR.
2018-02-10 13:44:32	axk4545	clime: oh, sweet
2018-02-10 13:44:39	clime	https://pagure.io/copr/copr/blob/master/f/rpmbuild
2018-02-10 13:45:46	clime	axk4545: the suggestion to implement it was here: https://pagure.io/copr/copr/issue/102
2018-02-10 13:46:15	axk4545	clime: that looks awesome. so if I understand this right, that eliminates the need for frontend and you can talk to the builder from a cli?
2018-02-10 13:46:39	clime	axk4545: oh, copr-cli only talks to frontend
2018-02-10 13:47:27	axk4545	clime: right but copr-rpmbuild short circuits talking to frontend and talks directly to builder or backend?
2018-02-10 13:47:59	clime	axk4545: copr-rpmbuild normally takes build tasks from frontend
2018-02-10 13:48:20	clime	but we can tweak it so that the task is specified on a command line
2018-02-10 13:48:33	clime	so that it doesn't need to contact frontend at all
2018-02-10 13:48:57	axk4545	clime: ok, and with that tweak would frontend be needed at all? 
2018-02-10 13:49:22	clime	nope, not even backend but you would need something that would invoke copr-rpmbuild on pushes
2018-02-10 13:49:43	clime	with the correct input on the command-line
2018-02-10 13:49:54	clime	(like clone-url, git hash, etc.)
2018-02-10 13:50:11	<--	imcleod (~imcleod@2601:249:8200:1ba1:5ad1:e2f1:c074:7b53) has quit (Ping timeout: 276 seconds)
2018-02-10 13:50:15	clime	backend normally also refreshes repodata so you would need to do it yourself.
2018-02-10 13:50:24	axk4545	clime: that is perfect for me use case. let me know if I can help. 
2018-02-10 13:50:25	clime	copr-rpmbuild just spits out srpms/rpms
2018-02-10 13:50:50	axk4545	clime: that is exactly what I need! :)
2018-02-10 13:50:56	clime	axk4545: okay, cool :)
2018-02-10 13:51:33	clime	axk4545: I guess we can do it like next week. There are one or two tricky parts in the implementation.
2018-02-10 13:52:07	axk4545	clime: ok. again let me know if I can be of assistance and I will try to help
2018-02-10 13:52:16	clime	axk4545: okay
2018-02-10 13:53:17	clime	I will try to to prepare it and then will see if I can delegate something.
2018-02-10 13:54:42	axk4545	clime: than you so much! This has taken me like six months to figure out lol
2018-02-10 13:55:05	axk4545	clime: are you on github so I can tag you in our tracking issue?
2018-02-10 13:55:09	axk4545	clime++
2018-02-10 13:55:10	zodbot	axk4545: Karma for clime changed to 1 (for the f27 release cycle):  https://badges.fedoraproject.org/tags/cookie/any
2018-02-10 13:55:16	axk4545	clime++
2018-02-10 13:55:18	clime	axk4545: well, it took us much longer to point when we can offer something like that:)
2018-02-10 13:55:53	clime	axk4545: you can file a new PR at pagure
2018-02-10 13:56:04	clime	axk4545: or just watch https://pagure.io/copr/copr/issue/102
2018-02-10 13:56:19	clime	I guess I can also update that issue when we are done.
2018-02-10 13:56:34	axk4545	clime: for us the six months was to decide on a way to automate RPM building 
2018-02-10 13:56:56	clime	axk4545: right, well, it takes time.
2018-02-10 14:02:52	axk4545	clime: may I log this conversation?
2018-02-10 14:03:35	clime	axk4545: sure
2018-02-10 14:04:01	axk4545	ok.