#168 mysterious rebuilds
Closed 6 years ago Opened 6 years ago by brianjmurrell.

I'm seeing rebuilds happening in my Fedora Copr project that I have no idea why they are happening. They happen to many different packages but to examine one in particular take a look at this one.

It rebuilt just 13h ago with no apparent change to the package. So why was it rebuilt?


Or this one which has rebuilt 3 times (here here and here) in the past 24h.

Metadata Update from @clime:
- Issue assigned to clime

6 years ago

They happen to many different packages but to examine one in particular take a look at this one.

We have some incoming webhook events for the project:

https://copr.fedorainfracloud.org/coprs/managerforlustre/manager-for-lustre/build/662929/
192.30.252.40 - - [16/Nov/2017:18:25:12 +0000] "POST /webhooks/github/14105/70da741d-c072-4456-a972-ac40e1b93f78/ HTTP/1.1" 200 2 "-" "GitHub-Hookshot/e20df6f"

https://copr.fedorainfracloud.org/coprs/managerforlustre/manager-for-lustre/build/663021/
192.30.252.34 - - [16/Nov/2017:21:16:45 +0000] "POST /webhooks/github/14105/70da741d-c072-4456-a972-ac40e1b93f78/ HTTP/1.1" 200 2 "-" "GitHub-Hookshot/e20df6f"

https://copr.fedorainfracloud.org/coprs/managerforlustre/manager-for-lustre/build/663033/
192.30.252.42 - - [16/Nov/2017:21:27:29 +0000] "POST /webhooks/github/14105/70da741d-c072-4456-a972-ac40e1b93f78/ HTTP/1.1" 200 2 "-" "GitHub-Hookshot/e20df6f"

Or this one which has rebuilt 3 times (here here and here) in the past 24h.

Also for this one:

https://copr.fedorainfracloud.org/coprs/managerforlustre/manager-for-lustre/build/662344/
192.30.252.42 - - [15/Nov/2017:18:42:34 +0000] "POST /webhooks/github/14105/70da741d-c072-4456-a972-ac40e1b93f78/ HTTP/1.1" 200 2 "-" "GitHub-Hookshot/e20df6f"

https://copr.fedorainfracloud.org/coprs/managerforlustre/manager-for-lustre/build/662352/
192.30.252.40 - - [15/Nov/2017:19:43:54 +0000] "POST /webhooks/github/14105/70da741d-c072-4456-a972-ac40e1b93f78/ HTTP/1.1" 200 2 "-" "GitHub-Hookshot/e20df6f"

https://copr.fedorainfracloud.org/coprs/managerforlustre/manager-for-lustre/build/662437/
192.30.252.34 - - [15/Nov/2017:22:40:20 +0000] "POST /webhooks/github/14105/70da741d-c072-4456-a972-ac40e1b93f78/ HTTP/1.1" 200 2 "-" "GitHub-Hookshot/95a0199"

Can you match those log lines with the respective events as seen in your Github project webhook settings?

https://copr.fedorainfracloud.org/coprs/managerforlustre/manager-for-lustre/build/662344/
192.30.252.42 - - [15/Nov/2017:18:42:34 +0000] "POST /webhooks/github/14105/70da741d-c072-4456-a972-ac40e1b93f78/ HTTP/1.1" 200 2 "-" "GitHub-Hookshot/e20df6f"

So this was a webhook event for this commit which was a push to a PR/branch in the repo which as you can see bumps the Version: in the specfile from 2.0.2 to 2.0.4. But the build ultimately still built 2.0.2. So why it built 2.0.2 again is puzzling, albeit our desired behaviour. :-)

Since this was a PR only, and not a land to master, we wouldn't ultimately want our Copr updated to that new version in the PR -- so that it simply rebuilt 2.0.2 is good. But still puzzling, and perhaps expected in so much as if you are going to build something as a result of a PR, it should NOT be the version in the PR.

But why build anything as a result of a PR/branch push? Seems like a waste of resources to do these "NOOP" builds for PR pushes. Of course, maybe it's mis-configuration on my side.

Looking at that Package definition I notice that whoever created it did not fill in a Commitish, which I usually set to master when I create new Packages. Could that cause these "NOOP" builds on PR (i.e. non-master) pushes?

Is it just happy accident/bug that in the case where the Commitish is not limited to master that PR/branch updates are doing "NOOP" builds of master?

Is it just happy accident/bug that in the case where the Commitish is not limited to master that PR/branch updates are doing "NOOP" builds of master?

This is not a coincidence. It builds pushes into any branch but defaults to master for checkout in the end. This is probably a bug. I think, empty 'Committish' field should mean 'master'.

Note that the 'Committish' setting will not however have effect for tag-create event. In this case, committish will be ignored and the given tag will be rebuilt (if tag-create events are being sent). This might come as a bit of a surprise but as tags can be across all branches and we don't which one it is actually, then any checking against 'Committish' is not really possible anyway.

Does this make sense to you or would you prefer some other behavior?

Were you able to track down also the other spawned builds? (So that we know there is no mystery build left :)). If so, I will probably just fix the code so that 'Committish' field defaults to master for ref checking purposes.

This is probably a bug. I think, empty 'Committish' field should mean 'master'.

Seems the least surprising.

Does this make sense to you or would you prefer some other behavior?

No, I think this is fine as it is.

Were you able to track down also the other spawned builds? (So that we know there is no mystery build left :)). If so, I will probably just fix the code so that 'Committish' field defaults to master for ref checking purposes.

I suspect they were all of the same cause. I fixed up those packages' Commitish field and now have not seen a mystery build for 13 days.

Thanks for looking into this one.

In the end, I decided to keep the current behavior with empty committish meaning any branch in package definition but if a push event arrives, that branch from the push event will be rebuilt, not always 'master' as before.

'Committish' in package definition is now basically a ref filter for push events (tag events ignore it). In build job description, 'committish' simply means what to build.

Fixed by https://pagure.io/copr/copr/c/d3dcbaa2e28ab33a11372c4471d99c3deab8629b?branch=master.

Metadata Update from @clime:
- Issue status updated to: Closed (was: Open)

6 years ago

Login to comment on this ticket.

Metadata