This looks like a race condition where the updates compose was running at the same time as the twoweek nightly atomic run.

updates compose ostree tasks started at 08:48:45 and ended at 09:27:07:

2018-06-25 08:48:45 [INFO    ] [BEGIN] ---------- PHASE: OSTREE ----------
...
2018-06-25 09:27:07 [INFO    ] [DONE ] ---------- PHASE: OSTREE ----------

twoweek compose image build tasks started at 09:17:42 and finished at 09:43:04:

2018-06-25 09:17:42 [INFO    ] [BEGIN] ---------- PHASE: IMAGE_BUILD ----------
...
2018-06-25 09:43:04 [IMAGE_BUILD     ] [INFO    ] [DONE ] Creating image (formats: qcow2-raw-xz, arches: aarch64-ppc64le-x86_64, variant: AtomicHost, subvariant: AtomicHost) (task id: 27853671)

mark this as another reason we want to generate the ISO and VM images inside the updates compose and not as a separate compose.

Started another compose

https://kojipkgs.fedoraproject.org/compose/twoweek/Fedora-Atomic-28-20180625.1/

This looks like a race condition where the updates compose was running at the same time as the twoweek nightly atomic run.
updates compose ostree tasks started at 08:48:45 and ended at 09:27:07:
2018-06-25 08:48:45 [INFO ] [BEGIN] ---------- PHASE: OSTREE ----------
...
2018-06-25 09:27:07 [INFO ] [DONE ] ---------- PHASE: OSTREE ----------

twoweek compose image build tasks started at 09:17:42 and finished at 09:43:04:
2018-06-25 09:17:42 [INFO ] [BEGIN] ---------- PHASE: IMAGE_BUILD ----------
...
2018-06-25 09:43:04 [IMAGE_BUILD ] [INFO ] [DONE ] Creating image (formats: qcow2-raw-xz, arches: aarch64-ppc64le-x86_64, variant: AtomicHost, subvariant: AtomicHost) (task id: 27853671)

Ah I see.
I wonder if it was a race condition, why did it pick OSTree version 28.20180623.0 instead of 28.20180624.0 ?

mark this as another reason we want to generate the ISO and VM images inside the updates compose and not as a separate compose.

I wonder if it was a race condition, why did it pick OSTree version 28.20180623.0 instead of 28.20180624.0 ?

I have no idea. It's really weird the 0624 compose is marked as finished but there is no 0624 commit in the repo (for x86_64):

commit fbed0e26736fc189129f80e9547bfd71497377ca49f0cfd421f173667f5ea825                                                                                                                                           
ContentChecksum:  c558c8d4f54308b71302a587ef8f65fa659dfca0e31f94d8a52f2d3a5fe9a08f                                                                                                                                
Date:  2018-06-25 08:58:55 +0000                                                                                                                                                                                  
Version: 28.20180625.0                                                                                                                                                                                            
(no subject)                                                                                                                                                                                                      

commit 5cd042553d8c2f5d49f91630279eaf29af30ffca99ce42bf9ee14bc4c233fdcd                                                                                                                                           
ContentChecksum:  1dc12f3d042ffbb4d1a3f485ebe2de9ff3fc43501fc14797c689f7009fcef75c                                                                                                                                
Date:  2018-06-23 18:55:35 +0000                                                                                                                                                                                  
Version: 28.20180623.0                                                                                                                                                                                            
(no subject)

and from the 0624 compose log:

2018-06-24 19:38:21 [INFO    ] [DONE ] OSTree phase for variant Everything, arch x86_64 (task id: 27842011)
2018-06-24 19:42:17 [ERROR   ] Unable to find commitid file

looks like it couldn't write the commit to the commitid file (which is what robosig uses to update the ref after it signs the commit).

We probably need to investigate this more.

The commitid file is written by rpm-ostree via the --write-commitid-to option. Hoever looking at logs from rpm-ostree there were no changes, hence no commit and thus no commit id was written. This is probably a bug in Pungi. What do you expect should happen in such case? (I expect it should do nothing.)

The commitid file is written by rpm-ostree via the --write-commitid-to option. Hoever looking at logs from rpm-ostree there were no changes, hence no commit and thus no commit id was written. This is probably a bug in Pungi. What do you expect should happen in such case? (I expect it should do nothing.)

ahh I see. We need to add the force_new_commit option to the cofig. @sinnykumari - can you do that?

The commitid file is written by rpm-ostree via the --write-commitid-to option. Hoever looking at logs from rpm-ostree there were no changes, hence no commit and thus no commit id was written. This is probably a bug in Pungi. What do you expect should happen in such case? (I expect it should do nothing.)

ahh I see. We need to add the force_new_commit option to the cofig. @sinnykumari - can you do that?

Yup, I will work on it.

OSTree version from Fedora-Atomic-28-20180625.1 compose looks good on all arches. They all have version 28.20180625.0