This repo is a mirror of the RHN channel ppc64-server for RHEL-6, have you tried just retrieving the rpm from there?

I changed the repo to local mirror and added EPEL repo, but it still doesn't initialize the buildroot:

DEBUG util.py:283: Error: Package: epel-release-6-8.noarch (epel)
DEBUG util.py:283: Requires: redhat-release >= 6
DEBUG util.py:283: You could try using --skip-broken to work around the problem
DEBUG util.py:283: You could try running: rpm -Va --nofiles --nodigest
DEBUG util.py:283: Error: Package: epel-release-6-8.noarch (epel)
DEBUG util.py:283: Requires: /bin/sh

The final mock.cfg is here:
http://pastebin.test.redhat.com/245054

I guess it still doesn't see the RHEL-6 server repo. Could you tell me what's wrong?

Anyway the mock-config command is useless. There should be mechanism, how to quickly replicate the buildroot locally.

Anyway the mock-config command is useless.

Well, no, it isn't.

For example:

{{{
$ koji mock-config --tag=f21-build --arch=x86_64
}}}

Gives me a perfectly working way to "quickly replicate the buildroot locally"

The only reason it doesn't work for EL buildroots is because RHEL binary RPMs aren't publically available, but this is nothing Fedora can help with.

Now, as you actually do have access to them, then youc an make it work. One problem I see with the mock config you are using is:

{{{
config_opts['chroot_setup_cmd'] = 'groupinstall build'
}}}

Change that to:

{{{
config_opts['chroot_setup_cmd'] = 'groupinstall buildsys-build'
}}}

In any case, I don't think this is an infrastructure issue.

Replying to [comment:3 bochecha]:

Thanks, it initializes now.

Anyway the mock-config command is useless.

Well, no, it isn't.

For example:

{{{
$ koji mock-config --tag=f21-build --arch=x86_64
}}}

Gives me a perfectly working way to "quickly replicate the buildroot locally"

Of course I meant EPEL.

The only reason it doesn't work for EL buildroots is because RHEL binary RPMs aren't publically available, but this is nothing Fedora can help with.

I don't think so, the 'groupinstall build' was generated by koji mock-config and I had to also add the EPEL snip.

Couldn't it be generated right by koji mock-config? And with e.g. comment that you need to point the RHEL repo to RHN?

There is still missing something, the install phase fails with:
--> libtool-2.2.6-15.5.el6.ppc64
Error: No Package found for libtool-ltdl-devel

Could you help me what's missing?

This is different error than in koji

If you are interested, this is the build failure I am trying to replicate:
http://koji.fedoraproject.org/koji/taskinfo?taskID=8062208

I don't think so, the 'groupinstall build' was generated by koji mock-config and I had to also add the EPEL snip.

Couldn't it be generated right by koji mock-config?

It is generated right.

The thing to understand is that in Koji maintains its own internal repositories. And in these, the group is called ''build''.

But then, you pointed away from the Koji repos, to mirrors. In those mirrors, the group is called ''buildsys-build''.

So Koji generated a perfectly working mock config, under the assumption that you have access to its internal repos, as for Fedora builds. If you don't, well, then you have to edit the mock config properly.

You could make a case for naming the groups identically in both Koji and the mirrors, but that's a very different question from what you were asking here, and it would be suited for the Rel-Eng trac.

Replying to [comment:7 bochecha]:

I am afraid we are going off-topic. The point of this bug report is, why I simply cannot do koji mock-config for EPEL the same way as you show for f21 and be set-up (I am on Red Hat IP).

The point of this bug report is, why I simply cannot do koji mock-config for EPEL the same way as you show for f21 and be set-up

Because RHEL RPMs aren't publicly accessible, as I told you earlier.

(I am on Red Hat IP).

So am I, but I don't see how that's relevant.

Replying to [comment:9 bochecha]:

The point of this bug report is, why I simply cannot do koji mock-config for EPEL the same way as you show for f21 and be set-up

Because RHEL RPMs aren't publicly accessible, as I told you earlier.

I am not asking for public access, I am asking for access for Red Hat developers (e.g. through HTTPS or anything).

Just for the record I finally resolved it all.

Regarding the "No Package found for libtool-ltdl-devel" I had to add RHEL-6 optional repo to the mock config. Then it built OK! Remember, the same build is failing on koji. Thus, by spending nearly afternoon on it, I still haven't 1:1 copy of the buildroot.

So I did it the (old good) hard way - compared all package versions in both buildroots. I found that the problem is in the xmlsec1 package. There are different versions of it in both RHEL-6 and EPEL. Koji takes the EPEL version, everything other take the RHEL version (which has higher NVR). The EPEL version of xmlsec1 is buggy and prevents oath-toolkit from building. I think it should be blocked in EPEL. So there are more bugs on different places, I will report them all appropriately.

This all convinces me even more that mechanism for quick 1:1 EPEL buildroots replication is needed (at least for Red Hat engineers).

If I understand the need and use-case, one thing which makes me quite chilly about this request is that it explicitly creates a difference between rh contributors and non-rh contributors in our community, which is something I think we should avoid as much as possible.

My 2cts.

the question is why does epel have it at all. The answer to that is it no longer has xmlsec1

I share pingou's concern.

Additionally, it may be difficult to restrict something to RH folks. All we could do is external IP based... if we get something wrong or someone spoofs the right IP we could be leaking RHEL binaries. ;(

This is very sad. Imagine situation xmlsec1 having same NVRs but different binary bits in EPEL and RHEL. There will be no chance to diagnose the problem. NVRs comparison of root.log will not help in such case, not counting that this trial-error approach is very time consuming. This is not the first time I have been diagnosing such obscure build failure. Each time it took hours or even days to resolve the issue. Having access to the buildroot or exact copy of the koji buildroot will allow to resolve such issues in minutes. Not knowing what is going there and inability to replicate the build process is serious blocker for me and I am thinking about stepping out from EPEL development in favour of RHEL or Fedora.

Well, I am sorry to hear it, and I hate saying no to things, but I really don't want us to distribute rhel binaries to people who shouldn't have them.

I really think we might be able to come up with another solution here that helps everyone.

For these cases would it have worked for you to just have the metadata? That would have shown the differeing version of that package?

Or perhaps we could make sure to find these things sooner and just clean them up.

Kevin, thanks for your understanding. This was only proposal - anything that will allow replication of the buildroot will be helpful for me (and very probably for other devels as well). Having metadata (with checksums for packages) will improve the situation as there will be no doubt which binary bits were used for building. Currently I don't know how to script build root reconstruction using this information, but at least I will be able to check if my buildroot is identical and find where (in which package) the difference is. Other proposals how to improve the current situation are also welcome.