#49570 Improve dependency download for developers
Opened 2 years ago by firstyear. Modified a year ago

Issue Description

Our current contributor guide has these steps:

http://www.port389.org/docs/389ds/contributing.html#get-the-dependencies

It's a pretty messy copy paste job right now - we should improve this!

Some goals:

  • Make rpm builds require "less dependencies" (ie incorporate a one shot "make rpms" that is easier to access)
  • Make dependencies easier to install, similar to "make deps", regardless of platform.
  • the actual make step should be one command, not multiple.

Metadata Update from @firstyear:
- Custom field component adjusted to None
- Custom field origin adjusted to None
- Custom field reviewstatus adjusted to review
- Custom field type adjusted to None
- Custom field version adjusted to None

2 years ago

I followed the instructions and I have an issue building on a clean Fedora 27. during 'sudo ./x.py install'

 /usr/bin/mkdir -p '/opt/dirsrv/share/dirsrv/updates'
 /usr/bin/install -c ldap/admin/src/scripts/exampleupdate.sh '/opt/dirsrv/share/dirsrv/updates'
make[2]: Leaving directory '/mnt/tests/rhds/tests/upstream/ds'
make[1]: Leaving directory '/mnt/tests/rhds/tests/upstream/ds'
cd ./src/lib389; python3 setup.py build
Traceback (most recent call last):
  File "setup.py", line 15, in <module>
    from setuptools import setup, find_packages
ImportError: cannot import name 'setup'
make: *** [Makefile:12859: lib389] Error 1
Traceback (most recent call last):
  File "./x.py", line 126, in <module>
    run_x()
  File "./x.py", line 113, in run_x
    install()
  File "./x.py", line 82, in install
    'make', 'lib389-install'
  File "/usr/lib64/python3.6/subprocess.py", line 291, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['make', 'lib389-install']' returned non-zero exit status 2.

Also, is it expected? during './x.py test'

make[3]: Entering directory '/mnt/tests/rhds/tests/upstream/ds'
PASS: test_slapd
PASS: test_libsds
./test-driver: line 107: 19711 Aborted                 (core dumped) "$@" > $log_file 2>&1
FAIL: test_nuncstans
PASS: test_nuncstans_stress_small
============================================================================
Testsuite summary for 389-ds-base 1.4.0.5.20180219git18ea647
============================================================================
# TOTAL: 4
# PASS:  3
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0
============================================================================
See ./test-suite.log
Please report to http://bugzilla.redhat.com/enter_bug.cgi?product=389
============================================================================
make[3]: *** [Makefile:12087: test-suite.log] Error 1
make[3]: Leaving directory '/mnt/tests/rhds/tests/upstream/ds'
make[2]: *** [Makefile:12195: check-TESTS] Error 2
make[2]: Leaving directory '/mnt/tests/rhds/tests/upstream/ds'
make[1]: *** [Makefile:12419: check-am] Error 2
make[1]: Leaving directory '/mnt/tests/rhds/tests/upstream/ds'
make: *** [Makefile:12421: check] Error 2
Traceback (most recent call last):
  File "./x.py", line 126, in <module>
    run_x()
  File "./x.py", line 117, in run_x
    test()
  File "./x.py", line 74, in test
    'make', 'check'
  File "/usr/lib64/python3.6/subprocess.py", line 291, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['make', 'check']' returned non-zero exit status 2.

Crash report: https://paste.fedoraproject.org/paste/lU0liHcgcMGyOvx~tnOKcQ

The first one looks like a missing dependency from the specfile perhaps. This normally indicates missing setup utils.

The second is unrelated to this change, but i'll investigate it.

Trust you to find an issue with it :) I thought this was simple code!

I've tried to catch you today morning but wasn't able.
I still have the issue with import when I am doing it in 'sudo ./x.py install'

from setuptools import setup, find_packages
ImportError: cannot import name 'setup'

Though it works from my terminal:

$ python -c 'from setuptools import setup, find_packages'
$ echo $?
0

I didn't have much time today for the investigation, I'll take a look tomorrow more.
And after the testing phase I want to take a look at #49572 issue, maybe I'll be able to resolve it. :)

@vashirov said he will take a look and will share the ideas he has regarding this issue.

Did we ever get anywhere with this @vashirov and @spichugi ?

Did we ever get anywhere with this @vashirov and @spichugi ?

I still have the same issues I've posted above...
So I've decided to try another way. I've built the packages with Mock like this:

useradd -m -G wheel -s /bin/bash spichugi
echo "spichugi ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers
su - spichugi
git clone https://pagure.io/389-ds-base.git
cd 389-ds-base/
sudo yum install bzip2 mock rpm-build -y
sudo usermod -a -G mock spichugi
sudo make -f rpm.mk srpms
mock -r fedora-27-x86_64 $SRC_RPM_PATH

Then I've installed the packages.

What do you think if we'll use something like this? It is cleaner and the result is more close to what customers have.
Also we can add an option for lib389 installation (through pip install, so we will have fresh packages)

Maybe, @vashirov will add something that I missed.

I am open to the discussion. Anyway, I think the tool is very nice and it's very cool to have it. :)
Thank you!

Well, provided these steps are encapsulated into the x.py script, I think I would be happy with it. But I also want to point out we can't be 100% fedora / rhel / centos focused. We could have other distros like gentoo, suse, debian, who want to add the support for their dependencies and more.

So I think for rpm build maybe we can do this, but we need to be careful about modification of sudo, and other related changes that people may not expect.

PS: Don't use echo to /etc/sudoers, use /etc/sudoers.d/<file>

Well, provided these steps are encapsulated into the x.py script, I think I would be happy with it. But I also want to point out we can't be 100% fedora / rhel / centos focused. We could have other distros like gentoo, suse, debian, who want to add the support for their dependencies and more.

Agree. It makes sense. We can have different distros support, so more people could contribute to the project. :)

So I think for rpm build maybe we can do this, but we need to be careful about modification of sudo, and other related changes that people may not expect.
PS: Don't use echo to /etc/sudoers, use /etc/sudoers.d/<file>

Sure, my script was just a draft for my local use.

Login to comment on this ticket.

Metadata