#1757 install mbs in the container images using pip
Merged 2 years ago by mikem. Opened 2 years ago by mikeb.
mikeb/fm-orchestrator pip-install  into  master

file modified
+48 -32
@@ -1,54 +1,70 @@ 

- FROM fedora:31 AS builder

+ FROM registry.fedoraproject.org/fedora:latest AS builder

  

- ARG EXTRA_RPMS=""

- ARG GIT_REPO=""

- ARG GIT_REF=""

- ARG VERSION=""

- ARG CREATED=""

  ARG DNF_CMD="dnf -y --setopt=deltarpm=0 --setopt=install_weak_deps=false --setopt=tsflags=nodocs"

  

  COPY . /src

  WORKDIR /src

  

- RUN ${DNF_CMD} install \

-       'dnf-command(builddep)' rpm-build rpmdevtools rpmlint \

-       python3-tox python3-pytest python3-pytest-cov python3-mock python3-flake8 bandit && \

-     ${DNF_CMD} builddep *.spec && \

-     ${DNF_CMD} clean all

- RUN rpmdev-setuptree && \

-     python3 setup.py sdist && \

-     rpmbuild --define "_sourcedir $PWD/dist" -ba *.spec && \

-     mv $HOME/rpmbuild/RPMS /srv

- RUN flake8 && \

-     bandit -r -ll -s B102,B104,B303,B411,B602 module_build_service && \

-     tox -v -e py3

- 

- 

- FROM fedora:31

+ RUN $DNF_CMD install \

+       python3-setuptools python3-wheel && \

+     $DNF_CMD clean all

+ RUN python3 setup.py bdist_wheel -u root -g root

+ 

+ FROM registry.fedoraproject.org/fedora:latest

+ 

+ ARG DNF_CMD="dnf -y --setopt=deltarpm=0 --setopt=install_weak_deps=false --setopt=tsflags=nodocs"

+ ARG EXTRA_RPMS=""

+ 

  LABEL \

      org.opencontainers.image.title="Backend for the Module Build Service (MBS)" \

      org.opencontainers.image.description="The MBS coordinates module builds. This image is to serve as the MBS backend." \

-     org.opencontainers.image.vendor="The Factory 2.0 Team" \

-     org.opencontainers.image.authors="The Factory 2.0 Team <pnt-factory2-devel@redhat.com>" \

+     org.opencontainers.image.vendor="The Brew Guild" \

+     org.opencontainers.image.authors="The Brew Guild <brew-devel@redhat.com>" \

      org.opencontainers.image.licenses="MIT" \

-     org.opencontainers.image.source="$GIT_REPO" \

-     org.opencontainers.image.revision="$GIT_REF" \

-     org.opencontainers.image.version="$VERSION" \

-     org.opencontainers.image.created="$CREATED" \

      org.opencontainers.image.url="https://pagure.io/fm-orchestrator" \

      org.opencontainers.image.documentation="https://pagure.io/fm-orchestrator" \

      distribution-scope="public"

  

- COPY --from=builder /srv/RPMS /srv/RPMS

- COPY repos/ /etc/yum.repos.d/

+ COPY --from=builder /src/dist /src/dist

  

  RUN $DNF_CMD install \

+       python3-celery \

+       python3-distro \

+       python3-dnf \

+       python3-docopt \

+       python3-dogpile-cache \

+       python3-fedmsg \

+       python3-flask \

+       python3-flask-migrate \

+       python3-flask-sqlalchemy \

+       python3-kobo \

+       python3-kobo-rpmlib \

+       python3-koji \

+       python3-ldap3 \

+       python3-libmodulemd \

+       python3-m2crypto \

+       python3-moksha-hub \

+       python3-munch \

+       python3-openidc-client \

+       python3-packaging \

+       python3-pip \

+       python3-prometheus_client \

        python3-psycopg2 \

+       python3-pungi \

+       python3-pyOpenSSL \

+       python3-pyparsing \

+       python3-requests \

        python3-service-identity \

-       /srv/*/*/*.rpm \

+       python3-six \

+       python3-solv \

+       python3-sqlalchemy \

+       git-core \

        $EXTRA_RPMS && \

-     $DNF_CMD clean all && \

-     rm -rf /srv/RPMS

+     rm -f /etc/fedmsg.d/* && \

+     $DNF_CMD clean all

+ # Install locally available python wheels only.

+ # Any new/updated dependencies should be installed via rpm.

+ RUN pip-3 install --no-deps /src/dist/*.whl

  

  USER 1001

  

file modified
+2 -10
@@ -1,22 +1,14 @@ 

  # See `../backend/` for building `mbs-backend:latest`

  FROM mbs-backend:latest

  

- ARG GIT_REPO=""

- ARG GIT_REF=""

- ARG VERSION=""

- ARG CREATED=""

  ARG DNF_CMD="dnf -y --setopt=deltarpm=0 --setopt=install_weak_deps=false --setopt=tsflags=nodocs"

  

  LABEL \

      org.opencontainers.image.title="Frontend for the Module Build Service (MBS)" \

      org.opencontainers.image.description="The MBS coordinates module builds. This image is to serve as the MBS frontend." \

-     org.opencontainers.image.vendor="The Factory 2.0 Team" \

-     org.opencontainers.image.authors="The Factory 2.0 Team <pnt-factory2-devel@redhat.com>" \

+     org.opencontainers.image.vendor="The Brew Guild" \

+     org.opencontainers.image.authors="The Brew Guild <brew-devel@redhat.com>" \

      org.opencontainers.image.licenses="MIT" \

-     org.opencontainers.image.source="$GIT_REPO" \

-     org.opencontainers.image.revision="$GIT_REF" \

-     org.opencontainers.image.version="$VERSION" \

-     org.opencontainers.image.created="$CREATED" \

      org.opencontainers.image.url="https://pagure.io/fm-orchestrator" \

      org.opencontainers.image.documentation="https://pagure.io/fm-orchestrator" \

      io.openshift.expose-services="8080:http,8443:https" \

Removing the requirement to build the rpm as part of the image build simplifies the
build process and reduces fragility introduced by trying to keep the rpm build and
image builds in sync.

Also, remove a bunch of args which are no longer used, and update the vendor and authors.

rebased onto 2e70c5f9bf43b9c9cf6a3980820691f97a2d80b5

2 years ago

rebased onto 4ad0e64cf8b85532b196b7d692d0e7e15a7aec6e

2 years ago

This long list had me thinking - shouldn't dependencies be installed during the wheel install? I tried running the build locally to see, and I hit a lot of missing dependencies, like cairo or gcc. I'm assuming you're installing this to avoid having to build the dependencies by ourselves?

pretty please pagure-ci rebuild

2 years ago

This long list had me thinking - shouldn't dependencies be installed during the wheel install?

As this image is likely to be a model for prod deployments, I do not want to use pip for resolving dependencies. Installing deps directly from pypi can be very convenient, but it is not anything I would use in production. Apart from the general lack of curation in pypi, we are very much focused on Fedora, CentOS, and RHEL as targets (including EPEL). We do not want to pull in any dependencies outside of these ecosystems.

So, this would either mean keeping the rpm build and letting dnf resolve deps or keeping them coded in the dockerfile.

rebased onto ecf7da7

2 years ago

pip-3 is now run with the --no-deps option, so it will not pull in any newly added dependencies. Ideally the integration tests will catch anything missing, prompting the developers to update the Dockerfile as necessary.

Note that there is one dependency this is no longer available in Fedora 36+: python-flask-script:

https://koji.fedoraproject.org/koji/packageinfo?packageID=15856

It is only used by manage.py so hopefully excluding it won't cause runtime issues.

Commit 7f6f584 fixes this pull-request

Pull-Request has been merged by mikem

2 years ago

Pull-Request has been merged by mikem

2 years ago