#1351 propose Generate.SoureceX.sh script
Opened a month ago by msuchy. Modified 12 days ago
msuchy/packaging-committee generate.sourceX.sh  into  master

@@ -83,19 +83,66 @@ 

  include::{examplesdir}/spectemplate-forge-multi.spec[]

  ----

  

- == Using Revision Control

+ == Manually create tarball

  

- In some cases you may want to pull sources from upstream's revision control system

- because there have been many changes since the last release

- and you think that a tarball that you generate from there will more accurately show how the package relates to upstream's development.

+ In some cases, you may need to create tarball manually. You should create

+ a script `Generate.Source1.sh` that creates the tarball:

+ 

+ .Generate.Source1.sh

+ [source, bash]

+ ....

+ # Requires: git

+ git clone --no-checkout https://github.com/cucumber/gherkin-ruby

+ git -C gherkin-ruby archive -v -o rubygem-gherkin-%{version}-testdata.txz v%{version} testdata/

+ ....

+ 

+ On the first line, in a comment that starts with "Requires:" you MUST list packages that are needed to generate

+ the tarball and that are not in a minimal buildroot. This can be interpreted by automation tools before executing following commands.

+ You can use variables: `%{name}`, `%{version}`, `%{release}` in the script. And only in this curly bracket form.

+ Other variables will not be available.

+ 

+ Then you can use in the Spec file:

+ 

+ .example.spec

+ [source, , rpm-spec]

+ ....

+ Source1: %{name}-%{version}-testdata.txz

+ ....

+ 

+ Previously, it was recommended that the steps be documented in a comment. This is allowed,

+ but maintainers are encouraged to use the `Generate.SourceX.sh` scripts:

+ 

+ .example.spec

+ [source, rpm-spec]

+ ....

+ # git clone --no-checkout https://github.com/cucumber/gherkin-ruby

+ # git -C gherkin-ruby archive -v -o rubygem-gherkin-22.0.0-testdata.txz v22.0.0 testdata/

+ Source1: %{name}-%{version}-testdata.txz

+ Source2: Generate.Source1.sh

+ ....

+ 

+ 

+ === Using Revision Control

+ In some cases, you may want to pull sources from the upstream's revision control system

+ because there have been many changes since the last release,

+ and you think that a tarball that you generate from there will more accurately show how the package relates to the upstream's development.

  Here's how you can use a comment to show where the source came from:

  

+ .Generate.Source.sh

+ [source, bash]

  ....

+ # Requires: svn

  # The source for this package was pulled from upstream's vcs.  Use the

  # following commands to generate the tarball:

- #  svn export -r 250 https://www.example.com/svn/foo/trunk foo-20070221

- #  tar -cJvf foo-20070221.tar.xz foo-20070221

+ svn export -r 250 https://www.example.com/svn/foo/trunk foo-20070221

+ tar -cJvf foo-20070221.tar.xz foo-20070221

+ ....

+ 

+ .example.spec

+ [source, rpm-spec]

+ ....

  Source: foo-20070221.tar.xz

+ Source2: Generate.Source.sh

  ....

  

  When pulling from revision control,
@@ -109,30 +156,26 @@ 

  In these cases you have to modify the source tarball to remove this code before you even upload it to the build system.

  Here's an example of using a script to document how you went from the upstream tarball to the one included in the package:

  

- From the spec:

- 

+ .Generate.Source1.sh

+ [source, bash]

  ....

- Source: libfoo-1.0-nopatents.tar.gz

  # libfoo contains patented code that we cannot ship.  Therefore we use

  # this script to remove the patented code before shipping it.

  # Download the upstream tarball and invoke this script while in the

  # tarball's directory:

- # ./generate-tarball.sh 1.0

- Source1: generate-tarball.sh

- ....

  

- generate-tarball.sh:

+ tar -xzvf libfoo-%{version}.tar.gz

+ rm libfoo-%{version}/src/patentedcodec.c

+ sed -i -e 's/patentedcodec.c//' libfoo-$VERSION/src/Makefile

  

+ tar -czvf libfoo-%version-nopatents.tar.gz libfoo-$VERSION

  ....

- #!/bin/sh

- 

- VERSION=$1

  

- tar -xzvf libfoo-$VERSION.tar.gz

- rm libfoo-$VERSION/src/patentedcodec.c

- sed -i -e 's/patentedcodec.c//' libfoo-$VERSION/src/Makefile

- 

- tar -czvf libfoo-$VERSION-nopatents.tar.gz libfoo-$VERSION

+ .example.spec

+ [source, rpm-spec]

+ ....

+ Source1: libfoo-1.0-nopatents.tar.gz

+ Source2: Generate.Source1.sh

  ....

  

  == Python Packages (pypi)

This will recommend that tarball that does not have URL must include
script that will generate the tarball.

This will allow automatic check of the content of the tarball.
And it will provide a record how the tarball was generated.

My personal observations (that I entered yesterday and yet never submitted):

I agree with this conceptually; we should at least provide suggestions for how this should be done, with an eye towards enabling more automation.

Following this shouldn't be any sort of requirement at this point. maybe later. The only requirement should be that if a script has the prescribed name, then it MUST follow the format

I don't understand why you would require something to substitute in %{version} and such when those could be passed in the environment or as positional arguments. Is there some kind of security consideration here that I'm not understanding?

What is the state of the various bits of automation which would use this? Is there a proposal elsewhere to set up automated checks based on this? Is there a ticket or PR against fedpkg or some adjacent tool to handle passing the information to the script to automate the downloads?

I don't understand why you would require something to substitute in %{version} and such when those could be passed in the environment or as positional arguments. Is there some kind of security consideration here that I'm not understanding?

The script will live outside of the spec file. So macros expansion will not work. Therefore I suggested to use only these ones. But yes, passing them as positional arguments is option too. And likely better option :)

What is the state of the various bits of automation which would use this? Is there a proposal elsewhere to set up automated checks based on this? Is there a ticket or PR against fedpkg or some adjacent tool to handle passing the information to the script to automate the downloads?

Packit will find this beneficial. Packit can create new version of package when there is upstream release, but only if SOURCE is URL. When it is file there's nothing to do.

And I guess QA can later use it too, to determine if there are potential differences.

Metadata Update from @tibbs:
- Pull-request tagged with: meeting

12 days ago