#286 RFE: `rust2rpm -p` for bundled vendored dependencies
Opened 5 months ago by dmellado. Modified 3 months ago

Extend rust2rpm -p support for vendored dependencies.

Enable a -p option in rust2rpm to activate a mode where:

  • Given a specific dependency to patch (e.g., fiat-crypto), it will locate the vendor directory and apply the patch to the existing Cargo structure. This will automate the manual process of:
    1. Navigating to the unpacked sources directory
    2. Running cargo vendor --versioned-dirs
    3. Modifying the fiat-crypto crate
    4. Repacking the tarball using tar -Jcvf vendor/ ../tarball.xz

  • Additional options will be introduced to specify:

  • The dependency to patch
  • The corresponding patch file

  • If the operation fails:

  • Exit with an error (exit code 1)
  • Do not open an editor
  • Display an appropriate error message before quitting

  • If successful:

  • Apply the patch (*.diff) to the vendored dependency
  • Repackage the tarball with the modified dependencies

Use Case

This extends the rust -p behavior to handle not only RPM-based dependencies but also simple vendored dependencies, with options for specifying the dependency and patch file, automating the process in a straightforward manner.


I've given this some thought, and I'm not sure that rust2rpm can (or should) accomodate every bespoke workflow here. It's already the most complicated piece of software that I've ever maintained, and the use case you're describing is very niche. To my knowledge, only 1-2 packages in Fedora use vendored sources and need to patch the contents of the vendor tarball.

Would something like this work for your use case instead:

  1. In addition to --vendor (which generates the vendor tarball internally without ability to influence the process), rust2rpm gains a new flag, something like rust2rpm --vendor-myself (name TBD).
  2. This flag causes a spec file to get generated in the same way as when using --vendor, but doesn't actually generate the vendor tarball automatically.
  3. A vendor tarball with the expected file name and internal directory structure can be created with a separate process / script.

This would allow for maximum flexibility without needing to implement all of it in rust2rpm itself.

We could even provide some kind of separate helper program (rust2rpm-vendor, name TBD) to automate parts of this process so it would be scriptable:

  • rust2rpm-vendor prepare: downloading vendored dependencies
  • (make modifications / apply patches / remove objectionable content here)
  • rust2rpm-vendor checksumfix: fixing up .cargo_checksum.json for modified files
  • rust2rpm-vendor compress: compress vendored dependencies into tarball with expected name / file structure

What do you think?

Metadata Update from @decathorpe:
- Issue assigned to decathorpe
- Issue set to the milestone: v27.0
- Issue tagged with: feature

4 months ago

Hi @decathorpe,

I think that could totally work for now. Having the ability to basically decouple the vendorfile from rust2rpm itself.

We can start by creating this ticket into the --vendor-myslef flag, which should be pretty much straightforward.

Regarding the rust2rpm-vendor tool those steps are basically what I also had in mind. Would it be possible to also have that under pagure.io/rust2rpm or would you prefer to host it elsewhere?

Thanks!

We can start by creating this ticket into the --vendor-myslef flag, which should be pretty much straightforward.

I hope so. This ticket is already on list for the rust2rpm v27.0.0 milestone, so I'll get to it eventually.

Regarding the rust2rpm-vendor tool those steps are basically what I also had in mind. Would it be possible to also have that under pagure.io/rust2rpm or would you prefer to host it elsewhere?

It can be in this same repository, but I'd like it to provide a different CLI entry point. The current CLI interface already does many different things, and I'm trying to remove unrelated things rather than adding more :)

I've implemented part of this RFE in this commit: https://pagure.io/fedora-rust/rust2rpm/c/429b54b

The --vendor flag from rust2rpm ≤ v26 becomes an option with an argument in rust2rpm v27 - accepting values of off, auto, and manual, with

  • --vendor=off and not passing the --vendor argument being equivalent to not passing --vendor in rust2rpm ≤ v26,
  • --vendor=auto being equivalent to passing --vendor in rust2rpm ≤ v26, and
  • --vendor=manual causing new behaviour:

The manual mode assumes usage of a vendor tarball (with the expected default name and folder structure) but does not actually create it.

This does not change the default behaviour (not passing --vendor works the same), and allows more control over rust2rpm behaviour in the non-default case.

With this, the parts that need to be implemented in rust2rpm directly are done - so I'm removing this from the v27 milestone. Other helper functionality for creating vendor tarballs can be implemented independently.

It would also be easy to add other modes in the future in a non-breaking way (for example, a "filtered" mode using cargo-vendor-filterer in a instead of plain "cargo vendor").

Metadata Update from @decathorpe:
- Issue set to the milestone: None (was: v27.0)

3 months ago

Log in to comment on this ticket.

Metadata