Skip to content

Add --outdir parameter fallback to extend container image download support to older pipelines#4212

Open
MatthiasZepper wants to merge 6 commits intonf-core:devfrom
MatthiasZepper:fix/param_outdir_inspect
Open

Add --outdir parameter fallback to extend container image download support to older pipelines#4212
MatthiasZepper wants to merge 6 commits intonf-core:devfrom
MatthiasZepper:fix/param_outdir_inspect

Conversation

@MatthiasZepper
Copy link
Copy Markdown
Member

@MatthiasZepper MatthiasZepper commented Apr 23, 2026

Since we switched from the regex-based container image extraction to running nextflow inspect on the downloaded pipelines, particularly older pipeline revisions caused issues.

By convention nf-core pipelines expect an --outdir parameter. If there was no default specified in the test profile of a revision, extraction of the container images failed:

ERROR    Running 'nextflow inspect' failed with the following error                                                                                                                              
ERROR    Command 'nextflow inspect -format json -profile singularity,test,test_full /home/matthias/.config/nfcore/nf-core/rnaseq/main.nf' returned non-zero error code '1':                      
         > [0;31mThe following invalid input values have been detected:                                                                                                                          
                                                                                                                                                                                                 
         * Missing required parameter: --outdir                                                                                                                                                  
         [0m                                                                                                                                                                                     
          -- Check script '/home/matthias/.config/nfcore/nf-core/rnaseq/main.nf' at line: 52 or see 'null' file for more details                                                                 
         ERROR ~ ERROR: Validation of pipeline parameters failed!

This PR now introduces a silent fallback exactly for this problem. For the user, the retry is silent, so they do not even notice that they can now successfully download a pipeline revision they could previously only obtain by pinning and using an older version of nf-core tools.

To test, e.g. try the following command with this patched version vs. the current dev:

nf-core pipelines download -r "3.14.0" -s "singularity" -u "amend" -x "none" rnaseq

PR checklist

  • This comment contains a description of changes (with reason)
  • CHANGELOG.md is updated
  • If you've fixed a bug or added code that should be tested, add tests!
  • Documentation in docs is updated

@MatthiasZepper MatthiasZepper changed the title Add --outdir parameter fallback to extend container download image support to older pipelines Add --outdir parameter fallback to extend container image download support to older pipelines Apr 23, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 23, 2026 •
edited
Loading

Codecov Report

❌ Patch coverage is 88.46154% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.36%. Comparing base (7744f96) to head (eacd60f).

Files with missing lines Patch % Lines
nf_core/pipelines/download/download.py 88.46% 3 Missing ⚠️

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@MatthiasZepper MatthiasZepper force-pushed the fix/param_outdir_inspect branch from d36cde1 to d38eb0d Compare April 23, 2026 10:28
@MatthiasZepper MatthiasZepper marked this pull request as ready for review April 23, 2026 10:28
@MatthiasZepper MatthiasZepper force-pushed the fix/param_outdir_inspect branch from d38eb0d to 97bc3a9 Compare April 23, 2026 10:32
Copilot AI reviewed Apr 23, 2026
View reviewed changes
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a fallback when extracting container images via nextflow inspect fails for older nf-core pipeline revisions that require --outdir to be set, improving download compatibility without user-visible changes.

Changes:

  • Retry nextflow inspect with a temporary -params-file providing outdir when the failure indicates a missing --outdir.
  • Add unit tests covering the retry path and non-retry error propagation.
  • Update CHANGELOG.md to mention the new compatibility behavior.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
nf_core/pipelines/download/download.py Implements nextflow inspect retry using a temporary params file when --outdir appears to be required.
tests/pipelines/download/test_download.py Adds tests validating the retry behavior and correct error handling for unexpected failures.
CHANGELOG.md Documents the download behavior change for older pipelines.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tests/pipelines/download/test_download.py Outdated
Comment thread CHANGELOG.md Outdated
Comment thread nf_core/pipelines/download/download.py Outdated
mashehu
mashehu approved these changes Apr 23, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@mashehu mashehu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Comment thread nf_core/pipelines/download/download.py Outdated
Comment thread CHANGELOG.md Outdated
@MatthiasZepper MatthiasZepper force-pushed the fix/param_outdir_inspect branch from 97bc3a9 to fe7bf46 Compare April 24, 2026 16:33
@MatthiasZepper @mashehu
Apply suggestions from code review
eacd60f 
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Matthias Hörtenhuber <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mashehu mashehu mashehu approved these changes

Copilot code review Copilot Copilot left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@MatthiasZepper @mashehu @nf-core-bot