Skip to content

Optmise full test resources for AWS megatest#878

Merged
jfy133 merged 7 commits intodevfrom
optmise-full_test_resources
Oct 14, 2025
Merged

Optmise full test resources for AWS megatest#878
jfy133 merged 7 commits intodevfrom
optmise-full_test_resources

Conversation

@jfy133
Copy link
Copy Markdown
Member

@jfy133 jfy133 commented Oct 11, 2025

During issues in the AWS megatest of our latest release (V5.0.0) resulting in lots of debugging, I realised just how badly the pipeline megatest was configured resource-wise.

I used Seqera Cloud/Platform's 'optimise config' button to refine the amount of resources each process needs, to allow it to run through at lowest cost.

PR checklist

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
  • If you've added a new tool - have you followed the pipeline conventions in the contribution docs
  • If necessary, also make a PR on the nf-core/mag branch on the nf-core/test-datasets repository.
  • Make sure your code lints (nf-core pipelines lint).
  • Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
  • Check for unexpected warnings in debug mode (nextflow run . -profile debug,test,docker --outdir <OUTDIR>).
  • Usage Documentation in docs/usage.md is updated.
  • Output Documentation in docs/output.md is updated.
  • CHANGELOG.md is updated.
  • README.md is updated (including new tool citations and authors/contributors).

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Oct 11, 2025 •
edited
Loading

nf-core pipelines lint overall result: Passed ✅ ⚠️

Posted for pipeline commit 66e99b1

+| ✅ 376 tests passed       |+
#| ❔   1 tests were ignored |#
!| ❗   6 tests had warnings |!
Details

❗ Test warnings:

  • pipeline_todos - TODO string in main.nf: Remove this line if you don't need a FASTA file [TODO: try and test using for --host_fasta and --host_genome]
  • pipeline_todos - TODO string in methods_description_template.yml: #Update the HTML below to your preferred methods description, e.g. add publication citation for this pipeline
  • pipeline_todos - TODO string in main.nf: Optionally add in-text citation tools to this list.
  • pipeline_todos - TODO string in main.nf: Optionally add bibliographic entries to this list.
  • pipeline_todos - TODO string in main.nf: Only uncomment below if logic in toolCitationText/toolBibliographyText has been filled!
  • pipeline_todos - TODO string in nextflow.config: Specify any additional parameters here

❔ Tests ignored:

  • files_unchanged - File ignored due to lint config: .github/PULL_REQUEST_TEMPLATE.md

✅ Tests passed:

Run details

  • nf-core/tools version 3.3.2
  • Run at 2025-10-14 14:28:10

dialvarezs
dialvarezs approved these changes Oct 11, 2025
View reviewed changes
Copy link
Copy Markdown
Member

@dialvarezs dialvarezs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

jfy133
jfy133 commented Oct 13, 2025
View reviewed changes
Copy link
Copy Markdown
Member Author

@jfy133 jfy133 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tweak based on last run

Comment thread conf/test_full.config Outdated
Comment thread conf/test_full.config Outdated
Comment thread conf/test_full.config Outdated
Comment thread conf/test_full.config Outdated
jfy133
jfy133 commented Oct 13, 2025
View reviewed changes
Comment thread CHANGELOG.md Outdated
jfy133
jfy133 commented Oct 13, 2025
View reviewed changes
Comment thread CHANGELOG.md Outdated
jfy133
jfy133 commented Oct 14, 2025
View reviewed changes
Copy link
Copy Markdown
Member Author

@jfy133 jfy133 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Last minor tweak

Comment thread conf/test_full.config Outdated
@jfy133 jfy133 merged commit 0d95ada into dev Oct 14, 2025
4 checks passed
@jfy133 jfy133 deleted the optmise-full_test_resources branch October 14, 2025 14:26
@jfy133 jfy133 mentioned this pull request Oct 25, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dialvarezs dialvarezs dialvarezs approved these changes

@d4straub d4straub Awaiting requested review from d4straub d4straub is a code owner

@prototaxites prototaxites Awaiting requested review from prototaxites prototaxites is a code owner

@muabnezor muabnezor Awaiting requested review from muabnezor muabnezor is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@jfy133 @dialvarezs