Conversation
Bumps [certifi](https://github.com/certifi/python-certifi) from 2021.10.8 to 2022.12.7. - [Release notes](https://github.com/certifi/python-certifi/releases) - [Commits](certifi/python-certifi@2021.10.08...2022.12.07) --- updated-dependencies: - dependency-name: certifi dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [cryptography](https://github.com/pyca/cryptography) from 37.0.1 to 38.0.3. - [Release notes](https://github.com/pyca/cryptography/releases) - [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst) - [Commits](pyca/cryptography@37.0.1...38.0.3) --- updated-dependencies: - dependency-name: cryptography dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
no scheduled scripts missig #! on lsf
* Updating versions in lock file. * Increase minimum version of python to 3.8 * Remove missed 3.7 version from matrix. * Update to readd 3.7 and remove fabric. * Remove version limit on pytest. * Updates to versions in lock. * Add 3.11 to the test matrix * Bump Sphinx version.
Signed-off-by: grosa1 <[email protected]>
* Adds discussion on the philosophy and motivations of Maestros design: reproducible computational science * Adds complete documentation of the current study specification * Improved discussion of dependency types available in env block * Documents Maestros minimal workflow language (tokens, step dependencies, ...) * Expands documentation of how to schedule studies using various HPC schedulers * Documentation of command line interface * Comprehensive, progressive tutorial building Maestro workflows from scratch and running them * Exposes multiple variants of the lulesh specs in the sample directory directly in the documentation * Adds mini tutorial for porting existing HPC batch workflows and extending them with Maestro features * Adds how-to guides with more advanced recipes/solutions * Add mkdocstrings to autogenerate api docs from source as part of docs build * Adds workflow topology diagrams using mermaid to better highlight runtime expansion of specs with parameters * New documentation organization/layout inspired by divio documentation system * Removes sphinx dependency --------- Co-authored-by: Jeremy White <[email protected]>
Switch to importlib to get version for maestro's cli from pyproject.toml and remove old out of sync version source
* Pin mermaid to 9.4 due to api breaking change in 10.x * Pin readthedocs poetry install to < 1.5 to avoid python2supports import failure in virtualenv pkg
Fixes dependabot flagged security issues: * Updates lock file to update to certifi version that removes e-Tugra root certificate. * Updates lock file to update requests dependency to patch unintended leak of proxy-authorization header
…it's resource/uri handling (#415) Casts flux's JobID objects into string form to enable the dag to be pickled when conductor saves state. JobID objects are tied to flux's use of cffi and are not pickle-able themselves. Base58 string form is used instead of the integer it's derived from so that logging is consistent with the view users have on the command line with flux reporting job id's in that format. This requires a separate re-instantiation of JobID objects upon waking to check the job status'. Additional fixes: * Improve the flexibility of resource specification in the flux adapter to align with the other adapters (i.e. not requiring nodes be specified). * Change uri handling so env vars are not the only way to schedule jobs * Updates to get working on flux only machines that require no bootstrapping or uri * Changes job submit call to be non-waitable by default on adapter 0.49 and newer as only owner of instance can do so (which is rarely the case on flux managed machines) * Adds documentation and recipes for how to use Maestro with flux machines and flux instances running on non-flux scheduled machines * Pins poetry version to 1.4.2 in CI to maintain support for python 3.7 * Fixes parsing of flux broker version to properly handle non-release versions of flux and logs that version into the flux batch script header comments and the maestro logs.
Adds pager functionality to rich rendered status layouts (not legacy layout) with ability to turn pager and theme on/off to remain compatible with systems/setups that don't have full support.
…hecking (#427) * Adds packaging to toml file as it is now a required non-dev dependency. Version is constrained to >=22.0 to reflect removal of LegacyVersion which can be returned from parse_version in older packaging versions. * Improve robustness of flux adapter version error handling by testing against base_version to allow pre-release builds to succeed. * Add version comparison unit tests for both full and base_version variants
Update v0.49 and v0.26 flux script adapters to adapt cancellation machinery to work with recent update to submit machinery that converts from flux JobID to native types. Update general cancellation behaviors: * Check for in progress steps before declaring cancelled successfully to delay until actual final states can be serialized * Update cancel logic to mirror failure logic: mark all steps downstream of a cancelled step to also be cancelled
Slurm adapter bugfixes * Adds prescribed column lists to both squeue and sacct to avoid parsing issues due to optional user environment variables 'SQUEUE_FORMAT' and 'SACCT_FORMAT'. * Reduces some unneeded columns in sacct to make it more robust (not all machines have accounts, which broke the parsing) Test additions * Enabled detection of slurm scheduler for live test subset (sched_slurm marker) with auto disablement of marked tests to remain quiet in github ci * Unit tests added for slurm adapter * check use of squeue and sacct job status checking options in presence of custom squeue/sacct format env vars * check functionality of job cancellation * Slurm 'hello world' integration test
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resetting/creating new branch structure to enable git flow/smoother release and mergebacks going forward. Main branch is replacing master due to history syncing issues on the release mergebacks into develop (1.1.7/1.1.8, etc end up on top due to some missed mergebacks/version ticks). Main has been spawned off the first 1.1.10dev commit. Master will essentially be frozen from this point on, using git flow with main and develop going forward.