improvements to controller/fsm disconnection handling#883
Open
improvements to controller/fsm disconnection handling#883
Conversation
make exclude work if disconnected
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes issue #516
Several improvements and fixes to the controller, particularly disconnection handling.
Type of change
Change log
statuscommand now checks that there is a valid connection to all children, showing them asdisconnectedwithout changing the internal state if they can't be reached. Disconnected children will no longer be marked asin errorexcludethe disconnected child if you want to carry on without it. If youexcludeall disconnected children, fsm commands can go ahead.statuswill show not in error, fsm commands are possible) see this comment for an example. This is done by refreshing the new rest endpoint from the connectivity service if a connection couldn't be established, similar to the connection refreshing for gRPC children.Suggested manual testing checklist
Note: you may need to do status twice, re-connection is asynchronous and doesn't happen instantly
Same as 2, observe that the parent
df-controllerwill be configuredDeveloper checklist
Prior to marking this as "Ready for Review"
Tests ran on: hep cluster from develop
Unit tests - some tests can't be ran on the CI. This is documented. If this PR checks a feature that can't be tested with CI, this has been marked appropriately.
Integration tests - the
daqsystemtest_integtest_bundlerequires a lot of resources, and connections to the EHN1 infrastructure. Check the cross referenced list if you can't run these. The developer needs to run at least the .pytest --marker) passeddaqsystemtest_integtest_bundle.sh -k minimal_system_quick_test.pydaqsystemtest_integtest_bundle.shFinal checklist prior to marking this as "Ready for Review"
Reviewer checklist
druncare in the log filesdruncfailure appears:Once the features are validated and both the unit and integration tests pass, the PRs is ready to be merged.
Choose one of the following an complete all substepsPrior to merging
Once completed, the reviewer can merge the PR.
Notification message for a Slack channel
Note - this should be to #dunedaq-integration for general workflow that isn't during a release candidate period, and to #daq-release-prep otherwise.
For an single merge that changes the user workflow
For co-ordinated merge