Skip to content

returning original object of outputfiles#2411

Open
sidharth-sinhasane wants to merge 2 commits intoganga-devs:developfrom
sidharth-sinhasane:bugfixing-job-outputfiles-can-not-be-extended
Open

returning original object of outputfiles#2411
sidharth-sinhasane wants to merge 2 commits intoganga-devs:developfrom
sidharth-sinhasane:bugfixing-job-outputfiles-can-not-be-extended

Conversation

@sidharth-sinhasane
Copy link
Copy Markdown
Contributor

fixes: #2190
Job output files can now be extended dynamically.

@sidharth-sinhasane sidharth-sinhasane temporarily deployed to Integrate Pull Request April 6, 2025 19:32 — with GitHub Actions Inactive
Copy link
Copy Markdown
Member

@egede egede left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am afraid that this will not work. In outputfiles we can have wildcards. So consider the following

j = Job()
j.application.exe='touch'
j.application.args=['abcd.txt']
j.outputfiles=['*.txt']

If I print the job object now, I see

Ganga In [11]: j.outputfiles
Ganga Out [11]: [LocalFile(namePattern='*.txt', localDir='')]

so the wildcard is still there. When I now submit the job and look at it after it has completed, I see something different

j.submit()
# wait
Ganga In [17]: j.outputfiles
Ganga Out [17]: [LocalFile(namePattern='abcd.txt', localDir='/home/egede/gangadir/workspace/egede/LocalXML/3/output')]

so you see that it now shows the actual file that was created and not the wildcard. However, if I copy the job (to create a new one)

j = j.copy()
Ganga In [18]: j.outputfiles
Ganga Out [18]: [LocalFile(namePattern='*.txt', localDir='')]

you see that the wildcard is back. This is the correct behaviour (i.e. the new job is a copy of the unsubmitted one).

I do not believe that your change here will have that behaviour. I am not sure if we have tes cases for this, but I will start them now.

@egede
Copy link
Copy Markdown
Member

egede commented Apr 8, 2025

As expected, the testing indeed shows that there now are multiple errors related to the use of wildcards for outputfiles.

@sidharth-sinhasane
Copy link
Copy Markdown
Contributor Author

While running Gangacore/test/Unit and Gangacore/test/GPI same test cases fail with or without these changes. I think we need to research more into it.

@sidharth-sinhasane sidharth-sinhasane temporarily deployed to Integrate Pull Request April 8, 2025 06:23 — with GitHub Actions Inactive
@sidharth-sinhasane
Copy link
Copy Markdown
Contributor Author

sidharth-sinhasane commented Apr 8, 2025

Tried returning proxy of original object.

outputfiles are now can be extended.
Screenshot from 2025-04-08 11-46-24

Also behaving as expected with wildcards.
Screenshot from 2025-04-08 11-37-55

Copy link
Copy Markdown
Member

@egede egede left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not think this latest change will work neither. In the last of your examples, the outputfiles should have "resolved" the wildcard and showed abcd.txt instead.

@sidharth-sinhasane
Copy link
Copy Markdown
Contributor Author

sidharth-sinhasane commented Apr 11, 2025

I tried same with inputfiles
image
showed similar output as outputfiles(unresolved).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Job outputfiles cannot be extended

2 participants