Skip to content

feat(internal/librarian/java): appends to versions.txt when new modules is added in generate#5660

Open
zhumin8 wants to merge 13 commits intogoogleapis:mainfrom
zhumin8:update-versions-txt
Open

feat(internal/librarian/java): appends to versions.txt when new modules is added in generate#5660
zhumin8 wants to merge 13 commits intogoogleapis:mainfrom
zhumin8:update-versions-txt

Conversation

@zhumin8
Copy link
Copy Markdown
Contributor

@zhumin8 zhumin8 commented Apr 27, 2026

Appends version line to versions.txt file for each new module added in generate. The implememtation takes advantage that syncPOMs already detects which module is missing, and alters java.Generate to return a list of newly created artifact version entries to be added to versions.txt. Then as a monorepo level post generate task, append these to versions.txt which is at the repo root.

Context: versions.txt is a release-please file that tracks the released version and snapshot version for artifacts. Until Java onboards to librarian for releases, we need to update this versions.txt file when new artifacts is added by generate. This happens when new library is introduced or new version API path is added to existing libraries.

Fix #4927

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request modifies the Java library generation process to track and record newly created artifact versions. It updates the Generate, postProcessLibrary, and syncPOMs functions to return a list of version strings, which are then appended to versions.txt during the PostGenerate phase. The review feedback highlights an empty test function that should be removed or implemented and suggests adding the os.O_CREATE flag when opening versions.txt to ensure the file is created if it does not exist.

Comment thread internal/librarian/java/pom_test.go Outdated
Comment thread internal/librarian/java/postgenerate.go Outdated
@zhumin8 zhumin8 marked this pull request as ready for review April 27, 2026 19:18
@zhumin8 zhumin8 requested a review from a team as a code owner April 27, 2026 19:18
Comment thread internal/filesystem/filesystem.go Outdated
return command.Run(ctx, "unzip", "-q", "-o", src, "-d", dest)
}

// AppendLines appends the given lines to an existing file, ensuring that it
Copy link
Copy Markdown
Contributor

@sofisl sofisl Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this function for linting? If so, a few questions (again mostly for me to learn):

  1. Is there no go linting tool that adds a newline automatically, if that's what we're trying to account for?
  2. Is there a go library that adds a newline?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. No, this function is used to append new entries to versions.txt file. versions.txt exists in google-cloud-java, and although not encouraged, can be edited by human and I believe does not have a check for trailing empty line.
  2. I am not aware of. I think we need to read in the file and write.

Comment thread internal/filesystem/filesystem_test.go Outdated
Comment thread internal/filesystem/filesystem.go Outdated
Comment thread internal/filesystem/filesystem.go Outdated
// AppendLines appends the given lines to an existing file, ensuring that it
// ends with a newline character before appending. It returns an error if the
// file does not exist.
func AppendLines(path string, lines []string) (err error) {
Copy link
Copy Markdown
Member

@julieqiu julieqiu Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about simplifying this function to this:

  func AppendLines(path string, lines []string) error {
        if len(lines) == 0 {
                return nil
        }
        existing, err := os.ReadFile(path)
        if err != nil {
                return err
        }

        var buf bytes.Buffer
        buf.Write(existing)
        if len(existing) > 0 && existing[len(existing)-1] != '\n' {
                buf.WriteByte('\n')
        }
        for _, line := range lines {
                buf.WriteString(line)
                buf.WriteByte('\n')
        }
        return os.WriteFile(path, buf.Bytes(), 0644)
  }

This will rewrite the whole file each time, but for a small file like versions.txt it seems cleaner

What I don't quite understand is what we need to append the newline at all though?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why we need to append the newline at all though?

It's more of a safety check. Although not encouraged, this versions.txt can be edited by human and I believe there is no check for trailing empty line in google-cloud-java. Alternatively, we can add a dedicated workflow in google-cloud-java to check for it. But I think it's easier to check when attempting to update.

Comment thread internal/filesystem/filesystem_test.go Outdated
Comment thread internal/librarian/java/pom.go Outdated
Comment thread internal/librarian/java/pom.go Outdated
}
}

releasedVersion, err := deriveLastReleasedVersion(library.Version)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what if a library has no versions to parse?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A Java library should always have version. librarian add sets a default version

lib.Version = defaultVersion

Comment thread internal/librarian/java/postprocess_test.go Outdated
@zhumin8 zhumin8 requested review from julieqiu and sofisl April 28, 2026 18:52
var allNewVersions []string
for _, library := range libraries {
if err := java.Generate(ctx, cfg, library, src); err != nil {
newVersions, err := java.Generate(ctx, cfg, library, src)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should try to keep the same API interface across languages for Generate. what about moving the post generate logic inside java.Generate?

// Generate generates a Java client library.
func Generate(ctx context.Context, cfg *config.Config, library *config.Library, srcs *sources.Sources) error {
// Generate generates a Java client library and returns a list of newly created
// artifact version entries to be added to versions.txt.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see we will stop adding multiple lines into version txt for proto- and grpc- artifacts #4650.

Copy link
Copy Markdown
Contributor Author

@zhumin8 zhumin8 Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope. #4650 is to unify the versions. No changes to versions.txt structure. We can potentially do that, but it does not change that we still need append to versions.txt when new library onboards.


// Generate generates a Java client library.
func Generate(ctx context.Context, cfg *config.Config, library *config.Library, srcs *sources.Sources) error {
// Generate generates a Java client library and returns a list of newly created
Copy link
Copy Markdown
Member

@suztomo suztomo Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you define the canonical way within Java to detect a library is new or not?

(I believe that the responsibility of detecting the condition is not in Generate function.)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@julieqiu your comment https://github.com/googleapis/librarian/pull/5660/changes#r3156502889 is related, I'll bring this discussion together:

  • Currently, Generate does post processing steps that generates pom.xml files when it is a new library (that is where new library/module is detected and bubbled up).
  • I choose to change Generate to return these version entries and let java.PostGenerate write to versions.txt, because versions.txt is located at repo root, and it's natural to update it once for the whole generation. It should be technically possible to move inside java.Generate and workout the parent dir of outdir, but it feels a bit wrong as a per library function updates a repo level file, and may be problematic if we seek to do parallel in the future.
  • By the time it gets to java.PostGenerate, pom.xml are already added, I don't see a way to identify which library is new. @suztomo any ideas you have in mind?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

java: add entry to versions.txt if new modules added

4 participants