Skip to content

Refactor Admin REST API Documentation to use OpenAPI annotations.#20669

Merged
mposolda merged 1 commit intokeycloak:mainfrom
jsorah:wip-update-rest-admin-api-doc-generation
Jun 29, 2023
Merged

Refactor Admin REST API Documentation to use OpenAPI annotations.#20669
mposolda merged 1 commit intokeycloak:mainfrom
jsorah:wip-update-rest-admin-api-doc-generation

Conversation

@jsorah
Copy link
Copy Markdown
Contributor

@jsorah jsorah commented May 30, 2023

Removes dependencies on swagger-doclet
Adds dependencies on microprofile-openapi-api
Plugins for smallrye-open-api-maven-plugin, openapi-generator-maven-plugin

Customized ascii doc template for openapi-generator-maven-plugin, to give similar feel to previous documentation.

OpenAPI annotations added to Admin REST API resources.

Closes #20433

@jsorah
Copy link
Copy Markdown
Contributor Author

jsorah commented May 30, 2023

Quick notes for generating the docs for comparison

cd services && mvn -s ../maven-settings.xml -Pjboss-release clean package

Docs should be generated at services/target/apidocs-rest/output/index.html

And then can be compared with last release API docs at https://www.keycloak.org/docs-api/21.1.1/rest-api/index.html

@jonkoops
Copy link
Copy Markdown
Contributor

Being part of the team that builds the UIs that consume these APIs this would be a huge help to us. It would essentially remove the need for us to maintain a separate Admin Client (JS), as we could use the schema as a means of generating the client itself.

Curious where this will lead.

@jonkoops
Copy link
Copy Markdown
Contributor

See also #10775, #12133, #12313 and #19132 for historical discussion on this feature.

@abstractj
Copy link
Copy Markdown
Contributor

abstractj commented Jun 2, 2023

@jonkoops @jsorah I just wonder if now that we are on Quarkus it would be better to benefit from quarkus-smallrye-openapi and quarkus-swagger-ui so that we would be able to document our endpoints like:

@GET
@Path("/some/endpoint")
@Operation(summary = "Returns a greeting message")
public String something() {
    return "something";
}

More details: https://quarkus.io/guides/openapi-swaggerui

@jsorah
Copy link
Copy Markdown
Contributor Author

jsorah commented Jun 2, 2023

@jonkoops @jsorah I just wonder if now that we are on Quarkus it would be better to benefit from quarkus-smallrye-openapi and quarkus-swagger-ui so that we would be able to document our endpoints like:

@GET
@Path("/some/endpoint")
@Operation(summary = "Returns a greeting message")
public String something() {
    return "something";
}

More details: https://quarkus.io/guides/openapi-swaggerui

That's a good question - I initially was scoping to just offline generation for the documentation and trying to keep the changes a little conservative (though they still involve annotations in all the resource classes). I am not sure if the offline generation would still be possible with the quarkus based annotations, but its something I can look into trying and dropping the other openapi dependency.

I'm a little cautious about inclusion of the Swagger UI - it has had a few XSS vulnerabilities in the past, so at a minimum probably both swagger-ui and openapi spec availability should be controlled via configuration somehow.

@jsorah jsorah force-pushed the wip-update-rest-admin-api-doc-generation branch 2 times, most recently from 84201bc to 7a0219b Compare June 22, 2023 17:55
@jsorah
Copy link
Copy Markdown
Contributor Author

jsorah commented Jun 22, 2023

@abstractj @jonkoops I've updated the branch to use the eclipse microprofile / smallrye-openapi variant of the annotations. The PR is already rather large considering the number of classes that needed to be annotated - I am thinking if we are OK with this approach I can make a separate PR that exposes the actual swagger-ui and openapi spec through the quarkus dist as "opt-in" properties. I have most of that work done locally already.

One issue I ran into had to do with the generic handling of OPTIONS / preflight requests to the admin resources and how that required an "Object" return type on the Realm Admin Resource, which resulted in the annotation scanning to fail finding the relevant classes. I was able to get around the typing issue using a subclass, though this is less than ideal -> https://github.com/keycloak/keycloak/pull/20669/files#diff-c1ffdb124c9481c03397c671cf35661db45cb91019da2a180f4fcaebe92d5b9d

I'm also not sure who else I should pull into this conversation / PR for feedback.

@stianst
Copy link
Copy Markdown
Contributor

stianst commented Jun 26, 2023

Focus for this PR should be on fixing the offline API documentation.

Considering online API documentation is something we can consider later, but there needs to be some reasoning for considering online API docs exposed by the Keycloak Server. Examples of this could be ability to try the API through the browser (would require figuring out how to authenticate though), or if we have dynamic APIs (that's a bigger effort, but in general APIs could get metadata on how the server is configured, what custom providers there are, etc. to generate a bespoke API for a given server/realm). We'd also need an option to enable/disable this, including removing things from the runtime when it's disabled.

@jsorah jsorah force-pushed the wip-update-rest-admin-api-doc-generation branch from 7a0219b to 6d7cea0 Compare June 27, 2023 14:18
@mposolda mposolda requested review from mposolda and rmartinc June 28, 2023 13:44
@mposolda mposolda self-assigned this Jun 28, 2023
@mposolda mposolda marked this pull request as ready for review June 28, 2023 13:44
@mposolda mposolda requested a review from a team as a code owner June 28, 2023 13:44
@mposolda mposolda changed the title DRAFT: Refactor Admin REST API Documentation to use OpenAPI annotations. Refactor Admin REST API Documentation to use OpenAPI annotations. Jun 28, 2023
@jsorah jsorah force-pushed the wip-update-rest-admin-api-doc-generation branch from 6d7cea0 to f5806e6 Compare June 28, 2023 14:09
Copy link
Copy Markdown
Contributor

@mposolda mposolda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jsorah The PR looks great, Thanks for it!

I've added only one minor comment inline, could you please check?

Additional comment (not a blocker for this PR, but rather some possible follow-up and/or discussion point): I am not so big fan of the fact that @Operations and @Parameter annotations need to repeat the same stuff, which is mentioned in the javadoc of the methods of Admin REST API. IMO it can be nice if the Java classes in Keycloak codebase do not need to have any openapi annotations. Then somehow (likely in some "pre-compile" step), we will be able to add the stuff from Javadoc automatically to those annotations and generate Java classes with the annotations at this pre-compile phase. Then finally compile phase will just compile those Java classes. We can investigate if there is some plugin, which can already do it, or we can perhaps try to do it on our own? AFAIK for instance new store team has some approach with some "Generated" java classes, which are then compiled. But IMO this can be a follow-up task. I don't see it as a blocker for this PR. Also not 100% sure what others thinks about this (Not 100% sure if this is relatively big effort with not so big small value. The amount of effort is something, which I don't know...).

Also I don't know where/if this can be documented in the other place than the comment on this github :-) I think we need to use this during the Keycloak release and maybe other places too. For now, I think it is good to have this PR as is. We can discuss documenting this as a follow-up if needed.

Comment thread services/pom.xml Outdated
Copy link
Copy Markdown
Contributor

@rmartinc rmartinc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the minor issue commented by @mposola about the dependency version, this looks good to me.

The PR works OK, it generates the index.html file and it's extraordinarily similar to the previous one. I have not checked all the methods one by one, but I couldn't find a missing piece, and they look nice. If there are minor problems or missing parts, we can tune/refine the texts/annotations later with subsequent issues/PRs.

About swagger annotations, I think that swagger just works this way, take it of leave it. There are things like enunciate, but I would go with the real swagger thing. My vote is for removing (not taking care of) the javadocs, to not maintain two different things, and relying only on swagger annotations for the REST api.

We can even add later a startup option (or similar) to give the online version in the server. We would need to manage the token/authentication to make it really usable.

Thanks @jsorah for the thorough PR! It's a very good work!

@mposolda
Copy link
Copy Markdown
Contributor

@rmartinc Yes, removing javadoc can be also an option. I am not sure how much people rely on the javadoc for example during coding. I am personally not much and since things would be documented in the code with openapi annotations, it is possibly fine? But no need to change anything in this PR (Just though for the future...)

Removes dependencies on swagger-doclet
Adds dependencies on microprofile-openapi-api
Plugins for smallrye-open-api-maven-plugin, openapi-generator-maven-plugin

Customized ascii doc template for openapi-generator-maven-plugin, to give similar feel to previous documentation.

OpenAPI annotations added to Admin REST API resources.

Closes keycloak#20433
@jsorah jsorah force-pushed the wip-update-rest-admin-api-doc-generation branch from f5806e6 to 28bba2b Compare June 29, 2023 12:06
@jsorah jsorah requested a review from a team as a code owner June 29, 2023 12:06
@ghost ghost added the team/ui label Jun 29, 2023
@jsorah
Copy link
Copy Markdown
Contributor Author

jsorah commented Jun 29, 2023

@rmartinc @mposolda - Thanks for the reviews - I've updated the dependency management bits as suggested. I was also not sure what to do about the Javadoc since now its basically duplicated but it may have been used by folks too. I'll have to look at what the store team has done and see if theres an approach we can go with as a followup.

I think @jonkoops your team is now tagged for review too because I did modify the rest/admin-ui-ext/pom.xml to take advantage of the dependency management in the top-level pom.xml - there is net-no change for that module as it is using the same versions as before.

Copy link
Copy Markdown
Contributor

@mposolda mposolda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jsorah Thanks! I am approving

@mposolda mposolda merged commit f695eea into keycloak:main Jun 29, 2023
@jonkoops
Copy link
Copy Markdown
Contributor

Thanks for landing this!

@abstractj
Copy link
Copy Markdown
Contributor

Thank you all for including this, it is amazing to see this happening. Thank you @jsorah.

@omasseau
Copy link
Copy Markdown

omasseau commented Oct 5, 2023

Just to understand, this means we have an OpenAPI endpoint in latest versions of Keycloak ?
Or the endpoint to access the OpenPAPI documentation is not available yet ?

If it is available, what is the endpoint ? I cannot find any documentation on this.

@jsorah
Copy link
Copy Markdown
Contributor Author

jsorah commented Oct 5, 2023

Just to understand, this means we have an OpenAPI endpoint in latest versions of Keycloak ? Or the endpoint to access the OpenPAPI documentation is not available yet ?

If it is available, what is the endpoint ? I cannot find any documentation on this.

Instead of using Javadoc to generate the REST Admin API documentation it was switched over to OpenAPI annotations to get something similar, because the previous plugin to do so had various incompatibility issues.

There isn't an OpenAPI spec published as part of this change except as an intermediate artifact in generating the documentation. I believe there is an open PR to add this generated spec to the documentation, but I think it probably needs some more work before someone generates actual clients from the spec.

@jsorah jsorah deleted the wip-update-rest-admin-api-doc-generation branch November 10, 2023 15:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Administration / Keycloak Admin REST API documentation can no longer be generated

7 participants