JUnit or QuarkusTest for custom Authenticator class? #11841

jfrantzius · 2022-05-05T07:00:56Z

jfrantzius
May 5, 2022

Hi,
we developed a custom org.keycloak.authentication.Authenticator class, and I am now looking for a way of writing automated tests for it. I looked at https://stackoverflow.com/questions/45492999/how-to-unit-test-a-new-keycloak-authenticator , which takes the road of mocking the inner parts of Keycloak that are required for the class to operate. That seems risky to me, as it is based on a lot of assumptions on the inner workings of Keycloak, which may simply differ from runtime reality, and which are also quite tedious to find out plus having the risk to stumble over some static Keycloak method that cannot be mocked.

Now that Keycloak has moved to Quarkus, and having worked with QuarkusTest a lot, I wonder if somebody already has managed to write a QuarkusTest that runs Keycloak e.g. against an in-memory database, so a custom class can be tested inside of a "real" running Keycloak?

I found QuarkusTest to be very very helpful in testing Quarkus applications, as they allow to easily test logic in the context a whole runtime stack, avoiding lots of mocking and the mistakes that may come with it. I think it would be very helpful for the development of custom Keycloak extensions if they could be tested using QuarkusTest!

Cheers to the Keycloak community,
Jörg

abstractj · 2022-05-11T15:55:11Z

abstractj
May 11, 2022
Maintainer

@jfrantzius I believe that we have been using QuarkusTest for the Keycloak Operator. You should be able to find some examples here: https://github.com/keycloak/keycloak/search?q=QuarkusTest

@andreaTP @DGuhr certainly know more than me about all the details.

0 replies

andreaTP · 2022-05-11T16:17:13Z

andreaTP
May 11, 2022

@jfrantzius in the Quarkus integration tests sub-module you can actually find 2 implementations of what you are describing, based on the "raw" Quarkus distribution and on Testcontainers:
https://github.com/keycloak/keycloak/tree/main/quarkus/tests

But those implementations are intended to be used in this very repository, and making them "portable" would require considerable effort.

Do you think that using Testcontainers is a viable alternative for you?
If that's the case you can get some inspiration from https://github.com/keycloak/keycloak/blob/main/quarkus/tests/integration/src/main/java/org/keycloak/it/utils/DockerKeycloakDistribution.java
It nicely plays with QuarkusTest too.

1 reply

andreaTP May 11, 2022

forgot to mention, starting the new distribution with an in-memory DB is as easy as passing the option --db=dev-mem

DGuhr · 2022-05-11T18:43:37Z

DGuhr
May 11, 2022

As the others have mentioned, please take a look at our testsuite, or e.g. projects like https://github.com/dasniko/testcontainers-keycloak or https://github.com/thomasdarimont/keycloak-project-example :)

0 replies

jfrantzius · 2022-05-12T12:44:35Z

jfrantzius
May 12, 2022
Author

Thank you @abstractj , @andreaTP and @DGuhr for your answers. Those solutions all seem to require building and running a Docker image first before running any test.

What I was looking for is that mvn test in my Keycloak extension's Maven project will start up Keycloak without having to build a Docker image first, and then run the QuarkusTest against it. This is the usual way of working when developing a Quarkus application, as it allows for very fast turnaround times. It should also be possible to run the Quarkus application (Keycloak) using mvn quarkus:dev (Quarkus Dev Mode), allowing to change Java classes at runtime without server restart.

I eventually found out how to do this. I modified my pom.xml to include the quarkus-maven-plugin and added this dependency:

        <dependency>
            <groupId>org.keycloak</groupId>
            <artifactId>keycloak-quarkus-server-app</artifactId>
            <version>${keycloak.version}</version>
        </dependency>

Keycloak can now be started in Quarkus Dev Mode like this, as documented in https://github.com/keycloak/keycloak/tree/main/quarkus#running-in-keycloak-development-mode :

mvn compile quarkus:dev -Dquarkus.args="start-dev"

In the log I can see that Keycloak is also picking up my extension Java class, which is great. I haven't written a QuarkusTest yet, but I'm positive that this will be possible now :)

0 replies

DGuhr · 2022-05-12T13:02:58Z

DGuhr
May 12, 2022

@jfrantzius unfortunately, that's not possible atm. I think @thomasdarimont opened up a discussion around this here on the quarkus-dev mailing list, but there was no follow-up, and sadly as of now we didn't come back to that.

cc @pedroigor

11 replies

thomasdarimont Jun 2, 2022
Collaborator

Hi, no you don't need to build the docker Image in order to test custom providers. The custom provider classes are mounted from target/classes via dynamically mounted shrinkwrap .jar generated via Testcontainers-keycloak.

dasniko Jun 2, 2022

Yes, I can confirm this (as I'm the maintainer of testcontainers-keylcoak and @thomasdarimont introduced some time ago the feature to test extensions with the testcontainer). No need to build a custom image. But still a Keycloak "build" step is needed (takes some time) and with every change at the extension code, the container has to be restarted to execute the "build" step again.

It would be great, if Keycloak would support some kind of auto-restart when "things" have changed when running in dev mode. This would increase the developer-experience a lot when developing for and with Keycloak.

jfrantzius Jun 2, 2022
Author

@thomasdarimont alright, sorry I was immediately thinking that if testcontainers is used, the extensions would have to be baked into the image.
That's of course great to avoid building time of the Docker image, but from a developer experience point of view I still think it would be easier to understand, setup and more "standard" Quarkus if one could simply use @QuarkusMainTest...

pedroigor Jun 2, 2022
Collaborator

@jfrantzius +1, that is the best experience we can provide. @thomasdarimont work is a not yet leveraging Quarkus capabilities but a very nice start towards productivity when developing custom providers.

What I would like to have from Keycloak, as a developer is (please share your thoughts too):

Create a brand new maven project (gradle later)
Import available dependencies and their versions from a Keycloak BOM file. The BOM file holds all steps available at runtime. Or perhaps a parent POM similar to those Spring starters.
Run a quarkus:dev or keycloak:dev maven goal to not only run in Quarkus dev mode but also to start a KC runtime for dev purposes in the same JVM. Or using Quarkus dev services and testcontainers. I think JVM would be ideal and faster.
Package my project and deploy the artifact to the providers dir.

As a bonus point, I would also like to have some tooling that is able to infer 3rd party libraries not available from the distribution so that I can also deploy them to the server and make them available at runtime.

From a testing perspective, I think we have a lot of things to share from our testsuite. We basically need to make it available for public usage.

jfrantzius Jun 3, 2022
Author

That sounds awesome :)

Concerning point 3, in my experiment here #11841 (comment) this seems to work just by adding a dependency and configuring Quarkus maven-plugin. Keycloak then runs in Quarkus Dev mode directly in a JVM, without a Docker container (if that is what you are referring to by "JVM would be ideal and faster").

Personally I'd mostly be looking for an example @QuarkusMainTest that leverages this, showing how to use things from the testsuite.

This is an excerpt from my pom.xml that I'm using for the above:

    <properties>
        <keycloak.version>18.0.0</keycloak.version>
        <jboss-jaxrs-api_2.1_spec>2.0.1.Final</jboss-jaxrs-api_2.1_spec>
        <maven.compiler.source>11</maven.compiler.source>
        <maven.compiler.target>11</maven.compiler.target>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <quarkus.platform.artifact-id>quarkus-universe-bom</quarkus.platform.artifact-id>
        <quarkus.platform.group-id>io.quarkus</quarkus.platform.group-id>
        <quarkus.platform.version>2.8.3.Final</quarkus.platform.version>
        <quarkus-plugin.version>2.8.3.Final</quarkus-plugin.version>
    </properties>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>${quarkus.platform.group-id}</groupId>
                <artifactId>${quarkus.platform.artifact-id}</artifactId>
                <version>${quarkus.platform.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <dependencies>
        <dependency>
            <groupId>org.keycloak</groupId>
            <artifactId>keycloak-quarkus-server-app</artifactId>
            <version>${keycloak.version}</version>
        </dependency>        
        <dependency>
            <groupId>org.keycloak</groupId>
            <artifactId>keycloak-server-spi</artifactId>
            <version>${keycloak.version}</version>
        </dependency>
        <dependency>
            <groupId>org.keycloak</groupId>
            <artifactId>keycloak-server-spi-private</artifactId>
            <version>${keycloak.version}</version>
        </dependency>
        <dependency>
            <groupId>org.keycloak</groupId>
            <artifactId>keycloak-core</artifactId>
            <version>${keycloak.version}</version>
        </dependency>
        <dependency>
            <groupId>org.keycloak</groupId>
            <artifactId>keycloak-services</artifactId>
            <version>${keycloak.version}</version>
        </dependency>
        <dependency>
            <groupId>org.jboss.spec.javax.ws.rs</groupId>
            <artifactId>jboss-jaxrs-api_2.1_spec</artifactId>
            <version>${jboss-jaxrs-api_2.1_spec}</version>
        </dependency>
    </dependencies>
    <build>
        <finalName>${project.artifactId}</finalName>
        <pluginManagement>
            <plugins>
                <plugin>
                    <groupId>io.quarkus</groupId>
                    <artifactId>quarkus-maven-plugin</artifactId>
                    <version>${quarkus-plugin.version}</version>
                    <extensions>true</extensions>
                    <executions>
                        <execution>
                            <goals>
                                <goal>build</goal>
                                <goal>generate-code</goal>
                                <goal>generate-code-tests</goal>
                            </goals>
                        </execution>
                    </executions>
                </plugin>
            </plugins>
        </pluginManagement>
        <plugins>
            <plugin>
              <groupId>io.quarkus</groupId>
              <artifactId>quarkus-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

JUnit or QuarkusTest for custom Authenticator class? #11841

Uh oh!

jfrantzius May 5, 2022

Replies: 5 comments · 12 replies

Uh oh!

abstractj May 11, 2022 Maintainer

Uh oh!

andreaTP May 11, 2022

Uh oh!

andreaTP May 11, 2022

Uh oh!

DGuhr May 11, 2022

Uh oh!

jfrantzius May 12, 2022 Author

Uh oh!

DGuhr May 12, 2022

Uh oh!

thomasdarimont Jun 2, 2022 Collaborator

Uh oh!

Uh oh!

dasniko Jun 2, 2022

Uh oh!

jfrantzius Jun 2, 2022 Author

Uh oh!

Uh oh!

pedroigor Jun 2, 2022 Collaborator

Uh oh!

Uh oh!

jfrantzius Jun 3, 2022 Author

jfrantzius
May 5, 2022

Replies: 5 comments 12 replies

abstractj
May 11, 2022
Maintainer

andreaTP
May 11, 2022

DGuhr
May 11, 2022

jfrantzius
May 12, 2022
Author

DGuhr
May 12, 2022

thomasdarimont Jun 2, 2022
Collaborator

jfrantzius Jun 2, 2022
Author

pedroigor Jun 2, 2022
Collaborator

jfrantzius Jun 3, 2022
Author