KC 17 should be out December yes, it may slip to January though.

KC 16 should have been out already, but we're having some challenges with WildFly upgrades. We hope to have it out very soon.

We are eagerly awaiting for KC 16 release to try out the front channel logout feature.

Taking reference from https://www.keycloak.org/2021/10/keycloak-x-update.html, As Keycloak 17 missed the timeline mentioned in Roadmap.
@stianst, @thomasdarimont Can I request to share some details related to the release for Keycloak 18? Can we still expect Keycloak 18 release in March 2022?

Yes, standard remote debugging into a KCX container also works for me, but I struggle with hot code replacement somehow.

A dedicated hot deployment support for reloading newly deployed providers would be very nice, indeed!
Perhaps this could be enabled with a switch or implicitly when enabling debugging with the DEBUG env var.
For me personally, the "exploded-jar" option is not that important, but that's just the way one (me) is used to work and develop. In any case it'd be a lovely feature!

@dasniko We want/need to improve our dev story. As you know, right now we are not leveraging all capabilities from Quarkus like live reload as well as dev services.

Once we have the core server bits in place, we should focus on this area.

@dasniko One thing we have been discussing is to provide a BOM and potentially some tooling (e.g.: maven/gradle plugins) that allows you to easily start a project and run an embedded Keycloak.

Right now, you can achieve live reload and develop custom providers if you write code within the server module. I think that is the experience we would like to achieve but without having to rely on the main code base.

@pedroigor Thanks for your answer.

We want/need to improve our dev story.

Great to hear 👍, I‘m really looking forward to a better Developer Experience! 🙂

One thing we have been discussing is to provide a BOM and potentially some tooling (e.g.: maven/gradle plugins) that allows you to easily start a project and run an embedded Keycloak.

Having a BOM and some tooling is nice, but most of it is convenience. 😇 For me, from a developer perspective, the killer feature is hot-/live-reload.

Right now, you can achieve live reload and develop custom providers if you write code within the server module. I think that is the experience we would like to achieve but without having to rely on the main code base.

Running an embedded Keycloak (or even developing in the code base, which is IMHO NOT an alternative) might be not enough in some cases. I have a few customers where I do a lot customization with various custom SPI implementations. Theses also require some Keycloak configuration and some other services/infrastructure around. So, having a "real" Keycloak running in a container network with other required services for the use case, deploying the SPI extensions to the container and having hot-/live-reload is necessary for these environments.

For simple use-cases it might be enough to have something like an embedded Keycloak, but I doubt that this will work for all the cases.

I see.

I think there are two main usages:

• Embedded
• Container

As each one has its own audience.

I think I agree that we should initially focus on container usage as it is more aligned with the goals of Keycloak.X.

That said, I think we should be able to leverage Keycloak Dev Services from Quarkus. If not fully, with some tweaks to make it aligned with our goals.

If not possible to reload code in a live container we should be able to at least restart it with the updated code (hoping we do fast restarts).

But yeah, I'm also expecting this a lot :)

Yes, take a look at: quay.io/keycloak/keycloak-x:15.1.0

🤦🏻 thank you! I was looking at keycloak/keycloak!

There is currently no explicit support for adding additional JVM options like JAVA_OPTS_APPEND, but you can use JAVA_TOOL_OPTIONS in the meantime.

Thanks, but this doesn't work for the memory settings (-Xmx/Xms). They get overwritten in the kc.sh script by setting the JAVA_OPTS.
Yes, I can set the JAVA_OPTS envvar, but then I have to declare all default values, too. As soon as the JAVA_OPTS is set, only the custom set envvar will be used.

@DGuhr @pedroigor WDYT? Will there be a JAVA_OPTS_APPEND option also for Keycloak.X? Or how is it supposed to be done for X?

@dasniko Sounds like a good addition to me. Do you want to create an issue for it?

Sure, done: #9188

I think you need to add the port number to the hostname name in the hostname parameter --hostname=id.acme.test:8443 - that's how it works for me.

It shouldn't be needed to add port to the hostname, so this should be fixed if that's the case. @mduchrow Could you open a bug for this one?

@pedroigor FIY

The port should be either the port from the request or hardcoded to 443 (assuming TLS enabled) if using proxy.

Thanks for the hints.
I eventually got it running by setting additional property kc.spi.hostname.default.admin = idp.mydomain.local:8943
However, I agree with @stianst and will open a bug for it.

Created issue #9133 and PR #9134 to fix this.

@ioemat Thanks for catching! Would be nice if you could look at @pedroigor s changes in PR #9134 and see if that would work for you. Your changes seem to have some unwanted side effects :)

I already commented in the PRs, the fixes from @pedroigor are fixing the issue, thanks for taking care!

@thomasdarimont could you split this into multiple comments, as that would make it easier to reply and have a conversations on the separate topics?

Better CLI help

--help-all doesnt work for the startcommand but forstart-dev. Why is there a help-alloption at all, wouldn't it make sense to print "all suitable" commands with just--help`?

The start does not support help-all. If you look at the help message this option is not available but from other commands such as build and start-dev.

By default, the help option only displays the main options for a particular command. For instance, build only shows build time properties. And start-dev only shows runtime properties. When using help-all you also have runtime properties to the build command help. The same thing with start-dev but in this case you have also access to build time options.

It is often unclear which option is a build-time option and needs a "config rebuild" and which is a runtime option. Also for beginners the help format does not make it clear whether you need --option=value or just --option value some examples might be helpful. The default values for the options somehow get lost in the when it is shown right after the option description.

It should not really matter if you are using space or = when using an option. Could you elaborate more on what you think can be improved?

--verbose command doesn't work with start-dev

...
Unknown option: '--verbose'
Creates a new and optimized server image.
...

The verbose option is not among the available options for start-dev command. You should use instead:

./kc.sh --verbose start-dev

See kc.sh -h for more details.

Ability to restrict access to admin and management endpoints

In Wildfly based Keycloak it was possible to expose the admin console or management endpoints via dedicated ports, and apply dedicated access constraints to specific endpoints via undertow filters / predicates. Support for dedicated access control is possible with custom components in Keycloak.X, however it would be helpful to have support for this out of the box. One alternative solution is to add the capability to explicitly enable / disable some management / admin endpoints for a particular keycloak instance via profiles or feature flags, but this might then force users to run additional instances just to expose the admin endpoints.

I think you have a PR for that, don't you?

Auto-build feature

The auto-build feature works quite well and should IMHO be the default for container based Keycloak deployments. The ability to create a static server configuration makes sense when user need even faster startup times and could be achived with a custom docker image.

Exactly ... That is the idea. For optimal runtime always prefer building the server first using a custom docker image (which btw was drastically simplified)

Ability to run an embedded Keycloak.X within Keycloak extension module

It would be helpful, if we could develop and test custom extensions in an embedded quarkus runtime. The IDELauncher from the quarkus/server module already provides initial support for this, but needs some tweaking to correctly discover extension classes and themes on the classpath. This would help to speed up Keycloak theme and extension development.

We should be focusing on Dev.X experience soon.

but not glad that I have to use Infinispan

But that's the point. The container is missing an "abstraction layer" for this (as Keycloak does in general). I don't want to configure infinispan or jgroups in the first place. I want to configure clustering, like in 'I want to run a cluster with x nodes and a fault tolerance up to y nodes down.' An API for the container (in terms of env vars) should hide these details of configuring infinispan or jgroups or whatever.
From my point of view, a more abstract API improves the experience for regular deployments to an extent. I has to be more opinionated to give some guidance and not only provide all the options that are available. I also see the need to be able to fine-tune everything, if needed, though.
Don't get me wrong here, getting rid of the intermediate Wildfly model is great(!!!), but it does not hide any of the complexity and therefore does not help with a better experience for many regular deployments.

Any of the available options have their corresponding env var format.

@pedroigor I think it would be good to mention that in the README for the container.

the container image is basically a wrapper on top of the distribution

I get this from technical point of view, but I don't know many users who see it this way. The wast majority of customers I work with use the container image instead of the distribution. The container is the product in their point of view. That said, they need a conveniant API for the container together with good documentation as it is availbe for the distribution.

It is the embedded Infinispan server that requires clustering, not Keycloak itself. Keycloak itself is stateless. So, having a "clustering" option in Keycloak just makes no sense in regard to Keycloak.X where we are refactoring the new store, which we are also looking at making the cache layer an optional thing. For small scale deployments having just Postgres is perfectly fine, and for large multi-region deployments just CockroachDB may be just fine. Then for inbetween scenarios Postgres + an external Infinispan cache may be the best option, again even in this scenario there's no "clustering" to configure for Keycloak, rather for the state services which are Postgres and Infinispan.

But that's the point. The container is missing an "abstraction layer" for this (as Keycloak does in general). I don't want to configure infinispan or jgroups in the first place. I want to configure clustering, like in 'I want to run a cluster with x nodes and a fault tolerance up to y nodes down.' An API for the container (in terms of env vars) should hide these details of configuring infinispan or jgroups or whatever. From my point of view, a more abstract API improves the experience for regular deployments to an extent. I has to be more opinionated to give some guidance and not only provide all the options that are available. I also see the need to be able to fine-tune everything, if needed, though. Don't get me wrong here, getting rid of the intermediate Wildfly model is great(!!!), but it does not hide any of the complexity and therefore does not help with a better experience for many regular deployments.

I think we are trying (perhaps not there yet) do achieve that level of abstraction by:

• Being more opinionated about cache configuration
• Being opinionated about the JGroups transport stack. See the --cache-stack option.

All that to say that to deploy in k8s, for instance, you should be able to leverage the default settings from Keycloak + ISPN and only set the --cache-stack=kubernetes option.

I agree we can do even more to make this configuration easier. And with the new store I would also expect more improvements in this area.

I get this from technical point of view, but I don't know many users who see it this way. The wast majority of customers I work with use the container image instead of the distribution. The container is the product in their point of view. That said, they need a convenient API for the container together with good documentation as it is available for distribution.

+1. And that should be our focus: Container-first. This requirement is also a good justification to have the container image as simple as possible and be just a wrapper on top of the distribution. With that in mind, you share the same experience when doing standalone and containers. Anything specific to containers should be properly documented.

@stianst

Keycloak itself is stateless

As of now that is not the case. As long as there is some kind of caching involved, Keycloak has state and needs inter-node communication at least for cache invalidation. This will also be the case if the caching is optional with the new store. If I want to apply this option then I need a way to configure it. This is a design decision and not specific to infinispan. The new store is a great simplification, fully agree! But it does not remove the need for a configuration API.
For example, with the old container I could set CACHE_OWNERS=3 and all caches were configured to have 3 owners. How do I do this with the Keycloak.X container? Do I have to configure each cache individually? Why do I even need to be aware of the fact that there are multiple caches? Of course, there are scenarios where we should be aware of that, but should not be needed for default setups.

Actually, I did not even want to start a discussion about caching/infinispan in general. I just took that as an example for container configuration. My point was about env vars which I understand as a configuration API for the container.

@pedroigor had some good points. I also think that options like --cache-stack=kubernetes are heading into the right direction. But why is this restricted to caching? I think something like STACK=kubernetes (or autodetect the stack) which then may set --cache-stack=kubernetes under the hood or automatically uses certificates (like it was) etc. would even be better.
And here is where I do not agree with @pedroigor

you share the same experience when doing standalone and containers

Customers in general do not have the need for the same experience with standalone and containers. This simplifies developers life, but is not very end-user/customer centric. They either use one or the other. And because there runtime environment is different in both cases, they may need a different configuration API.

@sventorben Clustering/stack/etc are all requirements to be configured for the embedded Infinispan cache, there's nothing else in Keycloak that requires this configuration, so makes no sense to make that a global config thing.

I also think having something text based is better - then you can also "grep" for it.

makes absolute sense. I'm not hard on that one. ;)

I agree with the text approach as well, but please don't include runtime. Have a marker for build-time properties only, so should be:

--db <vendor>  The database vendor. (build-time)
               ...
--db-url <jdbc-url>  The full database JDBC URL.
               ...

Not convinced about build-time though, but can't come up with a better label.

hum... (build) ? kinda uncreative on that one too, though.

I prefer just build over build time

I'm okay with just doing the profile feature flags for now. In the future we want to wrap that up as proper extensions that are enabled/disabled, but that's a future thing.

@dasniko Yer, I have the same feeling. And anything we add now should be there for a long time and hard to remove (therefore introducing legacy to something new).

There's not a chance we're putting the "/auth" thing back in there. It should never have been there, and this is just a really good opportunity to get rid of it :)

Isn't this really something that is solved by a good migration guide? We want to make it as easy as possible to migrate for sure, but we are not making trade-offs for the future to make it backwards compatible, or to eliminate some minor migration issues.

We have something between 200-1000 clients integrated with Keycloak instances and it is pretty safe to say adapting all these applications’ URLs is not going to happen. So having a config option to set the old path if needed is really necessary.

@sschu you have it, with KC_HTTP_RELATIVE_PATH/kc.http.relative-path/--http.relative-path=/auth

That’s what I wanted to hear!😉

@nicph We have some defaults for database settings, including username and password. By default, username is keycloak and password is password.

If the database was previously created using those credentials, passing them to their corresponding option should be enough.

Sorry for the confusion, but the actual default username is sa and the default password is keycloak. See here for more information: https://github.com/keycloak/keycloak/blob/main/quarkus/runtime/src/main/resources/META-INF/keycloak.properties

@nicph Sorry for that, should be fixed by #9089.

1. When running bin/kc.sh with Java 8, I see error message with long stacktrace like this:

Created #9215.

2. The README file at the root of the keycloak.x distribution mentions To run on Linux/Unix: $ bin/kc.sh . However when I run this command bin/kc.sh, server is not executed but the "help" page is shown. It looks that this README might be either removed or updated.

Created #9216.

3. From the stock distribution, I displayed the help for the start command: ./kc.sh start --help . It shows bunch of options for DB configuration in the DB section. Hence I tried to configure DB with the provided options. I've used the command with those options:

./kc.sh start --http-enabled=true --hostname-strict=true --db-url=jdbc:mysql://docker-mysql/keycloak --db-username=keycloak --db-password=keycloak

The command above should not run becaue hostname-strict is true and you did not define a hostname. Regardless, the error you might be experiencing is due to the fact that you need to choose a database prior to starting the server (if using any other than the default h2-file).

Perhaps this is better solved by a guide?

Or perhaps we can be a bit smarter here and say: "The choosen JDBC URL does not match with the current database installed". Or something like that?

4. After re-building with mysql and running the command above, the server was successfully started with MySQL DB. One minor point is, that there is no INFO message shown with my DB connection details. We display this for Wildfly distribution and I personally like it as DB connection is one of the INFO messages, which makes sense to display to administrator. In this concrete example, it is clear as I've used DB connections from the command line, but sometimes when DB connection details are configured in the other place, it is useful to display them in the log IMO. FYI. Wildfly displays this at the server startup:

13:50:09,444 INFO  [org.keycloak.connections.jpa.DefaultJpaConnectionProviderFactory] (ServerService Thread Pool -- 56) Database info: {databaseUrl=jdbc:h2:/home/mposolda/tmp/keycloak-16.0.0-SNAPSHOT/standalone/data/keycloak, databaseUser=SA, databaseProduct=H2 1.4.197 (2018-03-18), databaseDriver=H2 JDBC Driver 1.4.197 (2018-03-18)}

Yeah, we had that before (you added, btw :)). But we are trying to slim down bootstrapping including messages. You can still have that information by running using DEBUG level.

5. Maybe we discussed already, but IMO it may not be 100% clear for typical non-java administrator to understand the concept of build . Typical guy may not know when exactly he needs only to start and when he needs to both build + start . For example I consider it not-so-intuitive that for configuring DB, you need to build first and then start . The ./kc.sh --help does not clearify this IMO as it mentions this for the build:

  build                   Creates a new and optimized server image.

which does not imply to me that I need this to configure the DB. The build implies to me that it compiles something and build distribution, but here it is more often used for configure things and persist configurations. I wonder about rename build to something like: persist, augment, augment-server and also improve the initial --help to add some more info about this.

TBH I am not sure about perfect name either, but I just consider build as non-ideal :-)

As you know, previously it was config. After long, long, discussions we changed to build. Not sure if we are going to be able to re-discuss that and change at this point.

But one thing that I think you are missing is the --auto-build option, that serves as a middle term between UX and optimal runtime. By using this option, you don't need to care about buildfirst, etc.

6. Running ./kc.sh tools - I am not sure what it is used for and I am not able to decode it from help message (Intentionally did not look to the sources to just rely on help). Hence I used the "completion" suggested by "help". So when used ./kc.sh tools completion, the long bash is displayed. But I don't know what it is supposed to be used for TBH :-)

You should some examples in CLI about how to configure auto-complete. We have one in the help for the main command.

The idea behind the tools command is to serve as a repository for things like completion or whatever can help using the CLI.

What do you suggest to improve that?

7. There is option --profile for specifying profile. But it is not clear what are available profiles. Maybe the option to show what are available profiles can help (Although I understand that it likely depends on the used configuration file etc... So not 100% sure it is something, which can be improved here? Maybe at least display the latest augmented profile and the profiles supported by keycloak.properties configuration file?)

That should come with the guide/man pages.

But profiles are not hardcoded they can be whatever you want, with any configuration you want.

By default, we have two: dev and prod. The first related to running the server in dev mode, start-dev, and the second when running just start if no other profile was given.

8. The file in KEYCLOAK_DIST/conf/keycloak.properties contains properties for the prod and dev profile. I am not 100% sure if it is clear from the comments in this file that %prod and %dev prefixes refer to the name of the profile. IMO it can be useful to explicitly specify this that this is name of the profile and that folks are able to define their own configuration properties for their own profile in a similar way.

Perhaps a guide?

9. Minor: I tried to run ./kc.sh --help start-dev . It mentions help for the whole ./kc.sh command, not for the start-dev subcommand. I wonder if running the --help with arguments should either throw an error (to inform user that it was bad usage) or be smart enough to display the help for start-dev subcommand?

The help message states which options are available and how to use them. The format you tried is not correct hence the help message.

10. The help for --profile option mentions this: Use 'dev' profile to enable development mode. So I tried to run the server with the "dev" profile:

./kc.sh --profile=dev start

The output is error message (referred later as error1) :

ERROR: Failed to run 'start' command.
ERROR: You can not 'start' the server using the 'dev' configuration profile. Please re-build the server first, using 'kc.sh build' for the default production profile, or using 'kc.sh build --profile=<profile>' with a profile more suitable for production.

We should probably remove that from the description because it should be hidden and transparent to users when using start-dev. @DGuhr wdyt?

So I tried to build with dev profile and run the command:

./kc.sh build --profile=dev

But seeing another error:

Unknown option: '--profile=dev'
Possible solutions: --proxy

This is likely only bug in the error message error1 from above, which has switched "command" and "options". So I am switching it:

Isn't the format wrong? The profile is a option to the root command (kc.sh).

./kc.sh --profile=dev build

But seeing another error:

ERROR: Failed to run 'build' command.
ERROR: You can not 'build' the server using the 'dev' configuration profile. Please re-build the server first, using 'kc.sh build' for the default production profile, or using 'kc.sh build --profile=<profile>' with a profile more suitable for production.

So it seems I cannot use profile "dev" with any of build or start commands and hence it is question if it should be even mentioned in the help for the profile option?

That is on purpose, you can only build for production or any profile other than dev.

There is a clear separation between what should be done for production and for dev.

This should be solved by my first proposal to this particular item. Remove references to dev profile.

11. Running:

./kc.sh start-dev

works as expected and I can run server.

Then I stopped the server and I run this:

./kc.sh start

but seeing error message like:

ERROR: You can not 'start' the server using the 'dev' configuration profile. Please re-build the server first, using 'kc.sh build' for the default production profile, or using 'kc.sh build --profile=<profile>' with a profile more suitable for production.

which is different error message in comparison to the use of ./kc.sh start from the stock distribution... Now I decide to try this command:

./kc.sh --profile=prod start

and server is successfully started. Console even shows:

Profile prod activated.

However this is not very great as server was in fact started with the dev profile (For example "http" protocol is enabled).

I suppose this is because start-dev did not just started the server, but also did re-augmentation and did a build of the server with the "dev" properties (For example http.enabled is set to true). This seems to be confusing to me as start command seems never to re-augment the server, but start-dev does...

Yes, start should never rebuild the configuration. But start-dev.

You have here an interesting case that I did not try before. But I'm not sure if the behavior is wrong. Although you have indirectly built the server using the dev configuration (when running start-dev) you are now starting using prod profile, so it should work.

But yeah, we probably solve that by having a specific marker on the persisted properties to indicate that the image was built for dev mode. @DGuhr wdyt?

12. This is related to the previous point. I've configured this in my keycloak.properties file:

I'm not sure about your proposal. The way we have it is simple enough to what we need.

We can discuss it in more detail as well as discuss alternatives that could help to avoid any confusion. For instance, the profile option is not supposed to revert configuration, but as an additional configuration layer on top of what you already have built into the server image.

Again, this "new concept" is tricky and not natural for most people. But we can find some balance for sure.

Another idea: When I use start with --verbose, it can be nice if all used properties are shown exactly like when done with show-config command.

Sounds good to me. @DGuhr wdyt?

Hey Marek! Thanks for this feedback :) I generally go with @pedroigor s answers above. Some picks from my side:

6. Running ./kc.sh tools - I am not sure what it is used for and I am not able to decode it from help message (Intentionally did not look to the sources to just rely on help). Hence I used the "completion" suggested by "help". So when used ./kc.sh tools completion, the long bash is displayed. But I don't know what it is supposed to be used for TBH :-)

At the moment, the "tools" command in itself has nothing to do, and the completion command should always be used together with source cmd as stated when you type kc.sh -h. I agree this is not the best UX, we have different options here, from hiding these commands to let the tools command also show help for subcommands, to... - not sure what's the best way.

10. The help for --profile option mentions this: Use 'dev' profile to enable development mode. So I tried to run the server with the "dev" profile:

./kc.sh --profile=dev start

The output is error message (referred later as error1) :

ERROR: Failed to run 'start' command.
ERROR: You can not 'start' the server using the 'dev' configuration profile. Please re-build the server first, using 'kc.sh build' for the default production profile, or using 'kc.sh build --profile=<profile>' with a profile more suitable for production.

So I tried to build with dev profile and run the command:

./kc.sh build --profile=dev

But seeing another error:

Unknown option: '--profile=dev'
Possible solutions: --proxy

This is likely only bug in the error message error1 from above, which has switched "command" and "options". So I am switching it:

./kc.sh --profile=dev build

But seeing another error:

ERROR: Failed to run 'build' command.
ERROR: You can not 'build' the server using the 'dev' configuration profile. Please re-build the server first, using 'kc.sh build' for the default production profile, or using 'kc.sh build --profile=<profile>' with a profile more suitable for production.

So it seems I cannot use profile "dev" with any of build or start commands and hence it is question if it should be even mentioned in the help for the profile option?

@pedroigor I also think we shall remove the help msg. dev profile is atm exclusively "usable" with the start-dev command.

11. Running:

./kc.sh start-dev

works as expected and I can run server.

Then I stopped the server and I run this:

./kc.sh start

but seeing error message like:

ERROR: You can not 'start' the server using the 'dev' configuration profile. Please re-build the server first, using 'kc.sh build' for the default production profile, or using 'kc.sh build --profile=<profile>' with a profile more suitable for production.

which is different error message in comparison to the use of ./kc.sh start from the stock distribution... Now I decide to try this command:

./kc.sh --profile=prod start

and server is successfully started. Console even shows:

Profile prod activated.

However this is not very great as server was in fact started with the dev profile (For example "http" protocol is enabled).

I suppose this is because start-dev did not just started the server, but also did re-augmentation and did a build of the server with the "dev" properties (For example http.enabled is set to true). This seems to be confusing to me as start command seems never to re-augment the server, but start-dev does...

@pedroigor +1 for a "marker" solution. perhaps not only for dev profile, but always when --profile="..." is used, check if the server was previously built using a different profile, and add at least a warning that there's a mismatch, so the/some properties from the previously built profile are applied.

12. This is related to the previous point. I've configured this in my keycloak.properties file:

# Non-strict foo profile
%foo.http.enabled=true
%foo.hostname.strict=false

# More strict bar profile
%bar.http.enabled=false
%bar.hostname.strict=true

Now I used this command to build the foo profile and successfully started the server with it.

./kc.sh --profile=foo build
./kc.sh --profile=foo start

Then I decided to go with the other profile:

./kc.sh --profile=bar build
./kc.sh --profile=bar start

and it failed to start due the expected strict hostname resolution.

Hence I decided to revert to the working foo profile and so I wanted to just start the server with it:

./kc.sh --profile=foo start

but seeing the error about strict hostname resolution even if foo profile has this disabled in the configuration file.

I understand this is due the fact that "persisted" properties take preference over the properties from the configuration file. But not sure the order of preference is clear for typical admin (again). To improve this, I was wondering if we can support something like "Cache of augmented profiles" . In this case:

• When you build with foo profile, you will have persisted profile for foo .
• Then when you build with bar profile, you will have persisted profile for bar .
• And then when you start command with foo, the server will look into the "cache" and it will use persisted properties for the previous builded foo profile.
• When running show-config or start without profile, it would work same way like today and use last builded profile.

This will be nice, but not sure if realistic to do and if it is too much overhead for the server to support multiple augmented profiles? Just an idea...

The caching part is hard to achieve I guess, and perhaps we should focus on simplifying the ux here another way first, but I know what you mean.
technically / logically this is absolutely correct behaviour. "start" just starts the latest pre-built server image. When the server was pre-built using another profile, e.g. kc.sh -pf=bar build as in your example, you'd have a problem running start with another profile, bc. keycloak.properties ordinal < persisted.properties ordinal. (240/250 vs 300 to be precise). I still have to think this through, though. Tried to lower the ordinal of persisted to 200 in the past, but this has severe side effects (start failing) and I wasn't sure if that really solves it so i dropped it then, but @pedroigor imo we should look into this.

Another idea: When I use start with --verbose, it can be nice if all used properties are shown exactly like when done with show-config command.

@pedroigor sounds good to me. We didn't put much thought into some more value --verbose could bring, but this might be something worth to look into.

@flipp5b What is the most common authentication method used in Hashicorp? The one that is also considered the most secure?

Asking because I'm thinking about improving vault configuration a bit and for that, I would like to base it on the most common configuration people are using. The idea is to make database credentials from a vault a bit easier to set up and without exposing Quarkus properties.

@DGuhr, @pedroigor Sorry for the delay.

What is the most common authentication method used in Hashicorp? The one that is also considered the most secure?

We use AppRole for services/applications and deliver Role ID and Secret ID to applications through different channels (Secret ID is delivered at the startup time). But AFAIK the recommended way is to use platform-specific auth method (e.g. Kubernetes service accounts) if it's available (and it's not available in our case). So it's hard for me to suppose what method is more common.

would be great if you could check with latest main if it works for you now :)

I'll try to check it anytime soon! :)

@DGuhr, it looks like env variables take precedence over persisted settings now (for distro built from latest main). This is great!

But at the same time, it looks like keycloak.properties file mounted at startup time are totally ignored. E.g. in 16.1.0 it was possible to do something like this:

1. Run ./kc.sh build --db postgres ....
2. Place db.url.database=mydatabase to keycloak.properties.
3. Run ./kc.sh start.

Now, keycloak tries to use default value in such a scenario and fails to start. It's possible to override default value using KC_DB_URL_DATABASE. But it would be great to have an ability to override dynamic properties using keycloak.properties (or some other file).

@flipp5b yah well, that's bad timing i think 😅 we just changed the format of the properties from keycloak.properties to keycloak.confa few days ago - so yes, keycloak.properties will get ignored on latest main, sorry ;) Would you mind trying it out using keycloak.conf file?

See here for the discussion and here for the PR containg the changes.

@DGuhr yes, keycloak.conf works like a charm! :)

There are some difficulties configuring Vault as a credentials provider still. I can hardly wait upcoming improvements in this field :)

Thanks to the whole keycloak team!

@1Const1 By setting the proxy option you should get the same behavior as when setting the http.proxy-address-forwarding.

Also, proxy=edge should implicity set http-enabled to true.

I'm not sure what might be happening and we have tested a similar setup as yours: Ingress NGINX - Service - KXPods). Would you mind creating an issue and sharing the deployment spec?

Yes, i can for reproduce bug

Where is more correct to post issue about KeycloakX?
Here: https://github.com/keycloak/keycloak-containers/tree/main/server-x

OR

Here: https://github.com/keycloak/keycloak/issues

The latter, please.

thank you, now it described here: #9644

Awesome. Thanks, @thomasdarimont.

We should be fine then with the upcoming changes so that the missing dependencies are available from the distribution. We should probably update our guides with your examples.

@thomasdarimont Btw, due to the Infinispan upgrade. Using a custom JGroups stack was not possible. Something changed in latest Infinispan.

The fix is here https://github.com/keycloak/keycloak/pull/10053.`

Thanks a lot @thomasdarimont for your very detailed example, very useful.

It works also fine with DIGEST SASL authentication mechanism :

<distributed-cache name="sessions" owners="2">
    <expiration lifespan="-1"/>
    <remote-store cache="sessions" xmlns="urn:infinispan:config:store:remote:13.0"
                  fetch-state="false"
                  purge="false"
                  raw-values="true"
                  shared="true"
                  segmented="false"
                  connect-timeout="${env.KEYCLOAK_REMOTE_ISPN_CONN_TIMEOUT:10000}"
                  marshaller="org.keycloak.cluster.infinispan.KeycloakHotRodMarshallerFactory">  
        <remote-server host="infinispan" port="${infinispan.bind.port:11222}" />
        <remote-server host="infinispan2" port="${infinispan.bind.port:11222}"/>
        <security>
            <authentication>
                <digest username="test" password="changeme" realm="default"/>
            </authentication>
        </security>
    </remote-store>
</distributed-cache>

Thanks for the tip with digest authentication, just updated the example :)

We should be able to improve clustering configuration including when using an external Infinispan server.

Perhaps this discussions deserves its own discussion, but here are some thoughts:

• Have a cache-owners option to set the number of owners for distributable caches
• Have a cache-node and cache-site option to set both node and site names
• Have specific options to configure the number of entries in caches. We could avoid that too if we review our defaults that cover most of the cases
• Provide a cache-ispn-remote.xml file with the necessary configuration to connect to a external ISPN server
• Review key/value media types to make sure we are using the proper ones and following ISPN best practices. For instance, does it makes sense and it is possible to use protostream?

@thomasdarimont Wdyt?