Skip to content

KEYCLOAK-15773 Enable admin endpoints and admin-console via Feature flags#8369

Merged
pedroigor merged 1 commit intokeycloak:mainfrom
thomasdarimont:issue/KEYCLOAK-15773-toggle-admin-and-console-endpoints-via-profile
Sep 9, 2022
Merged

KEYCLOAK-15773 Enable admin endpoints and admin-console via Feature flags#8369
pedroigor merged 1 commit intokeycloak:mainfrom
thomasdarimont:issue/KEYCLOAK-15773-toggle-admin-and-console-endpoints-via-profile

Conversation

@thomasdarimont
Copy link
Copy Markdown
Contributor

@thomasdarimont thomasdarimont commented Aug 16, 2021

This PR adds the ability to control the availability of the admin-api endpoints and admin-console via feature flags.
This enables administrators to configure the /admin endpoint's availability, including the admin REST API and the admin-console per instance.
This allows admins to have public-facing worker instances and internal admin instances, which reduces the attack surface for the public-facing Keycloak instances.

New feature flags:

  • ADMIN_API
    Controls whether the AdminRoot JAX-RS resource are served via the /admin path

  • ADMIN
    Controls whether the legacy admin-console endpoints are served.
    This flag is marked as deprecated and will be removed once the legacy admin-console is removed.

Note that the existing ADMIN2 feature flag can also be used to control the availability of the "new" admin-console.

To run a Keycloak instance without admin-api and admin-console use:

bin/kc.sh --features-disabled=admin-api,admin,admin2

To run a Keycloak instance without admin-console use:

bin/kc.sh --features-disabled=admin,admin2

Comment thread common/src/main/java/org/keycloak/common/Profile.java Outdated
@thomasdarimont thomasdarimont force-pushed the issue/KEYCLOAK-15773-toggle-admin-and-console-endpoints-via-profile branch from b79bd39 to 3421c57 Compare October 11, 2021 12:00
Copy link
Copy Markdown
Contributor

@stianst stianst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wonder if we need ability to disable API and UI separately, or if we could just have a single ADMIN feature?

As a side-note the feature stuff is becoming a bit of a mess, and could do with a bit of improving/enhancing. Perhaps we could do a breaking change here on both names and how they are enabled/disabled in the Quarkus distribution. @pedroigor

@pedroigor
Copy link
Copy Markdown
Contributor

pedroigor commented Oct 20, 2021

I'm not 100% sure about this. At the same time, I understand the motivation and I agree that it helps to keep this form of access control within the server boundaries rather than forcing people to configure their proxies.

That said, I'm not against it if there is a consensus and demand from the community that it is going to make life easier when deploying and managing Keycloak in production.

W.r.t. to the proposed changes, I think it makes sense to have separate settings for controlling access to both console and API.

I'm also wondering how the admin console feature somewhat here overlaps with the adminUrl in the default hostname provider. It also helps to avoid access to the admin console if you are using a private URL.

Perhaps a more generic solution would be to have a JAX-RS filter that intercepts requests and checks whether an endpoint is enabled or not. It should help to avoid boilerplate code we have here as well as serve as a central point for managing access to whatever API we expose. Endpoints could be annotated with EnvironmentDependentProviderFactory (perhaps rename it to something more generic).

In Dist.X, we are currently managing features like:

./kc.sh --features-admin_api=disabled --features-admin_console=disabled

However, for this one, I think it would make more sense to provide specific configuration options like:

./kc.sh --no-admin-console
./kc.sh --no-admin-api

Not necessarily related to this problem, but perhaps we should also have a features command too? Similar to what we are discussing about providers?

./kc.sh features [--name] [enabled|disabled|list]

@stianst
Copy link
Copy Markdown
Contributor

stianst commented Oct 20, 2021

I'm not 100% sure about this. At the same time, I understand the motivation and I agree that it helps to keep this form of access control within the server boundaries rather than forcing people to configure their proxies.

That said, I'm not against it if there is a consensus and demand from the community that it is going to make life easier when deploying and managing Keycloak in production.

W.r.t. to the proposed changes, I think it makes sense to have separate settings for controlling access to both console and API.

I'm also wondering how the admin console feature somewhat here overlaps with the adminUrl in the default hostname provider. It also helps to avoid access to the admin console if you are using a private URL.

Perhaps a more generic solution would be to have a JAX-RS filter that intercepts requests and checks whether an endpoint is enabled or not. It should help to avoid boilerplate code we have here as well as serve as a central point for managing access to whatever API we expose. Endpoints could be annotated with EnvironmentDependentProviderFactory (perhaps rename it to something more generic).

In Dist.X, we are currently managing features like:

./kc.sh --features-admin_api=disabled --features-admin_console=disabled

However, for this one, I think it would make more sense to provide specific configuration options like:

./kc.sh --no-admin-console
./kc.sh --no-admin-api

Not necessarily related to this problem, but perhaps we should also have a features command too? Similar to what we are discussing about providers?

./kc.sh features [--name] [enabled|disabled|list]

I'm not 100% sure about this. At the same time, I understand the motivation and I agree that it helps to keep this form of access control within the server boundaries rather than forcing people to configure their proxies.

That said, I'm not against it if there is a consensus and demand from the community that it is going to make life easier when deploying and managing Keycloak in production.

W.r.t. to the proposed changes, I think it makes sense to have separate settings for controlling access to both console and API.

I'm also wondering how the admin console feature somewhat here overlaps with the adminUrl in the default hostname provider. It also helps to avoid access to the admin console if you are using a private URL.

Perhaps a more generic solution would be to have a JAX-RS filter that intercepts requests and checks whether an endpoint is enabled or not. It should help to avoid boilerplate code we have here as well as serve as a central point for managing access to whatever API we expose. Endpoints could be annotated with EnvironmentDependentProviderFactory (perhaps rename it to something more generic).

In Dist.X, we are currently managing features like:

./kc.sh --features-admin_api=disabled --features-admin_console=disabled

However, for this one, I think it would make more sense to provide specific configuration options like:

./kc.sh --no-admin-console
./kc.sh --no-admin-api

Not necessarily related to this problem, but perhaps we should also have a features command too? Similar to what we are discussing about providers?

./kc.sh features [--name] [enabled|disabled|list]

Let's think a bit broader about this than just the admin console, as the "--no-admin-console" doesn't make much sense to me. Ideally it would be very easy to disable capabilities you don't need from Keycloak. Don't want LDAP sure, just remove it. Don't want SAML, again sure just remove it. Want to have background service (like cleanup jobs) run in a separate set of nodes that don't handle request, again just do that.

Now if we align features/providers a bit more, and make it easy to define what you want to include in a consistent way, then we have an easy way to do that with Quarkus right, which basically removes features/providers completely from the runtime.

@pedroigor
Copy link
Copy Markdown
Contributor

Let's think a bit broader about this than just the admin console, as the "--no-admin-console" doesn't make much sense to me.

It is an attempt to make this setting more explicit when deploying into production rather than having to configure features (more verbose). Similarly to what we are planning for the hostname provider.

Ideally it would be very easy to disable capabilities you don't need from Keycloak. Don't want LDAP sure, just remove it. Don't want SAML, again sure just remove it. Want to have background service (like cleanup jobs) run in a separate set of nodes that don't handle request, again just do that.

Now if we align features/providers a bit more, and make it easy to define what you want to include in a consistent way, then we have an easy way to do that with Quarkus right, which basically removes features/providers completely from the runtime.

Sure, and that is related to the long-term modularization story we have been discussing. For now, we can't do much and my suggestions were based on what we have today. Dist.X can not help much because we have some constraints in our codebase that block us from properly adding/removing features. For instance, if you remove LDAP you also need to remove UIs, mappers, etc, all that spread in different modules.

If we had features (and their providers) self-contained, we could definitely have something in Dist.X. For instance, we would have a very basic distribution with only the core bits. Additional stuff like the LDAP, SAML, etc, would be deployed similarly to what we are doing for custom providers. With the possibility to fetch these extensions from some marketplace, etc. That would be awesome, ideed.

@stianst
Copy link
Copy Markdown
Contributor

stianst commented Oct 21, 2021

Let's think a bit broader about this than just the admin console, as the "--no-admin-console" doesn't make much sense to me.

It is an attempt to make this setting more explicit when deploying into production rather than having to configure features (more verbose). Similarly to what we are planning for the hostname provider.

Ideally it would be very easy to disable capabilities you don't need from Keycloak. Don't want LDAP sure, just remove it. Don't want SAML, again sure just remove it. Want to have background service (like cleanup jobs) run in a separate set of nodes that don't handle request, again just do that.
Now if we align features/providers a bit more, and make it easy to define what you want to include in a consistent way, then we have an easy way to do that with Quarkus right, which basically removes features/providers completely from the runtime.

Sure, and that is related to the long-term modularization story we have been discussing. For now, we can't do much and my suggestions were based on what we have today. Dist.X can not help much because we have some constraints in our codebase that block us from properly adding/removing features. For instance, if you remove LDAP you also need to remove UIs, mappers, etc, all that spread in different modules.

If we had features (and their providers) self-contained, we could definitely have something in Dist.X. For instance, we would have a very basic distribution with only the core bits. Additional stuff like the LDAP, SAML, etc, would be deployed similarly to what we are doing for custom providers. With the possibility to fetch these extensions from some marketplace, etc. That would be awesome, ideed.

My point there was more in terms on how Keycloak.X should allow users to enable/disable features/providers. We don't want to make that into an "API" that has to change in the future though, and I'm pretty much sure we don't want a "--no-" option ;)

Some random thoughts/ideas (I'm only thinking about how it would be configured from Keycloak.X, not the implementation/improvements there, as for now we can just wire it up to what we currently have):

  • Combine features/spis/providers into a single thing that can be enabled/disabled
  • Short-hand options to enable/disable a specific capabilities
  • Perhaps a "profile.properties" or something that can list what capabilities should be enabled/disabled separately

Let's for arguments sake that we call these capabilities not to conflict with existing things like features/spis/providers. Then we can build some hierarchy of capabilites, so something like:

  • admin
  • admin/api
  • admin/ui
  • federation/ldap
  • protocol/saml
  • db/postgres

Then instead of listing "providers" with kc you'd list capabilities, which could behind the scenes be just a profile feature, a complete spi, or a provider of an spi.

It could also align with setting the default provider, or with the high-level "type" of something we have, so some ideas on how this could look like in Keycloak.X:

kc --db=postgres --admin=disabled --protocol.saml=disabled

Or with a separate profile properties file or whatever:

myprofile

admin.ui=enabled
db=postgres
kc --profile=myprofile

@stianst
Copy link
Copy Markdown
Contributor

stianst commented Oct 21, 2021

@thomasdarimont sorry for taking you're simple PR and making it into a complex discussion ;)

@pedroigor
Copy link
Copy Markdown
Contributor

@stianst I see where you a heading to. As I mentioned before, the example you gave for admin=disabled is somewhat similar to what we support today using --features-admin=disabled.

Let us discuss that in a separate discussion, I'm going to create one.

@stianst
Copy link
Copy Markdown
Contributor

stianst commented Oct 21, 2021

Let us discuss that in a separate discussion, I'm going to create one.

Thanks

@thomasdarimont thomasdarimont force-pushed the issue/KEYCLOAK-15773-toggle-admin-and-console-endpoints-via-profile branch from 3421c57 to 99db6fb Compare December 16, 2021 00:06
@stianst stianst self-assigned this Aug 25, 2022
@stianst
Copy link
Copy Markdown
Contributor

stianst commented Aug 25, 2022

Would be nice to get this in. @thomasdarimont are you still interested in this one?

With regards to how to disable it that should just be done like we're disabling other features (https://www.keycloak.org/server/features#_disabling_features).

For the names of the features "ADMIN_API" is fine as that aligns with ACCOUNT_API, but not sure about ADMIN_CONSOLE, as we have ADMIN2 to enable the new admin console.

Perhaps what we should do is add "ADMIN" feature, that enables/disables the old admin console. This should just be marked as deprecated straight away.

Update so that it doesn't fallback on the old admin console if ADMIN2 is not enabled. Essentially that'd mean to use the old admin console you'd need to run:

bin/kc.sh --features=admin --features-disabled=admin2

Or, if you want to disable the admin console fully you'd just run:

bin/kc.sh --features-disabled=admin2

@stianst stianst requested a review from pedroigor August 25, 2022 12:59
@thomasdarimont thomasdarimont force-pushed the issue/KEYCLOAK-15773-toggle-admin-and-console-endpoints-via-profile branch from 99db6fb to bb468f2 Compare August 25, 2022 17:07
@thomasdarimont
Copy link
Copy Markdown
Contributor Author

@stianst I just rebased this branch and applied the suggested changes.

@thomasdarimont thomasdarimont force-pushed the issue/KEYCLOAK-15773-toggle-admin-and-console-endpoints-via-profile branch from bb468f2 to 89cfba9 Compare August 25, 2022 18:59
Comment thread services/src/main/java/org/keycloak/services/resources/WelcomeResource.java Outdated
Comment thread services/src/main/java/org/keycloak/services/resources/admin/AdminRoot.java Outdated
Comment thread common/src/main/java/org/keycloak/common/Profile.java Outdated
@thomasdarimont thomasdarimont force-pushed the issue/KEYCLOAK-15773-toggle-admin-and-console-endpoints-via-profile branch from 89cfba9 to bd33611 Compare August 25, 2022 20:06
Comment thread services/src/main/java/org/keycloak/services/resources/admin/AdminRoot.java Outdated
Comment thread common/src/main/java/org/keycloak/common/Profile.java Outdated
Comment thread common/src/main/java/org/keycloak/common/Profile.java Outdated
Comment thread services/src/main/java/org/keycloak/services/resources/admin/AdminRoot.java Outdated
Comment thread services/src/main/java/org/keycloak/services/resources/admin/AdminRoot.java Outdated
Comment thread services/src/main/java/org/keycloak/services/resources/admin/AdminRoot.java Outdated
Comment thread services/src/main/java/org/keycloak/services/resources/admin/AdminRoot.java Outdated
Comment thread services/src/main/java/org/keycloak/services/resources/admin/AdminRoot.java Outdated
Comment thread services/src/main/java/org/keycloak/services/resources/admin/AdminRoot.java Outdated
Comment thread services/src/main/java/org/keycloak/services/resources/admin/AdminRoot.java Outdated
stianst
stianst previously approved these changes Aug 26, 2022
@stianst
Copy link
Copy Markdown
Contributor

stianst commented Aug 26, 2022

I'm happy with the PR in it's current state. Let's stop nit-picking this one to death ;)

@stianst
Copy link
Copy Markdown
Contributor

stianst commented Aug 29, 2022

/rerun

@pedroigor
Copy link
Copy Markdown
Contributor

@thomasdarimont Looks like there are some related failures. Could you please check them out?

@thomasdarimont thomasdarimont force-pushed the issue/KEYCLOAK-15773-toggle-admin-and-console-endpoints-via-profile branch from e79b0af to 1a77225 Compare August 30, 2022 22:26
@thomasdarimont
Copy link
Copy Markdown
Contributor Author

I just rebased this branch on current main, squashed the commits and adapted the failing tests in ProfileTest.

stianst
stianst previously approved these changes Sep 1, 2022
@stianst
Copy link
Copy Markdown
Contributor

stianst commented Sep 1, 2022

/rerun

@stianst
Copy link
Copy Markdown
Contributor

stianst commented Sep 1, 2022

@thomasdarimont some tests are failing, which looks like real issues rather than the usual unstable tests. At least one thing I noticed is that adminConsoleEnabled is not added to https://github.com/keycloak/keycloak/blob/main/quarkus/runtime/src/main/java/org/keycloak/quarkus/runtime/services/resources/QuarkusWelcomeResource.java. Not sure why, but there's a different class that exposes the welcome-page on Quarkus.

@thomasdarimont
Copy link
Copy Markdown
Contributor Author

@stianst I adjusted the QuarkusWelcomeResource accordingly.

I just gave this a try:

With admin console and admin-api disabled --features-disabled=admin-api,admin2,admin:

Updating the configuration and installing your custom providers, if any. Please wait.
...
2022-09-02 10:15:59,178 WARN  [org.keycloak.quarkus.runtime.KeycloakMain] (main) Running the server in development mode. DO NOT use this configuration in production.

With invalid configurations like --features-disabled=admin-api and --features=admin --features-disabled=admin-api we get a lengthy stacktrace - perhaps we should just print a 1 line error message and exit.

$ bin/kc.sh start-dev --features=admin --features-disabled=admin-api             
Updating the configuration and installing your custom providers, if any. Please wait.
2022-09-02 10:17:51,966 WARN  [org.keycloak.common.Profile] (build-22) Deprecated feature enabled: admin
ERROR: Failed to run 'build' command.
ERROR: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
	[error]: Build step org.keycloak.quarkus.deployment.KeycloakProcessor#configureProviders threw an exception: java.lang.RuntimeException: Invalid value for feature: ADMIN_API needs to be enabled because it is required by [ADMIN, ADMIN2].
	at org.keycloak.common.Profile.<init>(Profile.java:96)
	at org.keycloak.quarkus.runtime.QuarkusProfile.<init>(QuarkusProfile.java:28)
	at org.keycloak.quarkus.deployment.KeycloakProcessor.configureProviders(KeycloakProcessor.java:296)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
	at java.base/java.lang.reflect.Method.invoke(Method.java:577)
	at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:909)
	at io.quarkus.builder.BuildContext.run(BuildContext.java:281)
	at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
	at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2449)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1478)
	at java.base/java.lang.Thread.run(Thread.java:833)
	at org.jboss.threads.JBossThread.run(JBossThread.java:501)

ERROR: Build failure: Build failed due to errors
	[error]: Build step org.keycloak.quarkus.deployment.KeycloakProcessor#configureProviders threw an exception: java.lang.RuntimeException: Invalid value for feature: ADMIN_API needs to be enabled because it is required by [ADMIN, ADMIN2].
	at org.keycloak.common.Profile.<init>(Profile.java:96)
	at org.keycloak.quarkus.runtime.QuarkusProfile.<init>(QuarkusProfile.java:28)
	at org.keycloak.quarkus.deployment.KeycloakProcessor.configureProviders(KeycloakProcessor.java:296)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
	at java.base/java.lang.reflect.Method.invoke(Method.java:577)
	at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:909)
	at io.quarkus.builder.BuildContext.run(BuildContext.java:281)
	at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
	at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2449)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1478)
	at java.base/java.lang.Thread.run(Thread.java:833)
	at org.jboss.threads.JBossThread.run(JBossThread.java:501)

ERROR: Invalid value for feature: ADMIN_API needs to be enabled because it is required by [ADMIN, ADMIN2].
For more details run the same command passing the '--verbose' option. Also you can use '--help' to see the details about the usage of the particular command.

--features=admin --features-disabled=admin-api yields:

2022-09-02 10:17:51,966 WARN  [org.keycloak.common.Profile] (build-22) Deprecated feature enabled: admin
...
followed by the stacktrace from above

--features=admin2 --features-disabled=admin-api yields:

Stacktrace from above

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that we currently cannot use Profile.isFeatureEnabled(..) here since this currently leads to an infinite recusrsion, that ends with a StackOverFlowError.

stianst
stianst previously approved these changes Sep 6, 2022
@pedroigor
Copy link
Copy Markdown
Contributor

@thomasdarimont Quarkus Test failures are basically about the help message. if you run HelpCommandTest you should see some files created at https://github.com/keycloak/keycloak/tree/0701922dad52e272c806ea29e24bd9faa0e89245/quarkus/tests/integration/src/test/resources/org/keycloak/it/cli/approvals/cli/help so that you can paste their content to their approved variants.

…a feature flags

Inline profile checks for enabled admin-console to avoid issues during
static initialization with quarkus.

Potentially Re-enable admin-api feature if admin-console is enabled
via the admin/admin2 feature flag.

Add legacy admin console as deprecated feature flag
Throw exception if admin-api feature is disabled but admin-console is enabled

Adapt ProfileTest

Consider adminConsoleEnabled flag in QuarkusWelcomeResource
Fix check for Admin-Console / Admin-API feature dependency.

Add new features to approved help output files

Co-authored-by: Stian Thorgersen <[email protected]>
Signed-off-by: Thomas Darimont <[email protected]>
@thomasdarimont thomasdarimont force-pushed the issue/KEYCLOAK-15773-toggle-admin-and-console-endpoints-via-profile branch from bbf8e62 to 0593adf Compare September 9, 2022 18:18
@thomasdarimont
Copy link
Copy Markdown
Contributor Author

@pedroigor thanks for the hint! I just fixed the HelpCommandTest, rebased the branch and squashed the commits into a single one.

@pedroigor pedroigor merged commit 962a685 into keycloak:main Sep 9, 2022
@pedroigor
Copy link
Copy Markdown
Contributor

@thomasdarimont Tks.

@jrivers96
Copy link
Copy Markdown

jrivers96 commented Sep 13, 2022

So if I want to keep our admin dashboard separate from the public path through a reverse proxy, I'd somehow configure a subset of keycloak instances to enable the administration dashboard and some of them to disable it?

Then I would route the public facing path to the keycloak instances that have the admin dashboard disabled?

We are using the codecentric chart. Maybe the keycloak operator has better configuration to be able to do this?
https://github.com/codecentric/helm-charts/blob/keycloak-18.3.0/charts/keycloak/README.md

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants