Skip to content

KEYCLOAK-11019 Initial support for lazy offline user-session loading#8033

Merged
hmlnarik merged 7 commits intokeycloak:masterfrom
martin-kanis:KEYCLOAK-11019
May 26, 2021
Merged

KEYCLOAK-11019 Initial support for lazy offline user-session loading#8033
hmlnarik merged 7 commits intokeycloak:masterfrom
martin-kanis:KEYCLOAK-11019

Conversation

@martin-kanis
Copy link
Copy Markdown
Contributor

This PR includes #7722 + rebase to current master + adds model tests

@hmlnarik hmlnarik self-assigned this May 12, 2021
@hmlnarik hmlnarik added area/storage Indicates an issue that touches storage (change in data layout or data manipulation) kind/enhancement Categorizes a PR related to an enhancement labels May 12, 2021
Copy link
Copy Markdown
Contributor

@vramik vramik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the PR. I went thru the original PR and there has been done huge amount of work. Thanks again.

Few notes:

  • There was some issues in GHA
    • model tests were cancelled after 329m. But I can see it passed in @martin-kanis's fork so I assume it is not related.
    • cluster tests are failing
  • I haven't found any documentation PR, is there a plan to send one?
  • the changeSet should be updated, please see inline.

Otherwise it looks good to me.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This changeSet should be moved to jpa-changelog-14.0.0.xml

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, moved.

@hmlnarik
Copy link
Copy Markdown
Contributor

model tests were cancelled after 329m. But I can see it passed in @martin-kanis's fork so I assume it is not related.

That's correct, it is an independent intermittent issue, tracked as https://issues.redhat.com/browse/KEYCLOAK-18091

@martin-kanis martin-kanis force-pushed the KEYCLOAK-11019 branch 2 times, most recently from 9214b5c to 1d6bb19 Compare May 14, 2021 11:24
@martin-kanis
Copy link
Copy Markdown
Contributor Author

The failing cluster tests were caused by a syntax error in SQL query on MySQL. The coalesce + cast functions didn't work as expected at least on MySQL in clustered environment with provided null values. I removed the coalesce from all queries because from the keycloak code we always called the queries with null values.
I also removed some newly introduced methods which weren't called from anywhere.

WARN  [org.hibernate.engine.jdbc.spi.SqlExceptionHelper] (ServerService Thread Pool -- 69) SQL Error: 1064, SQLState: 42000
2021-04-02T22:46:06.7960979Z AuthenticationSessionClusterTest ++ 22:45:45,473 ERROR [org.hibernate.engine.jdbc.spi.SqlExceptionHelper] (ServerService Thread Pool -- 69) You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'longtext), persistent0_.REALM_ID)' at line 1
2021-04-02T22:46:06.7965147Z AuthenticationSessionClusterTest ++ 22:45:45,484 FATAL [org.keycloak.services] (ServerService Thread Pool -- 69) Error during startup: javax.persistence.PersistenceException: org.hibernate.exception.SQLGrammarException: could not extract ResultSet
2021-04-02T22:46:06.7969007Z AuthenticationSessionClusterTest ++ 	at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:154)
2021-04-02T22:46:06.7972268Z AuthenticationSessionClusterTest ++ 	at org.hibernate.query.internal.AbstractProducedQuery.list(AbstractProducedQuery.java:1575)
2021-04-02T22:46:06.7976018Z AuthenticationSessionClusterTest ++ 	at org.hibernate.query.internal.AbstractProducedQuery.getSingleResult(AbstractProducedQuery.java:1614)
2021-04-02T22:46:06.7980927Z AuthenticationSessionClusterTest ++ 	at org.keycloak.models.jpa.session.JpaUserSessionPersisterProvider.getUserSessionsCount(JpaUserSessionPersisterProvider.java:460)
2021-04-02T22:46:06.7986423Z AuthenticationSessionClusterTest ++ 	at org.keycloak.models.session.UserSessionPersisterProvider.getUserSessionsCount(UserSessionPersisterProvider.java:133)
2021-04-02T22:46:06.7993299Z AuthenticationSessionClusterTest ++ 	at org.keycloak.models.sessions.infinispan.initializer.OfflinePersistentUserSessionLoader.computeLoaderContext(OfflinePersistentUserSessionLoader.java:67)
2021-04-02T22:46:06.8001438Z AuthenticationSessionClusterTest ++ 	at org.keycloak.models.sessions.infinispan.initializer.OfflinePersistentUserSessionLoader.computeLoaderContext(OfflinePersistentUserSessionLoader.java:37)
2021-04-02T22:46:06.8007571Z AuthenticationSessionClusterTest ++ 	at org.keycloak.models.sessions.infinispan.initializer.InfinispanCacheInitializer$2.run(InfinispanCacheInitializer.java:92)

@hmlnarik
Copy link
Copy Markdown
Contributor

thomasdarimont and others added 7 commits May 24, 2021 16:54
…ase query if in-memory lookup fails for offline sessions

Allow to configure pre-loading of offline user-sessions

We now allow to configure whether offline user-sessions should be
preloaded on startup from the database.

We now trigger offline user-sessions loading when
UserSessionProvider#getOfflineUserSessionsStream(realm,user) is called.

This ensures that the user can revoke the offline_access consent
via the account console. Additionally the (offline) user consents
are now displayed correctly for a user in the admin-console.

Offline user-sessions are now correctly display if offline user-sessions
are loaded lazily.

Add note about unsupported lazy offline session loading by broker info:
Currently we don't support lazy offline user-session lookup by
brokerSessionId and brokerUserId.

See: keycloak#7722 (review)

Keep behavior of 'logout all sessions' and don't delete dangling offline sessions.
As discussed in keycloak#7722 (comment)

Throw on unsupported dynamic offline user-session lookups from database
Removed redundant condition checking.

We use the `coalesce` function to make the queries work for a
specific realm / client or all, e.g.:
```
clientSess.clientId = coalesce(CAST(:clientId as text), clientSess.clientId)
```
Note that the explicit cast is required to treat null values as text to avoid
confusing databases.
@martin-kanis
Copy link
Copy Markdown
Contributor Author

@hmlnarik hmlnarik requested a review from vramik May 25, 2021 08:36
Copy link
Copy Markdown
Contributor

@vramik vramik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @martin-kanis!

Th PR looks good to me. From my POV there is just documentation PR missing and we've discussed with @hmlnarik that after those changes in queries the index analysis (which was done with the original PR) should re-checked.

@hmlnarik
Copy link
Copy Markdown
Contributor

Indices checked, they look good. @vramik , is the docs the only remaining concern?

Copy link
Copy Markdown
Contributor

@vramik vramik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hmlnarik yes, the docs, are the only remaining concern, thank you for checking the indices.

Copy link
Copy Markdown
Contributor

@hmlnarik hmlnarik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pipelines ran successfully. Thank you for the contribution, @thomasdarimont and @martin-kanis !

@DanielJoyce
Copy link
Copy Markdown

We're seeing the issue this PR is meant to fix. Has this made it into a release, or are there some setting changes?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/storage Indicates an issue that touches storage (change in data layout or data manipulation) kind/enhancement Categorizes a PR related to an enhancement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants