KEYCLOAK-18727 Improve user search query#8346
Conversation
|
I also wanted to add that i'm on vacation for three weeks from 14.08 until 5.09. I won't be able to respond during that time. |
714cec2 to
a622a50
Compare
|
Is there any chance this change can be merged? I also have this issue searching for users. |
vramik
left a comment
There was a problem hiding this comment.
Thank you @artur-baltabayev for the PR, could you please do the rebase? There has been done some changes in codebase what will probably affect your PR. Thanks.
a622a50 to
0680a01
Compare
|
@vramik Thank you for your response. I just rebased the branch and it's up to date with master. But somehow there is still a merge conflict displayed. Maybe it just needs time to synchronize. |
|
@artur-baltabayev I believe that it has been synced with master. Keycloak's main branch is now called |
|
I've started discussion regarding the change of the default behavior: #9507 |
0680a01 to
600c2d6
Compare
|
@hmlnarik Thanks for the hint. Now it's rebased on the main branch, |
Thank you @artur-baltabayev, it seems there has happened some issue as the branch you're sending the PR from seems to be 118 commits behind keycloak:main, could you please double-check it? |
vramik
left a comment
There was a problem hiding this comment.
Overall it seems good to me from the first look. I have 2 comments for now (please see inline) and I'll look at it once it'll be rebased, thank you @artur-baltabayev
There was a problem hiding this comment.
I've tried it and it seems the issue is not reproducible on current main.
There was a problem hiding this comment.
Needed for undertow. See comment below.
b4df847 to
5ae27ab
Compare
|
@vramik Implemented your feedback. I also removed some imports that were unused in the test classes. |
|
@vramik Ok i remember now that the single wildcard workaround was needed for undertow. As you can see, currently every undertow test that searches for a empty string fails. That is because it defaults to a 'empty' prefix search which is represented by a single wildcard character (i.e. '%'). We couldn't come up with another fix for this problem, so i would appreciate suggestions. Otherwise i can add the workaround again. |
|
Thank you @artur-baltabayev. Please correct me if I'm wrong, but I think search for empty string is equivalent to getting all users. In that case it could be checked in |
|
@vramik You mean we should move the workaround to the UsersResource instead of being MapProvider specific ? |
I don't think there should be added any kind of workaround. When you asked for suggestions regarding the empty string searches I've realized that empty string search should be considered as equal to getting all users. It should work for both legacy-jpa and map-storage, why exactly it is failing for map storage and not for legacy-jpa? |
|
@vramik My observation was that the test cases failed after i removed the workaround. But since you were pointing out (https://issues.redhat.com/browse/KEYCLOAK-18969), that the mentioned single wildcard issue no longer exists i will have to do some further testing before i can answer you question. |
|
Hello @vramik , after i did some more research i can now explain why you couldn't recreate the mentioned single wildcard issue (https://issues.redhat.com/browse/KEYCLOAK-18969) and why it's failing in the first place : You can only recreate the issue on this branch (not on main) since it is tied to how the new user search works. I recreated it locally by running the following command : mvn clean install -nsu -B -Pauth-server-undertow -Pmap-storage -Dtest=UserTest -Dkeycloak.logging.level=info -f testsuite/integration-arquillian/tests/base/pom.xml . The error is map storage specific because it comes from the 'like' and 'ilike' methods in the CriteriaOperator class. There is a substring call that wasn't made for input strings that only contain one character (in our case it's the single wildcard). If I have several suggestions to fix this issue :
|
|
Option 3 please. If |
5ae27ab to
9a1c9dc
Compare
vramik
left a comment
There was a problem hiding this comment.
Thank you @artur-baltabayev for the update. The code looks good to me. Approving.
|
@artur-baltabayev Thank you for the code changes, they now look good to me. Could you please prepare a PR for release notes in https://github.com/keycloak/keycloak-documentation/blob/main/release_notes/topics for version 17 (if not present, please introduce file |
|
@vramik @hmlnarik Thank you two a lot for all the feedback! I just made the documentation PR : keycloak/keycloak-documentation#1378 . |
|
Just out of curiosity using |
|
Good improvement. Just one question why it doesn't include the case |
It's maybe not that intuitive, but you should be able to handle that case like that: |
The purpose of this PR is to improve the performance of the user search (specifically when you search for a user in the UI),
since customers were facing bad performance on systems with a large amount of users. The implementation is following
a suggestion from @hmlnarik . You can find the full discussion here: https://groups.google.com/g/keycloak-dev/c/jn5Yy9Y4Lw4 .
TL;DR :
You can now search for a user in 3 different ways:
However, we would like to make the following remarks and like to hear your feedback:
Changing the default behavior of the user search endpoint is technically breaking the API. One might also consider making a new endpoint and let the UI use it instead.
For consistency, we also changed the user count (if a search string is part of the input) to work similarly to the user search.
We only adjusted the JpaUserProvider and MapUserProvider because we wanted to hear your feedback on whether it is also necessary to adjust the LdapStorageProvider.
While adjusting the JpaUserProvider and MapUserProvider, we decided that it would be consistent to unify their queries. This means the JpaUserProvider now searches for the search string on both the FIRST_NAME and LAST_NAME columns separately instead of concatenating them with coalesc. Also the search string gets split on both providers now.