Skip to content

Introduce unique index for enums stored by storages#12299

Merged
hmlnarik merged 1 commit intokeycloak:mainfrom
vramik:12277-enums
Jun 15, 2022
Merged

Introduce unique index for enums stored by storages#12299
hmlnarik merged 1 commit intokeycloak:mainfrom
vramik:12277-enums

Conversation

@vramik
Copy link
Copy Markdown
Contributor

@vramik vramik commented Jun 1, 2022

Closes #12277

@vramik vramik marked this pull request as ready for review June 2, 2022 07:52
@vramik vramik added area/storage Indicates an issue that touches storage (change in data layout or data manipulation) team/storage-sig labels Jun 2, 2022
@vramik vramik requested a review from hmlnarik June 2, 2022 09:48
Comment thread core/src/main/java/org/keycloak/util/EnumWithUnchangableIndex.java Outdated
Copy link
Copy Markdown
Contributor

@hmlnarik hmlnarik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is nice, thank you!

I've thought a little bit about naming, could you please see the comments inline?

I suggest replacing all occurences of unchangebleIndex and unchangableIndex with stableIndex

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is the return type Integer rather than int? If it is because of the jackson bug, can it be int now the the PR includes jackson upgrade?

Copy link
Copy Markdown
Contributor Author

@vramik vramik Jun 2, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was not due to jackson bug, the Integer came from assumption that we'd store Enums numeric representations on entity level. In that case we'd want to have possibility to store nulls due to tree store and store hierarchy.

But now, since it has been decided we let each store handle it separately it seems it might make sense to have int there. I'll update it.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* @return An unchangeable numeric index of {@link Enum} item.
* @return Unique numeric index which is stable in time and identifies an instance.
Reusing the same index for two distinct entries of the same class is forbidden even
if they cannot exist at the same time (e.g. one is deleted before other is introduced).

Comment thread core/src/main/java/org/keycloak/util/EnumWithUnchangeableIndex.java Outdated
Comment thread core/src/main/java/org/keycloak/util/EnumWithUnchangeableIndex.java Outdated
Comment thread core/src/main/java/org/keycloak/util/EnumWithUnchangeableIndex.java Outdated
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this method doesn't work with CHM and Hotrod because the MapAdminEventEntity interface has the get/setResourceType methods in String form. When I tested the events I noticed that the type passed to the mcb in this method must match the type found in the entity's interface (CHM uses comparators that simply don't match if the types are different) and I believe this particular case is not being tested properly at the moment. In other words, the AdminEventStoreProviderTest has no tests with searches that involve the resource type.

If you change the OperationType into String in the entity's interface, you will see that the same test fails with CHM because the CHM won't be able to retrieve the events when searching by operation type.

So in this case we either use the enum type in the entity's interface, or change the above to use Arrays.stream(resourceTypes).map(Enum::toString)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for pointing this out. I agree it is not very well tested. I believe we cannot use enum type in the entity interface due to custom event types (KEYCLOAK-9573). We have to be able to store exact string of the custom type (test AdminEventsStoreProviderTest.handleCustomResourceTypeEvents). Unless we'd store the value somewhere else (e.g. we can add attributes and put it there).

But with custom types querying starts to be a little bit tricky. In scenario when there is more than one custom resource type and admin events are queried by ResourceType.CUSTOM, all custom events are returned regardless of its actual value. Is this expected behavior?

If not we'd need to make additional changes anyway.

If so (and we don't want to store custom type value elsewhere) easiest way seems to be to use Arrays.stream(resourceTypes).map(Enum::toString) but in that case I'd also change fieldType to String
AdminEvent:

 public static final SearchableModelField<AdminEvent> RESOURCE_TYPE   = new SearchableModelField<>("resourceType", String.class);

Copy link
Copy Markdown
Contributor Author

@vramik vramik Jun 3, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or maybe the cleanest solution (in case we won't support search by custom types) would be to change implementation of MapFieldPredicates:

put(ADMIN_EVENTS_PREDICATES, AdminEvent.SearchableFields.RESOURCE_TYPE, mapAdminEventEntity -> ResourceType.valueOf(mapAdminEventEntity.getResourceType()));

Should this topic be put into its own separate PR?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But with custom types querying starts to be a little bit tricky. In scenario when there is more than one custom resource type and admin events are queried by ResourceType.CUSTOM, all custom events are returned regardless of its actual value. Is this expected behavior?

I honestly have no idea, but this is definitely a strange behavior in my opinion.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this topic be put into its own separate PR?

Yeah, I think so. This PR doesn't seem like the proper place to handle these resource type inconsistencies. We should probably discuss that as part of a separate issue.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there this issue filed already?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've created #12485

@vramik vramik requested a review from hmlnarik June 3, 2022 19:23
sguilhen
sguilhen previously approved these changes Jun 7, 2022
Copy link
Copy Markdown
Contributor

@sguilhen sguilhen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving as the changes here seem correct to me. The request type inconsistencies should be discussed in a separate thread/issue.

Copy link
Copy Markdown
Contributor

@hmlnarik hmlnarik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have one question spanning all the enum implementations, please see inline.

Comment thread server-spi-private/src/main/java/org/keycloak/events/admin/OperationType.java Outdated
Comment thread server-spi-private/src/main/java/org/keycloak/events/admin/OperationType.java Outdated
@vramik vramik force-pushed the 12277-enums branch 2 times, most recently from 9f588f3 to b993d4e Compare June 13, 2022 17:12
@vramik
Copy link
Copy Markdown
Contributor Author

vramik commented Jun 13, 2022

Rebased and incorporated suggestions, thanks @hmlnarik!

sguilhen
sguilhen previously approved these changes Jun 13, 2022
Copy link
Copy Markdown
Contributor

@sguilhen sguilhen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @vramik , it looks good to me!

@vramik
Copy link
Copy Markdown
Contributor Author

vramik commented Jun 14, 2022

Rebased again. KerberosLdapTest should be fixed now.

@vramik vramik requested a review from hmlnarik June 15, 2022 06:22
Copy link
Copy Markdown
Contributor

@hmlnarik hmlnarik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the fix!

@hmlnarik hmlnarik merged commit df41f23 into keycloak:main Jun 15, 2022
@vramik vramik deleted the 12277-enums branch November 5, 2023 13:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/storage Indicates an issue that touches storage (change in data layout or data manipulation)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Introduce unique index for enums stored by storages

3 participants