Skip to content

Moving changes from develop to master#80

Merged
klehmann merged 112 commits intomasterfrom
develop
Jul 7, 2024
Merged

Moving changes from develop to master#80
klehmann merged 112 commits intomasterfrom
develop

Conversation

@klehmann
Copy link
Copy Markdown
Owner

@klehmann klehmann commented Jul 7, 2024

Biggest update ever for Domino JNA!

DQL

  • DQL options to refresh the FT index before search and to rebuild the design catalog 0a34291
  • APIs to create/remove/list DQL index views bd33ee9

MIME

  • convenience function for richtext to HTML conversion with modern standards (e.g. CSS)
  • API to convert MIME to richtext 265a225
  • new project to read and write MIME data using MIME4J project, used by Domino JNA in XPages environments for MIME handling (domino-jna-mime-mime4j) 53b11d4
  • API to set alternative text formats in MIME data like html/plaintext 1a5742f
  • API to render a richtext item as mime objects with all attached files 3698cb5

Named docs

  • Added support for named docs similar to legacy Java/LS API a10f72f

NSF properties/features

  • new constant for DB encryption with AES256 1629538
  • Added methods to check for folder changes ea5203a
  • NotesDatabase.getNSFVersionInfo(), large text list support, more db options e.g. for large summary buffer abe2a49
  • API to mark a NotesNote read/unread 178cd68
  • API to get document attachments as ByteBuffer 0bd7d92
  • API to change view selection formula 2acffc3

Security

  • new API to hash and verify passwords c98d1a0

Misc

  • Added new method to write log messages to the server console a6fdea5
  • 32/64 bit bugfixes
  • Performance improvements

Virtual Views

We completely reinvented Domino view indexing and the QueryResultsProcessor in Java with a lot more features!

A Virtual View is an in-memory Java structure to store, analyze and quickly traverse Domino and any other data. Like Domino views, Virtual Views provide document summary data in a categorized and sorted way with total/average computation, handling of expanded/collapsed categories and options to display a subset of the view (e.g. single category or selected entries).

But the view content can be build from data, profile and design documents, with cross-database support and fast incremental view updates when NSF data is changed.
Compute your column values with formula code or Java functions, with additional data from external data sources (poor man's JOIN).
See this test case for a few examples how this feature is used: https://github.com/klehmann/domino-jna/blob/develop/domino-jna/src/test/java/com/mindoo/domino/jna/test/TestVirtualView.java

And here is the full feature list for the first release of the Virtual View API:

  • multi-DB views
  • view structure similar to Domino (multi level categorization, sorted columns)
  • support for sums / average values
  • compute column values via formula or Java code
  • incremental view updates, so no rebuilt required
  • view is populated by the server, shared across users
  • for each user we check which view entries the user is allowed to see (checks DB ACL level and compares user names list for each DB with computed list of document readers list)
  • for category entries we accumulate the readers of all descendant docs to quickly skip categories that would be empty for a user
  • these collected readers stats can be used for analysis purposes
  • several data sources can be combined to produce view data
  • datasource 1: run NSF search with a formula (incrementally), can search data and design documents, optional post processing with FT search
  • datasource 2: profile documents
  • datasource 3: read note ids from a folder (incrementally)
  • datasource 4: compute column values from any list of note ids
  • custom datasources can be implemented and added to the virtual view
  • VirtualViewNavigator to read the view entries that a user is allowed to see, either all entries/category descendants, just docs/category, support for expanded/selected entries, upwards/downwards/paging like NIFReadEntries, keyword and range lookups
  • fast (e.g. processes 40.000 fakename docs and builds the view in 2-3 seconds)
  • VirtualView currently stored in Java heap (each VirtualViewEntry with a ConcurrentSkipListMap for the sorted children), might add support for serialization to disk later on
  • not a Domino specific implementation, would work in other environments as well

klehmann added 30 commits May 31, 2022 15:22
Depending on the presence/absence of data we may skip creation of
multipart/mixed, multpart/alternative and multipart/related mimeparts
klehmann added 29 commits July 2, 2024 23:37
@klehmann klehmann merged commit a48d57a into master Jul 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant