Added

• Added configurable shard-level concurrent reconciliation by @zcross in #1124. It is controlled by the following operator configuration settings:

reconcile:
  runtime:
    # Max number of concurrent CHI reconciles in progress
    reconcileCHIsThreadsNumber: 10
    # Max number of concurrent shard reconciles in progress
    reconcileShardsThreadsNumber: 1
    # The maximum percentage of cluster shards that may be reconciled in parallel
    reconcileShardsMaxConcurrencyPercent: 50

• Added default configuration for ClickHouse system.trace_log table with 30 days TTL

Changed

• ZooKeeper manifests were rewritten to store configuration separately

Fixed

• Fixed a bug in metrics-exporter that might stop working on ClickHouse nodes with certain types of system.errors. Closes #1161
• Fixed a bug with ClickHouse major version detection for Altinity.Stable builds

Full Changelog: release-0.21.0...release-0.21.1

What's Changed

• Changed the way Operator applies ClickHouse settings. In the previous version, every change in settings required a restart via re-creating a StatefulSet. In this version it does not re-create StatefulSet anymore, but maintains a logic that decide if ClickHouse needs to be restarted in order to pickup a change or not. In case of restart it is performed using scaling stateful set down and up. The restart logic is controlled by configurationRestartPolicy configuration setting. Here is the default configuration:

    rules:
      - version: "*"
        rules:
          - settings/*: "yes"
          - settings/dictionaries_config: "no"
          - settings/logger: "no"
          - settings/macros/*: "no"
          - settings/max_server_memory_*: "no"
          - settings/max_*_to_drop: "no"
          - settings/max_concurrent_queries: "no"
          - settings/models_config: "no"
          - settings/user_defined_executable_functions_config: "no"

          - zookeeper/*: "yes"

          - files/config.d/*.xml: "yes"
          - files/config.d/*dict*.xml: "no"

          - profiles/default/background_*_pool_size: "yes"
          - profiles/default/max_*_for_server: "yes"
      - version: "21.*"
        rules:
          - settings/logger: "yes" 

• Improved reliability of schema creation for new shards/replicas
• Added "secure" option for Zookeeper connections by @Tvion in #1114
• Added "insecure" flag that closes insecure TCP/HTTP ClickHouse ports. See security_hardening.md for more detail
• Added an option to disable metrics exporter by @roimor in #1131
• Added system.errors scrapping into metrics-exporter
• Refactor Registry internals to make concurrent access safe by @zcross in #1115

Fixed

• Fixed a bug that might result in PDB being deleted. Closes #1139
• Fixed propagation of podTemplate environment variables from ClickHouseInstallationTemplate to ClickHouseInstallation
• Fixed propagation of startup probe from ClickHouseInstallationTemplate to ClickHouseInstallation

Non-operator changes

• Changed Grafana deployment to allow persisting custom dashboards
• Changed ZooKeeper version to 3.8.1

New Contributors

• @roimor made their first contribution in #1131
• @Tvion made their first contribution in #1114
• @zcross made their first contribution in #1115

Full Changelog: release-0.20.3...release-0.21.0

What's Changed

• Use alpine base image instead of UBI
• Fix error handling when table already exists in ZooKeeper for new ClickHouse versions

Full Changelog: release-0.20.2...release-0.20.3

What's Changed

• Added 'hostsCompleted' to the CHI status and events
• Changed some 'default' profile settings:
  • Enabled 'do_not_merge_across_partitions_select_final'
  • Set 'load_balancing' to 'nearest_hostname'
  • Set niceness ('os_thread_priority') to 2.
• Improved stability of metric-exporter when some ClickHouse nodes are responding slowly
• Changed sequence of LB service creation to avoid a situation when service exists with no available endpoints
• Added 'secure' flag at cluster level for enabling distributed queries over TLS
• Addressed CVEs in dependent packages

Note: datatype for 'secure' flag has been changed from boolean to String (accepting, 'true', 'yes', '1' etc.)

What's Changed

• Improvements to Grafana dashboards by @Slach
• Generate operator helm chart by @slamdev in #1049
• Added cluster to PodDesruptionBudget selector. Closes #996. Note: PDB requires Kubernetes version 1.21 or above.
• Added status.hostsCompleted to track reconcile progress
• Do not restart CHI with restart attribute when operator is restarted
• Address https://nvd.nist.gov/vuln/detail/CVE-2022-27664. Closes #1039
• Fixed RBAC permissions for secrets. Closes #1051

New Contributors

• @slamdev made their first contribution in #1049

Full Changelog: release-0.20.0...release-0.20.1

What's Changed

• Add support for setting secure communication between clickhouse instances. (Inspried by @chancez in #938). See examples:
  • auto secret
  • plain text secret
  • k8s secret reference
• Operator managed PV provisioning. (Inspired by @chhan-coupang in #947). Allows to re-scale volumes w/o a downtime. In order to enable, use:

  defaults:
    storageManagement:
      provisioner: Operator

• Use secret for clickhouse_operator credentials as default
• Improve zookeeper scaleout by @Slach in #984
• add loadaverage + network + disk panel to clickhouse dashboard by @Slach in #1015
• comment out chopusername check by @SuzyWangIBMer in #1009
• Reduced a size of status field
• Improved logging

Fixed

• Fixed a bug with cluster definitions being reset when CHI has been updated via k8s API POST method.
• Fixed a problem of metrics exporter not being able to scrape any managed cluster when one of clickhouse nodes is slow
• Fix zookeeper advanced setup with empty dir by @luluz66 in #993
• Delete duplicate return by @Abirdcfly in #987
• Fixes #991 - fix watch namespace regex by @mcgrawia in #992
• Address CVE-2022-32149 by @bkuschel in #1035

New Contributors

• @Abirdcfly made their first contribution in #987
• @mcgrawia made their first contribution in #992
• @chhan-coupang made their first contribution in #947
• @luluz66 made their first contribution in #993
• @chancez made their first contribution in #938
• @antip00 made their first contribution in #1034

Full Changelog: release-0.19.2...release-0.20.0

What's Changed

• Fixed a possible loose of cluster state when operator has been restarted in the middle of reconcile.
• Backport of #1009
• add loadaverage + network + disk panel to clickhouse dashboard by @Slach in #1015

NOTE: There is a change in ClickHouseInstallation custom resource type, so make sure you deploy full installation manifest when upgrading.

Full Changelog: release-0.19.2...release-0.19.3

What's Changed

• Fixed a bug in 0.19.0 and 0.19.1 versions when operator could be rebuilding remote-servers.xml from scratch after restart. When it happened, distributed queries might behave unexpectedly.

Full Changelog: release-0.19.1...release-0.19.2

What's Changed

• Fixed MVs in Atomic databases support for ClickHouse versions 22.6 and above. Closes #989
• Fixed tables with '{uuid}' macro in Atomic databases for ClickHouse versions 22.6 and above.
• Fixed watch namespaces regexp by #992. Closes #991
• Added "clickhouse.altinity.com/ready: yes|no" annotations for replica services for the use by external load balancer.
• Updated podDisruptionBudget API version

Full changelog: release-0.19.0...release-0.19.1

What's Changed

• Now 'clickhouse-operator' user is restricted to an IP address of the pod where clickhouse-operator is running. It can not be used from anywhere else.
• In addition to hostRegexp, 'default' user is restricted to IP addresses of pods of the CHI. This fixes connectivity issues between shards in GKE.
• ClickHouse 'interserver_http_host' is set to the service name and matches 'remote_servers'. It can be changed to full names with 'replicasUseFQDN' setting if needed. This fixes connectivity issues between replicas in GKE.
• Changed the way schema is propagated to new shards. It is currently controlled by two cluster level settings:

  schemaPolicy:
    replica: None | All (default)
    shard: None | All (default) | DistributedTablesOnly

Previously, when adding shards only distributed tables and dependent objects were created (DistributedTablesOnly).

• Added support for adding custom CA for operator connection to ClickHouse by @SuzyWangIBMer in #930
• Fixed support of '{uuid}' macro for Atomic databases. Closes #857
• Fixed a bug when replica sometimes was not removed from ZooKeeper when scaling down the cluster
• Set container-log on ubi instead of busybox by @SuzyWangIBMer in #940
• upgrade vertamedia-clickhouse-grafana to 2.5.0 in dashboard and grafana 7.5.16 by @Slach in #951
• Improve grafana dashboards by @Slach in #952
• Improve grafana-operator + prometheus-operator by @Slach in #953

New Contributors

• @roshanths made their first contribution in #936
• @SuzyWangIBMer made their first contribution in #940
• @meob made their first contribution in #949

Full Changelog: 0.18.5...release-0.19.0