Skip to content

Improve zookeeper scaleout #984

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Slach merged 13 commits into from
Sep 5, 2022
Merged

Improve zookeeper scaleout #984

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
114ef9d
Merge branch '0.19.0' of github.com:Altinity/clickhouse-operator into…
Slach May 17, 2022
43e3bda
Merge branch '0.19.0' of github.com:Altinity/clickhouse-operator into…
Slach May 23, 2022
445b37b
fix `test_keeper`, add test_zookeeper_operator_probes_workload and te…
Slach May 23, 2022
816569b
Merge branch '0.19.0' of github.com:Altinity/clickhouse-operator into…
Slach Jun 21, 2022
fb090e2
Merge branch 'master' into improve_zookeeper_scaleout
Slach Aug 3, 2022
0773ab3
Merge remote-tracking branch 'origin/improve_zookeeper_scaleout' into…
Slach Aug 3, 2022
7f9d290
remove unnecessary step from create-prometheus.sh, fix 22.7 `ReadOnly…
Slach Aug 4, 2022
19502b7
`test_clickhouse_keeper_rescale` pass, but `test_clickhouse_keeper_pr…
Slach Aug 4, 2022
88c36b2
clickhouse-keeper fail after insert 10000 parts, look https://github.…
Slach Aug 4, 2022
5154a9a
log cleanup rename table to keeper_bench
Slach Aug 5, 2022
492c2b2
Merge branch '0.20.0' of github.com:Altinity/clickhouse-operator into…
Slach Aug 21, 2022
1bf8fee
Merge branch 'master' of github.com:Altinity/clickhouse-operator into…
Slach Sep 5, 2022
55222ad
Merge branch '0.20.0' of github.com:Altinity/clickhouse-operator into…
Slach Sep 5, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 8 additions & 13 deletions deploy/clickhouse-keeper/clickhouse-keeper-1-node-256M-for-test-only.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,7 @@ spec:
containers:
- name: clickhouse-keeper
imagePullPolicy: Always
image: "clickhouse/clickhouse-server:head"
image: "clickhouse/clickhouse-keeper:head-alpine"
resources:
requests:
memory: "256M"
Expand Down Expand Up @@ -181,18 +181,13 @@ spec:
echo "</keeper_server></yandex>"
} > /tmp/clickhouse-keeper/config.d/generated-keeper-settings.xml &&
cat /tmp/clickhouse-keeper/config.d/generated-keeper-settings.xml &&
clickhouse keeper --config-file=/etc/clickhouse-keeper/keeper_config.xml --force-recovery
readinessProbe:
tcpSocket:
port: 9444
timeoutSeconds: 5
periodSeconds: 60
failureThreshold: 3
livenessProbe:
tcpSocket:
port: 2181
timeoutSeconds: 5
periodSeconds: 30
clickhouse-keeper --config-file=/etc/clickhouse-keeper/keeper_config.xml --force-recovery
# readinessProbe:
# tcpSocket:
# port: 9444
# timeoutSeconds: 5
# periodSeconds: 60
# failureThreshold: 3
- name: zookeeper-exporter
imagePullPolicy: IfNotPresent
image: dabealu/zookeeper-exporter:latest
Expand Down
4 changes: 2 additions & 2 deletions deploy/clickhouse-keeper/clickhouse-keeper-1-node.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ spec:
containers:
- name: clickhouse-keeper
imagePullPolicy: IfNotPresent
image: "clickhouse/clickhouse-server:latest"
image: "clickhouse/clickhouse-keeper:latest"
resources:
requests:
memory: "256M"
Expand Down Expand Up @@ -162,7 +162,7 @@ spec:
echo "</keeper_server></yandex>"
} > /tmp/clickhouse-keeper/config.d/generated-keeper-settings.xml &&
cat /tmp/clickhouse-keeper/config.d/generated-keeper-settings.xml &&
clickhouse keeper --config-file=/etc/clickhouse-keeper/keeper_config.xml --force-recovery
clickhouse-keeper --config-file=/etc/clickhouse-keeper/keeper_config.xml --force-recovery
readinessProbe:
tcpSocket:
port: 9444
Expand Down
23 changes: 9 additions & 14 deletions deploy/clickhouse-keeper/clickhouse-keeper-3-nodes-256M-for-test-only.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,7 @@ spec:
containers:
- name: clickhouse-keeper
imagePullPolicy: Always
image: "clickhouse/clickhouse-server:head"
image: "clickhouse/clickhouse-keeper:head-alpine"
resources:
requests:
memory: "256M"
Expand Down Expand Up @@ -182,21 +182,16 @@ spec:
} > /tmp/clickhouse-keeper/config.d/generated-keeper-settings.xml &&
cat /tmp/clickhouse-keeper/config.d/generated-keeper-settings.xml &&
if [[ "1" == "$MY_ID" ]]; then
clickhouse keeper --config-file=/etc/clickhouse-keeper/keeper_config.xml --force-recovery
clickhouse-keeper --config-file=/etc/clickhouse-keeper/keeper_config.xml --force-recovery
else
clickhouse keeper --config-file=/etc/clickhouse-keeper/keeper_config.xml
clickhouse-keeper --config-file=/etc/clickhouse-keeper/keeper_config.xml
fi
readinessProbe:
tcpSocket:
port: 9444
timeoutSeconds: 5
periodSeconds: 60
failureThreshold: 3
livenessProbe:
tcpSocket:
port: 2181
timeoutSeconds: 5
periodSeconds: 30
# readinessProbe:
# tcpSocket:
# port: 9444
# timeoutSeconds: 5
# periodSeconds: 60
# failureThreshold: 3
- name: zookeeper-exporter
imagePullPolicy: IfNotPresent
image: dabealu/zookeeper-exporter:latest
Expand Down
6 changes: 3 additions & 3 deletions deploy/clickhouse-keeper/clickhouse-keeper-3-nodes.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ spec:
containers:
- name: clickhouse-keeper
imagePullPolicy: IfNotPresent
image: "clickhouse/clickhouse-server:latest"
image: "clickhouse/clickhouse-keeper:latest"
resources:
requests:
memory: "256M"
Expand Down Expand Up @@ -163,9 +163,9 @@ spec:
} > /tmp/clickhouse-keeper/config.d/generated-keeper-settings.xml &&
cat /tmp/clickhouse-keeper/config.d/generated-keeper-settings.xml &&
if [[ "1" == "$MY_ID" ]]; then
clickhouse keeper --config-file=/etc/clickhouse-keeper/keeper_config.xml --force-recovery
clickhouse-keeper --config-file=/etc/clickhouse-keeper/keeper_config.xml --force-recovery
else
clickhouse keeper --config-file=/etc/clickhouse-keeper/keeper_config.xml
clickhouse-keeper --config-file=/etc/clickhouse-keeper/keeper_config.xml
fi
readinessProbe:
tcpSocket:
Expand Down
5 changes: 0 additions & 5 deletions deploy/prometheus/create-prometheus.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,11 +31,6 @@ kubectl create namespace "${PROMETHEUS_NAMESPACE}" || true
BASE_PATH="https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/${PROMETHEUS_OPERATOR_BRANCH}"

echo "Setup prometheus-operator into '${PROMETHEUS_NAMESPACE}' namespace."
kubectl --namespace="${PROMETHEUS_NAMESPACE}" apply --server-side --validate="${VALIDATE_YAML}" -f <( \
wget -qO- "${BASE_PATH}"/bundle.yaml | \
sed "s/namespace: default/namespace: ${PROMETHEUS_NAMESPACE}/" \
)

kubectl --namespace="${PROMETHEUS_NAMESPACE}" apply --server-side --validate="${VALIDATE_YAML}" -f <( \
wget -qO- "${BASE_PATH}"/bundle.yaml | \
sed "s/namespace: default/namespace: ${PROMETHEUS_NAMESPACE}/" | \
Expand Down
Loading