Skip to content

Extract entrypoint command from bin scripts, try native-image zookeeper #311

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 41 commits into from
Jul 12, 2020
Merged

Extract entrypoint command from bin scripts, try native-image zookeeper #311

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
066477a
Makes static (pre-generated) configuration files possible
solsson Mar 6, 2020
56cf85a
Use generated entrypoints, no shell script around the java process
solsson Mar 8, 2020
86b2990
Append to args, not to command, now that we use images' entrypoints
solsson Mar 8, 2020
cc4e968
Nonroot zookeeper had to be restarted once after create
solsson Mar 10, 2020
98663a2
Kafka 2.4.1
solsson Mar 15, 2020
76eee47
kafka-minion ec29ba8c30cb4cdf9bbd04dc55a854b686f85a97
solsson Mar 9, 2020
bc2896a
Unlike broker config zookeeper's needs to know the replica count
solsson Apr 21, 2020
7f5c78c
Merge branch 'static-props-some-day' into nonroot-entrypoints
solsson Apr 22, 2020
d91a92a
Here we need to know the number of replicas, not as two envs
solsson Apr 22, 2020
50e28a6
Transition to a single REPLICAS env (phasing out pzoo)
solsson Apr 22, 2020
6d86264
Crazy fast zookeeper start, crazy cheap so we can scale up
solsson Apr 22, 2020
bec515c
Kafka scales up nicely with no need to restart existing pods
solsson Apr 22, 2020
f2a302e
Connect by name resolution to individual zookeeper pods instead of
solsson Apr 23, 2020
a2802e7
Guaranteed, with limits that will only work given quite even load
solsson Apr 23, 2020
f42afde
AdoptOpenJDK 11.0.7 and a smaller nonroot user layer
solsson Apr 23, 2020
377a0c3
Kafka 2.5.0 with the default (no extra envs) entrypoint
solsson Apr 24, 2020
ebc5042
Breaking: renames the statefulset's headless service so
solsson Apr 25, 2020
2aef643
Merge pull request #315 from Yolean/entrypoints-args-override-listene…
solsson Apr 25, 2020
df954ae
This probe caused log entries "Unable to read additional data from
solsson Apr 25, 2020
a29a19d
New variant suitable for a 3-node 3-zone cluster
solsson Apr 25, 2020
b1199fa
Current zookeeper doesn't log anything from factory on ruok
solsson Apr 25, 2020
6d0e41b
Zookeeper docs say: In container environment, especially in Kubernetes,
solsson Apr 25, 2020
1320641
Fixes scale-1 (and thus dev-small) with the env change from 50e28a6
solsson May 18, 2020
83af767
Native-image zookeeper for scale-1 and dev-small environments,
solsson May 18, 2020
728acff
Nonroot init-containers, seems to fix #318 on minikube and k3s
solsson May 18, 2020
4ab73a0
Adapts to the new entrypoint strategy, and distroless native zookeper
solsson May 18, 2020
502d483
Speeds up kafka-configs.sh and kafka-topics.sh with --zookeeper
solsson May 18, 2020
9e63e93
Current zookeeper build, with graalvm 20.1 and --install-exit-handlers,
solsson Jun 17, 2020
7e0f456
Fixes mistake in the change from commmand to args override
solsson Jul 12, 2020
4207d20
Merge branch 'master' into nonroot-entrypoints
solsson Jul 12, 2020
372497f
We've never had limits on zookeeper cpu, and we've requested 10
solsson Jul 12, 2020
d5f12ab
Increases max connections per client from 2 to 3 after I had
solsson May 3, 2020
8c5b3fc
cwiki.apache.org/confluence/display/KAFKA/KIP-537
solsson May 5, 2020
7790ee0
Theory: if the name matches the logged "Server environment:host.name"
solsson May 12, 2020
2d83408
Zookeeper 3.5.8
solsson Jul 12, 2020
2eeaa00
Merge branch 'zookeeper-3.5.8' into nonroot-entrypoints
solsson Jul 12, 2020
2c05b72
Fixes the zoo service names. Only pzoo was correct.
solsson Jul 12, 2020
959723f
Documents the upcoming merge
solsson Jul 12, 2020
e409686
Uses actual zookeeper hostnames (see 7790ee0 + 2c05b72),
solsson Jul 12, 2020
d1715d4
Fix changelog markdown
solsson Jul 12, 2020
cee5278
Uses the last builds from solsson/dockerfiles#29
solsson Jul 12, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Unlike broker config zookeeper's needs to know the replica count
  • Loading branch information
@solsson
solsson committed Apr 21, 2020
commit bc2896a348fe6d2d0b23a7cb33435d7d0679dd20
3 changes: 2 additions & 1 deletion zookeeper/10zookeeper-config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,14 @@ data:
export ZOOKEEPER_SERVER_ID=$((${HOSTNAME##*-} + $ID_OFFSET))
echo "${ZOOKEEPER_SERVER_ID:-1}" | tee /var/lib/zookeeper/data/myid
cp -Lur /etc/kafka-configmap/* /etc/kafka/
REPLICAS=$(( PZOO_REPLICAS + ZOO_REPLICAS ))
[ ! -z "$PZOO_REPLICAS" ] && [ ! -z "$ZOO_REPLICAS" ] && {
sed -i "s/^server\\./#server./" /etc/kafka/zookeeper.properties
for N in $(seq $PZOO_REPLICAS); do echo "server.$N=pzoo-$(( $N - 1 )).pzoo:2888:3888:participant" >> /etc/kafka/zookeeper.properties; done
for N in $(seq $ZOO_REPLICAS); do echo "server.$(( $PZOO_REPLICAS + $N ))=zoo-$(( $N - 1 )).zoo:2888:3888:participant" >> /etc/kafka/zookeeper.properties; done
}
sed -i "s/server\.$ZOOKEEPER_SERVER_ID\=[a-z0-9.-]*/server.$ZOOKEEPER_SERVER_ID=0.0.0.0/" /etc/kafka/zookeeper.properties
ln -s /etc/kafka/zookeeper.properties /etc/kafka/zookeeper.properties.$POD_NAME
ln -s /etc/kafka/zookeeper.properties /etc/kafka/zookeeper.properties.scale-$REPLICAS.$POD_NAME

zookeeper.properties: |
4lw.commands.whitelist=ruok
Expand Down
2 changes: 1 addition & 1 deletion zookeeper/50pzoo.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ spec:
value: -Dlog4j.configuration=file:/etc/kafka/log4j.properties
command:
- ./bin/zookeeper-server-start.sh
- /etc/kafka/zookeeper.properties.$(POD_NAME)
- /etc/kafka/zookeeper.properties.scale-5.$(POD_NAME)
lifecycle:
preStop:
exec:
Expand Down
2 changes: 1 addition & 1 deletion zookeeper/51zoo.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ spec:
value: -Dlog4j.configuration=file:/etc/kafka/log4j.properties
command:
- ./bin/zookeeper-server-start.sh
- /etc/kafka/zookeeper.properties.$(POD_NAME)
- /etc/kafka/zookeeper.properties.scale-5.$(POD_NAME)
lifecycle:
preStop:
exec:
Expand Down