Skip to content
Thom van Kalkeren edited this page Dec 18, 2019 · 10 revisions

Import

In the container ./bin/ori_api

Cleanup

This is needed when getting disk space full errors while seemingly still having space left. This is because the FS inodes are depleted (df -ih). Deleting old versions will free up inodes.

./bin/ori_api --clean-old-versions

Re-process

Prerequisites

  1. Install kafka (includes the cli tools)
  2. Create a kafka.props file (prevents leaking information to history)
ssl.endpoint.identification.algorithm=https
sasl.mechanism=PLAIN
request.timeout.ms=20000
bootstrap.servers=<server without http but with port>
retry.backoff.ms=500
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="<username>" password="<password>";
security.protocol=SASL_SSL
  1. sudo chmod go-rwx ./kafka.props
  2. Ensure all the consumers in the group have disconnected (kafka will reject otherwise).

Execution

kafka-consumer-groups --bootstrap-server="<server without http but with port>" --command-config kafka.props --group ori_api --topic ori-delta --reset-offsets --to-datetime "2019-10-01T01:00:00.000Z" --dry-run

Change the timestamp, ensure the formatting is correct (it's very particular about that).

Replace --dry-run with --execute to actually execute.

Clone this wiki locally