RheaKV: 调用 putIfAbsent 抛出异常 RouteTableException message: fail to find region by key

[en-hui]

Describe the bug

使用 putIfAbsent 报错
2025-10-11T08:28:18,603 [dp-oraclesourcetask-1359-2-reader] ERROR com.alipay.sofa.jraft.rhea.client.RegionRouteTable - There is a high probability that the data in the region table is corrupted. {}
2025-10-11T08:28:18,603 [dp-oraclesourcetask-1359-2-reader] ERROR com.alipay.sofa.jraft.rhea.client.RegionRouteTable - --------------------------------------------------------------------------- {}
2025-10-11T08:28:18,603 [dp-oraclesourcetask-1359-2-reader] ERROR com.alipay.sofa.jraft.rhea.client.RegionRouteTable - * RelatedKey: 5B53504C49545F494E49542D313335392D302D3133323737302D313736303137313134353735345D2D32. {}
2025-10-11T08:28:18,605 [dp-oraclesourcetask-1359-2-reader] ERROR com.alipay.sofa.jraft.rhea.client.RegionRouteTable - * RangeTable: {}. {}
2025-10-11T08:28:18,605 [dp-oraclesourcetask-1359-2-reader] ERROR com.alipay.sofa.jraft.rhea.client.RegionRouteTable - * RegionTable: {}. {}
2025-10-11T08:28:18,605 [dp-oraclesourcetask-1359-2-reader] ERROR com.alipay.sofa.jraft.rhea.client.RegionRouteTable - --------------------------------------------------------------------------- {}

Caused by: com.alipay.sofa.jraft.rhea.errors.RouteTableException: key: 5B53504C49545F494E49542D313335392D302D3133323737302D313736303137313134353735345D2D38, message: fail to find region by key
at com.alipay.sofa.jraft.rhea.client.RegionRouteTable.reject(RegionRouteTable.java:335)
at com.alipay.sofa.jraft.rhea.client.RegionRouteTable.findRegionByKeyWithoutLock(RegionRouteTable.java:199)
at com.alipay.sofa.jraft.rhea.client.RegionRouteTable.findRegionByKey(RegionRouteTable.java:188)
at com.alipay.sofa.jraft.rhea.client.pd.AbstractPlacementDriverClient.findRegionByKey(AbstractPlacementDriverClient.java:137)
at com.alipay.sofa.jraft.rhea.client.DefaultRheaKVStore.internalPutIfAbsent(DefaultRheaKVStore.java:1311)
at com.alipay.sofa.jraft.rhea.client.DefaultRheaKVStore.putIfAbsent(DefaultRheaKVStore.java:1290)
at com.alipay.sofa.jraft.rhea.client.DefaultRheaKVStore.putIfAbsent(DefaultRheaKVStore.java:1296)
at com.alipay.sofa.jraft.rhea.client.DefaultRheaKVStore.bPutIfAbsent(DefaultRheaKVStore.java:1306)
at com.datapipeline.base.coordinator.rheakv.kvstore.DpRheaKVStore.lambda$bPutIfAbsent$2(DpRheaKVStore.java:49)
at com.datapipeline.base.coordinator.rheakv.kvstore.DpRheaKVStore.executeWithRetry(DpRheaKVStore.java:114)
... 13 more

参考了https://github.com/sofastack/sofa-jraft/issues/984的回答，但我只设置了一个regionId为-1，且没有配置该region的数据start和end

我记录的并不是数字类型的key，5B53504C49545F494E49542D313335392D302D3133323737302D313736303137313134353735345D2D38 转为字符串为：[SPLIT_INIT-1359-0-132770-1760171145754]-8
这是使用问题，还是代码bug呢

Expected behavior

Actual behavior

Steps to reproduce

Minimal yet complete reproducer code (or GitHub URL to code)

    RheaKVStoreOptions opts = new RheaKVStoreOptions();
    config.fillRheaKvStoreOptions(opts);
    kvStore.addLeaderStateListener(-1, rheaKvStatusListener);
    if (kvStore.init(opts)) {

// 参数设置
public void fillRheaKvStoreOptions(RheaKVStoreOptions kvStoreOptions) {
final NodeOptions nodeOptions = getNodeOptions();

Endpoint endpoint = parseServerAddress(this.serverAddress);
StoreEngineOptionsConfigured storeOptsConfigured =
    StoreEngineOptionsConfigured.newConfigured()
        .withStorageType(StorageType.RocksDB)
        .withRocksDBOptions(
            RocksDBOptionsConfigured.newConfigured().withDbPath(this.dbPath).config())
        .withRaftDataPath(this.dataPath)
        .withServerAddress(endpoint)
        .withCommonNodeOptions(nodeOptions);
if (!enableMetrics) {
  storeOptsConfigured.withMetricsReportPeriod(-1);
}
StoreEngineOptions storeOpts = storeOptsConfigured.config();
final PlacementDriverOptions pdOptions =
    PlacementDriverOptionsConfigured.newConfigured().withFake(true).config();

kvStoreOptions.setClusterName(groupId);
kvStoreOptions.setUseParallelCompress(true);
kvStoreOptions.setInitialServerList(this.initialServerAddress);
kvStoreOptions.setStoreEngineOptions(storeOpts);
kvStoreOptions.setPlacementDriverOptions(pdOptions);

}

Environment

• SOFAJRaft version: 1.3.15.bugfix
• JVM version (e.g. java -version): jdk17
• OS version (e.g. uname -a):
• Maven version:
• IDE version:

[en-hui]

@shihuili1218 @fengjiachun 大佬，能帮忙看下吗？
用了很长时间了都没出问题，突然在一个客户那边遇到了，第一次遇到，把db目录删除重启后能恢复了一段时间，然后过了会又出现了

[fengjiachun]

错误就是 key 不在路由表的范围内，应该是配置问题，你给的似乎是 server 配置，没看到 client 的，参考一下 example 的吧 https://github.com/sofastack/sofa-jraft/blob/373c087824cba920cc87c67592c3bb7ec1d866f1/jraft-example/src/main/java/com/alipay/sofa/jraft/example/rheakv/Client.java

[en-hui]

我给的是server的配置，server和client都没有显式的配置路由表。

我发现可能是init失败了，因为init会返回一个boolean值，但当前没处理false的情况。
假如出现init返回false，好像不能知道失败的原因，内部的LOG.info 都不会输出到日志文件，这个我还要在排查一下，或者您能给出一些提示吗 @fengjiachun

[en-hui]

client的配置如下

public static synchronized RheaKVStore getRheaKVStore() {
if (rheaKVStore == null) {
// init
final List regionRouteTableOptionsList =
MultiRegionRouteTableOptionsConfigured.newConfigured()
.withInitialServerList(-1, COORDINATOR_NODE_RAFT_INITIAL_SERVER_ADDRESS)
.config();
final PlacementDriverOptions pdOpts =
PlacementDriverOptionsConfigured.newConfigured()
.withFake(true)
.withRegionRouteTableOptionsList(regionRouteTableOptionsList)
.config();
final RheaKVStoreOptions opts =
RheaKVStoreOptionsConfigured.newConfigured()
.withClusterName(COORDINATOR_NODE_RAFT_GROUP_ID)
.withPlacementDriverOptions(pdOpts)
.withFutureTimeoutMillis(COORDINATOR_NODE_RAFT_FUTURE_TIMEOUT_MILLIS)
.withFailoverRetries(COORDINATOR_NODE_RAFT_FAILOVER_RETRIES)
.config();

  SystemPropertyUtil.setProperty("jraft.bolt.conn.reconnect", JRAFT_BOLT_CONN_RECONNECT);
  SystemPropertyUtil.setProperty(
      "rhea.default_future_timeout", JRAFT_RHEA_CLIENT_FUTURE_TIMEOUT_MILLIS);
  log.info("[JRaft RheaKV] init options: {}", opts);
  rheaKVStore = new DefaultRheaKVStore();
  rheaKVStore.init(opts);
  try {
    // init 的检查
    rheaKVStore.bGet("test-init");
  } catch (RheaRuntimeException e) {
    if (e.getMessage().contains("rhea kv is not started or shutdown")) {
      rheaKVStore = null;
      log.error("[JRaft RheaKV] rheaKVStore init failed!", e);
      throw new RheaKVException("rheaKVStore init failed!");
    }
  }
}
return rheaKVStore;

}

[fengjiachun]

@en-hui 你这个问题应该不是偶现，而是必现的，所以我猜你本地按照你自己的配置拉起服务，测一把， debug 或者打打日志应该就能找到问题了

[en-hui]

@fengjiachun 他还真不是毕现的，测试环境，我尝试debug抛出错误，可以看到异常里面会打印RouteTable的信息，不是空的。而报错的时候，这里是没有的。我现在想打印info日志，让生产环境出错的时候可以有排查的方案