Skip to content

[Improve] Compatibility improvement for parsing Flink config file #4264

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 22, 2025

Conversation

wolfboys
Copy link
Member

[Improve] Compatibility improvement for parsing Flink config file

Contribution Checklist

Enhanced compatibility for parsing Flink configuration files (supporting both config.yml and flink-conf.yml)

What changes were proposed in this pull request

Issue Number: close #xxx

Brief change log

Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

Does this pull request potentially affect one of the following parts

  • Dependencies (does it add or upgrade a dependency): (yes / no)

Copy link

@wolfboys wolfboys requested a review from Copilot June 21, 2025 19:00
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This pull request improves configuration file parsing compatibility by migrating calls from the old PropertiesUtils to new utility classes (FlinkConfigurationUtils and SparkConfigurationUtils). Key changes include updating argument extraction in the Flink client trait and various console service files, and enhancing Flink configuration parsing in FlinkEnv based on Flink version.

Reviewed Changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated no comments.

Show a summary per file
File Description
streampark-flink-client-core/src/main/scala/org/apache/streampark/flink/client/trait/FlinkClientTrait.scala Replaced PropertiesUtils.extractArguments with FlinkConfigurationUtils.extractArguments.
streampark-console-service/src/main/java/org/apache/streampark/console/core/watcher/FlinkK8sWatcherWrapper.java Updated dynamic property extraction by switching to FlinkConfigurationUtils.
streampark-console-service/src/main/java/org/apache/streampark/console/core/service/impl/FlinkSavepointServiceImpl.java Updated dynamic property extraction and removed import for obsolete utility.
streampark-console-service/src/main/java/org/apache/streampark/console/core/service/application/impl/SparkApplicationActionServiceImpl.java Replaced multiple PropertiesUtils calls with SparkConfigurationUtils methods.
streampark-console-service/src/main/java/org/apache/streampark/console/core/service/application/impl/FlinkApplicationActionServiceImpl.java Updated dynamic property extraction to utilize FlinkConfigurationUtils.
streampark-console-service/src/main/java/org/apache/streampark/console/core/runner/QuickStartRunner.java Replaced PropertiesUtils.extractMultipleArgumentsAsJava with FlinkConfigurationUtils equivalent.
streampark-common/src/test/scala/org/apache/streampark/common/util/PropertiesUtilsTestCase.scala Updated test to use FlinkConfigurationUtils methods instead of PropertiesUtils.
streampark-common/src/main/scala/org/apache/streampark/common/util/SparkConfigurationUtils.scala & FlinkConfigurationUtils.scala Added new utility classes for configuration parsing, replacing deprecated PropertiesUtils methods.
streampark-console-service/src/main/java/org/apache/streampark/console/core/entity/SparkEnv.java & FlinkEnv.java Updated config parsing logic and removed several redundant version helper methods in SparkEnv; introduced legacy versus new parsing logic in FlinkEnv based on Flink version.
Comments suppressed due to low confidence (1)

streampark-console/streampark-console-service/src/main/java/org/apache/streampark/console/core/entity/FlinkEnv.java:178

  • The methods getVersionOfFirst() and getVersionOfMiddle() are being invoked but are not defined in FlinkEnv, which may lead to a compile-time error. Consider implementing these helper methods or refactoring the version parsing logic to directly process the version string.
            return Float.parseFloat(getVersionOfFirst() + "." + getVersionOfMiddle());

@wolfboys wolfboys merged commit 4ab0db5 into dev Jun 22, 2025
60 of 62 checks passed
@wolfboys wolfboys deleted the flink-new-conf branch July 5, 2025 15:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants