Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 69 additions & 21 deletions .github/workflows/coverage_runner.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ jobs:
- ${{ needs.python-versions.outputs.earliest-python-version }}
- ${{ needs.python-versions.outputs.latest-python-version }}
os: [ ubuntu-latest, windows-latest ]
hz_version: [ "5.6.0" ]
fail-fast: false

steps:
Expand Down Expand Up @@ -74,10 +75,6 @@ jobs:
with:
ref: refs/pull/${{ github.event.inputs.pr_number }}/merge

- name: Install dependencies
run: |
pip install -r requirements-test.txt

- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v4
with:
Expand All @@ -90,13 +87,74 @@ jobs:
with:
secret-ids: |
HAZELCAST_ENTERPRISE_KEY,CN/HZ_LICENSE_KEY

- name: Checkout to certificates
uses: actions/checkout@v3
with:
repository: hazelcast/private-test-artifacts
path: certs
ref: data
token: ${{ secrets.GH_TOKEN }}

- name: Create the test jar with certificates (Linux)
if: matrix.os == 'ubuntu-latest'
working-directory: certs
run: |
zip -r -j certs.jar $GITHUB_WORKSPACE/tests/integration/backward_compatible/ssl_tests/hostname_verification/*.p12
cp certs.jar ../hazelcast-enterprise-${{ matrix.hz_version }}-tests.jar

- name: Run tests
- name: Create the test jar with certificates (Windows)
if: matrix.os == 'windows-latest'
working-directory: certs
run: |
$compress = @{
Path = "../tests/integration/backward_compatible/ssl_tests/hostname_verification/*.p12"
CompressionLevel = "Fastest"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit - can we omit and use implicit defaults?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The default is different, but I guess Fastest doesn't hurt.

DestinationPath = "certs.jar"
}
Compress-Archive -Update @compress
cp certs.jar ../hazelcast-enterprise-${{ matrix.hz_version }}-tests.jar

- name: Download RCD (Linux)
if: matrix.os == 'ubuntu-latest'
shell: bash
run: |
wget -q https://client-rcd-download.s3.us-east-2.amazonaws.com/rcd-ubuntu-latest

- name: Download RCD (Windows)
if: matrix.os == 'windows-latest'
run: |
$ProgressPreference = 'SilentlyContinue'
Invoke-WebRequest https://client-rcd-download.s3.us-east-2.amazonaws.com/rcd-windows-latest.exe -OutFile rcd-windows-latest.exe

- name: Install dependencies
run: |
pip install -r requirements-test.txt

- name: Run tests (Linux)
if: matrix.os == 'ubuntu-latest'
env:
HZ_SNAPSHOT_INTERNAL_USERNAME: ${{ secrets.HZ_SNAPSHOT_INTERNAL_USERNAME }}
HZ_SNAPSHOT_INTERNAL_PASSWORD: ${{ secrets.HZ_SNAPSHOT_INTERNAL_PASSWORD }}
run: python run_tests.py

HZ_VERSION: ${{ matrix.hz_version }}
run: |
chmod +x rcd-ubuntu-latest
./rcd-ubuntu-latest -version $HZ_VERSION &
# wait for a bit for RCD to download artifacts
sleep 10
pytest --verbose --cov=hazelcast --cov-report=xml

- name: Run tests (Windows)
if: matrix.os == 'windows-latest'
env:
HZ_VERSION: ${{ matrix.hz_version }}
run: |
Start-Process -FilePath .\rcd-windows-latest -ArgumentList '-version', $Env:HZ_VERSION -RedirectStandardOutput rcd-stdout.log -RedirectStandardError rcd-stderr.log
# wait for a bit for RCD to download artifacts
sleep 10
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit - comment explaining why sleep necessary (presumably for RCD to start?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added comments at: 3111e1b

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok - not ideal to sleep rather than polling for something expected. Could we query the hz healthcheck or something to check when it's up?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could check for the port to be OK to read, but sleeping is simple and worked so far.

echo "RCD Log:"
cat rcd-stdout.log
cat rcd-stderr.log
pytest --verbose --cov=hazelcast --cov-report=xml

- name: Publish results to Codecov for PR coming from hazelcast organization
if: ${{ matrix.python-version == needs.python-versions.outputs.latest-python-version && matrix.os == 'ubuntu-latest' && github.event_name == 'pull_request_target' }}
uses: codecov/codecov-action@v5
Expand All @@ -105,15 +163,15 @@ jobs:
files: ./coverage.xml
override_pr: ${{ github.event.pull_request.number }}
fail_ci_if_error: true

- name: Publish results to Codecov for Push
if: ${{ matrix.python-version == needs.python-versions.outputs.latest-python-version && matrix.os == 'ubuntu-latest' && github.event_name == 'push' }}
uses: codecov/codecov-action@v5
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: ./coverage.xml
fail_ci_if_error: true

- name: Publish result to Codecov for PR coming from community
if: ${{ matrix.python-version == needs.python-versions.outputs.latest-python-version && matrix.os == 'ubuntu-latest' && github.event_name == 'workflow_dispatch' }}
uses: codecov/codecov-action@v5
Expand All @@ -122,13 +180,3 @@ jobs:
files: ./coverage.xml
override_pr: ${{ github.event.inputs.pr_number }}
fail_ci_if_error: true

- name: Upload remote controller logs if test run fails
uses: actions/upload-artifact@v4
if: failure()
with:
name: rc-logs-${{ matrix.python-version }}-${{ matrix.os }}
path: |
rc_stderr.log
rc_stdout.log

Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
<network>
<ssl enabled="true">
<factory-class-name>
com.hazelcast.nio.ssl.BasicSSLContextFactory
com.hazelcast.nio.ssl.ClasspathSSLContextFactory
</factory-class-name>
<properties>
<property name="keyStore">%s</property>
Expand Down Expand Up @@ -130,6 +130,6 @@ async def start_client_with(
)

def start_member_with(self, keystore_name: str) -> None:
config = MEMBER_CONFIG % get_abs_path(current_directory, keystore_name)
config = MEMBER_CONFIG % keystore_name
self.cluster = self.create_cluster(self.rc, config)
self.cluster.start_member()
Original file line number Diff line number Diff line change
Expand Up @@ -163,7 +163,4 @@ async def test_ma_optional_with_no_cert_file(self):
def read_config(self, is_ma_required):
file_path = self.ma_req_xml if is_ma_required else self.ma_opt_xml
with open(file_path, "r") as f:
xml_config = f.read()
keystore_path = get_abs_path(self.current_directory, "server1.keystore")
truststore_path = get_abs_path(self.current_directory, "server1.truststore")
return xml_config % (keystore_path, truststore_path)
return f.read()
10 changes: 2 additions & 8 deletions tests/integration/asyncio/ssl_tests/ssl_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -121,14 +121,8 @@ async def test_ssl_enabled_with_protocol_mismatch(self):

def read_default_ca_config(self):
with open(self.default_ca_xml, "r") as f:
xml_config = f.read()

keystore_path = get_abs_path(self.current_directory, "keystore.jks")
return xml_config % (keystore_path, keystore_path)
return f.read()

def read_ssl_config(self):
with open(self.hazelcast_ssl_xml, "r") as f:
xml_config = f.read()

keystore_path = get_abs_path(self.current_directory, "server1.keystore")
return xml_config % keystore_path
return f.read()
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,15 @@
<network>
<ssl enabled="true">
<factory-class-name>
com.hazelcast.nio.ssl.BasicSSLContextFactory
com.hazelcast.nio.ssl.ClasspathSSLContextFactory
</factory-class-name>
<properties>
<property name="keyStore">%s</property>
<property name="keyStore">com/hazelcast/nio/ssl/letsencrypt.jks
</property>
<property name="keyStorePassword">123456</property>
<property name="keyManagerAlgorithm">SunX509</property>
<property name="trustStore">%s</property>
<property name="trustStore">com/hazelcast/nio/ssl/letsencrypt.jks
</property>
<property name="truestStorePassword">123456</property>
<property name="trustManagerAlgorithm">SunX509</property>
<property name="protocol">TLSv1.2</property>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,13 @@
<network>
<ssl enabled="true">
<factory-class-name>
com.hazelcast.nio.ssl.BasicSSLContextFactory
com.hazelcast.nio.ssl.ClasspathSSLContextFactory
</factory-class-name>
<properties>
<property name="keyStore">%s</property>
<property name="keyStore">com/hazelcast/nio/ssl-mutual-auth/server1.keystore
</property>
<property name="keyStorePassword">password</property>
<property name="trustStore">%s</property>
<property name="trustStore">com/hazelcast/nio/ssl-mutual-auth/server1_knows_client1/server1.truststore</property>
<property name="trustStorePassword">password</property>
<property name="trustManagerAlgorithm">SunX509</property>
<property name="javax.net.ssl.mutualAuthentication">OPTIONAL</property>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,12 @@
<network>
<ssl enabled="true">
<factory-class-name>
com.hazelcast.nio.ssl.BasicSSLContextFactory
com.hazelcast.nio.ssl.ClasspathSSLContextFactory
</factory-class-name>
<properties>
<property name="keyStore">%s</property>
<property name="keyStore">com/hazelcast/nio/ssl-mutual-auth/server1.keystore</property>
<property name="keyStorePassword">password</property>
<property name="trustStore">%s</property>
<property name="trustStore">com/hazelcast/nio/ssl-mutual-auth/server1_knows_client1/server1.truststore</property>
<property name="trustStorePassword">password</property>
<property name="trustManagerAlgorithm">SunX509</property>
<property name="javax.net.ssl.mutualAuthentication">REQUIRED</property>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@
<network>
<ssl enabled="true">
<factory-class-name>
com.hazelcast.nio.ssl.BasicSSLContextFactory
com.hazelcast.nio.ssl.ClasspathSSLContextFactory
</factory-class-name>
<properties>
<property name="keyStore">%s</property>
<property name="keyStore">com/hazelcast/nio/ssl-mutual-auth/server1.keystore</property>
<property name="keyStorePassword">password</property>
<property name="keyManagerAlgorithm">SunX509</property>
<property name="protocol">TLSv1.2</property>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
<network>
<ssl enabled="true">
<factory-class-name>
com.hazelcast.nio.ssl.BasicSSLContextFactory
com.hazelcast.nio.ssl.ClasspathSSLContextFactory
</factory-class-name>
<properties>
<property name="keyStore">%s</property>
Expand Down Expand Up @@ -137,6 +137,6 @@ def start_client_with(
)

def start_member_with(self, keystore_name: str) -> None:
config = MEMBER_CONFIG % get_abs_path(current_directory, keystore_name)
config = MEMBER_CONFIG % keystore_name
self.cluster = self.create_cluster(self.rc, config)
self.cluster.start_member()
Binary file modified tests/integration/backward_compatible/ssl_tests/keystore.jks
Binary file not shown.
Original file line number Diff line number Diff line change
Expand Up @@ -167,9 +167,4 @@ def test_ma_optional_with_no_cert_file(self):
def read_config(self, is_ma_required):
file_path = self.ma_req_xml if is_ma_required else self.ma_opt_xml
with open(file_path, "r") as f:
xml_config = f.read()

curr_dir = os.path.dirname(__file__)
keystore_path = get_abs_path(curr_dir, "server1.keystore")
truststore_path = get_abs_path(curr_dir, "server1.truststore")
return xml_config % (keystore_path, truststore_path)
return f.read()
10 changes: 2 additions & 8 deletions tests/integration/backward_compatible/ssl_tests/ssl_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -117,14 +117,8 @@ def test_ssl_enabled_with_protocol_mismatch(self):

def read_default_ca_config(self):
with open(self.default_ca_xml, "r") as f:
xml_config = f.read()

keystore_path = get_abs_path(self.current_directory, "keystore.jks")
return xml_config % (keystore_path, keystore_path)
return f.read()

def read_ssl_config(self):
with open(self.hazelcast_ssl_xml, "r") as f:
xml_config = f.read()

keystore_path = get_abs_path(self.current_directory, "server1.keystore")
return xml_config % keystore_path
return f.read()
Loading