NetBox container 4.1.0 (or newer) SSL crypto error on startup

[tjmullicani]

Current Behavior

I get the following error starting NetBox on docker using tag v4.1.0 or newer (running on Kubernetes). Image tag v4.0.11 or older works without issue.

ssl.SSLError: [CRYPTO] unknown error (_ssl.c:3076)

Expected Behavior

Container successfully runs

Docker Compose Version

kubernetes 1.30.5 helm chart version netbox-5.0.0-beta.139

Azure Linux with FIPS enabled

Docker Version

n/a

The git Revision

n/a

The git Status

n/a

Startup Command

n/a

NetBox Logs

Traceback (most recent call last):
  File "/opt/netbox/netbox/./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/core/management/__init__.py", line 382, in execute
    settings.INSTALLED_APPS
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/conf/__init__.py", line 89, in __getattr__
    self._setup(name)
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/conf/__init__.py", line 76, in _setup
    self._wrapped = Settings(settings_module)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/conf/__init__.py", line 190, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/opt/netbox/netbox/netbox/settings.py", line 10, in <module>
    import requests
  File "/opt/netbox/venv/lib/python3.12/site-packages/requests/__init__.py", line 164, in <module>
    from .api import delete, get, head, options, patch, post, put, request
  File "/opt/netbox/venv/lib/python3.12/site-packages/requests/api.py", line 11, in <module>
    from . import sessions
  File "/opt/netbox/venv/lib/python3.12/site-packages/requests/sessions.py", line 15, in <module>
    from .adapters import HTTPAdapter
  File "/opt/netbox/venv/lib/python3.12/site-packages/requests/adapters.py", line 80, in <module>
    _preloaded_ssl_context = create_urllib3_context()
                             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.12/site-packages/urllib3/util/ssl_.py", line 292, in create_urllib3_context
    context = SSLContext(PROTOCOL_TLS_CLIENT)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/ssl.py", line 438, in __new__
    self = _SSLContext.__new__(cls, protocol)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ssl.SSLError: [CRYPTO] unknown error (_ssl.c:3076)
⏳ Waiting on DB... (0s / 30s)

Content of docker-compose.override.yml

n/a

[tjmullicani]

similar error certbot/certbot#10044

[Pyromonkey83]

Have a similar issue on one of my hosts, and I'm unsure as to why. I have 1 dev environment and 3 prod environments running the same image. Works on dev, works on 2 prod environments. Third prod environment throws this error, despite the build being identical.

EDIT: Builds were not identical. FIPS mode was enabled on the third host throwing the errors, but disabled on the other prod units. Disabling FIPS mode allows the container to start.

[alexandrud]

I had a similar issue, but I also loaded some custom ROOT CAs and apparently there was a newline missing when concatenating the certs. Maybe it helps.

[timothy-mullican]

A workaround is to set the environment variable OPENSSL_FORCE_FIPS_MODE to 0. Example for the NetBox helm chart values.yaml:

...
extraEnvs:
  - name: OPENSSL_FORCE_FIPS_MODE
     value: "0"
worker:
  extraEnvs:
    - name: OPENSSL_FORCE_FIPS_MODE
       value: "0"

[timothy-mullican]

I am also attaching a work in progress Dockerfile that runs rootless NetBox on Rocky Linux 9 (unable to use RHEL UBI since it lacks the xmlsec1-devel package). It should not be affected by this SSL crypto issue.

Dockerfile.txt

[Codykilpatrick]

A workaround is to set the environment variable OPENSSL_FORCE_FIPS_MODE to 0. Example for the NetBox helm chart values.yaml:

...
extraEnvs:
  - name: OPENSSL_FORCE_FIPS_MODE
     value: "0"
worker:
  extraEnvs:
    - name: OPENSSL_FORCE_FIPS_MODE
       value: "0"

Huge save for me thank you a bunch for this.

[jo-bzh]

Thanks @timothy-mullican for sharing your rootless Netbox Dockerfile,
By any chance did you plan to opensourced it in another location ?