Allow to use system CA pool for certificate verification
authorDaniel Gustafsson <[email protected]>
Wed, 5 Apr 2023 21:22:17 +0000 (23:22 +0200)
committerDaniel Gustafsson <[email protected]>
Wed, 5 Apr 2023 21:22:17 +0000 (23:22 +0200)
This adds a new option to libpq's sslrootcert, "system", which will load
the system trusted CA roots for certificate verification. This is a more
convenient way to achieve this than pointing to the system CA roots
manually since the location can differ by installation and be locally
adjusted by env vars in OpenSSL.

When sslrootcert is set to system, sslmode is forced to be verify-full
as weaker modes aren't providing much security for public CAs.

Changing the location of the system roots by setting environment vars is
not supported by LibreSSL so the tests will use a heuristic to determine
if the system being tested is LibreSSL or OpenSSL.

The workaround in .cirrus.yml is required to handle a strange interaction
between homebrew and the openssl@3 formula; hopefully this can be removed
in the near future.

The original patch was written by Thomas Habets, which was later revived
by Jacob Champion.

Author: Jacob Champion <[email protected]>
Author: Thomas Habets <[email protected]>
Reviewed-by: Jelte Fennema <[email protected]>
Reviewed-by: Andrew Dunstan <[email protected]>
Reviewed-by: Magnus Hagander <[email protected]>
Discussion: https://www.postgresql.org/message-id/flat/CA%2BkHd%2BcJwCUxVb-Gj_0ptr3_KZPwi3%2B67vK6HnLFBK9MzuYrLA%40mail.gmail.com

.cirrus.yml
doc/src/sgml/libpq.sgml
doc/src/sgml/runtime.sgml
src/interfaces/libpq/fe-connect.c
src/interfaces/libpq/fe-secure-openssl.c
src/interfaces/libpq/t/001_uri.pl
src/test/ssl/ssl/server-cn-only+server_ca.crt [new file with mode: 0644]
src/test/ssl/sslfiles.mk
src/test/ssl/t/001_ssltests.pl

index 04786174ed4ca6e17142d65a21c3160351ad945e..d3f88821a85a8510cc19a22ec48ef7b4b62fbc6a 100644 (file)
@@ -477,12 +477,24 @@ task:
       make \
       meson \
       openldap \
-      openssl \
+      openssl@3 \
       python \
       tcl-tk \
       zstd
 
     brew cleanup -s # to reduce cache size
+
+    # brew cleanup removes the empty certs directory in OPENSSLDIR, causing
+    # OpenSSL to report unexpected errors ("unregistered scheme") during
+    # verification failures. Put it back for now as a workaround.
+    #
+    #   https://github.com/orgs/Homebrew/discussions/4030
+    #
+    # Note that $(brew --prefix openssl) will give us the opt/ prefix but not
+    # the etc/ prefix, so we hardcode the full path here. openssl@3 is pinned
+    # above to try to minimize the chances of this changing beneath us, but it's
+    # brittle...
+    mkdir -p "/opt/homebrew/etc/openssl@3/certs"
   upload_caches: homebrew
 
   ccache_cache:
index 9f72dd29d89725f0a1e6d7a0ca3851147b89c97a..faa8aa3187e349d901fc9dc962bc8a3a930d523d 100644 (file)
@@ -1876,6 +1876,30 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
         to be signed by one of these authorities.  The default is
         <filename>~/.postgresql/root.crt</filename>.
        </para>
+       <para>
+        The special value <literal>system</literal> may be specified instead, in
+        which case the system's trusted CA roots will be loaded. The exact
+        locations of these root certificates differ by SSL implementation and
+        platform. For <productname>OpenSSL</productname> in particular, the
+        locations may be further modified by the <envar>SSL_CERT_DIR</envar>
+        and <envar>SSL_CERT_FILE</envar> environment variables.
+       </para>
+       <note>
+        <para>
+         When using <literal>sslrootcert=system</literal>, the default
+         <literal>sslmode</literal> is changed to <literal>verify-full</literal>,
+         and any weaker setting will result in an error. In most cases it is
+         trivial for anyone to obtain a certificate trusted by the system for a
+         hostname they control, rendering <literal>verify-ca</literal> and all
+         weaker modes useless.
+        </para>
+        <para>
+         The magic <literal>system</literal> value will take precedence over a
+         local certificate file with the same name. If for some reason you find
+         yourself in this situation, use an alternative path like
+         <literal>sslrootcert=./system</literal> instead.
+        </para>
+       </note>
       </listitem>
      </varlistentry>
 
index 149e9b33d4841f2bf2588c5c84c895b12815900a..dbe23db54f05b0628007fca975b4d480b41d0b02 100644 (file)
@@ -2007,7 +2007,11 @@ pg_dumpall -p 5432 | psql -d postgres -p 5433
    (<xref linkend="ssl-tcp"/>). The TCP client must connect using
    <literal>sslmode=verify-ca</literal> or
    <literal>verify-full</literal> and have the appropriate root certificate
-   file installed (<xref linkend="libq-ssl-certificates"/>).
+   file installed (<xref linkend="libq-ssl-certificates"/>). Alternatively the
+   system CA pool can be used using <literal>sslrootcert=system</literal>; in
+   this case, <literal>sslmode=verify-full</literal> is forced for safety, since
+   it is generally trivial to obtain certificates which are signed by a public
+   CA.
   </para>
 
   <para>
index bb7347cb0c0ab2e4a390387206de36b7ddfa223b..40fef0e2c816779184880dc7ac0dc44290e4e843 100644 (file)
@@ -1465,6 +1465,23 @@ connectOptions2(PGconn *conn)
            goto oom_error;
    }
 
+#ifndef USE_SSL
+
+   /*
+    * sslrootcert=system is not supported. Since setting this changes the
+    * default sslmode, check this _before_ we validate sslmode, to avoid
+    * confusing the user with errors for an option they may not have set.
+    */
+   if (conn->sslrootcert
+       && strcmp(conn->sslrootcert, "system") == 0)
+   {
+       conn->status = CONNECTION_BAD;
+       libpq_append_conn_error(conn, "sslrootcert value \"%s\" invalid when SSL support is not compiled in",
+                               conn->sslrootcert);
+       return false;
+   }
+#endif
+
    /*
     * validate sslmode option
     */
@@ -1511,6 +1528,22 @@ connectOptions2(PGconn *conn)
            goto oom_error;
    }
 
+#ifdef USE_SSL
+
+   /*
+    * If sslrootcert=system, make sure our chosen sslmode is compatible.
+    */
+   if (conn->sslrootcert
+       && strcmp(conn->sslrootcert, "system") == 0
+       && strcmp(conn->sslmode, "verify-full") != 0)
+   {
+       conn->status = CONNECTION_BAD;
+       libpq_append_conn_error(conn, "weak sslmode \"%s\" may not be used with sslrootcert=system (use verify-full)",
+                               conn->sslmode);
+       return false;
+   }
+#endif
+
    /*
     * Validate TLS protocol versions for ssl_min_protocol_version and
     * ssl_max_protocol_version.
@@ -6236,6 +6269,8 @@ static bool
 conninfo_add_defaults(PQconninfoOption *options, PQExpBuffer errorMessage)
 {
    PQconninfoOption *option;
+   PQconninfoOption *sslmode_default = NULL,
+              *sslrootcert = NULL;
    char       *tmp;
 
    /*
@@ -6252,6 +6287,9 @@ conninfo_add_defaults(PQconninfoOption *options, PQExpBuffer errorMessage)
     */
    for (option = options; option->keyword != NULL; option++)
    {
+       if (strcmp(option->keyword, "sslrootcert") == 0)
+           sslrootcert = option;   /* save for later */
+
        if (option->val != NULL)
            continue;           /* Value was in conninfo or service */
 
@@ -6294,6 +6332,13 @@ conninfo_add_defaults(PQconninfoOption *options, PQExpBuffer errorMessage)
                }
                continue;
            }
+
+           /*
+            * sslmode is not specified. Let it be filled in with the compiled
+            * default for now, but if sslrootcert=system, we'll override the
+            * default later before returning.
+            */
+           sslmode_default = option;
        }
 
        /*
@@ -6326,6 +6371,27 @@ conninfo_add_defaults(PQconninfoOption *options, PQExpBuffer errorMessage)
        }
    }
 
+   /*
+    * Special handling for sslrootcert=system with no sslmode explicitly
+    * defined. In this case we want to strengthen the default sslmode to
+    * verify-full.
+    */
+   if (sslmode_default && sslrootcert)
+   {
+       if (sslrootcert->val && strcmp(sslrootcert->val, "system") == 0)
+       {
+           free(sslmode_default->val);
+
+           sslmode_default->val = strdup("verify-full");
+           if (!sslmode_default->val)
+           {
+               if (errorMessage)
+                   libpq_append_error(errorMessage, "out of memory");
+               return false;
+           }
+       }
+   }
+
    return true;
 }
 
index 4d1e4009ef133dc4370fd7c4ad9837c6e319440f..00b203cbfa3ab4682b20fc3e746377a346e24de1 100644 (file)
@@ -1060,8 +1060,29 @@ initialize_SSL(PGconn *conn)
    else
        fnbuf[0] = '\0';
 
-   if (fnbuf[0] != '\0' &&
-       stat(fnbuf, &buf) == 0)
+   if (strcmp(fnbuf, "system") == 0)
+   {
+       /*
+        * The "system" sentinel value indicates that we should load whatever
+        * root certificates are installed for use by OpenSSL; these locations
+        * differ by platform. Note that the default system locations may be
+        * further overridden by the SSL_CERT_DIR and SSL_CERT_FILE
+        * environment variables.
+        */
+       if (SSL_CTX_set_default_verify_paths(SSL_context) != 1)
+       {
+           char       *err = SSLerrmessage(ERR_get_error());
+
+           libpq_append_conn_error(conn, "could not load system root certificate paths: %s",
+                                   err);
+           SSLerrfree(err);
+           SSL_CTX_free(SSL_context);
+           return -1;
+       }
+       have_rootcert = true;
+   }
+   else if (fnbuf[0] != '\0' &&
+            stat(fnbuf, &buf) == 0)
    {
        X509_STORE *cvstore;
 
@@ -1122,10 +1143,10 @@ initialize_SSL(PGconn *conn)
             */
            if (fnbuf[0] == '\0')
                libpq_append_conn_error(conn, "could not get home directory to locate root certificate file\n"
-                                  "Either provide the file or change sslmode to disable server certificate verification.");
+                                       "Either provide the file, use the system's trusted roots with sslrootcert=system, or change sslmode to disable server certificate verification.");
            else
                libpq_append_conn_error(conn, "root certificate file \"%s\" does not exist\n"
-                                  "Either provide the file or change sslmode to disable server certificate verification.", fnbuf);
+                                       "Either provide the file, use the system's trusted roots with sslrootcert=system, or change sslmode to disable server certificate verification.", fnbuf);
            SSL_CTX_free(SSL_context);
            return -1;
        }
index 2ab537f97f11627c2b3eace80efdaf19457a72a0..cd659bc1b0fc998f793125f1eacba75a3c97da0c 100644 (file)
@@ -8,7 +8,9 @@ use IPC::Run;
 
 
 # List of URIs tests. For each test the first element is the input string, the
-# second the expected stdout and the third the expected stderr.
+# second the expected stdout and the third the expected stderr. Optionally,
+# additional arguments may specify key/value pairs which will override
+# environment variables for the duration of the test.
 my @tests = (
    [
        q{postgresql://uri-user:secret@host:12345/db},
@@ -209,20 +211,44 @@ my @tests = (
        q{postgres://%2Fvar%2Flib%2Fpostgresql/dbname},
        q{dbname='dbname' host='/var/lib/postgresql' (local)},
        q{},
+   ],
+   # Usually the default sslmode is 'prefer' (for libraries with SSL) or
+   # 'disable' (for those without). This default changes to 'verify-full' if
+   # the system CA store is in use.
+   [
+       q{postgresql://host?sslmode=disable},
+       q{host='host' sslmode='disable' (inet)},
+       q{},
+       PGSSLROOTCERT => "system",
+   ],
+   [
+       q{postgresql://host?sslmode=prefer},
+       q{host='host' sslmode='prefer' (inet)},
+       q{},
+       PGSSLROOTCERT => "system",
+   ],
+   [
+       q{postgresql://host?sslmode=verify-full},
+       q{host='host' (inet)},
+       q{},
+       PGSSLROOTCERT => "system",
    ]);
 
 # test to run for each of the above test definitions
 sub test_uri
 {
    local $Test::Builder::Level = $Test::Builder::Level + 1;
+   local %ENV = %ENV;
 
    my $uri;
    my %expect;
+   my %envvars;
    my %result;
 
-   ($uri, $expect{stdout}, $expect{stderr}) = @$_;
+   ($uri, $expect{stdout}, $expect{stderr}, %envvars) = @$_;
 
    $expect{'exit'} = $expect{stderr} eq '';
+   %ENV = (%ENV, %envvars);
 
    my $cmd = [ 'libpq_uri_regress', $uri ];
    $result{exit} = IPC::Run::run $cmd, '>', \$result{stdout}, '2>',
diff --git a/src/test/ssl/ssl/server-cn-only+server_ca.crt b/src/test/ssl/ssl/server-cn-only+server_ca.crt
new file mode 100644 (file)
index 0000000..9870e8c
--- /dev/null
@@ -0,0 +1,38 @@
+-----BEGIN CERTIFICATE-----
+MIIDAzCCAesCCCAhAwMUEgcBMA0GCSqGSIb3DQEBCwUAMEIxQDA+BgNVBAMMN1Rl
+c3QgQ0EgZm9yIFBvc3RncmVTUUwgU1NMIHJlZ3Jlc3Npb24gdGVzdCBzZXJ2ZXIg
+Y2VydHMwHhcNMjEwMzAzMjIxMjA3WhcNNDgwNzE5MjIxMjA3WjBGMR4wHAYDVQQL
+DBVQb3N0Z3JlU1FMIHRlc3Qgc3VpdGUxJDAiBgNVBAMMG2NvbW1vbi1uYW1lLnBn
+LXNzbHRlc3QudGVzdDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBANWz
+VPMk7i5f+W0eEadRE+TTAtsIK08CkLMUnjs7zJkxnnm6RGBXPx6vK3AkAIi+wG4Y
+mXjYP3GuMiXaLjnWh2kzBSfIRQyNbTThnhSu3nDjAVkPexsSrPyiKimFuNgDfkGe
+5dQKa9Ag2SuVU4vd9SYxOMAiIFIC4ts4MLWWJf5D/PehdSuc0e5Me+91Nnbz90nl
+ds4lHvuDR+aKnZlTHmch3wfhXv7lNQImIBzfwl36Kd/bWB0fAEVFse3iZWmigaI/
+9FKh//WIq43TNLxn68OCQoyMe/HGjZDR/Xwo3rE6jg6/iAwSWib9yabfYPKbqq2G
+oFy6aYmmEquaDgLuX7kCAwEAATANBgkqhkiG9w0BAQsFAAOCAQEA2AZrD9cTQXTW
+4j2tT8N/TTc6WK2ncN4h22NTte6vK7MVwsZJCtw5ndYkmxcWkXAqiclzWyMdayds
+WOa12CEH7jKAhivF4Hcw3oO3JHM5BA6KzLWBVz9uZksOM6mPqn29DTKvA/Y1V8tj
+mxK/KUA68h/u6inu3mo4ywBpb/tqHxxg2cjyR0faCmM0pwRM0HBr/16fUMfO83nj
+QG8g9J/bybu5sYso/aSoC5nUNp4XjmDMdVLdqg/nTe/ejS8IfFr0WQxBlqooqFgx
+MSE+kX2e2fHsuOWSU/9eClt6FpQrwoC2C8F+/4g1Uz7Liqc4yMHPwjgeP9ewrrLO
+iIhlNNPqpQ==
+-----END CERTIFICATE-----
+-----BEGIN CERTIFICATE-----
+MIIDFDCCAfygAwIBAgIIICEDAxQSBwAwDQYJKoZIhvcNAQELBQAwQDE+MDwGA1UE
+Aww1VGVzdCByb290IENBIGZvciBQb3N0Z3JlU1FMIFNTTCByZWdyZXNzaW9uIHRl
+c3Qgc3VpdGUwHhcNMjEwMzAzMjIxMjA3WhcNNDgwNzE5MjIxMjA3WjBCMUAwPgYD
+VQQDDDdUZXN0IENBIGZvciBQb3N0Z3JlU1FMIFNTTCByZWdyZXNzaW9uIHRlc3Qg
+c2VydmVyIGNlcnRzMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4kp2
+GW5nPb6QrSrtbClfZeutyQnHrm4TMPBoNepFdIVxBX/04BguM5ImDRze/huOWA+z
+atJAQqt3R7dsDwnOnPKUKCOuHX6a1aj5L86HtVgaWTXrZFE5NtL9PIzXkWu13UW0
+UesHtbPVRv6a6fB7Npph6hHy7iPZb009A8/lTJnxSPC39u/K/sPqjrVZaAJF+wDs
+qCxCZTUtAUFvWFnR/TeXLWlFzBupS1djgI7PltbJqSn6SKTAgNZTxpRJbu9Icp6J
+/50ELwT++0n0KWVXNHrDNfI5Gaa+SxClAsPsei2jLKpgR6QFC3wn38Z9q9LjAVuC
++FWhoN1uhYeoricEXwIDAQABoxAwDjAMBgNVHRMEBTADAQH/MA0GCSqGSIb3DQEB
+CwUAA4IBAQCdCA/EoXrustoV4jJGbkdXDuOUkBurwggSNBAqUBSDvCohRoD77Ecb
+QVuzPNxWKG+E4PwfUq2ha+2yPONEJ28ZgsbHq5qlJDMJ43wlcjn6wmmAJNeSpO8F
+0V9d2X/4wNZty9/zbwTnw26KChgDHumQ0WIbCoBtdqy8KDswYOvpgws6dqc021I7
+UrFo6vZek7VoApbJgkDL6qYADa6ApfW43ThH4sViFITeYt/kSHgmy2Udhs34jMM8
+xsFP/uYpRi1b1glenwSIKiHjD4/C9vnWQt5K3gRBvYukEj2Bw9VkNRpBVCi0cOoA
+OuwX3bwzNYNbZQv4K66oRpvuoEjCNeHg
+-----END CERTIFICATE-----
index e63342469d3a5c4bbde63ce3ce0372175a0322c9..f7ababe42c9237ca7cc6bcacac55ff8fa58ad49b 100644 (file)
@@ -61,7 +61,8 @@ COMBINATIONS := \
    ssl/root+server.crl \
    ssl/root+client_ca.crt \
    ssl/root+client.crl \
-   ssl/client+client_ca.crt
+   ssl/client+client_ca.crt \
+   ssl/server-cn-only+server_ca.crt
 
 CERTIFICATES := root_ca server_ca client_ca $(SERVERS) $(CLIENTS)
 STANDARD_CERTS := $(CERTIFICATES:%=ssl/%.crt)
@@ -150,6 +151,9 @@ ssl/root+client_ca.crt: ssl/root_ca.crt ssl/client_ca.crt
 # and for the client, to present to the server
 ssl/client+client_ca.crt: ssl/client.crt ssl/client_ca.crt
 
+# for the server, to present to a client that only knows the root
+ssl/server-cn-only+server_ca.crt: ssl/server-cn-only.crt ssl/server_ca.crt
+
 # If a CRL is used, OpenSSL requires a CRL file for *all* the CAs in the
 # chain, even if some of them are empty.
 ssl/root+server.crl: ssl/root.crl ssl/server.crl
index dc43b8f81aaa77f03e162699c191fe79234f92c3..6fdc040cfc2e8b21a43837a24c38aca4daab945c 100644 (file)
@@ -33,6 +33,12 @@ sub switch_server_cert
 {
    $ssl_server->switch_server_cert(@_);
 }
+
+# Determine whether this build uses OpenSSL or LibreSSL. As a heuristic, the
+# HAVE_SSL_CTX_SET_CERT_CB macro isn't defined for LibreSSL. (Nor for OpenSSL
+# 1.0.1, but that's old enough that accommodating it isn't worth the cost.)
+my $libressl = not check_pg_config("#define HAVE_SSL_CTX_SET_CERT_CB 1");
+
 #### Some configuration
 
 # This is the hostname used to connect to the server. This cannot be a
@@ -461,6 +467,43 @@ $node->connect_fails(
    expected_stderr =>
      qr/could not get server's host name from server certificate/);
 
+# Test system trusted roots.
+switch_server_cert($node,
+   certfile => 'server-cn-only+server_ca',
+   keyfile => 'server-cn-only',
+   cafile => 'root_ca');
+$common_connstr =
+  "$default_ssl_connstr user=ssltestuser dbname=trustdb sslrootcert=system hostaddr=$SERVERHOSTADDR";
+
+# By default our custom-CA-signed certificate should not be trusted.
+$node->connect_fails(
+   "$common_connstr sslmode=verify-full host=common-name.pg-ssltest.test",
+   "sslrootcert=system does not connect with private CA",
+   expected_stderr => qr/SSL error: certificate verify failed/);
+
+# Modes other than verify-full cannot be mixed with sslrootcert=system.
+$node->connect_fails(
+   "$common_connstr sslmode=verify-ca host=common-name.pg-ssltest.test",
+   "sslrootcert=system only accepts sslmode=verify-full",
+   expected_stderr => qr/weak sslmode "verify-ca" may not be used with sslrootcert=system/);
+
+SKIP:
+{
+   skip "SSL_CERT_FILE is not supported with LibreSSL" if $libressl;
+
+   # We can modify the definition of "system" to get it trusted again.
+   local $ENV{SSL_CERT_FILE} = $node->data_dir . "/root_ca.crt";
+   $node->connect_ok(
+       "$common_connstr sslmode=verify-full host=common-name.pg-ssltest.test",
+       "sslrootcert=system connects with overridden SSL_CERT_FILE");
+
+   # verify-full mode should be the default for system CAs.
+   $node->connect_fails(
+       "$common_connstr host=common-name.pg-ssltest.test.bad",
+       "sslrootcert=system defaults to sslmode=verify-full",
+       expected_stderr => qr/server certificate for "common-name.pg-ssltest.test" does not match host name "common-name.pg-ssltest.test.bad"/);
+}
+
 # Test that the CRL works
 switch_server_cert($node, certfile => 'server-revoked');