Bootstrapped databases with best practice role setup

charts/postgres-operator/crds/postgresqls.yaml

@@ -273,6 +273,26 @@ spec:
                   type: object
                   additionalProperties:
                     type: string
+            preparedDatabases:
+              type: object
+              additionalProperties:
+                type: object
+                properties:
+                  defaultUsers:
+                    type: boolean
+                  extensions:
+                    type: object
+                    additionalProperties:
+                      type: string
+                  schemas:
+                    type: object
+                    additionalProperties:
+                      type: object
+                      properties:
+                        defaultUsers:
+                          type: boolean
+                        defaultRoles:
+                          type: boolean
             replicaLoadBalancer:  # deprecated
               type: boolean
             resources:

docs/user.md

@@ -94,7 +94,10 @@ created on every cluster managed by the operator.
 * `teams API roles`: automatically create users for every member of the team
 owning the database cluster.
 
-In the next sections, we will cover those use cases in more details.
+In the next sections, we will cover those use cases in more details. Note, that
+the Postgres Operator can also create databases with pre-defined owner, reader
+and writer roles which saves you the manual setup. Read more in the next
+chapter.
 
 ### Manifest roles
 
@@ -216,6 +219,166 @@ to choose superusers, group roles, [PAM configuration](https://github.com/CyberD
 etc. An OAuth2 token can be passed to the Teams API via a secret. The name for
 this secret is configurable with the `oauth_token_secret_name` parameter.
 
+## Prepared databases with roles and default privileges
+
+The `users` section in the manifests only allows for creating database roles
+with global privileges. Fine-grained data access control or role membership can
+not be defined and must be set up by the user in the database. But, the Postgres
+Operator offers a separate section to specify `preparedDatabases` that will be
+created with pre-defined owner, reader and writer roles for each individual
+database and, optionally, for each database schema, too. `preparedDatabases`
+also enable users to specify PostgreSQL extensions that shall be created in a
+given database schema.
+
+### Default database and schema
+
+A prepared database is already created by adding an empty `preparedDatabases`
+section to the manifest. The database will then be called like the Postgres
+cluster manifest (`-` are replaced with `_`) and will also contain a schema
+called `data`.
+
+```yaml
+spec:
+  preparedDatabases: {}
+```
+
+### Default NOLOGIN roles
+
+Given an example with a specified database and schema:
+
+```yaml
+spec:
+  preparedDatabases:
+    foo:
+      schemas:
+        bar: {}
+```
+
+Postgres Operator will create the following NOLOGIN roles:
+
+| Role name      | Member of      | Admin         |
+| -------------- | -------------- | ------------- |
+| foo_owner      |                | admin         |
+| foo_reader     |                | foo_owner     |
+| foo_writer     | foo_reader     | foo_owner     |
+| foo_bar_owner  |                | foo_owner     |
+| foo_bar_reader |                | foo_bar_owner |
+| foo_bar_writer | foo_bar_reader | foo_bar_owner |
+
+The `<dbname>_owner` role is the database owner and should be used when creating
+new database objects. All members of the `admin` role, e.g. teams API roles, can
+become the owner with the `SET ROLE` command. [Default privileges](https://www.postgresql.org/docs/12/sql-alterdefaultprivileges.html)
+are configured for the owner role so that the `<dbname>_reader` role
+automatically gets read-access (SELECT) to new tables and sequences and the
+`<dbname>_writer` receives write-access (INSERT, UPDATE, DELETE on tables,
+USAGE and UPDATE on sequences). Both get USAGE on types and EXECUTE on
+functions.
+
+The same principle applies for database schemas which are owned by the
+`<dbname>_<schema>_owner` role. `<dbname>_<schema>_reader` is read-only,
+`<dbname>_<schema>_writer` has write access and inherit reading from the reader
+role. Note, that the `<dbname>_*` roles have access incl. default privileges on
+all schemas, too. If you don't need the dedicated schema roles - i.e. you only
+use one schema - you can disable the creation like this:
+
+```yaml
+spec:
+  preparedDatabases:
+    foo:
+      schemas:
+        bar:
+          defaultRoles: false
+```
+
+Then, the schemas are owned by the database owner, too.
+
+### Default LOGIN roles
+
+The roles described in the previous paragraph can be granted to LOGIN roles from
+the `users` section in the manifest. Optionally, the Postgres Operator can also
+create default LOGIN roles for the database an each schema individually. These
+roles will get the `_user` suffix and they inherit all rights from their NOLOGIN
+counterparts.
+
+| Role name           | Member of      | Admin         |
+| ------------------- | -------------- | ------------- |
+| foo_owner_user      | foo_owner      | admin         |
+| foo_reader_user     | foo_reader     | foo_owner     |
+| foo_writer_user     | foo_writer     | foo_owner     |
+| foo_bar_owner_user  | foo_bar_owner  | foo_owner     |
+| foo_bar_reader_user | foo_bar_reader | foo_bar_owner |
+| foo_bar_writer_user | foo_bar_writer | foo_bar_owner |
+
+These default users are enabled in the manifest with the `defaultUsers` flag:
+
+```yaml
+spec:
+  preparedDatabases:
+    foo:
+      defaultUsers: true
+      schemas:
+        bar:
+          defaultUsers: true
+```
+
+### Database extensions
+
+Prepared databases also allow for creating Postgres extensions. They will be
+created by the database owner in the specified schema.
+
+```yaml
+spec:
+  preparedDatabases:
+    foo:
+      extensions:
+        pg_partman: public
+        postgis: data
+```
+
+Some extensions require SUPERUSER rights on creation unless they are not
+whitelisted by the [pgextwlist](https://github.com/dimitri/pgextwlist)
+extension, that is shipped with the Spilo image. To see which extensions are
+on the list check the `extwlist.extension` parameter in the postgresql.conf
+file.
+
+```bash
+SHOW extwlist.extensions;
+```
+
+Make sure that `pgextlist` is also listed under `shared_preload_libraries` in
+the PostgreSQL configuration. Then the database owner should be able to create
+the extension specified in the manifest.
+
+### From `databases` to `preparedDatabases`
+
+If you wish to create the role setup described above for databases listed under
+the `databases` key, you have to make sure that the owner role follows the
+`<dbname>_owner` naming convention of `preparedDatabases`. As roles are synced
+first, this can be done with one edit:
+
+```yaml
+# before
+spec:
+  databases:
+    foo: db_owner
+
+# after
+spec:
+  databases:
+    foo: foo_owner
+  preparedDatabases:
+    foo:
+      schemas:
+        my_existing_schema: {}
+```
+
+Adding existing database schemas to the manifest to create roles for them as
+well is up the user and not done by the operator. Remember that if you don't
+specify any schema a new database schema called `data` will be created. When
+everything got synced (roles, schemas, extensions), you are free to remove the
+database from the `databases` section. Note, that the operator does not delete
+database objects or revoke privileges when removed from the manifest.
+
 ## Resource definition
 
 The compute resources to be used for the Postgres containers in the pods can be
@@ -586,8 +749,8 @@ don't know the value, use `103` which is the GID from the default spilo image
 OpenShift allocates the users and groups dynamically (based on scc), and their
 range is different in every namespace. Due to this dynamic behaviour, it's not
 trivial to know at deploy time the uid/gid of the user in the cluster.
-Therefore, instead of using a global `spilo_fsgroup` setting, use the `spiloFSGroup` field
-per Postgres cluster.
+Therefore, instead of using a global `spilo_fsgroup` setting, use the
+`spiloFSGroup` field per Postgres cluster.
 
 Upload the cert as a kubernetes secret:
 ```sh

manifests/complete-postgres-manifest.yaml

@@ -21,6 +21,17 @@ spec:
   - 127.0.0.1/32
   databases:
     foo: zalando
+  preparedDatabases:
+    bar:
+      defaultUsers: true
+      extensions:
+        pg_partman: public
+        pgcrypto: public
+      schemas:
+        data: {}
+        history:
+          defaultRoles: true
+          defaultUsers: false
   postgresql:
     version: "12"
     parameters: # Expert section

manifests/minimal-postgres-manifest.yaml

@@ -15,5 +15,7 @@ spec:
     foo_user: []  # role for application foo
   databases:
     foo: zalando  # dbname: owner
+  preparedDatabases:
+    bar: {}
   postgresql:
     version: "12"

manifests/postgresql.crd.yaml

@@ -237,6 +237,26 @@ spec:
                   type: object
                   additionalProperties:
                     type: string
+            preparedDatabases:
+              type: object
+              additionalProperties:
+                type: object
+                properties:
+                  defaultUsers:
+                    type: boolean
+                  extensions:
+                    type: object
+                    additionalProperties:
+                      type: string
+                  schemas:
+                    type: object
+                    additionalProperties:
+                      type: object
+                      properties:
+                        defaultUsers:
+                          type: boolean
+                        defaultRoles:
+                          type: boolean
             replicaLoadBalancer:  # deprecated
               type: boolean
             resources:

pkg/apis/acid.zalan.do/v1/crds.go

@@ -421,6 +421,43 @@ var PostgresCRDResourceValidation = apiextv1beta1.CustomResourceValidation{
 							},
 						},
 					},
+					"preparedDatabases": {
+						Type: "object",
+						AdditionalProperties: &apiextv1beta1.JSONSchemaPropsOrBool{
+							Schema: &apiextv1beta1.JSONSchemaProps{
+								Type: "object",
+								Properties: map[string]apiextv1beta1.JSONSchemaProps{
+									"defaultUsers": {
+										Type: "boolean",
+									},
+									"extensions": {
+										Type: "object",
+										AdditionalProperties: &apiextv1beta1.JSONSchemaPropsOrBool{
+											Schema: &apiextv1beta1.JSONSchemaProps{
+												Type: "string",
+											},
+										},
+									},
+									"schemas": {
+										Type: "object",
+										AdditionalProperties: &apiextv1beta1.JSONSchemaPropsOrBool{
+											Schema: &apiextv1beta1.JSONSchemaProps{
+												Type: "object",
+												Properties: map[string]apiextv1beta1.JSONSchemaProps{
+													"defaultUsers": {
+														Type: "boolean",
+													},
+													"defaultRoles": {
+														Type: "boolean",
+													},
+												},
+											},
+										},
+									},
+								},
+							},
+						},
+					},
 					"replicaLoadBalancer": {
 						Type:        "boolean",
 						Description: "Deprecated",

pkg/apis/acid.zalan.do/v1/postgresql_type.go

@@ -50,24 +50,25 @@ type PostgresSpec struct {
 	// load balancers' source ranges are the same for master and replica services
 	AllowedSourceRanges []string `json:"allowedSourceRanges"`
 
-	NumberOfInstances     int32                `json:"numberOfInstances"`
-	Users                 map[string]UserFlags `json:"users"`
-	MaintenanceWindows    []MaintenanceWindow  `json:"maintenanceWindows,omitempty"`
-	Clone                 CloneDescription     `json:"clone"`
-	ClusterName           string               `json:"-"`
-	Databases             map[string]string    `json:"databases,omitempty"`
-	Tolerations           []v1.Toleration      `json:"tolerations,omitempty"`
-	Sidecars              []Sidecar            `json:"sidecars,omitempty"`
-	InitContainers        []v1.Container       `json:"initContainers,omitempty"`
-	PodPriorityClassName  string               `json:"podPriorityClassName,omitempty"`
-	ShmVolume             *bool                `json:"enableShmVolume,omitempty"`
-	EnableLogicalBackup   bool                 `json:"enableLogicalBackup,omitempty"`
-	LogicalBackupSchedule string               `json:"logicalBackupSchedule,omitempty"`
-	StandbyCluster        *StandbyDescription  `json:"standby"`
-	PodAnnotations        map[string]string    `json:"podAnnotations"`
-	ServiceAnnotations    map[string]string    `json:"serviceAnnotations"`
-	TLS                   *TLSDescription      `json:"tls"`
-	AdditionalVolumes     []AdditionalVolume   `json:"additionalVolumes,omitempty"`
+	NumberOfInstances     int32                       `json:"numberOfInstances"`
+	Users                 map[string]UserFlags        `json:"users"`
+	MaintenanceWindows    []MaintenanceWindow         `json:"maintenanceWindows,omitempty"`
+	Clone                 CloneDescription            `json:"clone"`
+	ClusterName           string                      `json:"-"`
+	Databases             map[string]string           `json:"databases,omitempty"`
+	PreparedDatabases     map[string]PreparedDatabase `json:"preparedDatabases,omitempty"`
+	Tolerations           []v1.Toleration             `json:"tolerations,omitempty"`
+	Sidecars              []Sidecar                   `json:"sidecars,omitempty"`
+	InitContainers        []v1.Container              `json:"initContainers,omitempty"`
+	PodPriorityClassName  string                      `json:"podPriorityClassName,omitempty"`
+	ShmVolume             *bool                       `json:"enableShmVolume,omitempty"`
+	EnableLogicalBackup   bool                        `json:"enableLogicalBackup,omitempty"`
+	LogicalBackupSchedule string                      `json:"logicalBackupSchedule,omitempty"`
+	StandbyCluster        *StandbyDescription         `json:"standby"`
+	PodAnnotations        map[string]string           `json:"podAnnotations"`
+	ServiceAnnotations    map[string]string           `json:"serviceAnnotations"`
+	TLS                   *TLSDescription             `json:"tls"`
+	AdditionalVolumes     []AdditionalVolume          `json:"additionalVolumes,omitempty"`
 
 	// deprecated json tags
 	InitContainersOld       []v1.Container `json:"init_containers,omitempty"`
@@ -84,6 +85,19 @@ type PostgresqlList struct {
 	Items []Postgresql `json:"items"`
 }
 
+// PreparedDatabase describes elements to be bootstrapped
+type PreparedDatabase struct {
+	PreparedSchemas map[string]PreparedSchema `json:"schemas,omitempty"`
+	DefaultUsers    bool                      `json:"defaultUsers,omitempty" defaults:"false"`
+	Extensions      map[string]string         `json:"extensions,omitempty"`
+}
+
+// PreparedSchema describes elements to be bootstrapped per schema
+type PreparedSchema struct {
+	DefaultRoles *bool `json:"defaultRoles,omitempty" defaults:"true"`
+	DefaultUsers bool  `json:"defaultUsers,omitempty" defaults:"false"`
+}
+
 // MaintenanceWindow describes the time window when the operator is allowed to do maintenance on a cluster.
 type MaintenanceWindow struct {
 	Everyday  bool