Consolidate Redis roles (#99)

* Consolidate Redis roles

* Use handlers in redis role
Separate out basic configuration tasks in redis role
Fix issue with determining the Redis version when installing from source

* Update Redis readme

* Remove redis_replication and redis_auth references

---------

Co-authored-by: Steven Schattenberg <[email protected]>

Author: kvelarde-itential

docs/redis_guide.md

@@ -1,20 +1,18 @@
 # Overview
 
-The playbook and roles in this section install and configure Redis for the Itential Automation Platform.  There are currently three Redis-related roles:
+The playbook and role in this section install and configure Redis for the Itential Automation Platform.  There is one Redis-related role:
 
-* `redis` – Installs Redis and performs a base configuration.
-* `redis_auth` – Configures Redis authentication.
-* `redis_replication` – Configures Redis replication.
+* `redis` – Installs Redis and performs a base configuration.  Optionally configures authentication and replication.
 
-# Roles
+# Redis Role
 
-## Redis Role
+## Base Install
 
-The `redis` role performs a base install of Redis including any OS packages required. It will compile and install any custom SELinux profiles. It creates the appropriate Linux users, directories, log files, and systemd services. It uses a template to generate a configuration file with some potential features available in other roles commented out. It will start the redis service when complete.
+The `redis` role performs a base install of Redis including any OS packages required. It will compile and install any custom SELinux profiles. It creates the appropriate Linux users, directories, log files, and systemd services. It uses a template to generate a configuration file with some potential features available in other roles commented out. It will start the Redis service when complete.
 
-## Redis Auth Role
+## Authentication
 
-The `redis_auth` role performs tasks to require authentication (username and password) when communicating with the Redis server. It adjusts the Redis config file and adds each of the required users and applies appropriate ACLs (see table). The "default" Redis user is disabled. It modifies the Redis config file to use the appropriate user while doing replication. It adjusts the Sentinel config file to enable the correct Sentinel user to monitor the redis cluster, if required. It disables the default user in both Redis and Redis Sentinel. It will restart the redis service and the Sentinel service (if required) when complete.
+Optionally, the `redis` role performs tasks to require authentication (username and password) when communicating with the Redis server. It adjusts the Redis config file and adds each of the required users and applies appropriate ACLs (see table). The "default" Redis user is disabled. It modifies the Redis config file to use the appropriate user while doing replication. It adjusts the Sentinel config file to enable the correct Sentinel user to monitor the redis cluster, if required. It disables the default user in both Redis and Redis Sentinel.
 
 More info on Redis authorization: https://redis.io/docs/manual/security/
 
@@ -24,12 +22,13 @@ More info on Redis authorization: https://redis.io/docs/manual/security/
 | itential | itential | Has access to all keys, all channels, and all commands except: -asking -cluster -readonly -readwrite -bgrewriteaof -bgsave -failover -flushall -flushdb -psync -replconf -replicaof -save -shutdown -sync
 | repluser | repluser | Has access to the minimum set of commands to perform replication.
 | sentineluser | sentineluser | Has access to the minimum set of commands to perform sentinel monitoring.
+| prometheus | prometheus | Has access to the minimum set of commands to perform Redis and Sentinel monitoring with Prometheus. Required by the optional redis_exporter service.
 
 :::(Warning) (⚠ Warning: ) It is assumed that these default passwords will be changed to meet more rigorous standards. These are intended to be defaults strictly used just for ease of the installation.  It is highly recommended that sensitive data be encrypted using Ansible Vault.
 
-## Redis Replication Role
+## Replication
 
-The `redis_replication` role performs the steps required to create a Redis replica set. It uses a template to generate a Redis Sentinel config file. It modifies the Redis config file to turn off protected-mode. It assumes that the first host defined in the inventory file is the initial primary. It will update the config file for the non-primary Redis servers to replicate from the primary using hostname. It will restart Redis and Redis Sentinel when complete.
+Optionally, the `redis` role performs the steps required to create a Redis replica set. It uses a template to generate a Redis Sentinel config file. It modifies the Redis config file to turn off protected-mode. It assumes that the first host defined in the inventory file is the initial primary. It will update the config file for the non-primary Redis servers to replicate from the primary using hostname. It will start Redis Sentinel when complete.
 
 For more information on Redis replication: https://redis.io/docs/manual/replication/
 
@@ -57,15 +56,15 @@ The following table lists the default variables that are shared between the Redi
 
 | Variable | Group | Type | Description | Default Value
 | :------- | :---- | :--- | :---------- | :------------
-| `redis_auth` | `all` | Boolean | Flag to enable Redis authentication. When set to to `true`, the `redis_auth` role will be executed. | `false`
-| `redis_replication` | `all` | Boolean | Flag to enable Redis replication. When set to `true`, the `redis_replication` role will be executed. | `false`
+| `redis_auth` | `all` | Boolean | Flag to enable Redis authentication. When set to to `true`, Redis authentication will be configured. | `false`
+| `redis_replication` | `all` | Boolean | Flag to enable Redis replication. When set to `true`, Redis replication will be configured and the Redis Sentinel service started. | `false`
 | `redis_tls` | `all` | Boolean | Flag to enable TLS connections. | `false`
 
 ## Redis Role Variables
 
 The variables in this section may be overridden in the inventory in the `redis` group vars.
 
-The following table lists the default variables located in `roles/redis/defaults/main.yml`.
+The following table lists the default variables located in `roles/redis/defaults/main/redis.yml` and `roles/redis/defaults/main/sentinel.yml`.
 
 | Variable | Group | Type | Description | Default Value
 | :------- | :---- | :--- | :---------- | :------------
@@ -82,23 +81,8 @@ The following table lists the default variables located in `roles/redis/defaults
 | `redis_bind_ipv6` | `redis` | Boolean | Flag to enable IPv6. | `true`
 | `redis_bind_addr_source` | `redis` | String | The bind address source. Will default to the Ansible `inventory_hostname` unless explicitly set to `default_ipv4_address`. | `inventory_hostname`
 | `redis_bind_addrs` | `redis` | String | A space-separated list of hostnames/IP addresses on which Redis listeners will be created.  If `redis_bind_ipv6` is set to `true`, `::1` will be added to the addresses.  The `redis_bind_addr_source` will also be added to the addresses. | `127.0.0.1`
-| `iap_redis_packages` | `redis` | List of Strings | The Linux packages to install. | `redis`<br>`jemalloc`
 | `redis_install_method` | `redis` | String | The method to use to install Redis.<br>Set to `remi_repo` to use the Remi repo.<br>Set to `source` to install from source. | `remi_repo`
-| `epel_repo_url` | `redis` | String | The URL of the EPEL repo RPM.<br>Note: this is only used when the `redis_install_method` is set to `remi_repo`. | `https://dl.fedoraproject.org/pub/epel/epel-release-latest-{{ ansible_distribution_major_version }}.noarch.rpm`
-
-
-## Redis Auth Role Variables
-
-There are no default variables for the Redis Auth role other than the Redis common default variables.
-
-## Redis Replication Role Variables
-
-The variables in this section may be overridden in the inventory in the `redis` group vars.
-
-The following table lists the default variables located in `roles/redis_replication/defaults/main.yml`.
-
-| Variable | Group | Type | Description | Default Value
-| :------- | :---- | :--- | :---------- | :------------
+| `redis_epel_repo_url` | `redis` | String | The URL of the EPEL repo RPM.<br>Note: this is only used when the `redis_install_method` is set to `remi_repo`. | `https://dl.fedoraproject.org/pub/epel/epel-release-latest-{{ ansible_distribution_major_version }}.noarch.rpm`
 | `redis_sentinel_conf_file` | `redis` | String | The location of the Redis Sentinel configuration file. | `{{ redis_conf_path }}/sentinel.conf`
 | `redis_sentinel_port` | `redis` | Integer | The Redis Sentinel listen port | `26379`
 
@@ -165,32 +149,8 @@ all:
 
 # Running the Playbook
 
-To execute all Redis roles, run the `redis` playbook:
+To execute the Redis role, run the `redis` playbook:
 
 ```
 ansible-playbook itential.deployer.redis -i <inventory>
 ```
-
-You can also run select Redis roles by using the following tags:
-
-* `redis_install`
-* `redis_auth`
-* `redis_replication`
-
-To execute only the `redis` role (skipping the `redis_auth` and `redis_replication` roles), run the `itential.deployer.redis` playbook with the `redis_install` tag:
-
-```
-ansible-playbook itential.deployer.redis -i <inventory> --tags redis_install
-```
-
-To execute only the Redis Auth role (skipping the Redis and Redis Replication roles), run the `itential.deployer.redis` playbook with the `redis_auth` tag:
-
-```
-ansible-playbook itential.deployer.redis -i <inventory> --tags redis_auth
-```
-
-To execute only the Redis Replication role (skipping the Redis and Redis Auth roles), run the `itential.deployer.redis` playbook with the `redis_replication` tag:
-
-```
-ansible-playbook itential.deployer.redis -i <inventory> --tags redis_replication
-```

playbooks/install_active_standby.yml

@@ -6,27 +6,10 @@
   become: true
   roles:
     ### REDIS
-    # Perform a base installation of Redis
+    # Perform a base installation of Redis and optionally configure authorization and replication
     - role: itential.deployer.redis
       tags:
         - redis
-        - redis_install
-
-    # Perform installation of Redis Sentinel for Redis HA and replication
-    # https://redis.io/docs/manual/sentinel/
-    - role: itential.deployer.redis_replication
-      when: redis_replication | bool
-      tags:
-        - redis
-        - redis_replication
-
-    # Configure Redis to require a username & password (authorization)
-    # https://redis.io/docs/manual/security/acl/
-    - role: itential.deployer.redis_auth
-      when: redis_auth | bool
-      tags:
-        - redis
-        - redis_auth
 
 - name: Install RabbitMQ
   hosts: rabbitmq, rabbitmq_secondary

playbooks/redis.yml

@@ -8,27 +8,6 @@
   roles:
     # Pull in the common vars
     - role: itential.deployer.common_vars
-      tags:
-        - always
 
-    # Perform a base installation of Redis
+    # Perform a base installation of Redis and optionally configure authorization and replication
     - role: itential.deployer.redis
-      tags:
-        - redis
-        - redis_install
-
-    # Perform installation of Redis Sentinel for Redis HA and replication
-    # https://redis.io/docs/manual/sentinel/
-    - role: itential.deployer.redis_replication
-      when: redis_replication | bool
-      tags:
-        - redis
-        - redis_replication
-
-    # Configure Redis to require a username & password (authorization)
-    # https://redis.io/docs/manual/security/acl/
-    - role: itential.deployer.redis_auth
-      when: redis_auth | bool
-      tags:
-        - redis
-        - redis_auth

roles/redis/defaults/main.yml renamed to roles/redis/defaults/main/redis.yml

@@ -39,10 +39,6 @@ redis_bind_addr_source: inventory_hostname
 # to the redis_bind_addrs depending on the redis_bind_addr_source.
 redis_bind_addrs: 127.0.0.1
 
-iap_redis_packages:
-  - redis
-  - jemalloc
-
 # Offline install settings
 # Refer to the offline variables in common_vars for additional settings
 packages_path: "{{ itential_packages_path }}/{{ iap_release }}/redis"
@@ -52,4 +48,4 @@ packages_path: "{{ itential_packages_path }}/{{ iap_release }}/redis"
 redis_install_method: source
 
 # The EPEL repo is only required when the redis_install_method is set to 'remi_repo'
-epel_repo_url: "https://dl.fedoraproject.org/pub/epel/epel-release-latest-{{ ansible_distribution_major_version }}.noarch.rpm"
+redis_epel_repo_url: "https://dl.fedoraproject.org/pub/epel/epel-release-latest-{{ ansible_distribution_major_version }}.noarch.rpm"

roles/redis_replication/defaults/main.yml renamed to roles/redis/defaults/main/sentinel.yml

@@ -1,14 +1,18 @@
 # Copyright (c) 2024, Itential, Inc
 # GNU General Public License v3.0+ (see LICENSE or https://www.gnu.org/licenses/gpl-3.0.txt)
 ---
-- name: Restart Redis
+- name: Enable and Start Redis
+  throttle: 1
   ansible.builtin.systemd:
     name: redis
     enabled: true
     state: restarted
+    daemon_reload: true
 
-- name: Restart Sentinel
+- name: Enable and Start Redis Sentinel
+  throttle: 1
   ansible.builtin.systemd:
     name: redis-sentinel
     enabled: true
     state: restarted
+    daemon_reload: true

roles/redis/handlers/main.yml

@@ -0,0 +1,79 @@
+# Copyright (c) 2024, Itential, Inc
+# GNU General Public License v3.0+ (see LICENSE or https://www.gnu.org/licenses/gpl-3.0.txt)
+---
+# Kernel Adjust
+# Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition.
+# Being disabled, it can can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328
+- name: Adjust Memory Overcommit
+  ansible.posix.sysctl:
+    name: vm.overcommit_memory
+    value: 1
+
+- name: Install custom SELinux profiles
+  ansible.builtin.include_role:
+    name: selinux
+  tags: configure_selinux
+
+# Check if firewalld is running, if it is then open the appropriate ports
+- name: Gather service facts
+  ansible.builtin.service_facts:
+
+- name: Open Ports on FirewallD Public Zone
+  ansible.posix.firewalld:
+    port: "{{ item }}"
+    permanent: true
+    state: enabled
+    zone: public
+    immediate: true
+  loop:
+    - "{{ redis_port }}/tcp"
+    - "{{ redis_replication | bool | ternary(rabbitmq_mgt_console_port ~ '/tcp', omit) }}"
+  when:
+    - ansible_facts.services["firewalld.service"] is defined
+    - ansible_facts.services["firewalld.service"].state == "running"
+    - ansible_facts.services["firewalld.service"].status == "enabled"
+
+- name: Create Redis group
+  ansible.builtin.group:
+    name: "{{ redis_group }}"
+
+- name: Create Redis user
+  ansible.builtin.user:
+    name: "{{ redis_owner }}"
+    group: "{{ redis_group }}"
+    state: present
+
+- name: Create Redis data directory
+  ansible.builtin.file:
+    state: directory
+    path: "{{ redis_data_dir }}"
+    owner: "{{ redis_owner }}"
+    group: "{{ redis_group }}"
+    mode: "0755"
+  when: redis_data_dir != "/var/lib/redis" or redis_install_method == "source"
+
+- name: Create Redis log directory
+  ansible.builtin.file:
+    state: directory
+    path: "{{ redis_log_dir }}"
+    owner: "{{ redis_owner }}"
+    group: "{{ redis_group }}"
+    mode: "0755"
+  when: redis_log_dir != "/var/log/redis" or redis_install_method == "source"
+
+- name: Create Redis pid directory
+  ansible.builtin.file:
+    state: directory
+    path: "{{ redis_pid_dir }}"
+    owner: "{{ redis_owner }}"
+    group: "{{ redis_group }}"
+    mode: "0755"
+  when: redis_pid_dir != "/var/run" or redis_install_method == "source"
+
+- name: Create Redis configuration directory
+  ansible.builtin.file:
+    path: "{{ redis_conf_path }}"
+    state: directory
+    owner: "{{ redis_owner }}"
+    group: "{{ redis_group }}"
+    mode: "0760"

roles/redis/tasks/basic-configuration.yml

@@ -0,0 +1,41 @@
+# Copyright (c) 2024, Itential, Inc
+# GNU General Public License v3.0+ (see LICENSE or https://www.gnu.org/licenses/gpl-3.0.txt)
+---
+- name: Configure Sentinel
+  notify: Enable and Start Redis Sentinel
+  block:
+    - name: Create Redis Sentinel systemd file
+      ansible.builtin.template:
+        src: redis-sentinel.service.j2
+        dest: /usr/lib/systemd/system/redis-sentinel.service
+        owner: root
+        group: root
+        mode: "0644"
+
+    - name: Use template to generate sentinel.conf
+      ansible.builtin.template:
+        src: sentinel.conf.j2
+        dest: "{{ redis_sentinel_conf_file }}"
+        owner: "{{ redis_owner }}"
+        group: "{{ redis_group }}"
+        mode: "0640"
+        backup: true
+      when:
+        - groups['redis'] is defined
+        - inventory_hostname in groups['redis']
+      vars:
+        master_name: "{{ hostvars[groups['redis'][0]].inventory_hostname }}"
+
+    - name: Use template to generate sentinel.conf for secondary DR
+      ansible.builtin.template:
+        src: sentinel.conf.j2
+        dest: "{{ redis_sentinel_conf_file }}"
+        owner: "{{ redis_owner }}"
+        group: "{{ redis_group }}"
+        mode: "0640"
+        backup: true
+      when:
+        - groups['redis_secondary'] is defined
+        - inventory_hostname in groups['redis_secondary']
+      vars:
+        master_name: "{{ hostvars[groups['redis_secondary'][0]].inventory_hostname }}"

roles/redis/tasks/configure-sentinel.yml

@@ -30,7 +30,7 @@
   block:
     - name: Install EPEL repo
       ansible.builtin.dnf:
-        name: "{{ epel_repo_url }}"
+        name: "{{ redis_epel_repo_url }}"
         state: present
         update_cache: true
         disable_gpg_check: true