Bài 7: Cài đặt Patroni

Mục tiêu

Sau bài học này, bạn sẽ:

• Cài đặt Python và các dependencies cần thiết
• Cài đặt Patroni qua pip
• Hiểu cấu trúc file patroni.yml
• Tạo systemd service cho Patroni
• Cài đặt Patroni trên 3 nodes

1. Giới thiệu

Patroni là một Python application, do đó cần Python runtime và các dependencies. Trong bài này, chúng ta sẽ:

1. Cài đặt Python 3 và pip
2. Cài đặt Patroni package
3. Cấu hình Patroni
4. Tạo systemd service để quản lý Patroni daemon

Kiến trúc mục tiêu:

┌──────────────────────────────────┐
│      etcd Cluster (3 nodes)      │
│         ✅ RUNNING                │
└──────────────────────────────────┘
         │        │        │
    ┌────┴────┐  │  ┌─────┴─────┐
    ▼         ▼  ▼  ▼           ▼
┌─────────┐ ┌─────────┐ ┌─────────┐
│Patroni 1│ │Patroni 2│ │Patroni 3│ ← Installing now
│  + PG   │ │  + PG   │ │  + PG   │
└─────────┘ └─────────┘ └─────────┘

2. Cài đặt Python Dependencies

2.1. Cài đặt Python trên Ubuntu/Debian

Thực hiện trên TẤT CẢ 3 nodes.

Bước 1: Cài đặt Python 3 và pip

# Update package list
sudo apt update

# Cài Python 3, pip, và development tools
sudo apt install -y python3 python3-pip python3-dev python3-venv

# Kiểm tra version
python3 --version
# Output: Python 3.10.x (hoặc 3.11.x hoặc 3.12.x)

pip3 --version
# Output: pip 22.x.x

Bước 2: Cài đặt system dependencies

# Cài các libraries cần thiết cho Patroni và PostgreSQL Python modules
sudo apt install -y \
  libpq-dev \
  gcc \
  python3-psycopg2

# libpq-dev: PostgreSQL client library headers
# gcc: Compiler cho building Python packages
# python3-psycopg2: PostgreSQL adapter cho Python

2.2. Cài đặt Python trên CentOS/RHEL

# Python 3 thường đã có sẵn, nhưng cần pip và dev tools
sudo dnf install -y python3 python3-pip python3-devel

# System dependencies
sudo dnf install -y \
  postgresql18-devel \
  gcc \
  python3-psycopg2

# Kiểm tra
python3 --version
pip3 --version

2.3. Upgrade pip (khuyến nghị)

# Upgrade pip to latest version
sudo pip3 install --upgrade pip

# Verify
pip3 --version

3. Cài đặt Patroni qua pip

3.1. Cài đặt Patroni

Thực hiện trên TẤT CẢ 3 nodes.

# Cài Patroni với etcd support
sudo pip3 install patroni[etcd]

# Hoặc specify version cụ thể
sudo pip3 install patroni[etcd]==3.2.0

# Kiểm tra installation
patroni --version
# Output: patroni 3.2.0

patronictl --version
# Output: patronictl 3.2.0

Giải thích [etcd]:

• Patroni hỗ trợ nhiều DCS backends (etcd, consul, zookeeper)
• [etcd] install thêm python-etcd client library
• Có thể cài nhiều backends: patroni[etcd,consul,zookeeper]

3.2. Các packages được cài đặt

# Liệt kê các packages liên quan
pip3 list | grep -E "(patroni|etcd|psycopg)"

# Output:
# patroni          3.2.0
# python-etcd      0.4.5
# psycopg2         2.9.x
# psycopg2-binary  2.9.x

3.3. Verify Patroni commands

# Check patroni binary
which patroni
# Output: /usr/local/bin/patroni

# Check patronictl binary
which patronictl
# Output: /usr/local/bin/patronictl

# List patronictl commands
patronictl --help

Output:

Usage: patronictl [OPTIONS] COMMAND [ARGS]...

Commands:
  list        Show cluster members
  switchover  Perform planned switchover
  failover    Perform manual failover
  reinit      Reinitialize cluster member
  restart     Restart cluster member
  reload      Reload cluster member configuration
  pause       Disable auto-failover
  resume      Enable auto-failover
  edit-config Edit cluster configuration
  ...

4. Cấu trúc file patroni.yml

4.1. Tổng quan về patroni.yml

File patroni.yml là file cấu hình chính của Patroni, bao gồm:

• Scope: Tên cluster
• Namespace: Prefix cho keys trong DCS
• Node information: Tên node, REST API config
• DCS connection: etcd endpoints
• Bootstrap: Initial cluster setup
• PostgreSQL: Database configuration
• Tags: Node metadata
• Watchdog: Optional hardware watchdog

4.2. Cấu trúc cơ bản

scope: postgres
namespace: /service/
name: node1

restapi:
  listen: 0.0.0.0:8008
  connect_address: 10.0.1.11:8008

etcd:
  hosts: 10.0.1.11:2379,10.0.1.12:2379,10.0.1.13:2379

bootstrap:
  dcs:
    ttl: 30
    loop_wait: 10
    retry_timeout: 10
    maximum_lag_on_failover: 1048576
    postgresql:
      use_pg_rewind: true
      parameters:
        wal_level: replica
        hot_standby: "on"
        max_wal_senders: 10
        max_replication_slots: 10

  initdb:
    - encoding: UTF8
    - data-checksums

  pg_hba:
    - host replication replicator 10.0.1.0/24 scram-sha-256
    - host all all 0.0.0.0/0 scram-sha-256

postgresql:
  listen: 0.0.0.0:5432
  connect_address: 10.0.1.11:5432
  data_dir: /var/lib/postgresql/18/data
  bin_dir: /usr/lib/postgresql/18/bin
  authentication:
    replication:
      username: replicator
      password: replicator_password
    superuser:
      username: postgres
      password: postgres_password

tags:
  nofailover: false
  noloadbalance: false
  clonefrom: false
  nosync: false

4.3. Giải thích các sections chính

Section: Global

scope: postgres          # Cluster name (unique identifier)
namespace: /service/     # DCS key prefix
name: node1             # Unique node name trong cluster

Section: REST API

restapi:
  listen: 0.0.0.0:8008              # Listen trên tất cả interfaces
  connect_address: 10.0.1.11:8008   # Address để nodes khác connect
  # authentication:                  # Optional: Basic auth
  #   username: admin
  #   password: secret

REST API endpoints:

• GET /: Cluster info
• GET /health: Health check (200 = healthy)
• GET /primary: Check if node is primary
• GET /replica: Check if node is replica
• POST /restart: Restart PostgreSQL
• POST /reload: Reload configuration

Section: etcd (DCS)

etcd:
  hosts: 10.0.1.11:2379,10.0.1.12:2379,10.0.1.13:2379  # etcd endpoints
  # protocol: http      # http hoặc https
  # username: user      # Optional: etcd authentication
  # password: pass

Section: Bootstrap

bootstrap:
  dcs:
    ttl: 30                           # Leader lock TTL (seconds)
    loop_wait: 10                     # Check interval (seconds)
    retry_timeout: 10                 # DCS operation timeout
    maximum_lag_on_failover: 1048576  # Max lag (bytes) để eligible for failover
    postgresql:
      use_pg_rewind: true             # Enable pg_rewind for fast recovery
      parameters:                     # PostgreSQL parameters
        wal_level: replica
        hot_standby: "on"
        wal_keep_size: "1GB"
        max_wal_senders: 10
        max_replication_slots: 10
        wal_log_hints: "on"           # Required for pg_rewind

  initdb:                             # pg_initdb options
    - encoding: UTF8
    - data-checksums                  # Enable page checksums
    - locale: en_US.UTF-8

  pg_hba:                             # pg_hba.conf entries
    - host replication replicator 10.0.1.0/24 scram-sha-256
    - host all all 0.0.0.0/0 scram-sha-256

  users:                              # Create users during bootstrap
    admin:
      password: admin_password
      options:
        - createrole
        - createdb

Lưu ý: Bootstrap section chỉ áp dụng khi khởi tạo cluster lần đầu.

Section: PostgreSQL

postgresql:
  listen: 0.0.0.0:5432                    # PostgreSQL listen address
  connect_address: 10.0.1.11:5432         # Address để connect từ bên ngoài
  data_dir: /var/lib/postgresql/18/data   # Data directory
  bin_dir: /usr/lib/postgresql/18/bin     # PostgreSQL binaries
  pgpass: /tmp/pgpass                     # Optional: pgpass file
  
  authentication:
    replication:
      username: replicator
      password: replicator_password
    superuser:
      username: postgres
      password: postgres_password
    rewind:                                # Optional: dedicated user for pg_rewind
      username: rewind_user
      password: rewind_password
  
  parameters:                              # PostgreSQL runtime parameters
    max_connections: 100
    shared_buffers: 2GB
    effective_cache_size: 6GB
    maintenance_work_mem: 512MB
    checkpoint_completion_target: 0.9
    wal_buffers: 16MB
    default_statistics_target: 100
    random_page_cost: 1.1
    effective_io_concurrency: 200
    work_mem: 16MB
    min_wal_size: 1GB
    max_wal_size: 2GB
  
  pg_hba:                                  # Additional pg_hba.conf entries
    - host all all 10.0.2.0/24 scram-sha-256
  
  callbacks:                               # Optional: callback scripts
    on_start: /etc/patroni/scripts/on_start.sh
    on_stop: /etc/patroni/scripts/on_stop.sh
    on_role_change: /etc/patroni/scripts/on_role_change.sh

Section: Tags

tags:
  nofailover: false      # false = có thể become primary
  noloadbalance: false   # false = có thể nhận read traffic
  clonefrom: false       # false = có thể clone từ node này
  nosync: false          # false = có thể become synchronous standby
  # Custom tags
  datacenter: dc1
  environment: production

Section: Watchdog (Optional)

watchdog:
  mode: required         # required, automatic, hoặc off
  device: /dev/watchdog  # Hardware watchdog device
  safety_margin: 5       # Seconds before watchdog triggers

5. Tạo Patroni configuration files

5.1. Tạo thư mục cấu hình

Trên TẤT CẢ 3 nodes:

# Tạo directory cho Patroni config
sudo mkdir -p /etc/patroni

# Tạo directory cho callback scripts (optional)
sudo mkdir -p /etc/patroni/scripts

# Set ownership
sudo chown -R postgres:postgres /etc/patroni

5.2. Node 1 - /etc/patroni/patroni.yml

scope: postgres
namespace: /service/
name: node1

restapi:
  listen: 0.0.0.0:8008
  connect_address: 10.0.1.11:8008

etcd:
  hosts: 10.0.1.11:2379,10.0.1.12:2379,10.0.1.13:2379

bootstrap:
  dcs:
    ttl: 30
    loop_wait: 10
    retry_timeout: 10
    maximum_lag_on_failover: 1048576
    postgresql:
      use_pg_rewind: true
      parameters:
        wal_level: replica
        hot_standby: "on"
        wal_keep_size: "1GB"
        max_wal_senders: 10
        max_replication_slots: 10
        wal_log_hints: "on"

  initdb:
    - encoding: UTF8
    - data-checksums

  pg_hba:
    - host replication replicator 10.0.1.11/32 scram-sha-256
    - host replication replicator 10.0.1.12/32 scram-sha-256
    - host replication replicator 10.0.1.13/32 scram-sha-256
    - host all all 10.0.1.0/24 scram-sha-256
    - host all all 0.0.0.0/0 md5

  users:
    admin:
      password: admin123
      options:
        - createrole
        - createdb

postgresql:
  listen: 0.0.0.0:5432
  connect_address: 10.0.1.11:5432
  data_dir: /var/lib/postgresql/18/data
  bin_dir: /usr/lib/postgresql/18/bin
  authentication:
    replication:
      username: replicator
      password: replicator_password
    superuser:
      username: postgres
      password: postgres_password
  parameters:
    max_connections: 100
    shared_buffers: 2GB
    effective_cache_size: 6GB
    maintenance_work_mem: 512MB
    checkpoint_completion_target: 0.9

tags:
  nofailover: false
  noloadbalance: false
  clonefrom: false
  nosync: false

5.3. Node 2 - /etc/patroni/patroni.yml

scope: postgres
namespace: /service/
name: node2

restapi:
  listen: 0.0.0.0:8008
  connect_address: 10.0.1.12:8008

etcd:
  hosts: 10.0.1.11:2379,10.0.1.12:2379,10.0.1.13:2379

bootstrap:
  dcs:
    ttl: 30
    loop_wait: 10
    retry_timeout: 10
    maximum_lag_on_failover: 1048576
    postgresql:
      use_pg_rewind: true
      parameters:
        wal_level: replica
        hot_standby: "on"
        wal_keep_size: "1GB"
        max_wal_senders: 10
        max_replication_slots: 10
        wal_log_hints: "on"

  initdb:
    - encoding: UTF8
    - data-checksums

  pg_hba:
    - host replication replicator 10.0.1.11/32 scram-sha-256
    - host replication replicator 10.0.1.12/32 scram-sha-256
    - host replication replicator 10.0.1.13/32 scram-sha-256
    - host all all 10.0.1.0/24 scram-sha-256
    - host all all 0.0.0.0/0 md5

  users:
    admin:
      password: admin123
      options:
        - createrole
        - createdb

postgresql:
  listen: 0.0.0.0:5432
  connect_address: 10.0.1.12:5432
  data_dir: /var/lib/postgresql/18/data
  bin_dir: /usr/lib/postgresql/18/bin
  authentication:
    replication:
      username: replicator
      password: replicator_password
    superuser:
      username: postgres
      password: postgres_password
  parameters:
    max_connections: 100
    shared_buffers: 2GB
    effective_cache_size: 6GB
    maintenance_work_mem: 512MB
    checkpoint_completion_target: 0.9

tags:
  nofailover: false
  noloadbalance: false
  clonefrom: false
  nosync: false

5.4. Node 3 - /etc/patroni/patroni.yml

scope: postgres
namespace: /service/
name: node3

restapi:
  listen: 0.0.0.0:8008
  connect_address: 10.0.1.13:8008

etcd:
  hosts: 10.0.1.11:2379,10.0.1.12:2379,10.0.1.13:2379

bootstrap:
  dcs:
    ttl: 30
    loop_wait: 10
    retry_timeout: 10
    maximum_lag_on_failover: 1048576
    postgresql:
      use_pg_rewind: true
      parameters:
        wal_level: replica
        hot_standby: "on"
        wal_keep_size: "1GB"
        max_wal_senders: 10
        max_replication_slots: 10
        wal_log_hints: "on"

  initdb:
    - encoding: UTF8
    - data-checksums

  pg_hba:
    - host replication replicator 10.0.1.11/32 scram-sha-256
    - host replication replicator 10.0.1.12/32 scram-sha-256
    - host replication replicator 10.0.1.13/32 scram-sha-256
    - host all all 10.0.1.0/24 scram-sha-256
    - host all all 0.0.0.0/0 md5

  users:
    admin:
      password: admin123
      options:
        - createrole
        - createdb

postgresql:
  listen: 0.0.0.0:5432
  connect_address: 10.0.1.13:5432
  data_dir: /var/lib/postgresql/18/data
  bin_dir: /usr/lib/postgresql/18/bin
  authentication:
    replication:
      username: replicator
      password: replicator_password
    superuser:
      username: postgres
      password: postgres_password
  parameters:
    max_connections: 100
    shared_buffers: 2GB
    effective_cache_size: 6GB
    maintenance_work_mem: 512MB
    checkpoint_completion_target: 0.9

tags:
  nofailover: false
  noloadbalance: false
  clonefrom: false
  nosync: false

5.5. Set permissions

Trên TẤT CẢ 3 nodes:

# Set ownership
sudo chown postgres:postgres /etc/patroni/patroni.yml

# Secure permissions (file chứa passwords)
sudo chmod 600 /etc/patroni/patroni.yml

# Verify
ls -l /etc/patroni/patroni.yml
# Output: -rw------- 1 postgres postgres ... patroni.yml

6. Tạo systemd service cho Patroni

6.1. Tạo systemd unit file

Tạo file /etc/systemd/system/patroni.service trên TẤT CẢ 3 nodes:

[Unit]
Description=Patroni PostgreSQL HA manager
Documentation=https://patroni.readthedocs.io/
After=syslog.target network.target etcd.service

[Service]
Type=simple
User=postgres
Group=postgres

# Start Patroni
ExecStart=/usr/local/bin/patroni /etc/patroni/patroni.yml

# Reload configuration
ExecReload=/bin/kill -HUP $MAINPID

# Restart behavior
Restart=on-failure
RestartSec=5

# Limits
LimitNOFILE=65536
LimitNPROC=65536

# Logging
StandardOutput=journal
StandardError=journal

# Working directory
WorkingDirectory=/var/lib/postgresql

# Environment
Environment=PATH=/usr/lib/postgresql/18/bin:/usr/local/bin:/usr/bin:/bin

[Install]
WantedBy=multi-user.target

6.2. Giải thích systemd unit file

6.3. Enable và verify service

Trên TẤT CẢ 3 nodes:

# Reload systemd
sudo systemctl daemon-reload

# Enable Patroni service (auto-start on boot)
sudo systemctl enable patroni

# Verify service file
systemctl cat patroni

# Check status (should be inactive/dead - not started yet)
systemctl status patroni

7. Verify installation

7.1. Check Patroni installation

Trên TẤT CẢ 3 nodes:

# Check Patroni version
patroni --version

# Check patronictl
patronictl --help

# Verify config file
sudo -u postgres cat /etc/patroni/patroni.yml

# Validate YAML syntax
python3 -c "import yaml; yaml.safe_load(open('/etc/patroni/patroni.yml'))"
# No output = valid YAML

7.2. Check etcd connectivity

# Test etcd từ mỗi node
etcdctl endpoint health --cluster

# Should see all 3 etcd nodes healthy

7.3. Pre-flight checklist

Trước khi khởi động Patroni, verify:

# ✅ PostgreSQL installed nhưng NOT running
systemctl status postgresql
# Should be: inactive (dead)

# ✅ etcd cluster healthy
etcdctl endpoint health --cluster

# ✅ Patroni config file exists và có permissions đúng
ls -l /etc/patroni/patroni.yml

# ✅ Data directory có ownership đúng
ls -ld /var/lib/postgresql/18/data
# Owner: postgres:postgres

# ✅ systemd service enabled
systemctl is-enabled patroni
# Output: enabled

# ✅ Firewall rules (nếu có firewall)
sudo ufw status | grep -E "(5432|8008)"
# Hoặc
sudo firewall-cmd --list-ports | grep -E "(5432|8008)"

8. Troubleshooting

8.1. Issue: pip install fails

# Error: "No module named 'setuptools'"
# Solution:
sudo apt install python3-setuptools

# Or upgrade pip
sudo pip3 install --upgrade pip setuptools

8.2. Issue: Cannot find PostgreSQL binaries

# Error: "could not find postgres binary"
# Solution: Check bin_dir in patroni.yml

# Find PostgreSQL bin directory
which postgres
# Output: /usr/lib/postgresql/18/bin/postgres

# Update patroni.yml
postgresql:
  bin_dir: /usr/lib/postgresql/18/bin

8.3. Issue: Permission denied on data directory

# Error: "FATAL:  data directory ... has wrong ownership"
# Solution:
sudo chown -R postgres:postgres /var/lib/postgresql/18/data
sudo chmod 700 /var/lib/postgresql/18/data

8.4. Issue: YAML syntax error

# Validate YAML
python3 -c "import yaml; yaml.safe_load(open('/etc/patroni/patroni.yml'))"

# Common issues:
# - Mixed tabs and spaces (use spaces only)
# - Incorrect indentation
# - Missing quotes around special characters
# - Duplicate keys

9. Tổng kết

Key Takeaways

✅ Patroni: Python application, cài qua pip

✅ Dependencies: Python 3, pip, psycopg2, python-etcd

✅ Configuration: patroni.yml chứa tất cả settings

✅ systemd service: Quản lý Patroni daemon

✅ Security: File config có permissions 600 (chứa passwords)

Checklist sau Lab

•  Python 3 và pip đã cài trên cả 3 nodes
•  Patroni 3.2.0+ đã cài trên cả 3 nodes
•  File /etc/patroni/patroni.yml đã tạo trên mỗi node với config riêng
•  systemd service patroni.service đã tạo và enabled
•  Permissions đúng cho config file (600, owner postgres)
•  etcd cluster đang running và healthy

Kiến trúc hiện tại

✅ 3 VMs prepared (Bài 4)
✅ PostgreSQL 18 installed (Bài 5)
✅ etcd cluster running (Bài 6)
✅ Patroni installed and configured (Bài 7)

Next: Bootstrap cluster lần đầu

Chuẩn bị cho Bài 8

Bài 8 sẽ đi sâu vào cấu hình Patroni chi tiết:

• Phân tích từng section trong patroni.yml
• Bootstrap configuration options
• PostgreSQL parameters tuning
• Authentication setup
• Tags và constraints

Bài 9: Bootstrap cluster

Sau khi Patroni đã cài và cấu hình xong, Bài 9 sẽ hướng dẫn:

• Khởi động Patroni lần đầu
• Quá trình bootstrap tự động
• Kiểm tra cluster status
• Troubleshootin