NGINX Plus 설치 가이드 w/HA (Active-Active) : Technical Service Center

구성 환경
- BareMetal
- Cent 7.9
- Network Access OK
- RealIP : 192.168.3.105~106 / VIP : 192.168.3.107~8

설치 Flow
- OS 환경 설정
  - 필요 Package 설치
  - Selinux OFF
- Nginx-ha-setup 설치 / Nginx-sync 설치
- Nginx-ha 설치 (A/S)
- Nginx-Sync 구성 ( Nginx Config Sync 용 )

구성
- NGINX Plus 양 노드 설치
- NGINX-ha-keepalived 설치
```
yum install nginx-ha-keepalived
```
- NGINX Sync 설치
```
yum install nginx-sync
```
- nginx-ha-setup 실행 ( Master Node )

[root@105 ~]# nginx-ha-setup
Thank you for using NGINX Plus!

This script is intended for use with RHEL/CentOS/SLES/Debian/Ubuntu-based systems.
It will configure highly available NGINX Plus environment in Active/Passive pair.

NOTE: you will need the following in order to continue:
 - 2 running systems (nodes) with static IP addresses
 - one free IP address to use as Cluster IP endpoint

It is strongly recommended to run this script simultaneously on both nodes,
e.g. use two terminal windows and switch between them step by step.

It is recommended to run this script under screen(1) in order to allow
installation process to continue in case of unexpected session disconnect.

Press <Enter> to continue...

Step 1: configuring internal management IP addresses.

In order to communicate with each other, both nodes must have at least one IP address.

The guessed primary IP of this node is: 192.168.3.105/24

Do you want to use this address for internal cluster communication? (y/n)
Please use 'y' or 'n'
Please use 'y' or 'n'
IP address of this host is set to: 192.168.3.105/24
Primary network interface: ens192

Now please enter IP address of a second node: 192.168.3.106/24
You entered: 192.168.3.106/24
Is it correct? (y/n)
IP address of the second node is set to: 192.168.3.106/24

Press <Enter> to continue...

Step 2: creating keepalived configuration

Now you have to choose cluster IP address.
This address will be used as en entry point to all your cluster resources.
The chosen address must not be one already associated with a physical node.

Enter cluster IP address: 192.168.3.107
You entered: 192.168.3.107
Is it correct? (y/n)

You must choose which node should have the MASTER role in this cluster.

Please choose what the current node role is:
1) MASTER
2) BACKUP

(on the second node you should choose the opposite variant)

Press 1 or 2.
This is the MASTER node.

Step 3: starting keepalived

Starting keepalived...
keepalived has been successfully started.

Press <Enter> to continue...

Step 4: configuring cluster

Enabling keepalived and nginx at boot time...
알림: 'systemctl enable nginx.service'에 요청을 전송하고 있습니다.
알림: 'systemctl enable keepalived.service'에 요청을 전송하고 있습니다.
Created symlink from /etc/systemd/system/multi-user.target.wants/keepalived.service to /usr/lib/systemd/system/keepalived.service.
Initial configuration complete!

keepalived logs are written to syslog and located here:
/var/log/messages

Further configuration may be required according to your needs
and environment.
Main configuration file for keepalived can be found at:
 /etc/keepalived/keepalived.conf

To control keepalived, use 'service keepalived' command:
 service keepalived status

keepalived documentation can be found at:
<http://www.keepalived.org/>

NGINX-HA-keepalived documentation can be found at:
/usr/share/doc/nginx-ha-keepalived/README

Thank you for using NGINX Plus!

[root@105 ~]#

nginx-ha-setup 실행 ( 2nd Node )

[root@106 packages]# nginx-ha-setup
Thank you for using NGINX Plus!

This script is intended for use with RHEL/CentOS/SLES/Debian/Ubuntu-based systems.
It will configure highly available NGINX Plus environment in Active/Passive pair.

NOTE: you will need the following in order to continue:
 - 2 running systems (nodes) with static IP addresses
 - one free IP address to use as Cluster IP endpoint

It is strongly recommended to run this script simultaneously on both nodes,
e.g. use two terminal windows and switch between them step by step.

It is recommended to run this script under screen(1) in order to allow
installation process to continue in case of unexpected session disconnect.

Press <Enter> to continue...

Step 1: configuring internal management IP addresses.

In order to communicate with each other, both nodes must have at least one IP address.

The guessed primary IP of this node is: 192.168.3.106/24

Do you want to use this address for internal cluster communication? (y/n)
IP address of this host is set to: 192.168.3.106/24
Primary network interface: ens192

Now please enter IP address of a second node: 192.168.3.105/24
You entered: 192.168.3.105/24
Is it correct? (y/n)
IP address of the second node is set to: 192.168.3.105/24

Press <Enter> to continue...

Step 2: creating keepalived configuration

Now you have to choose cluster IP address.
This address will be used as en entry point to all your cluster resources.
The chosen address must not be one already associated with a physical node.

Enter cluster IP address: 192.168.3.107
You entered: 192.168.3.107
Is it correct? (y/n)

You must choose which node should have the MASTER role in this cluster.

Please choose what the current node role is:
1) MASTER
2) BACKUP

(on the second node you should choose the opposite variant)

Press 1 or 2.
This is the BACKUP node.

Step 3: starting keepalived

Starting keepalived...
keepalived has been successfully started.

Press <Enter> to continue...

Step 4: configuring cluster

Enabling keepalived and nginx at boot time...
알림: 'systemctl enable nginx.service'에 요청을 전송하고 있습니다.
알림: 'systemctl enable keepalived.service'에 요청을 전송하고 있습니다.
Created symlink from /etc/systemd/system/multi-user.target.wants/keepalived.service to /usr/lib/systemd/system/keepalived.service.
Initial configuration complete!

keepalived logs are written to syslog and located here:
/var/log/messages

Further configuration may be required according to your needs
and environment.
Main configuration file for keepalived can be found at:
 /etc/keepalived/keepalived.conf

To control keepalived, use 'service keepalived' command:
 service keepalived status

keepalived documentation can be found at:
<http://www.keepalived.org/>

NGINX-HA-keepalived documentation can be found at:
/usr/share/doc/nginx-ha-keepalived/README

Thank you for using NGINX Plus!

[root@106 packages]#

nginx-sync 구성 (# Master Node)
- vi /etc/nginx-sync.conf 설정
  - Node IP / Sync 대상 Config Path / Sync 예외 Config Path 설정
  - 기타 상세 설정도 있음
```
NODES="192.168.3.106" 
CONFPATHS="/etc/nginx/nginx.conf /etc/nginx/conf.d" 
EXCLUDE="default.conf"
```
- 양 노드 root SSH Access 설정
  - Master / Slave Node 모두 root ssh passwd 없이 allow 가능해야함

ssh-keygen -t rsa -b 2048 ## 키 생성
cat /root/.ssh/id_rsa.pub ## 키 확인 
## 상대방 노드에 Key 복사 
echo 'from="192.168.3.105" ssh-rsa 
AAAAB3NzaC1yc2EAAAADAQABAAABAQDIOS+modOCOqzDgcTP5RnGGNFwFg+VUnVrDIMZDpU54R6pRQMm4xQfXzeEmaS1jMRIRbqW8hEBYAfoXueD/jhDJhqNrrhcbOYhvW+C2P6frSZ5wPknvmOkY8aZboGUDIk64VELUdn6VOUD9fSdelG2TFnK8J8g+jg6tQkpZBWzV9in3WPax/Yw5WDVWKUTq8ZNI2LNA4EWPyOcQu/ThUY8+RIC9xDGV6ebx2nczJAfaAb4pTf7xkVlFeed5GSN/NotfEtB5vvnOXAtjZYcpJOZeUXwOt2HBVZcY0A7YNJjXyoDpu19+5kZsgiJjuUAm3JoOfebPbFhDvJp6gkPG8Gdd7i" ROOTu@106.nginx-test.com root@192.168.3.105' >> /root/.ssh/authorized_keys

Active - Active 구성
- 조건 : VIP 2EA ( 각각의 Node 에서, VIP 로 Traffic 를 받음 )
A/A 구성방법
- Keepalived.conf 변경 ( # Node 1 )

global_defs {
    vrrp_version 3
}

vrrp_script chk_manual_failover {
    script "/usr/libexec/keepalived/nginx-ha-manual-failover"
    interval 10
    weight 50
}

vrrp_script chk_nginx_service {
    script "/usr/libexec/keepalived/nginx-ha-check"
    interval 3
    weight 50
}

vrrp_instance VI_1 {
    interface ens192
    priority 101 ## Priority 가 높으면 Master 
    virtual_router_id 51
    advert_int 1
    accept
    garp_master_refresh 5
    garp_master_refresh_repeat 1
    unicast_src_ip 192.168.3.105/24
    unicast_peer {
        192.168.3.106
    }
    virtual_ipaddress {
        192.168.3.107
    }
    track_script {
        chk_nginx_service
        chk_manual_failover
    }
    notify "/usr/libexec/keepalived/nginx-ha-notify"
}
### 추가 Config ( 108 VIP , Slave 설정 )
vrrp_instance VI_2 {
    interface         ens192
    priority          99  ## Priority 가 낮으면 Slave 
    virtual_router_id 61
    advert_int        1
    accept
    unicast_src_ip    192.168.3.105/24

    unicast_peer {
        192.168.3.106
    }

    virtual_ipaddress {
        192.168.3.108
    }

    track_script {
        chk_nginx_service
    }

    notify "/usr/lib/keepalived/nginx-ha-notify"
}

Keepalived.conf 변경 ( # Node 2 )

global_defs {
    vrrp_version 3
}

vrrp_script chk_manual_failover {
    script "/usr/libexec/keepalived/nginx-ha-manual-failover"
    interval 10
    weight 50
}

vrrp_script chk_nginx_service {
    script "/usr/libexec/keepalived/nginx-ha-check"
    interval 3
    weight 50
}

vrrp_instance VI_1 {
    interface ens192
    priority 100
    virtual_router_id 51
    advert_int 1
    accept
    garp_master_refresh 5
    garp_master_refresh_repeat 1
    unicast_src_ip 192.168.3.106/24
    unicast_peer {
        192.168.3.105
    }
    virtual_ipaddress {
        192.168.3.107
    }
    track_script {
        chk_nginx_service
        chk_manual_failover
    }
    notify "/usr/libexec/keepalived/nginx-ha-notify"
}

### 추가 Config ( 108 VIP , Master 설정 ) 
vrrp_instance VI_2 {
    interface         ens192
    priority          100
    virtual_router_id 61
    advert_int        1
    accept
    unicast_src_ip    192.168.3.106/24

    unicast_peer {
        192.168.3.105
    }

    virtual_ipaddress {
        192.168.3.108
    }

    track_script {
        chk_nginx_service
    }

    notify "/usr/lib/keepalived/nginx-ha-notify"
}

Processes 재기동 ( 양 노드 )

```
systemctl restart keepalived.service
```
- 재기동 이후 확인사항

ip addr 
## 각각의 Node 에 VIP 할당 여부 확인

NGINX Plus 설치 가이드 w/HA (Active-Active) 인쇄

관련 아티클