서버에 동시 접속자가 많아져 서버 한대로 처리가 불가능 할 경우 부하 분산을 통해서 서비스를 하게 된다.
일반적으로 처음에는 한 대로 서비스 하던 것을 두 대로 늘여서 서비스를 시도하게 된다.
이때 Load Balancer 역활을 하는 서버를 일반적으로는 분리를 해서 논리적으로 상위단에 두고 들어 오는 패킷을 Real Server 단으로 분산을 시켜 주는데 그렇게 되면 LB 서버용도로 따로 서버가 한대 더 들어가게 되므로 초기 투자 비용이 증가하게 된다. 그래서 리눅스로 서비스를 운영하는 경우라면 LB 서버와 Real Server 를 합쳐서 두가지 일을 동시에 시킬 수 있다.
즉 서버 두대가 LB의 master/backup 역활을 하면서 동시에 Real Server1, Real Server2 의 역활을 수행하게 된다.
이번에는 keepalived.conf에 VRRP와 LVS 설정이 동시에 들어가게 된다.
중요한 부분 한가지를 짚고 넘어가자.
두 서버가 데이터 동기화는 어떻게 할 것인가?
Storage를 사용하면 된다고 대답을 할 수 있으나 LB 서버도 비용을 줄이고자 Real Server와 합쳐 놓은 상태인데 Storage를 구매할 여력이 남아 있을리 없다고 본다. rsync 등을 사용해서 최대한 짧게 동기화 하는것도 방법이긴 하다.
이런 분들을 위해서 DRBD를 추천한다. DRBD + GFS2를 사용하면 read/write 를 전체 노드에서 사용이 가능 하다.
DRBD 관련한 글을 포스팅 한 것이 있으니 찾아서 같이 응용을 하면 더욱 좋은 구성을 만들게 될 것이다.
이 글에서는 DRBD와 연동 설명은 제외한다.
다음과 같이 가정을 한다.
Real Server 1 : 10.30.6.111
Real Server 2 : 10.30.6.112
VIP : 10.30.6.110
Service : http
우선 커널을 따로 컴파일 해서 사용하는 경우라면 LVS를 커널에서 지원이 되도록 만들어야 한다.
모듈로 이미 들어가 있거나 debian 기본 커널 사용시에는 그냥 넘어간다.
-*- Networking support --->
Networking options --->
[*] Network packet filtering framework (Netfilter) --->
<*> IP virtual server support --->
--- IP virtual server support
[*] IP virtual server debugging
(12) IPVS connection table size (the Nth power of 2)
*** IPVS transport protocol load balancing support ***
[*] TCP load balancing support
[*] UDP load balancing support
[*] ESP load balancing support
[*] AH load balancing support
*** IPVS scheduler ***
<*> round-robin scheduling
<*> weighted round-robin scheduling
<*> least-connection scheduling
<*> weighted least-connection scheduling
<*> locality-based least-connection scheduling
<*> locality-based least-connection with replication scheduling
<*> destination hashing scheduling
<*> source hashing scheduling
<*> shortest expected delay scheduling
<*> never queue scheduling
*** IPVS application helper ***
<*> FTP protocol helper
|
keepalived 를 설치한다.
두 서버 모두 동일하게 설치 한다.
dongho-vm111:~# apt-get install keepalived
Reading package lists... Done
Building dependency tree... Done
The following NEW packages will be installed:
keepalived
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 0B/111kB of archives.
After unpacking 340kB of additional disk space will be used.
Selecting previously deselected package keepalived.
(Reading database ... 9199 files and directories currently installed.)
Unpacking keepalived (from .../keepalived_1.1.12-1_i386.deb) ...
Setting up keepalived (1.1.12-1) ...
Starting keepalived: keepalived.
|
http 서비스를 한다고 가정을 하였으니 apache2 도 설치를 한다.
두 서버 모두 동일하게 설치를 한다.
|
dongho-vm111:~# apt-get install apache2
Reading package lists... Done
Building dependency tree... Done
The following extra packages will be installed:
apache2-mpm-worker apache2-utils apache2.2-common libapr1 libaprutil1
libexpat1 libpq4 libsqlite3-0
The following NEW packages will be installed:
apache2 apache2-mpm-worker apache2-utils apache2.2-common libapr1
libaprutil1 libexpat1 libpq4 libsqlite3-0
0 upgraded, 9 newly installed, 0 to remove and 0 not upgraded.
Need to get 2497kB of archives.
After unpacking 6492kB of additional disk space will be used.
Do you want to continue [Y/n]?
Get:1 http://ftp.debian.org etch/main libapr1 1.2.7-8.2 [109kB]
Get:2 http://ftp.debian.org etch/main libexpat1 1.95.8-3.4 [62.9kB]
Get:3 http://ftp.debian.org etch/main libpq4 8.1.15-0etch1 [290kB]
Get:4 http://ftp.debian.org etch/main libsqlite3-0 3.3.8-1.1 [194kB]
Get:5 http://ftp.debian.org etch/main libaprutil1 1.2.7+dfsg-2 [68.5kB]
Get:6 http://ftp.debian.org etch/main apache2-utils 2.2.3-4+etch6 [344kB]
Get:7 http://ftp.debian.org etch/main apache2.2-common 2.2.3-4+etch6 [963kB]
Get:8 http://ftp.debian.org etch/main apache2-mpm-worker 2.2.3-4+etch6 [425kB]
Get:9 http://ftp.debian.org etch/main apache2 2.2.3-4+etch6 [41.3kB]
Fetched 2497kB in 5s (476kB/s)
Selecting previously deselected package libapr1.
(Reading database ... 9237 files and directories currently installed.)
Unpacking libapr1 (from .../libapr1_1.2.7-8.2_i386.deb) ...
Selecting previously deselected package libexpat1.
Unpacking libexpat1 (from .../libexpat1_1.95.8-3.4_i386.deb) ...
Selecting previously deselected package libpq4.
Unpacking libpq4 (from .../libpq4_8.1.15-0etch1_i386.deb) ...
Selecting previously deselected package libsqlite3-0.
Unpacking libsqlite3-0 (from .../libsqlite3-0_3.3.8-1.1_i386.deb) ...
Selecting previously deselected package libaprutil1.
Unpacking libaprutil1 (from .../libaprutil1_1.2.7+dfsg-2_i386.deb) ...
Selecting previously deselected package apache2-utils.
Unpacking apache2-utils (from .../apache2-utils_2.2.3-4+etch6_i386.deb) ...
Selecting previously deselected package apache2.2-common.
Unpacking apache2.2-common (from .../apache2.2-common_2.2.3-4+etch6_i386.deb) ...
Selecting previously deselected package apache2-mpm-worker.
Unpacking apache2-mpm-worker (from .../apache2-mpm-worker_2.2.3-4+etch6_i386.deb) ...
Selecting previously deselected package apache2.
Unpacking apache2 (from .../apache2_2.2.3-4+etch6_all.deb) ...
Setting up libapr1 (1.2.7-8.2) ...
Setting up libexpat1 (1.95.8-3.4) ...
Setting up libpq4 (8.1.15-0etch1) ...
Setting up libsqlite3-0 (3.3.8-1.1) ...
Setting up libaprutil1 (1.2.7+dfsg-2) ...
Setting up apache2-utils (2.2.3-4+etch6) ...
Setting up apache2.2-common (2.2.3-4+etch6) ...
Setting Apache2 to Listen on port 80. If this is not desired, please edit /etc/apache2/ports.conf as desired. Note that the Port directive no longer works.
Module alias installed; run /etc/init.d/apache2 force-reload to enable.
Module autoindex installed; run /etc/init.d/apache2 force-reload to enable.
Module dir installed; run /etc/init.d/apache2 force-reload to enable.
Module env installed; run /etc/init.d/apache2 force-reload to enable.
Module mime installed; run /etc/init.d/apache2 force-reload to enable.
Module negotiation installed; run /etc/init.d/apache2 force-reload to enable.
Module setenvif installed; run /etc/init.d/apache2 force-reload to enable.
Module status installed; run /etc/init.d/apache2 force-reload to enable.
Module auth_basic installed; run /etc/init.d/apache2 force-reload to enable.
Module authz_default installed; run /etc/init.d/apache2 force-reload to enable.
Module authz_user installed; run /etc/init.d/apache2 force-reload to enable.
Module authz_groupfile installed; run /etc/init.d/apache2 force-reload to enable.
Module authn_file installed; run /etc/init.d/apache2 force-reload to enable.
Module authz_host installed; run /etc/init.d/apache2 force-reload to enable.
Setting up apache2-mpm-worker (2.2.3-4+etch6) ...
Starting web server (apache2)...apache2: apr_sockaddr_info_get() failed for dongho-vm111.freeb.net
apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1 for ServerName
.
Setting up apache2 (2.2.3-4+etch6) ...
|
이제 keepalived.conf 설정을 한다.
설정중 중간에 체크를 하여 추가로 넣어야 하는 값도 있다.
아래는 Master 에서의 설정 값이다.
|
dongho-vm111:~# cat /etc/keepalived/keepalived.conf
global_defs {
notification_email {
uheung@gmail.com
}
notification_email_from keepalived@ha-lvs
smtp_server localhost
smtp_connect_timeout 30
router_id HA_LVS
}
vrrp_instance ETH0 {
state MASTER
interface eth0
lvs_sync_daemon_interface eth0
virtual_router_id 10
priority 200
garp_master_delay 2
advert_int 1
smtp_alert
authentication {
auth_type PASS
auth_pass p@ssw0rd
}
virtual_ipaddress {
10.30.6.110/24
}
notify_master "/etc/keepalived/redirect.sh del 10.30.6.110"
notify_backup "/etc/keepalived/redirect.sh add 10.30.6.110"
notify_fault "/etc/keepalived/redirect.sh add 10.30.6.110"
}
virtual_server 10.30.6.110 80 {
delay_loop 3
lb_algo rr
lb_kind DR
#persistence_timeout 600
protocol TCP
virtualhost keepalive.lovediary.co.kr
real_server 10.30.6.111 80 {
weight 100
HTTP_GET {
url {
path /apache2-default/
digest c7b4690c8c46625ef0f328cd7a24a0a3
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 10.30.6.112 80 {
weight 100
HTTP_GET {
url {
path /apache2-default/
digest c7b4690c8c46625ef0f328cd7a24a0a3
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
|
Backup 서버쪽에서의 keepalived.conf 값이다.
Backup 쪽은 MASTER 대신 BACKUP 이라고 적어 주고 priority 값을 Master 보다 적게 주면 된다.
|
dongho-vm112:~# cat /etc/keepalived/keepalived.conf
global_defs {
notification_email {
uheung@gmail.com
}
notification_email_from keepalived@ha-lvs
smtp_server localhost
smtp_connect_timeout 30
router_id HA_LVS
}
vrrp_instance ETH0 {
state BACKUP
interface eth0
lvs_sync_daemon_interface eth0
virtual_router_id 10
priority 100
garp_master_delay 2
advert_int 1
smtp_alert
authentication {
auth_type PASS
auth_pass p@ssw0rd
}
virtual_ipaddress {
10.30.6.110/24
}
notify_master "/etc/keepalived/redirect.sh del 10.30.6.110"
notify_backup "/etc/keepalived/redirect.sh add 10.30.6.110"
notify_fault "/etc/keepalived/redirect.sh add 10.30.6.110"
}
virtual_server 10.30.6.110 80 {
delay_loop 3
lb_algo rr
lb_kind DR
#persistence_timeout 600
protocol TCP
virtualhost keepalive.lovediary.co.kr
real_server 10.30.6.111 80 {
weight 100
HTTP_GET {
url {
path /apache2-default/
digest c7b4690c8c46625ef0f328cd7a24a0a3
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 10.30.6.112 80 {
weight 100
HTTP_GET {
url {
path /apache2-default/
digest c7b4690c8c46625ef0f328cd7a24a0a3
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
|
위 keepalived.conf 값 중 HTTP_GET 에서의 digest 값은 다음과 같이 구할 수 있다.
genhash 는 keepalived 패키지에 기본 포함되어 있다.
-v 옵션을 주어 상세히 값을 찍어 본 것이지만 일반적으로 아래처럼 -v 옵션을 줄 필요까지는 없다.
그리고 --use-virtualhost 값을 따로 준 이유는 Real Server 에서 도메인을 한개만 운영 할 수도 있지만 여러개 운영하는 경우 어느 도메인에서 값을 받아 올찌 헤더에 던저줘야 할 때도 있어 같이 적은 것이다.
|
dongho-vm111:~# genhash -s 10.30.6.112 -p 80 --url /apache2-default/ --use-virtualhost keepalived.lovediary.co.kr -v
-----------------------[ HTTP Header Buffer ]-----------------------
0000 48 54 54 50 2f 31 2e 31 - 20 32 30 30 20 4f 4b 0d HTTP/1.1 200 OK.
0010 0a 44 61 74 65 3a 20 54 - 75 65 2c 20 31 37 20 46 .Date: Tue, 17 F
0020 65 62 20 32 30 30 39 20 - 30 30 3a 31 37 3a 31 32 eb 2009 00:17:12
0030 20 47 4d 54 0d 0a 53 65 - 72 76 65 72 3a 20 41 70 GMT..Server: Ap
0040 61 63 68 65 2f 32 2e 32 - 2e 33 20 28 44 65 62 69 ache/2.2.3 (Debi
0050 61 6e 29 0d 0a 4c 61 73 - 74 2d 4d 6f 64 69 66 69 an)..Last-Modifi
0060 65 64 3a 20 53 61 74 2c - 20 32 30 20 4e 6f 76 20 ed: Sat, 20 Nov
0070 32 30 30 34 20 32 30 3a - 31 36 3a 32 34 20 47 4d 2004 20:16:24 GM
0080 54 0d 0a 45 54 61 67 3a - 20 22 31 63 30 30 61 2d T..ETag: "1c00a-
0090 32 63 2d 34 63 32 33 62 - 36 30 30 22 0d 0a 41 63 2c-4c23b600"..Ac
00a0 63 65 70 74 2d 52 61 6e - 67 65 73 3a 20 62 79 74 cept-Ranges: byt
00b0 65 73 0d 0a 43 6f 6e 74 - 65 6e 74 2d 4c 65 6e 67 es..Content-Leng
00c0 74 68 3a 20 34 34 0d 0a - 43 6f 6e 6e 65 63 74 69 th: 44..Connecti
00d0 6f 6e 3a 20 63 6c 6f 73 - 65 0d 0a 43 6f 6e 74 65 on: close..Conte
00e0 6e 74 2d 54 79 70 65 3a - 20 74 65 78 74 2f 68 74 nt-Type: text/ht
00f0 6d 6c 3b 20 63 68 61 72 - 73 65 74 3d 55 54 46 2d ml; charset=UTF-
0100 38 0d 0a 0d 0a - 8....
-----------------------[ HTTP Header Ascii Buffer ]-----------------------
HTTP/1.1 200 OK
Date: Tue, 17 Feb 2009 00:17:12 GMT
Server: Apache/2.2.3 (Debian)
Last-Modified: Sat, 20 Nov 2004 20:16:24 GMT
ETag: "1c00a-2c-4c23b600"
Accept-Ranges: bytes
Content-Length: 44
Connection: close
Content-Type: text/html; charset=UTF-8
-----------------------[ HTML Buffer ]-----------------------
0000 3c 68 74 6d 6c 3e 3c 62 - 6f 64 79 3e 3c 68 31 3e <html><body><h1>
0010 49 74 20 77 6f 72 6b 73 - 21 3c 2f 68 31 3e 3c 2f It works!</h1></
0020 62 6f 64 79 3e 3c 2f 68 - 74 6d 6c 3e body></html>
-----------------------[ HTML MD5 resulting ]-----------------------
0000 c7 b4 69 0c 8c 46 62 5e - f0 f3 28 cd 7a 24 a0 a3 ..i..Fb^..(.z$..
-----------------------[ HTML MD5 final resulting ]-----------------------
MD5SUM = c7b4690c8c46625ef0f328cd7a24a0a3
Global response time for [/apache2-default/] =10845
|
다른쪽 서버에서도 체크를 한다.
아래는 apache2 의 기본 페이지라서 결과값이 같게 나오고 있다.
|
dongho-vm112:~# genhash -s 10.30.6.111 -p 80 --url /apache2-default/ --use-virtualhost keepalived.lovediary.co.kr -v
-----------------------[ HTTP Header Buffer ]-----------------------
0000 48 54 54 50 2f 31 2e 31 - 20 32 30 30 20 4f 4b 0d HTTP/1.1 200 OK.
0010 0a 44 61 74 65 3a 20 54 - 75 65 2c 20 31 37 20 46 .Date: Tue, 17 F
0020 65 62 20 32 30 30 39 20 - 30 30 3a 32 30 3a 33 31 eb 2009 00:20:31
0030 20 47 4d 54 0d 0a 53 65 - 72 76 65 72 3a 20 41 70 GMT..Server: Ap
0040 61 63 68 65 2f 32 2e 32 - 2e 33 20 28 44 65 62 69 ache/2.2.3 (Debi
0050 61 6e 29 0d 0a 4c 61 73 - 74 2d 4d 6f 64 69 66 69 an)..Last-Modifi
0060 65 64 3a 20 53 61 74 2c - 20 32 30 20 4e 6f 76 20 ed: Sat, 20 Nov
0070 32 30 30 34 20 32 30 3a - 31 36 3a 32 34 20 47 4d 2004 20:16:24 GM
0080 54 0d 0a 45 54 61 67 3a - 20 22 31 63 30 30 61 2d T..ETag: "1c00a-
0090 32 63 2d 34 63 32 33 62 - 36 30 30 22 0d 0a 41 63 2c-4c23b600"..Ac
00a0 63 65 70 74 2d 52 61 6e - 67 65 73 3a 20 62 79 74 cept-Ranges: byt
00b0 65 73 0d 0a 43 6f 6e 74 - 65 6e 74 2d 4c 65 6e 67 es..Content-Leng
00c0 74 68 3a 20 34 34 0d 0a - 43 6f 6e 6e 65 63 74 69 th: 44..Connecti
00d0 6f 6e 3a 20 63 6c 6f 73 - 65 0d 0a 43 6f 6e 74 65 on: close..Conte
00e0 6e 74 2d 54 79 70 65 3a - 20 74 65 78 74 2f 68 74 nt-Type: text/ht
00f0 6d 6c 3b 20 63 68 61 72 - 73 65 74 3d 55 54 46 2d ml; charset=UTF-
0100 38 0d 0a 0d 0a - 8....
-----------------------[ HTTP Header Ascii Buffer ]-----------------------
HTTP/1.1 200 OK
Date: Tue, 17 Feb 2009 00:20:31 GMT
Server: Apache/2.2.3 (Debian)
Last-Modified: Sat, 20 Nov 2004 20:16:24 GMT
ETag: "1c00a-2c-4c23b600"
Accept-Ranges: bytes
Content-Length: 44
Connection: close
Content-Type: text/html; charset=UTF-8
-----------------------[ HTML Buffer ]-----------------------
0000 3c 68 74 6d 6c 3e 3c 62 - 6f 64 79 3e 3c 68 31 3e <html><body><h1>
0010 49 74 20 77 6f 72 6b 73 - 21 3c 2f 68 31 3e 3c 2f It works!</h1></
0020 62 6f 64 79 3e 3c 2f 68 - 74 6d 6c 3e body></html>
-----------------------[ HTML MD5 resulting ]-----------------------
0000 c7 b4 69 0c 8c 46 62 5e - f0 f3 28 cd 7a 24 a0 a3 ..i..Fb^..(.z$..
-----------------------[ HTML MD5 final resulting ]-----------------------
MD5SUM = c7b4690c8c46625ef0f328cd7a24a0a3
Global response time for [/apache2-default/] =4243
|
그리고 redirect.sh 스크립트를 작성을 한다.
역활은 VIP 로 들어오는 패킷을 자신이 받아서 처리를 할 수 있게 하기 위해서 추가 한다.
양쪽 서버에 모두 추가한다.
|
dongho-vm111:~# cat /etc/keepalived/redirect.sh
#!/bin/bash
VIP="$2"
case "$1" in
add)
/sbin/iptables -A PREROUTING -t nat -d $VIP -p tcp -j REDIRECT
;;
del)
/sbin/iptables -D PREROUTING -t nat -d $VIP -p tcp -j REDIRECT
;;
*)
echo "Usage: $0 {add|del} ipaddress"
exit 1
esac
exit 0
|
redirect.sh 파일에 실행 권한을 준다.
양쪽 서버 모두 작업한다.
dongho-vm111:~# cd /etc/keepalived/
dongho-vm111:/etc/keepalived# chmod 700 redirect.sh
|
failover 될때 메일을 받기 위해 postfix 를 설치한다.
아래는 이미 설치가 되어 있다고 나오는 메시지이다.
양쪽 서버 모두 작업한다.
dongho-vm111:~# apt-get install postfix
Reading package lists... Done
Building dependency tree... Done
postfix is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
|
그리고 마지막으로 packet 을 포워딩을 해야 하므로 net.ipv4.ip_forward=1 을 sysctl.conf 에 추가하여 적용한다.
양쪽 서버에도 동일하게 작업한다.
|
dongho-vm111:~# grep -v ^# /etc/sysctl.conf
net.ipv4.ip_forward=1
dongho-vm111:~# sysctl -p
net.ipv4.ip_forward = 1
dongho-vm111:~# sysctl -a |grep net.ipv4.ip_forward
net.ipv4.ip_forward = 1
|
이제 keepalived 데몬을 시작 한다.
먼저 Master 쪽 부터 데몬을 시작하고 Backup에서 데몬을 시작 한다.
dongho-vm111:~# /etc/init.d/keepalived start
Starting keepalived: keepalived.
dongho-vm112:~# /etc/init.d/keepalived start
Starting keepalived: keepalived.
|
Master 쪽에 VIP 가 정상적으로 올라 온 것을 볼 수 있다.
dongho-vm111:~# ip addr show eth0
2: eth0: <BROADCAST,MULTICAST,UP,10000> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:16:3e:4a:8b:c9 brd ff:ff:ff:ff:ff:ff
inet 10.30.6.111/24 brd 10.30.6.255 scope global eth0
inet 10.30.6.110/24 scope global secondary eth0
inet6 fe80::216:3eff:fe4a:8bc9/64 scope link
valid_lft forever preferred_lft forever
|
그리고 Master 쪽에서 ipvsadm 명령을 통해 정상적으로 잘 부하 분산을 할 준비가 되었는지 살펴 본다.
dongho-vm111:~# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.30.6.110:80 rr
-> 10.30.6.112:80 Route 100 0 0
-> 10.30.6.111:80 Local 100 0 0
|
Backup 쪽 서버에는 VIP에서 들어오는 패킷을 받아서 처리를 하도록 iptables 에서 redirect 가 아래처럼 되어야 한다.
|
dongho-vm112:~# iptables -L -n -t nat -v
Chain PREROUTING (policy ACCEPT 145 packets, 9529 bytes)
pkts bytes target prot opt in out source destination
0 0 REDIRECT tcp -- * * 0.0.0.0/0 10.30.6.110
Chain POSTROUTING (policy ACCEPT 254 packets, 15191 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 254 packets, 15191 bytes)
pkts bytes target prot opt in out source destination
|
간단히 정상적으로 부하 분산을 하는지 apache 의 ab 툴을 사용하여 테스트를 해보자.
아래는 총 10만개의 request 를 보낼때 동시에 200개씩 보내라는 명령이다.
|
uheung@freeb:~$ ab -n 100000 -c 200 http://10.30.6.110/
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/
Benchmarking 10.30.6.110 (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Finished 100000 requests
Server Software: Apache/2.2.3
Server Hostname: 10.30.6.110
Server Port: 80
Document Path: /
Document Length: 295 bytes
Concurrency Level: 200
Time taken for tests: 56.834278 seconds
Complete requests: 100000
Failed requests: 0
Write errors: 0
Non-2xx responses: 100061
Total transferred: 51731537 bytes
HTML transferred: 29517995 bytes
Requests per second: 1759.50 [#/sec] (mean)
Time per request: 113.669 [ms] (mean)
Time per request: 0.568 [ms] (mean, across all concurrent requests)
Transfer rate: 888.88 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 1 51 201.3 36 7599
Processing: 4 61 24.0 60 323
Waiting: 3 44 20.4 45 305
Total: 8 112 203.4 102 7672
Percentage of the requests served within a certain time (ms)
50% 102
66% 107
75% 108
80% 109
90% 113
95% 195
98% 240
99% 267
100% 7672 (longest request)
|
Load Balancer Master 쪽에서 정상적으로 부하 분산을 한다면 round robin 방식이므로 대략 절반씩 분산을 하게 될 것이다.
즉 200개의 동시 접속이므로 100개 정도씩 분산을 하게 된다.
아래를 살펴보자.
여러번 명령을 날려 보더라도 대략 100에 가까운 수치에서 절반씩 나누어 주는 것을 볼 수 있다.
dongho-vm111:~# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.30.6.110:80 rr
-> 10.30.6.112:80 Route 100 99 5902
-> 10.30.6.111:80 Local 100 98 5902
dongho-vm111:~# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.30.6.110:80 rr
-> 10.30.6.112:80 Route 100 99 6924
-> 10.30.6.111:80 Local 100 100 6923
dongho-vm111:~# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.30.6.110:80 rr
-> 10.30.6.112:80 Route 100 93 9835
-> 10.30.6.111:80 Local 100 94 9834
dongho-vm111:~# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.30.6.110:80 rr
-> 10.30.6.112:80 Route 100 92 14024
-> 10.30.6.111:80 Local 100 92 14024
dongho-vm111:~# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.30.6.110:80 rr
-> 10.30.6.112:80 Route 100 94 14022
-> 10.30.6.111:80 Local 100 97 14019
dongho-vm111:~# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.30.6.110:80 rr
-> 10.30.6.112:80 Route 100 96 14020
-> 10.30.6.111:80 Local 100 96 14020
|
실제 웹 서버에서 접속이 있었는지 웹로그를 살펴 보자.
아래 Master 와 Backup 서버 모두 정상적으로 접속이 있었던 것을 볼 수 있다.
dongho-vm111:~# tail -200 /var/log/apache2/access.log |grep ApacheBench
10.30.103.33 - - [17/Feb/2009:01:31:03 +0000] "GET / HTTP/1.0" 302 295 "-" "ApacheBench/2.0.40-dev"
10.30.103.33 - - [17/Feb/2009:01:31:03 +0000] "GET / HTTP/1.0" 302 295 "-" "ApacheBench/2.0.40-dev"
10.30.103.33 - - [17/Feb/2009:01:31:03 +0000] "GET / HTTP/1.0" 302 295 "-" "ApacheBench/2.0.40-dev"
dongho-vm112:~# tail -200 /var/log/apache2/access.log |grep ApacheBench
10.30.103.33 - - [17/Feb/2009:01:31:03 +0000] "GET / HTTP/1.0" 302 295 "-" "ApacheBench/2.0.40-dev"
10.30.103.33 - - [17/Feb/2009:01:31:03 +0000] "GET / HTTP/1.0" 302 295 "-" "ApacheBench/2.0.40-dev"
10.30.103.33 - - [17/Feb/2009:01:31:03 +0000] "GET / HTTP/1.0" 302 295 "-" "ApacheBench/2.0.40-dev"
|
Health Check 가 정상적으로 잘 되는지 테스트해 본다.
간단히 apache2 데몬을 내리고 살펴 보자.
아래처럼 내리면 ipvsadm 에서 분산될때 서버가 빠지는 것을 볼 수 있다.
