侧边栏壁纸
博主头像
WinJay博主等级

博观而约取 厚积而薄发

  • 累计撰写 306 篇文章
  • 累计创建 165 个标签
  • 累计收到 1 条评论
隐藏侧边栏

keepalived+nginx实现nginx的高可用

WinJay
2013-09-26 / 0 评论 / 0 点赞 / 59 阅读 / 12,271 字 / 正在检测是否收录...
温馨提示:
本文最后更新于 2021-04-08,若内容或图片失效,请留言反馈。部分素材来自网络,若不小心影响到您的利益,请联系我们删除。

keepalived+nginx实现nginx的高可用

===================================

nginx的高可用

nginx实现后端realserver的负载均衡

===================================

实验环境:OS:Centos 6.4(redhat 6.4)

yum源:

1
2
3
4
5
6
7
8
9
10
11
`[centos]`
`name=sohu-centos`
`baseurl=http:``//mirrors``.sohu.com``/centos/``$releasever``/os/``$basearch`
`gpgcheck=1`
`enable``=0`
`gpgkey=http:``//mirrors``.sohu.com``/centos/RPM-GPG-KEY-CentOS-6`
`[epel]`
`name=sohu-epel`
`baseurl=http:``//mirrors``.sohu.com``/fedora-epel/``$releasever/$basearch/`
`enable``=1`
`gpgcheck=0`
` `
**拓扑图: **

p154-001

拓扑图的规划:

IP地址 软件
Master 172.16.22.1(VIP172.16.22.100) keepalived+nginx
Backup 172.16.22.2(VIP172.16.22.100) keepalived+nginx
apache1 172.16.22.3 httpd
apache2 172.16.22.4 httpd
**此架构需考虑的问题**

1)、Master没挂,则Master占有vip且nginx运行在Master上

2)、Master挂了,则backup抢占vip且在backup上运行nginx服务

3)、如果master服务器上的nginx服务挂了,则vip资源转移到backup服务器上

4)、检测后端服务器的健康状态

Master和Backup两边都开启nginx服务,无论Master还是Backup,当其中的一个keepalived服务停止后,vip都会漂移到keepalived服务还在的节点上,

如果要想使nginx服务挂了,vip也漂移到另一个节点,则必须用脚本或者在配置文件里面用shell命令来控制。

首先必须明确后端服务器的健康状态检测keepalived在这种架构上是无法检测的,后端服务器的健康状态检测是有nginx来判断的,但是nginx 的检测机制有一定的缺陷,后端服务器某一个宕机之后,nginx还是会分发请求给它,在一定的时间内后端服务响应不了,nginx则会发给另外一个服务 器,然后当客户的请求来了,nginx会一段时间内不会把请求分发给已经宕机的服务器,但是过一段时间后,nginx还是会把分发请求发给宕机的服务器 上。

一、安装keepalived+nginx

Master:

1、安装keepalived和编译安装nginx

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
`[root@jie1 ~]``# yum -y install keepalived`
`[root@jie1 ~]``#tar xf nginx-1.4.2.tar.gz`
`[root@jie1 ~]``#yum -y groupinstall "Development tools" "Server  Platform Development"`
`[root@jie1 ~]``#yum -y install pcre-devel`
`[root@jie1 ~]``# cd nginx-1.4.2`
`[root@jie1 nginx-1.4.2]``# groupadd nginx`
`[root@jie1 nginx-1.4.2]``# useradd -r -g nginx nginx`
`[root@jie1 nginx-1.4.2]``#./configure \`
`--prefix=``/usr``\`
`--sbin-path=``/usr/sbin/nginx``\`
`--conf-path=``/etc/nginx/nginx``.conf \`
`--error-log-path=``/var/log/nginx/error``.log \`
`--http-log-path=``/var/log/nginx/access``.log \`
`--pid-path=``/var/run/nginx/nginx``.pid  \`
`--lock-path=``/var/lock/nginx``.lock \`
`--user=nginx \`
`--group=nginx \`
`--with-http_ssl_module \`
`--with-http_flv_module \`
`--with-http_stub_status_module \`
`--with-http_gzip_static_module \`
`--http-client-body-temp-path=``/var/tmp/nginx/client/``\`
`--http-proxy-temp-path=``/var/tmp/nginx/proxy/``\`
`--http-fastcgi-temp-path=``/var/tmp/nginx/fcgi/``\`
`--http-uwsgi-temp-path=``/var/tmp/nginx/uwsgi``\`
`--http-scgi-temp-path=``/var/tmp/nginx/scgi``\`
`--with-pcre`
`[root@jie1 nginx-1.4.2]``# make && make install`
**2、提供nginx的system V服务脚本文件**
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
`[root@jie1 nginx-1.4.2]``# vim /etc/rc.d/init.d/nginx`
`#!/bin/sh`
`#`
`# nginx - this script starts and stops the nginx daemon`
`#`
`# chkconfig:   - 85 15`
`# description:  Nginx is an HTTP(S) server, HTTP(S) reverse \`
`#               proxy and IMAP/POP3 proxy server`
`# processname: nginx`
`# config:      /etc/nginx/nginx.conf`
`# config:      /etc/sysconfig/nginx`
`# pidfile:     /var/run/nginx.pid`
`# Source function library.`
`. ``/etc/rc``.d``/init``.d``/functions`
`# Source networking configuration.`
`. ``/etc/sysconfig/network`
`# Check that networking is up.`
`[ ``"$NETWORKING"``= ``"no"``] && exit0`
`nginx=``"/usr/sbin/nginx"`
`prog=$(``basename``$nginx)`
`NGINX_CONF_FILE=``"/etc/nginx/nginx.conf"`
`[ -f ``/etc/sysconfig/nginx``] && . ``/etc/sysconfig/nginx`
`lockfile=``/var/lock/subsys/nginx`
`make_dirs() {`
`# make required directories`
`user=`nginx -V 2>&1 | ``grep``"configure arguments:"``| ``sed``'s/[^*]*--user=\([^ ]*\).*/\1/g'``-``
`options=`$nginx -V 2>&1 | ``grep``'configure arguments:'````
`foropt ``in``$options; ``do`
`if``[ ```echo``$opt | ``grep``'.*-temp-path'``` ]; ``then`
`value=```echo``$opt | ``cut``-d ``"="``-f 2``
`if``[ ! -d ``"$value"``]; ``then`
`# echo "creating" $value`
`mkdir``-p $value && ``chown``-R $user $value`
`fi`
`fi`
`done`
`}`
`start() {`
`[ -x $nginx ] || exit5`
`[ -f $NGINX_CONF_FILE ] || exit6`
`make_dirs`
`echo``-n $``"Starting $prog: "`
`daemon $nginx -c $NGINX_CONF_FILE`
`retval=$?`
`echo`
`[ $retval -eq0 ] && ``touch``$lockfile`
`return``$retval`
`}`
`stop() {`
`echo``-n $``"Stopping $prog: "`
`killproc $prog -QUIT`
`retval=$?`
`echo`
`[ $retval -eq0 ] && ``rm``-f $lockfile`
`return``$retval`
`}`
`restart() {`
`configtest || ``return``$?`
`stop`
`sleep1`
`start`
`}`
`reload() {`
`configtest || ``return``$?`
`echo``-n $``"Reloading $prog: "`
`killproc $nginx -HUP`
`RETVAL=$?`
`echo`
`}`
`force_reload() {`
`restart`
`}`
`configtest() {`
`$nginx -t -c $NGINX_CONF_FILE`
`}`
`rh_status() {`
`status $prog`
`}`
`rh_status_q() {`
`rh_status >``/dev/null2``>&1`
`}`
`case``"$1"``in`
`start)`
`rh_status_q && exit0`
`$1`
`;;`
`stop)`
`rh_status_q || exit0`
`$1`
`;;`
`restart|configtest)`
`$1`
`;;`
`reload)`
`rh_status_q || exit7`
`$1`
`;;`
`force-reload)`
`force_reload`
`;;`
`status)`
`rh_status`
`;;`
`condrestart|try-restart)`
`rh_status_q || exit0`
`;;`
`*)`
`echo``$``"Usage: $0 {start|stop|status|restart|condrestart|try-restart|reload|force-reload|configtest}"`
`exit2`
`esac`
`[root@jie1 nginx-1.4.2]``# chmod +x /etc/rc.d/init.d/nginx`
`[root@jie1 nginx-1.4.2]``# service nginx start`
`Starting nginx:                                            [  OK  ]`
`[root@jie1 nginx-1.4.2]``# scp -p /etc/rc.d/init.d/nginx  172.16.22.2:/etc/rc.d/init.d    #把nginx的服务脚本复制到backup上,-p是保持原有的权限`
**3、修改配置文件**
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
`[root@jie1 ~]``# cd /etc/keepalived/`
`[root@jie1 keepalived]``# vim keepalived.conf`
`global_defs {`
`   ``notification_email {`
`     ``root@localhost`
`   ``}`
`   ``notification_email_from admin@localhost`
`   ``smtp_server 127.0.0.1`
`   ``smtp_connect_timeout 30`
`   ``router_id LTT`
`}`
`vrrp_script chk_nginx {  ``#检测nginx服务是否在运行有很多方式,比如进程,用脚本检测等等`
`   ``script ``"killall -0 nginx"` `#用shell命令检查nginx服务是否存在`
`   ``interval 1  ``#时间间隔为1秒检测一次`
`   ``weight -2   ``#当nginx的服务不存在了,就把当前的权重-2`
`   ``fall 2      ``#测试失败的次数`
`   ``rise 1      ``#测试成功的次数`
`}`
`vrrp_instance IN_1 {`
`    ``state MASTER`
`    ``interface eth0`
`    ``virtual_router_id 22`
`    ``priority 100`
`    ``advert_int 1`
`    ``authentication {`
`        ``auth_type PASS`
`        ``auth_pass aaaa`
`    ``}`
`    ``virtual_ipaddress {`
`        ``172.16.22.100`
`    ``}`
`   ``track_script {`
`    ``chk_nginx   ``#引用上面的vrrp_script定义的脚本名称`
`}`
`}`
`[root@jie1 keepalived]``#scp keepalived.conf 172.16.22.2:/etc/keepalived #把配置文件copy到Backup服务器上,copy之前要保证Backup服务器上面已经安装了keepalived`
**4、开启keepalived和nginx的服务**
1
2
3
4
5
6
7
8
`[root@jie1 keepalived]``# service keepalived start`
`Starting keepalived:                                       [  OK  ]`
`[root@jie1 keepalived]``# chkconfig --add keepalived`
`[root@jie1 keepalived]``# chkconfig keepalived on`
`[root@jie1 ~]``# service nginx start`
`Starting nginx:                                            [  OK  ]`
`[root@jie1 ~]``# chkconfig --add nginx`
`[root@jie1 ~]``# chkconfig  nginx on`
**Backup:**

1、安装keepalived和编译安装nginx

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
`[root@jie2 ~]``# yum -y install keepalived`
`[root@jie2 ~]``#tar xf nginx-1.4.2.tar.gz`
`[root@jie2 ~]``#yum -y groupinstall "Development tools" "Server  Platform Development"`
`[root@jie2 ~]``#yum -y install pcre-devel`
`[root@jie2 ~]``# cd nginx-1.4.2`
`[root@jie2 nginx-1.4.2]``# groupadd nginx`
`[root@jie2 nginx-1.4.2]``# useradd -r -g nginx nginx`
`[root@jie2 nginx-1.4.2]``#./configure \`
`  ``--prefix=``/usr` `\`
`  ``--sbin-path=``/usr/sbin/nginx` `\`
`  ``--conf-path=``/etc/nginx/nginx``.conf \`
`  ``--error-log-path=``/var/log/nginx/error``.log \`
`  ``--http-log-path=``/var/log/nginx/access``.log \`
`  ``--pid-path=``/var/run/nginx/nginx``.pid  \`
`  ``--lock-path=``/var/lock/nginx``.lock \`
`  ``--user=nginx \`
`  ``--group=nginx \`
`  ``--with-http_ssl_module \`
`  ``--with-http_flv_module \`
`  ``--with-http_stub_status_module \`
`  ``--with-http_gzip_static_module \`
`  ``--http-client-body-temp-path=``/var/tmp/nginx/client/` `\`
`  ``--http-proxy-temp-path=``/var/tmp/nginx/proxy/` `\`
`  ``--http-fastcgi-temp-path=``/var/tmp/nginx/fcgi/` `\`
`  ``--http-uwsgi-temp-path=``/var/tmp/nginx/uwsgi` `\`
`  ``--http-scgi-temp-path=``/var/tmp/nginx/scgi` `\`
`  ``--with-pcre`
`[root@jie2 nginx-1.4.2]``# make && make install`
**2、之前 已经从Master复制了nginx的system V服务脚本文件,启动nginx****服务**
1
2
3
4
`[root@jie2 ~]``# service nginx start`
`Starting nginx:                                            [  OK  ]`
`[root@jie2 ~]``# chkconfig --add nginx`
`[root@jie2 ~]``# chkconfig  nginx on`
**3、修改配置文件**
1
2
3
4
`[root@jie2 ~]``# cd /etc/keepalived/`
`[root@jie2 keepalived]``# vim keepalived.conf   #此配置文件是从Master服务器上copy过来,只需小小改动`
`state BACKUP  ``#把这里原先的MASTER改成BACKUP`
`priority 99   ``#把这里原先的100改成99`
**4、开启服务**
1
2
3
4
`[root@jie2 keepalived]``# service keepalived start`
`Starting keepalived:                                       [  OK  ]`
`[root@jie2 keepalived]``# chkconfig --add keepalived`
`[root@jie2 keepalived]``# chkconfig keepalived on`
**apache1:**

1、安装(博主这里用rpm包安装,各位朋友可以用源码编译安装)

1
`[root@jie3 ~]# yum -y install httpd`
**2、建立测试网页文件**
1
2
3
`[root@jie3 ~]# cd /``var``/www/html/`
`[root@jie3 html]# cat index.html #建一个测试网页`
`

``this` `is` `apache1

`
**3、开启服务**
1
2
3
4
`[root@jie3 html]# service httpd start`
`Starting httpd:                 [  OK  ]`
`[root@jie3 html]# chkconfig --add httpd`
`[root@jie3 html]# chkconfig  httpd on`
**apache2:**

1、安装(博主这里用rpm包安装,各位朋友可以用源码编译安装)

1
`[root@jie4 ~]# yum -y install httpd`
**2、建立测试网页文件**
1
2
3
`[root@jie4 ~]# cd /``var``/www/html/`
`[root@jie4 html]# cat index.html #建一个测试网页`
`

``this` `is` `apache2

`
**3、开启服务**
1
2
3
4
`[root@jie4 html]# service httpd start`
`Starting httpd:                 [  OK  ]`
`[root@jie4 html]# chkconfig --add httpd`
`[root@jie4 html]# chkconfig  httpd on`
**此致所有安装已经完成。**
**二、nginx实现后端realserver的负载均衡,由于两边的配置文件必须保持一致,所以在Master配置完后直接copy到Backup上**

Master:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
`[root@jie1 ~]``# cd /etc/nginx/`
`[root@jie1 nginx]``# grep -v "#" nginx.conf | grep -v "^$"`
`worker_processes  1;`
`events {`
`    ``worker_connections  1024;`
`}`
`http {`
`    ``include       mime.types;`
`    ``default_type  application``/octet-stream``;`
`    ``sendfile        on;`
`    ``keepalive_timeout  65;`
`    ``upstream apacheweb {  ``#定义负载均衡的模块`
`        ``server 172.16.22.3:80 max_fails=3 fail_timeout=2s;`
`        ``server 172.16.22.4:80 max_fails=3 fail_timeout=2s;`
`     ``}`
`    ``server {`
`        ``listen       80;`
`        ``server_name  localhost;`
`        ``location / {`
`            ``root   html;`
`            ``index  index.html index.htm;`
`        ``}`
`        ``error_page   500 502 503 504  ``/50x``.html;`
`        ``location = ``/50x``.html {`
`            ``root   html;`
`        ``}`
`        ``location ~ \.(html|htm|gif|jpg|jpeg|bmp|png|ico|txt|js|css)$ {`
`            ``root        ``/var/www/html``;  ``#此处定义后端服务器网页存放路径`
`            ``proxy_pass   http:``//apacheweb``;`
`        ``}`
`    ``}`
`}`
`[root@jie1 nginx]``# scp nginx.conf 172.16.22.2:/etc/nginx`
**两边分别重启服务**
1
2
3
4
5
6
7
8
9
10
`[root@jie2 nginx]``# service nginx restart`
`nginx: the configuration ``file` `/etc/nginx/nginx``.conf syntax is ok`
`nginx: configuration ``file` `/etc/nginx/nginx``.conf ``test` `is successful`
`Stopping nginx:                                            [  OK  ]`
`Starting nginx:                                            [  OK  ]`
`[root@jie2 nginx]``# service nginx restart`
`nginx: the configuration ``file` `/etc/nginx/nginx``.conf syntax is ok`
`nginx: configuration ``file` `/etc/nginx/nginx``.conf ``test` `is successful`
`Stopping nginx:                                            [  OK  ]`
`Starting nginx:                                            [  OK  ]`
**验证负载均衡:**

p154-002

p154-003

本博客只是简单的用nginx做了方向代理和静态页面的负载均衡,keepalived+nginx实现高可用的nginx的动静分离,读写分离,后续会持续更新

0

评论区