keepalived+nginx实现nginx的高可用
===================================
nginx的高可用
nginx实现后端realserver的负载均衡
===================================
实验环境：OS：Centos 6.4(redhat 6.4)
yum源:
1 2 3 4 5 6 7 8 9 10 11 | `[centos]` `name=sohu-centos` `baseurl=http:``//mirrors``.sohu.com``/centos/``$releasever``/os/``$basearch` `gpgcheck=1` `enable``=0` `gpgkey=http:``//mirrors``.sohu.com``/centos/RPM-GPG-KEY-CentOS-6` `[epel]` `name=sohu-epel` `baseurl=http:``//mirrors``.sohu.com``/fedora-epel/``$releasever/$basearch/` `enable``=1` `gpgcheck=0` ` ` |
拓扑图的规划：
| IP地址 | 软件 | |
| Master | 172.16.22.1(VIP：172.16.22.100) | keepalived+nginx |
| Backup | 172.16.22.2(VIP：172.16.22.100) | keepalived+nginx |
| apache1 | 172.16.22.3 | httpd |
| apache2 | 172.16.22.4 | httpd |
1）、Master没挂，则Master占有vip且nginx运行在Master上
2）、Master挂了，则backup抢占vip且在backup上运行nginx服务
3）、如果master服务器上的nginx服务挂了，则vip资源转移到backup服务器上
4）、检测后端服务器的健康状态
Master和Backup两边都开启nginx服务，无论Master还是Backup，当其中的一个keepalived服务停止后，vip都会漂移到keepalived服务还在的节点上，
如果要想使nginx服务挂了，vip也漂移到另一个节点，则必须用脚本或者在配置文件里面用shell命令来控制。
首先必须明确后端服务器的健康状态检测keepalived在这种架构上是无法检测的，后端服务器的健康状态检测是有nginx来判断的，但是nginx 的检测机制有一定的缺陷，后端服务器某一个宕机之后，nginx还是会分发请求给它，在一定的时间内后端服务响应不了，nginx则会发给另外一个服务 器，然后当客户的请求来了，nginx会一段时间内不会把请求分发给已经宕机的服务器，但是过一段时间后，nginx还是会把分发请求发给宕机的服务器 上。
一、安装keepalived+nginx
Master：
1、安装keepalived和编译安装nginx
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 | `[root@jie1 ~]``# yum -y install keepalived` `[root@jie1 ~]``#tar xf nginx-1.4.2.tar.gz` `[root@jie1 ~]``#yum -y groupinstall "Development tools" "Server Platform Development"` `[root@jie1 ~]``#yum -y install pcre-devel` `[root@jie1 ~]``# cd nginx-1.4.2` `[root@jie1 nginx-1.4.2]``# groupadd nginx` `[root@jie1 nginx-1.4.2]``# useradd -r -g nginx nginx` `[root@jie1 nginx-1.4.2]``#./configure \` `--prefix=``/usr``\` `--sbin-path=``/usr/sbin/nginx``\` `--conf-path=``/etc/nginx/nginx``.conf \` `--error-log-path=``/var/log/nginx/error``.log \` `--http-log-path=``/var/log/nginx/access``.log \` `--pid-path=``/var/run/nginx/nginx``.pid \` `--lock-path=``/var/lock/nginx``.lock \` `--user=nginx \` `--group=nginx \` `--with-http_ssl_module \` `--with-http_flv_module \` `--with-http_stub_status_module \` `--with-http_gzip_static_module \` `--http-client-body-temp-path=``/var/tmp/nginx/client/``\` `--http-proxy-temp-path=``/var/tmp/nginx/proxy/``\` `--http-fastcgi-temp-path=``/var/tmp/nginx/fcgi/``\` `--http-uwsgi-temp-path=``/var/tmp/nginx/uwsgi``\` `--http-scgi-temp-path=``/var/tmp/nginx/scgi``\` `--with-pcre` `[root@jie1 nginx-1.4.2]``# make && make install` |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 | `[root@jie1 nginx-1.4.2]``# vim /etc/rc.d/init.d/nginx` `#!/bin/sh` `#` `# nginx - this script starts and stops the nginx daemon` `#` `# chkconfig: - 85 15` `# description: Nginx is an HTTP(S) server, HTTP(S) reverse \` `# proxy and IMAP/POP3 proxy server` `# processname: nginx` `# config: /etc/nginx/nginx.conf` `# config: /etc/sysconfig/nginx` `# pidfile: /var/run/nginx.pid` `# Source function library.` `. ``/etc/rc``.d``/init``.d``/functions` `# Source networking configuration.` `. ``/etc/sysconfig/network` `# Check that networking is up.` `[ ``"$NETWORKING"``= ``"no"``] && exit0` `nginx=``"/usr/sbin/nginx"` `prog=$(``basename``$nginx)` `NGINX_CONF_FILE=``"/etc/nginx/nginx.conf"` `[ -f ``/etc/sysconfig/nginx``] && . ``/etc/sysconfig/nginx` `lockfile=``/var/lock/subsys/nginx` `make_dirs() {` `# make required directories` `user=`nginx -V 2>&1 | ``grep``"configure arguments:"``| ``sed``'s/[^*]*--user=\([^ ]*\).*/\1/g'``-`` `options=`$nginx -V 2>&1 | ``grep``'configure arguments:'```` `foropt ``in``$options; ``do` `if``[ ```echo``$opt | ``grep``'.*-temp-path'``` ]; ``then` `value=```echo``$opt | ``cut``-d ``"="``-f 2`` `if``[ ! -d ``"$value"``]; ``then` `# echo "creating" $value` `mkdir``-p $value && ``chown``-R $user $value` `fi` `fi` `done` `}` `start() {` `[ -x $nginx ] || exit5` `[ -f $NGINX_CONF_FILE ] || exit6` `make_dirs` `echo``-n $``"Starting $prog: "` `daemon $nginx -c $NGINX_CONF_FILE` `retval=$?` `echo` `[ $retval -eq0 ] && ``touch``$lockfile` `return``$retval` `}` `stop() {` `echo``-n $``"Stopping $prog: "` `killproc $prog -QUIT` `retval=$?` `echo` `[ $retval -eq0 ] && ``rm``-f $lockfile` `return``$retval` `}` `restart() {` `configtest || ``return``$?` `stop` `sleep1` `start` `}` `reload() {` `configtest || ``return``$?` `echo``-n $``"Reloading $prog: "` `killproc $nginx -HUP` `RETVAL=$?` `echo` `}` `force_reload() {` `restart` `}` `configtest() {` `$nginx -t -c $NGINX_CONF_FILE` `}` `rh_status() {` `status $prog` `}` `rh_status_q() {` `rh_status >``/dev/null2``>&1` `}` `case``"$1"``in` `start)` `rh_status_q && exit0` `$1` `;;` `stop)` `rh_status_q || exit0` `$1` `;;` `restart|configtest)` `$1` `;;` `reload)` `rh_status_q || exit7` `$1` `;;` `force-reload)` `force_reload` `;;` `status)` `rh_status` `;;` `condrestart|try-restart)` `rh_status_q || exit0` `;;` `*)` `echo``$``"Usage: $0 {start|stop|status|restart|condrestart|try-restart|reload|force-reload|configtest}"` `exit2` `esac` `[root@jie1 nginx-1.4.2]``# chmod +x /etc/rc.d/init.d/nginx` `[root@jie1 nginx-1.4.2]``# service nginx start` `Starting nginx: [ OK ]` `[root@jie1 nginx-1.4.2]``# scp -p /etc/rc.d/init.d/nginx 172.16.22.2:/etc/rc.d/init.d #把nginx的服务脚本复制到backup上，-p是保持原有的权限` |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 | `[root@jie1 ~]``# cd /etc/keepalived/` `[root@jie1 keepalived]``# vim keepalived.conf` `global_defs {` ` ``notification_email {` ` ``root@localhost` ` ``}` ` ``notification_email_from admin@localhost` ` ``smtp_server 127.0.0.1` ` ``smtp_connect_timeout 30` ` ``router_id LTT` `}` `vrrp_script chk_nginx { ``#检测nginx服务是否在运行有很多方式，比如进程，用脚本检测等等` ` ``script ``"killall -0 nginx"` `#用shell命令检查nginx服务是否存在` ` ``interval 1 ``#时间间隔为1秒检测一次` ` ``weight -2 ``#当nginx的服务不存在了，就把当前的权重-2` ` ``fall 2 ``#测试失败的次数` ` ``rise 1 ``#测试成功的次数` `}` `vrrp_instance IN_1 {` ` ``state MASTER` ` ``interface eth0` ` ``virtual_router_id 22` ` ``priority 100` ` ``advert_int 1` ` ``authentication {` ` ``auth_type PASS` ` ``auth_pass aaaa` ` ``}` ` ``virtual_ipaddress {` ` ``172.16.22.100` ` ``}` ` ``track_script {` ` ``chk_nginx ``#引用上面的vrrp_script定义的脚本名称` `}` `}` `[root@jie1 keepalived]``#scp keepalived.conf 172.16.22.2:/etc/keepalived #把配置文件copy到Backup服务器上，copy之前要保证Backup服务器上面已经安装了keepalived` |
1 2 3 4 5 6 7 8 | `[root@jie1 keepalived]``# service keepalived start` `Starting keepalived: [ OK ]` `[root@jie1 keepalived]``# chkconfig --add keepalived` `[root@jie1 keepalived]``# chkconfig keepalived on` `[root@jie1 ~]``# service nginx start` `Starting nginx: [ OK ]` `[root@jie1 ~]``# chkconfig --add nginx` `[root@jie1 ~]``# chkconfig nginx on` |
1、安装keepalived和编译安装nginx
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 | `[root@jie2 ~]``# yum -y install keepalived` `[root@jie2 ~]``#tar xf nginx-1.4.2.tar.gz` `[root@jie2 ~]``#yum -y groupinstall "Development tools" "Server Platform Development"` `[root@jie2 ~]``#yum -y install pcre-devel` `[root@jie2 ~]``# cd nginx-1.4.2` `[root@jie2 nginx-1.4.2]``# groupadd nginx` `[root@jie2 nginx-1.4.2]``# useradd -r -g nginx nginx` `[root@jie2 nginx-1.4.2]``#./configure \` ` ``--prefix=``/usr` `\` ` ``--sbin-path=``/usr/sbin/nginx` `\` ` ``--conf-path=``/etc/nginx/nginx``.conf \` ` ``--error-log-path=``/var/log/nginx/error``.log \` ` ``--http-log-path=``/var/log/nginx/access``.log \` ` ``--pid-path=``/var/run/nginx/nginx``.pid \` ` ``--lock-path=``/var/lock/nginx``.lock \` ` ``--user=nginx \` ` ``--group=nginx \` ` ``--with-http_ssl_module \` ` ``--with-http_flv_module \` ` ``--with-http_stub_status_module \` ` ``--with-http_gzip_static_module \` ` ``--http-client-body-temp-path=``/var/tmp/nginx/client/` `\` ` ``--http-proxy-temp-path=``/var/tmp/nginx/proxy/` `\` ` ``--http-fastcgi-temp-path=``/var/tmp/nginx/fcgi/` `\` ` ``--http-uwsgi-temp-path=``/var/tmp/nginx/uwsgi` `\` ` ``--http-scgi-temp-path=``/var/tmp/nginx/scgi` `\` ` ``--with-pcre` `[root@jie2 nginx-1.4.2]``# make && make install` |
1 2 3 4 | `[root@jie2 ~]``# service nginx start` `Starting nginx: [ OK ]` `[root@jie2 ~]``# chkconfig --add nginx` `[root@jie2 ~]``# chkconfig nginx on` |
1 2 3 4 | `[root@jie2 ~]``# cd /etc/keepalived/` `[root@jie2 keepalived]``# vim keepalived.conf #此配置文件是从Master服务器上copy过来，只需小小改动` `state BACKUP ``#把这里原先的MASTER改成BACKUP` `priority 99 ``#把这里原先的100改成99` |
1 2 3 4 | `[root@jie2 keepalived]``# service keepalived start` `Starting keepalived: [ OK ]` `[root@jie2 keepalived]``# chkconfig --add keepalived` `[root@jie2 keepalived]``# chkconfig keepalived on` |
1、安装(博主这里用rpm包安装，各位朋友可以用源码编译安装)
1 | `[root@jie3 ~]# yum -y install httpd` |
1 2 3 | `[root@jie3 ~]# cd /``var``/www/html/` `[root@jie3 html]# cat index.html #建一个测试网页` ` ``this` `is` `apache1` |
1 2 3 4 | `[root@jie3 html]# service httpd start` `Starting httpd: [ OK ]` `[root@jie3 html]# chkconfig --add httpd` `[root@jie3 html]# chkconfig httpd on` |
1、安装(博主这里用rpm包安装，各位朋友可以用源码编译安装)
1 | `[root@jie4 ~]# yum -y install httpd` |
1 2 3 | `[root@jie4 ~]# cd /``var``/www/html/` `[root@jie4 html]# cat index.html #建一个测试网页` ` ``this` `is` `apache2` |
1 2 3 4 | `[root@jie4 html]# service httpd start` `Starting httpd: [ OK ]` `[root@jie4 html]# chkconfig --add httpd` `[root@jie4 html]# chkconfig httpd on` |
Master：
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 | `[root@jie1 ~]``# cd /etc/nginx/` `[root@jie1 nginx]``# grep -v "#" nginx.conf | grep -v "^$"` `worker_processes 1;` `events {` ` ``worker_connections 1024;` `}` `http {` ` ``include mime.types;` ` ``default_type application``/octet-stream``;` ` ``sendfile on;` ` ``keepalive_timeout 65;` ` ``upstream apacheweb { ``#定义负载均衡的模块` ` ``server 172.16.22.3:80 max_fails=3 fail_timeout=2s;` ` ``server 172.16.22.4:80 max_fails=3 fail_timeout=2s;` ` ``}` ` ``server {` ` ``listen 80;` ` ``server_name localhost;` ` ``location / {` ` ``root html;` ` ``index index.html index.htm;` ` ``}` ` ``error_page 500 502 503 504 ``/50x``.html;` ` ``location = ``/50x``.html {` ` ``root html;` ` ``}` ` ``location ~ \.(html|htm|gif|jpg|jpeg|bmp|png|ico|txt|js|css)$ {` ` ``root ``/var/www/html``; ``#此处定义后端服务器网页存放路径` ` ``proxy_pass http:``//apacheweb``;` ` ``}` ` ``}` `}` `[root@jie1 nginx]``# scp nginx.conf 172.16.22.2:/etc/nginx` |
1 2 3 4 5 6 7 8 9 10 | `[root@jie2 nginx]``# service nginx restart` `nginx: the configuration ``file` `/etc/nginx/nginx``.conf syntax is ok` `nginx: configuration ``file` `/etc/nginx/nginx``.conf ``test` `is successful` `Stopping nginx: [ OK ]` `Starting nginx: [ OK ]` `[root@jie2 nginx]``# service nginx restart` `nginx: the configuration ``file` `/etc/nginx/nginx``.conf syntax is ok` `nginx: configuration ``file` `/etc/nginx/nginx``.conf ``test` `is successful` `Stopping nginx: [ OK ]` `Starting nginx: [ OK ]` |
本博客只是简单的用nginx做了方向代理和静态页面的负载均衡，keepalived+nginx实现高可用的nginx的动静分离，读写分离，后续会持续更新
评论区