Nginx负载均衡
1.负载均衡类型
1.Nginx负载均衡按层划分
负载均衡按层划分应用场景: 四层负载均衡
负载均衡按层划分应用场景: 七层负载均衡, Nginx最常用Nginx负载均衡配置场景
Nginx实现负载均衡需要用到proxy_pass代理模块配置.
Nginx负载均衡是将客户端请求代理转发至一组upstream虚拟服务池
nginx 用yum源安装的，stream模块也用yum源安装
yum install -y nginx-mod-stream安装所有模块
yum install -y nginx-all-modules.
2.负载均衡反向代理优化
1.Nginx代理配置语法
Syntax: proxy_pass URL; Default: — Context: location, if in location, limit_except http://localhost:8000/uri/ http://192.168.56.11:8000/uri/ http://unix:/tmp/backend.socket:/uri/2.url跳转修改返回Location[不常用]参考URL
Syntax: proxy_redirect default; proxy_redirect off;proxy_redirect redirect replacement; Default: proxy_redirect default; Context: http, server, location3.添加发往后端服务器的请求头信息
Syntax: proxy_set_header field value; Default: proxy_set_header Host $proxy_host; proxy_set_header Connection close; Context: http, server, location # 用户请求的时候HOST的值是www.bgx.com, 那么代理服务会像后端传递请求的还是www.bgx.com proxy_set_header Host $http_host; # 将$remote_addr的值放进变量X-Real-IP中，$remote_addr的值为客户端的ip proxy_set_header X-Real-IP $remote_addr; # 客户端通过代理服务访问后端服务, 后端服务通过该变量会记录真实客户端地址 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;4.代理到后端的TCP连接、响应、返回等超时时间, 如图
//nginx代理与后端服务器连接超时时间(代理连接超时) Syntax: proxy_connect_timeout time; Default: proxy_connect_timeout 60s; Context: http, server, location //nginx代理等待后端服务器的响应时间 Syntax: proxy_read_timeout time; Default: proxy_read_timeout 60s; Context: http, server, location //后端服务器数据回传给nginx代理超时时间 Syntax: proxy_send_timeout time; Default: proxy_send_timeout 60s; Context: http, server, location5.proxy_buffer代理缓冲区, 如图
//nignx会把后端返回的内容先放到缓冲区当中，然后再返回给客户端,边收边传, 不是全部接收完再传给客户端 Syntax: proxy_buffering on | off; Default: proxy_buffering on; Context: http, server, location //设置nginx代理保存用户头信息的缓冲区大小 Syntax: proxy_buffer_size size; Default: proxy_buffer_size 4k|8k; Context: http, server, location //proxy_buffers 缓冲区 Syntax: proxy_buffers number size; Default: proxy_buffers 8 4k|8k; Context: http, server, location6.Proxy代理网站常用优化配置如下，将配置写入新文件，调用时使用include引用即可
[root@Nginx ~]# vim /etc/nginx/proxy_params proxy_set_header Host $http_host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_connect_timeout 30; proxy_send_timeout 60; proxy_read_timeout 60; proxy_buffering on; proxy_buffer_size 32k; proxy_buffers 4 128k;7.代理配置location时调用, 方便后续多个Location重复使用
location / { proxy_pass http://127.0.0.1:8080; include proxy_params; }.
3.负载均衡相关参数配置
轮询 # 按时间顺序逐一分配到不同的后端服务器(默认) weight # 加权轮询,weight值越大,分配到的访问几率越高 ip_hash # 每个请求按访问IP的hash结果分配,这样来自同一IP的固定访问一个后端服务器 url_hash # 按照访问URL的hash结果来分配请求,是每个URL定向到同一个后端服务器 least_conn # 最少链接数,那个机器链接数少就分发1.Nginx负载均衡[wrr]轮询具体配置
upstream load_pass { server 10.0.0.7:80; server 10.0.0.8:80; }2.Nginx负载均衡[weight]权重轮询具体配置
upstream load_pass { server 10.0.0.7:80 weight=5; server 10.0.0.8:80; }3.Nginx负载均衡ip_hash具体配置, 不能和weight一起使用。
//如果客户端都走相同代理, 会导致某一台服务器连接过多 upstream load_pass { ip_hash; server 10.0.0.7:80 weight=5; server 10.0.0.8:80; }后端Web服务器在前端Nginx负载均衡调度中的状态
down # 当前的server暂时不参与负载均衡 backup # 预留的备份服务器 max_fails # 允许请求失败的次数 fail_timeout # 经过max_fails失败后, 服务暂停时间 max_conns # 限制最大的接收连接数1.测试down状态, 测试该Server不参与负载均衡的调度
upstream load_pass { //不参与任何调度, 一般用于停机维护 server 10.0.0.7:80 down; }2.测试backup以及down状态
upstream load_pass { server 10.0.0.7:80 down; server 10.0.0.8:80 backup; server 10.0.0.9:80 max_fails=1 fail_timeout=10s; } location / { proxy_pass http://load_pass; include proxy_params; }3.测试max_fails失败次数和fail_timeout多少时间内失败多少次则标记down
upstream load_pass { server 10.0.0.7:80; server 10.0.0.8:80 max_fails=2 fail_timeout=10s; }4.测试max_conns最大TCP连接数
upstream load_pass { server 10.0.0.7:80; server 10.0.0.8:80 max_conns=1; }.
4.七层负载均衡
Nginx upstream虚拟配置语法
Syntax: upstream name { ... } Default: - Context: http //upstream例子 upstream backend { server backend1.example.com weight=5; server backend2.example.com:8080; server unix:/tmp/backend3; server backup1.example.com:8080 backup; } server { location / { proxy_pass http://backend; } }3.配置文件1
# 定义后端资源池 upstream proxy { server 172.16.1.7:80 max_fails=2 fail_timeout=10s; server 172.16.1.8:80 max_fails=2 fail_timeout=10s; } # https配置 server { listen 443 ssl; server_name opxdev.com; ssl_certificate /etc/nginx/ssl_key/server.crt; ssl_certificate_key /etc/nginx/ssl_key/server.key; ssl_session_timeout 5m; ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE:ECDH:AES:HIGH:!NULL:!aNULL:!MD5:!ADH:!RC4; ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_prefer_server_ciphers on; location /{ proxy_pass https://proxy; proxy_set_header Host $host:$server_port; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; client_max_body_size 100m; } error_log /var/log/proxy.log; error_page 404 /404.html; location = /404.html { root /usr/share/nginx/html; } error_page 500 502 503 504 /50x.html; location = /50x.html { root /usr/share/nginx/html; } location ~ /\.ht { deny all; } } # 用户http请求跳转至https server { listen 80; server_name opxdev.com; return 302 https://$server_name$request_uri; }4.配置文件2
upstream site { server 81.70.158.62:8180; } server { listen 80; server_name www.opsxdev.com; rewrite ^(.*)$ https://$host$1 permanent; } server { listen 80; server_name opsxdev.com; rewrite ^(.*)$ https://www.opsxdev.com$1 permanent; } server { listen 443 ssl; server_name opsxdev.com; ssl_certificate /etc/nginx/ssl_key/6098420_www.opsxdev.com.pem; ssl_certificate_key /etc/nginx/ssl_key/6098420_www.opsxdev.com.key; ssl_session_timeout 5m; ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE:ECDH:AES:HIGH:!NULL:!aNULL:!MD5:!ADH:!RC4; ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_prefer_server_ciphers on; rewrite ^/(?!theme|console|admin|api|init) /blogs/opsxdev$uri break; location /{ proxy_pass http://site/; proxy_set_header Host $host:$server_port; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; client_max_body_size 10m; } error_log /var/log/proxy.log; error_page 404 /404.html; location = /404.html { root /usr/share/nginx/html; } error_page 500 502 503 504 /50x.html; location = /50x.html { root /usr/share/nginx/html; } location ~ /\.ht { deny all; } }5.优化，调用时使用include引用
server { listen 80; server_name nginx.oldboy.com; location / { proxy_pass http://172.16.1.7:8080; include proxy_params; } } # vim /etc/nginx/proxy_params proxy_set_header Host $http_host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_connect_timeout 30; proxy_send_timeout 60; proxy_read_timeout 60; proxy_buffering on; proxy_buffer_size 32k; proxy_buffers 4 128k;方便后续多个Location重复使用
5.四层负载均衡
Nginx负载均衡TCP实践
- 配置Nginx4层负载均衡实现如下需求
- 1.通过访问负载均衡的5555端口，实际是后端的web01的22端口在提供服务。
- 2.通过访问负载均衡的6666端口，实际是后端的mysql的3306端口在提供服务。
1.需要用到 proxy_pass 代理模块配置
nginx 用 yum 源安装的，stream 模块也用 yum 源安装
yum install -y nginx-mod-stream安装所有模块
yum install -y nginx-all-modules2.Nginx四层负载均衡配置语法
stream { upstream backend { hash $remote_addr consistent; server backend1.example.com:12345 weight=5; server 127.0.0.1:12345 max_fails=3 fail_timeout=30s; server unix:/tmp/backend3; } server { listen 12345; proxy_connect_timeout 1s; proxy_timeout 3s; proxy_pass backend; } }3.线上四层代理示例
主配置文件，nginx.conf 开启 stream 模块
worker_processes 1; load_module lib64/nginx/modules/ngx_stream_module.so; events { worker_connections 1024; use epoll; } http { include mime.types; default_type application/octet-stream; map $http_upgrade $connection_upgrade { default upgrade; '' close; } sendfile on; keepalive_timeout 65; include conf.d/*.conf; server { listen 80; server_name localhost; location / { root /srv/www/htdocs/; index index.html index.htm; } error_page 500 502 503 504 /50x.html; location = /50x.html { root /srv/www/htdocs/; } } include vhosts.d/*.conf; } stream { log_format proxy '$remote_addr [$time_local] ' '$protocol $status $bytes_sent $bytes_received ' '$session_time "$upstream_addr" ' '"$upstream_bytes_sent" "$upstream_bytes_received" "$upstream_connect_time"'; access_log /var/log/nginx/tcp-access.log proxy ; open_log_file_cache off; include /etc/nginx/conf.d/*.stream; }子配置文件配置，xxxx.stream 配置反向代理
[admin@OPS /etc/nginx/conf.d] SUSE云端服务器 $cat hotwooolsession.stream # 添加socket转发的代理 upstream session_proxy { hash $remote_addr consistent; # 转发的目的地址和端口 server 192.168.49.2:32713 weight=5 max_fails=3 fail_timeout=60s; } # 提供转发的服务，即访问localhost:9001，会跳转至代理socket_proxy指定的转发地址 server { listen 20013; proxy_connect_timeout 3s; proxy_timeout 30s; proxy_pass session_proxy; } [admin@OPS /etc/nginx/conf.d] SUSE云端服务器 $cat hotwooolgateway.stream # 添加socket转发的代理 upstream gateway_proxy { hash $remote_addr consistent; # 转发的目的地址和端口 server 192.168.49.2:31742 weight=5 max_fails=3 fail_timeout=60s; } # 提供转发的服务，即访问localhost:9001，会跳转至代理socket_proxy指定的转发地址 server { listen 20021; proxy_connect_timeout 3s; proxy_timeout 30s; proxy_pass gateway_proxy; }4.Nginx四层负载均衡实战
[root@lb01 ~]# mkdir -p /etc/nginx/conf.c [root@lb01 ~]# vim /etc/nginx/nginx.conf # 在events层下面，http层上面配置include include /etc/nginx/conf.c/*.conf; # 编写四层代理配置 [root@lb01 ~]# cd /etc/nginx/conf.c/ [root@lb01 conf.c]# cat stream.conf stream { #1.定义虚拟资源池 upstream ssh { server 172.16.1.7:22; } upstream mysql { server 172.16.1.51:3306; } #2.调用虚拟资源池 server { listen 5555; proxy_connect_timeout 1s; proxy_timeout 300s; proxy_pass ssh; } server { listen 6666; proxy_connect_timeout 1s; proxy_timeout 300s; proxy_pass mysql; } } [root@lb01 conf.c]# systemctl restart nginx.
6.动静分离
动静分离, 通过中间件将动态请求和静态请求进行分离, 分离资源, 减少不必要的请求消耗, 减少请求延时。
好处: 动静分离后, 即使动态服务不可用, 但静态资源不会受到影响通过中间件将动态请求和静态请求分离
Nginx动静分离应用案例
0.环境准备
系统 服务 服务 地址 CentOS7.5 负载均衡 Nginx Proxy 10.0.0.5 CentOS7.5 静态资源 Nginx Static 10.0.0.7 CentOS7.5 动态资源 Tomcat Server 10.0.0.8 1.在10.0.0.7服务器上配置静态资源
[root@web01 conf.d]# cat ds_oldboy.conf server{ listen 80; server_name ds.oldboy.com; root /soft/code; index index.html; location ~* .*\.(png|jpg|gif)$ { root /soft/code/images; } } # 准备目录, 以及静态相关图片 [root@web01 ~]# mkdir /soft/code/images -p [root@web01 ~]# wget -O /soft/code/images/nginx.png http://nginx.org/nginx.png [root@web01 ~]# systemctl restart nginx2.在10.0.0.8服务器上配置动态资源
[root@web01 ~]# yum install -y tomcat [root@web01 ~]# systemctl start tomcat [root@web01 ~]# mkdir /usr/share/tomcat/webapps/ROOT [root@web01 ~]# vim /usr/share/tomcat/webapps/ROOT/java_test.jsp <%@ page language="java" import="java.util.*" pageEncoding="utf-8"%> <HTML> <HEAD> <TITLE>JSP Test Page</TITLE> </HEAD> <BODY> <% Random rand = new Random(); out.println("<h1>Random number:</h1>"); out.println(rand.nextInt(99)+100); %> </BODY> </HTML>3.在负载均衡10.0.0.5上配置调度, 实现访问jsp和png
root@lb01 conf.d]# cat ds_proxy.conf upstream static { server 10.0.0.7:80; } upstream java { server 10.0.0.8:8080; } server { listen 80; server_name ds.oldboy.com; location / { root /soft/code; index index.html; } location ~ .*\.(png|jpg|gif)$ { proxy_pass http://static; include proxy_params; } location ~ .*\.jsp$ { proxy_pass http://java; include proxy_params; } } [root@lb01 conf.d]# systemctl restart nginx4.通过负载测试访问静态资源
5.在负载均衡10.0.0.5上整合动态和静态资源的html文件[root@lb01 ~]# mkdir /soft/code -p [root@lb01 ~]# cat /soft/code/index.html <html lang="en"> <head> <meta charset="UTF-8" /> <title>测试ajax和跨域访问</title> <script src="http://libs.baidu.com/jquery/2.1.4/jquery.min.js"></script> </head> <script type="text/javascript"> $(document).ready(function(){ $.ajax({ type: "GET", url: "http://ds.oldboy.com/java_test.jsp", success: function(data) { $("#get_data").html(data) }, error: function() { alert("fail!!,请刷新再试!"); } }); }); </script> <body> <h1>测试动静分离</h1> <img src="http://ds.oldboy.com/nginx.png"> <div id="get_data"></div> </body> </html>6.测试动态和静态资源是否能正常加载在一个html文件中
7.当使用systemctl stop nginx停止Nginx后, 会发现静态内容无法访问, 动态内容依旧运行正常
8.当使用systemctl stop tomcat停止tomcat后, 静态内容依旧能正常访问, 动态内容将不会被请求到
7.Nginx通过来源设备拆分访问页面
2.根据不同的浏览器, 以及不同的手机, 访问的效果都将不一样。
//通过浏览器来分别连接不同的浏览器访问不同的效果。 http { ... upstream firefox { server 172.31.57.133:80; } upstream chrome { server 172.31.57.133:8080; } upstream iphone { server 172.31.57.134:8080; } upstream android { server 172.31.57.134:8081; } upstream default { server 172.31.57.134:80; } ... } //server根据判断来访问不同的页面 server { listen 80; server_name www.xuliangwei.com; #safari浏览器访问的效果 location / { if ($http_user_agent ~* "Safari"){ proxy_pass http://dynamic_pools; } #firefox浏览器访问效果 if ($http_user_agent ~* "Firefox"){ proxy_pass http://static_pools; } #chrome浏览器访问效果 if ($http_user_agent ~* "Chrome"){ proxy_pass http://chrome; } #iphone手机访问效果 if ($http_user_agent ~* "iphone"){ proxy_pass http://iphone; } #android手机访问效果 if ($http_user_agent ~* "android"){ proxy_pass http://and; } #其他浏览器访问默认规则 proxy_pass http://dynamic_pools; include proxy.conf; } } }3.根据访问不同目录, 代理不同的服务器
//默认动态，静态直接找设置的static，上传找upload upstream static_pools { server 10.0.0.9:80 weight=1; } upstream upload_pools { server 10.0.0.10:80 weight=1; } upstream default_pools { server 10.0.0.9:8080 weight=1; } server { listen 80; server_name www.xuliangwei.com; #url: http://www.xuliangwei.com/ location / { proxy_pass http://default_pools; include proxy.conf; } #url: http://www.xuliangwei.com/static/ location /static/ { proxy_pass http://static_pools; include proxy.conf; } #url: http://www.xuliangwei.com/upload/ location /upload/ { proxy_pass http://upload_pools; include proxy.conf; } } //方案2：以if语句实现。 if ($request_uri ~* "^/static/(.*)$") { proxy_pass http://static_pools/$1; } if ($request_uri ~* "^/upload/(.*)$") { proxy_pass http://upload_pools/$1; } location / { proxy_pass http://default_pools; include proxy.conf; }.
1
1