nginx永中文档在线转换预览
yozodcs 人气:0永中文档在线转换预览双活实现方案
永中文档在线转换预览服务 是永中软件股份有限公司基于十多年核心技术积累、面向移动互联领域推出的 一款文档处理软件。永中采用自主可控核心技术，具备快速技术和服务响应能力，把文档原样输出为 HTML，图片等，即点即得、无需下载、保护文档隐私，快速高效，轻松实现文档在线安全阅读。
实现目标
- 通过nginx实现业务fcsserver负载均衡，后端可以动态扩展应用服务器
- nginx通过keepalived实现高可用，解决单点故障
- 通过keepalived实现nginx双活配置，解决nginx主备资源使用率50%问题
- 通过dns轮询解析域名到双活vip，达到负载均衡效果
- 后期nginx达到瓶颈应考虑lvs+keepalived+nginx架构，动态扩展nginx服务器
环境准备
- 服务器可以连通外网或者有内网yum源服务器，本次实验服务器可连通外网
- nginx01和nginx02服务器需要在同一网段的网络内
- 保证各服务器之前网络互通
- 保证服务器的防火墙和selinux关闭
- 必须对外提供域名访问，否则只能使用其中一个vip
- 内网部署DNS服务器，模拟DNS轮询解析
1、网络架构
2、实验服务器分布
| 主机 | ip | 操作系统 | 软件 | 端口 | vip |
|---|---|---|---|---|---|
| nginx01 | 192.168.56.101 | Centos7.6 | nginx keepalived | 80 | 192.168.56.200 |
| nginx02 | 192.168.56.106 | Centos7.6 | nginx keepalived | 80 | 192.168.56.201 |
| fcs01 | 192.168.56.101 | Centos7.6 | tomcat | 8080 | |
| fcs02 | 192.168.56.106 | Centos7.6 | tomcat | 8080 | |
| 共享存储 | 192.168.56.108 | Centos7.6 | nfs | ||
| 缓存 | 192.168.56.108 | Centos7.6 | redis | 6379 | |
| 内部DNS | 192.168.56.108 | Centos7.6 | bind | 53 |
3、redis
部署服务器：192.168.56.108
3.1 配置redis
/etc/redis.conf
# 监听地址 bind 0.0.0.0 # 认证密码 requirepass yozosoft
3.2 启动redis
systemctl enable redis --now && systemctl status redis
4、nfs
4.1 配置nfs
192.168.56.108（服务端）
# 创建存储目录 mkdir -p /opt/yozo/data # 修改权限 chown -R nfsnobody.nfsnobody /opt/yozo/data # 修改配置文件 vim /etc/exports /opt/yozo/data 192.168.56.0/24(rw,sync,all_squash)
192.168.56.101/192.168.56.106（客户端）
# 创建挂载点 mkdir -p /opt/yozo/data # 挂载nfs共享目录 mount -t nfs 192.168.56.108:/opt/yozo/data /opt/yozo/data
4.2 启动nfs
systemctl enable rpcbind nfs --now && systemctl status rpcbind nfs
5、fcsserver
部署服务器：
192.168.56.101（fcsserver01）
192.168.56.106（fcsserver02）
5.1 部署fcs
以tomcat为中间件，本次实验fcs安装目录
/opt/yozo/fcsserver/webapps/fcsserver，/opt/yozo/fcsserver为解压后的tomcat
部署项目包
mkdir opt/yozo/fcsserver/webapps/fcsserver -p unzip fcscloud.war -d opt/yozo/fcsserver/webapps/fcsserver
修改fcsserver配置文件
# /opt/yozo/fcsserver/webapps/fcsserver/WEB-INF/classes/config.properties inputDir=/opt/yozo/data/fcsdata/input # 必须指定挂载共享存储目录的 outputDir=/opt/yozo/data/fcsdata/output # 如果不提供域名，就只能配置其中一个vip；如果配置为域名，则缓存后，通过vip和fcsserver的ip将不能访问转换的缓存文件 viewDomain=http://www.fcsserver.com/fcsserver/
# /opt/yozo/fcsserver/webapps/fcsserver/WEB-INF/classes/application.yml
cache: #采用哪种缓存数据方式
type: redis #local,redis,mysql(local模式只适用于单机,集群部署不支持)
redis:
enable: true #redis开关(预览设置权限时需要使用redis,并发和异步waitting机制使用redis,必开)
redis:
database: 1 # Redis数据库索引（默认为0）
timeout: 10000 #设置客户端超时时间，单位是毫秒，默认为2000
password: yozosoft #密码
#单机版
host: 192.168.56.108
port: 6379
# /opt/yozo/fcsserver/webapps/fcsserver/WEB-INF/classes/ <property name="LOG_HOME">../logs</property>
5.2 配置fcs系统服务
配置fcsserver.service
# cat /usr/lib/systemd/system/fcsserver.service [Unit] Description=fcsserver Service. After=network.target [Service] Type=forking Environment="PATH=/opt/yozo/jdk-8u251-amd64/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin" WorkingDirectory=/opt/yozo/fcsserver/bin ExecStart=/opt/yozo/fcsserver/bin/startup.sh Restart=always PrivateTmp=true # 可以指定相关用户启动fcsserver # User=yozo # Group=yozo LimitNOFILE=65536 [Install] WantedBy=multi-user.target
如果以普通用户启动，需要将tomcat目录属主属组设置成相应账户
# 此次实验以yozo用户启动 chown -R yozo. /opt/yozo/fcsserver
5.3 启动fcs
systemctl enable fcsserver --now && systemctl status fcsserver
6、nginx
部署服务器：
192.168.56.101（nginx01）
192.168.56.106（nginx02）
6.1 配置nginx
/etc/nginx/conf.d/fcsserver.conf配置
server {
....
location ~ /fcsserver {
add_header Cache-Control private,no-store,max-age=0,no-cache,must-revalidate,post-check=0,pre-check=0;
proxy_redirect off;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded_Proto "http";
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_request_buffering off;
proxy_read_timeout 7200;
proxy_send_timeout 7200;
proxy_pass http://fcsserver;
}
....
}
upstream fcsserver {
server 192.168.56.101:8080 fail_timeout=60s;
server 192.168.56.106:8080 fail_timeout=60s;
keepalive 256;
}
6.2 启动nginx
systemctl enable nginx --now && systemctl status nginx
7、keepalived
部署服务器：
192.168.56.101（nginx01）
192.168.56.106（nginx02）
7.1 配置keepalived
7.1.1 nginx01配置
/etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
router_id proxy1
}
vrrp_script chk_nginx {
script "/etc/keepalived/check_nginx.sh"
interval 2
weight 20
fall 3
rise 2
}
vrrp_instance VI_1 {
state MASTER
interface enp0s3
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.56.200
}
track_script {
chk_nginx
}
}
vrrp_instance VI_2 {
state BACKUP
interface enp0s3
virtual_router_id 52
priority 90
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.56.201
}
track_script {
chk_nginx
}
}
7.1.2 nginx02配置
# 备份keepalived.conf mv /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.confbak
/etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
router_id proxy2
}
vrrp_script chk_nginx {
script "/etc/keepalived/check_nginx.sh"
interval 2
weight 20
fall 3
rise 2
}
vrrp_instance VI_1 {
state BACKUP
interface enp0s8
virtual_router_id 51
priority 90
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.56.200
}
track_script {
chk_nginx
}
}
vrrp_instance VI_2 {
state MASTER
interface enp0s8
virtual_router_id 52
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.56.201
}
track_script {
chk_nginx
}
}
双机/etc/keepalived/check_nginx.sh
#!/bin/bash
#时间变量，用于记录日志
d=`date --date today +%Y%m%d_%H:%M:%S`
#计算nginx进程数量
n=`ps -C nginx --no-heading|wc -l`
#如果进程为0，则启动nginx，并且再次检测nginx进程数量，
#如果还为0，说明nginx无法启动，此时需要关闭keepalived
if [ $n -eq "0" ]; then
/etc/init.d/nginx start
n2=`ps -C nginx --no-heading|wc -l`
if [ $n2 -eq "0" ]; then
echo "$d nginx down,keepalived will stop" >> /var/log/check_ng.log
systemctl stop keepalived
fi
fi
7.2 启动keepalived
systemctl enable keepalived --now && systemctl status keepalived
nginx01
nginx02
7.3 模拟故障
7.3.1 nginx01手动关闭nginx
nginx故障后，keepalived会自动启动nginx
7.3.2 nginx01手动关闭keepalived
nginx01的vip将会绑定到nginx02上面，nginx02将会出现2个vip，2个vip均可以访问fcsserver
8、DNS服务器部署
用于模拟公网DNS轮询解析，实际使用中是在域名提供商处配置指向映射外网ip
8.1 配置dns
/etc/named.conf
zone "fcsserver.com" IN {
type master;
file "fcsserver.com.zone";
};
zone "56.168.192.in-addr.arpa" IN {
type master;
file "192.168.56.zone";
};
/var/named/fcsserver.com.zone
$TTL 86400
@ IN SOA tom jerry ( ; tom & jerry 这两个参数本应是主机名和邮件地址，这里随便填写，没有问题
42 ; serial (d. adams)
3H ; refresh
15M ; retry
1W ; expiry
1D ) ; minimum
IN NS ns.fcsserver.com. ; notice : don't forget the dot in the end
IN MX 10 mail.fcsserver.com.
www IN A 192.168.56.200
www IN A 192.168.56.201
ns IN A 192.168.56.108
mail IN A 192.168.56.108
/var/named/192.168.56.zone
$TTL 86400
@ IN SOA ns.fcsserver.com. root (
1997022700 ; Serial
28800 ; Refresh
14400 ; Retry
3600000 ; Expire
86400 ) ; Minimum
IN NS ns.fcsserver.com.
200 IN PTR www.fcsserver.com.
201 IN PTR www.fcsserver.com.
108 IN PTR mail.fcsserver.com.
108 IN PTR ns.fcsserver.com.
修改权限
chown named. /var/named/ -R
8.2 启动dns服务
systemctl enable named --now && systemctl status named
9、验证
目标：本次实验有2个vip 192.168.56.200 和 192.168.56.201，需要验证www.fcsserver.com分别解析到2个vip上，并确认每个vip后端服务正常可用
9.1 客户端dns配置
测试期间，禁用其他网卡，只留虚拟机网卡
添加DNS
9.2 DNS缓存清理
9.3 域名访问测试
测试之前需要确认此次DNS解析是否指向所需测试的VIP，如果不是请刷新DNS缓存
需要测试www.fcsserver.com --> 192.168.56.200和www.fcsserver.com --> 192.168.56.201
文件转换测试
转换文件访问测试
注：fcsserver的配置文件中viewDomain配置为域名，故转换文件预览连接只能通过www.fcsserver.com可以正常访问，通过vip、nginx ip、fcsserver ip均不可以访问
加载全部内容