redis实现sentinel哨兵架构的方法

时间:2022-11-13 gy001 人气:0

1、redis哨兵（Sentinel）

1.1、redis集群介绍

前面文章讲的主从复制集群是无法实现master和slave角色的自动切换的，如果master节点出现现redis服务异常、主机断电、磁盘损坏等问题导致master无法使用，而redis主从复制无法实现自动的故障转移(将slave 自动提升为新master)，需要手动修改环境配置,才能切换到slave redis服务器，另外当单台Redis服务器性能无法满足业务写入需求的时候,也无法横向扩展Redis服务的并行写入性能。
需要解决以上的两个核心问题：

master和slave角色的无缝切换，让业务无感知从而不影响业务使用；
可横向动态扩展Redis服务器，从而实现多台服务器并行写入以实现更高并发的目的。

Redis集群实现的方式：

客户端分片: 由应用决定将不同的KEY发送到不同的Redis服务器
代理分片: 由代理决定将不同的KEY发送到不同的Redis服务器,代理程序如:codis,twemproxy等
Redis Cluster

1.2、redis哨兵（Sentinel）的工作原理

Sentinel可以管理多个redis主从集群

Sentinel 进程是用于监控redis集群中Master主服务器工作的状态，在Master主服务器发生故障的时候，可以实现Master和Slave服务器的切换，保证系统的高可用，此功能在redis2.6+的版本已引用，Redis的哨兵模式到了2.8版本之后就稳定了下来。一般在生产环境也建议使用Redis的2.8版本的以后版本

哨兵(Sentinel) 是一个分布式系统，可以在一个架构中运行多个哨兵(sentinel) 进程，这些进程使用流言协议(gossip protocols)来接收关于Master主服务器是否下线的信息，并使用投票协议(Agreement Protocols)来决定是否执行自动故障迁移,以及选择哪个Slave作为新的Master

每个哨兵(Sentinel)进程会向其它哨兵(Sentinel)、Master、Slave定时发送消息，以确认对方是否”活”着，如果发现对方在指定配置时间(此项可配置)内未得到回应，则暂时认为对方已离线，也就是所谓的”主观认为宕机” (主观:是每个成员都具有的独自的而且可能相同也可能不同的意识)，英文名称：Subjective Down，简称SDOWN

有主观宕机，对应的有客观宕机。当“哨兵群”中的多数Sentinel进程在对Master主服务器做出SDOWN 的判断，并且通过 SENTINEL is-master-down-by-addr 命令互相交流之后，得出的Master Server下线判断，这种方式就是“客观宕机”(客观:是不依赖于某种意识而已经实际存在的一切事物)，英文名称是：Objectively Down，简称 ODOWN
通过一定的vote算法，从剩下的slave从服务器节点中，选一台提升为Master服务器节点，然后自动修改相关配置，并开启故障转移（failover）

Sentinel 机制可以解决master和slave角色的自动切换问题，但单个 Master 的性能瓶颈问题无法解决,类似于MySQL中的MHA功能

Redis Sentinel中的Sentinel节点个数应该为大于等于3且最好为奇数

客户端初始化时连接的是Sentinel节点集合，不再是具体的Redis节点，但Sentinel只是配置中心不是代理。

Redis Sentinel节点与普通redis没有区别,要实现读写分离依赖于客户端程序

redis 3.0之前版本中,生产环境一般使用哨兵模式,3.0后推出redis cluster功能,可以支持更大规模的生产环境sentinel中的三个定时任务：

1.2.1sentinel中的三个定时任务：

每10秒每个sentinel对master和slave执行info，发现slave节点，确认主从关系。
每2秒每个sentinel通过master节点的channel交换信息(pub/sub)通过sentinel__:hello频道交互，交互对节点的“看法”和自身信息
每1秒每个sentinel对其他sentinel和redis执行ping

1.3、实现哨兵

环境准备：

准备三台主机搭建主从集群，再在每个机器上搭建sentinel

IP	redis版本	主机名
10.0.0.101	redis-5.0.14	node1
10.0.0.102	redis-5.0.14	node2
10.0.0.103	redis-5.0.14	node3

1.3.1、实现哨兵需要先实现一下主从复制的架构

哨兵的前提是已经实现了一个redis的主从复制的运行环境，从而实现一个一主两从基于哨兵的高可用redis架构
注意: master的配置文件中masterauth和slave都必须相同

主从复制上篇文章已经实现了，这里继续用上一篇的环境

#主节点
[root@master etc]#redis-cli -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:2
slave0:ip=10.0.0.102,port=6379,state=online,offset=14,lag=1
slave1:ip=10.0.0.103,port=6379,state=online,offset=14,lag=0
master_replid:e9310fdb8dd9f91d265d4c9a8621a6879e0262ff
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:14
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:14
127.0.0.1:6379>
#slave1节点
[root@slave1 ~]#redis-cli -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:6379> info replication
# Replication
role:slave
master_host:10.0.0.101
master_port:6379
master_link_status:up
master_last_io_seconds_ago:8
master_sync_in_progress:0
slave_repl_offset:224
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:e9310fdb8dd9f91d265d4c9a8621a6879e0262ff
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:224
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:224
#slave2节点
[root@slave2 ~]#redis-cli -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:6379> info replication
# Replication
role:slave
master_host:10.0.0.101
master_port:6379
master_link_status:up
master_last_io_seconds_ago:7
master_sync_in_progress:0
slave_repl_offset:2016
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:ee9d24063f9d79d698e875634517923d6a9c2a10
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:2016
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:2016

1.3.2、编辑哨兵的配置文件

Sentinel实际上是一个特殊的redis服务器,有些redis指令支持,但很多指令并不支持.默认监听在26379/tcp端口.所有redis节点使用相同的以下的配置文件.

#这里因为我是源码编译安装的，所以要人源码的路径下找到sentinel.conf将其复制到redis安装的目录下
[root@master]# cd /usr/local/src/redis-5.0.14[root@master /usr/local/src/redis-5.0.14# ls
00-RELEASENOTES  CONTRIBUTING  INSTALL    README.md   runtest-cluster    sentinel.conf  TLS.md
BUGS             COPYING       Makefile   redis.conf  runtest-moduleapi  src            utils
CONDUCT          deps          MANIFESTO  runtest     runtest-sentinel   tests
 [root@master etc]#sed -i "/^#/d" redis-sentinel.conf
 [root@master etc]#sed -i "/^$/d" redis-sentinel.conf
 [root@master etc]#vim redis-sentinel.con

bind 0.0.0.0
port 26379
daemonize no
pidfile /apps/redis/run/redis-sentinel.pid
logfile /apps/redis/log/sentinel.log
dir /tmp
sentinel monitor mymaster 10.0.0.100 6379 2    #mymaster是集群的名称，此行指定当前mymaster集群中master服务器的地址和端口
#2为法定人数限制(quorum)，即有几个sentinel认为master down了就进行故障转移，一般此值是所有sentinel节点(一般总数是>=3的 奇数,如:3,5,7等)的一半以上的整数值，比如，总数是3，即3/2=1.5，取整为2,是master的ODOWN客观下线的依据
sentinel auth-pass mymaster wm521314    #mymaster集群中master的密码，注意此行要在上面行的下面
sentinel down-after-milliseconds mymaster 3000    #(SDOWN)判断mymaster集群中所有节点的主观下线的时间，单位：毫秒，建议3000
sentinel parallel-syncs mymaster 1    #发生故障转移后，可以同时向新master同步数据的slave的数量，数字越小总同步时间越长，但可以减轻新master的负载压力
sentinel failover-timeout mymaster 180000    #所有slaves指向新的master所需的超时时间，单位：毫秒
sentinel deny-scripts-reconfig yes    #禁止修改脚本

[root@master etc]# scp /apps/redis/etc/redis-sentinel.conf 10.0.0.102:/apps/redis/etc/[root@master etc]# scp /apps/redis/etc/redis-sentinel.conf 10.0.0.103:/apps/redis/etc/

三个哨兵服务器的配置都如下：

[root@slave2 etc]#cat redis-sentinel.conf
port 26379
daemonize no
pidfile /apps/redis/run/redis-sentinel.pid
logfile /apps/redis/log/sentinel.log
dir /tmp
sentinel monitor mymaster 10.0.0.101 6379 2
sentinel down-after-milliseconds mymaster 30000
sentinel parallel-syncs mymaster 1
sentinel failover-timeout mymaster 180000
sentinel deny-scripts-reconfig yes

1.3.2、启动哨兵

#我这里是编译安装，所以在所有节点生成新的service文件
[root@master ~]#vim /lib/systemd/system/redis-sentinel.service
[Unit]
Description=Redis Sentinel
After=network.target

[Service]
ExecStart=/apps/redis/bin/redis-sentinel /apps/redis/etc/redis-sentinel.conf --supervised systemd
ExecStop=/bin/kill -s QUIT $MAINPID
User=redis
Group=redis
RuntimeDirectory=redis
RuntimeDirectoryMode=0755

[Install]
WantedBy=multi-user.target
[root@master ~]#scp /lib/systemd/system/redis-sentinel.service 10.0.0.102:/lib/systemd/system/
[root@master ~]#scp /lib/systemd/system/redis-sentinel.service 10.0.0.103:/lib/systemd/system/
#在所有节点修改一下文件的所有者所属组，不然无法启动
chown -R redis.redis /apps/redis/
#在所有节点启动哨兵
  [root@master ~]#systemctl daemon-reload
  [root@master ~]#systemctl enable --now redis-sentinel.service

1.3.3、验证哨兵端口

[root@master ~]#ss -nlt
State     Recv-Q     Send-Q         Local Address:Port          Peer Address:Port    Process
LISTEN    0          511                  0.0.0.0:26379              0.0.0.0:*                 #已经启动，另外两个节点同样查看
LISTEN    0          511                  0.0.0.0:6379               0.0.0.0:*
LISTEN    0          1024           127.0.0.53%lo:53                 0.0.0.0:*
LISTEN    0          128                  0.0.0.0:22                 0.0.0.0:*
LISTEN    0          128                127.0.0.1:6010               0.0.0.0:*
LISTEN    0          511                     [::]:26379                 [::]:*
LISTEN    0          128                     [::]:22                    [::]:*
LISTEN    0          128                    [::1]:6010                  [::]:*

1.3.4、查看哨兵日志

[root@master ~]#tail -f /apps/redis/log/sentinel.log #查看主节点日志
37184:X 04 Nov 2022 23:32:07.023 # Configuration loaded
37184:X 04 Nov 2022 23:32:07.023 # systemd supervision requested, but NOTIFY_SOCKET not found
37184:X 04 Nov 2022 23:32:07.024 * Increased maximum number of open files to 10032 (it was originally set to 1024).
37184:X 04 Nov 2022 23:32:07.024 * Running mode=sentinel, port=26379.
37184:X 04 Nov 2022 23:32:07.032 # Sentinel ID is 2f9b5e2cb38fc5b3db19417b9626f0ebcc3a7120
37184:X 04 Nov 2022 23:32:07.032 # +monitor master mymaster 10.0.0.101 6379 quorum 2
37184:X 04 Nov 2022 23:32:07.033 * +slave slave 10.0.0.102:6379 10.0.0.102 6379 @ mymaster 10.0.0.101 6379
37184:X 04 Nov 2022 23:32:07.036 * +slave slave 10.0.0.103:6379 10.0.0.103 6379 @ mymaster 10.0.0.101 6379
37184:X 04 Nov 2022 23:32:07.595 * +sentinel sentinel f505346a3640b4942963b26829e5d99a1fa6691e 10.0.0.102 26379 @ mymaster 10.0.0.101 6379
37184:X 04 Nov 2022 23:32:12.779 * +sentinel sentinel b6aebc495fe5d6e81e57ffd5d4f5bef795b3819a 10.0.0.103 26379 @ mymaster 10.0.0.101 6379

#查看从节点日志

[root@slave1 ~]#tail -f /apps/redis/log/sentinel.log
6376:X 04 Nov 2022 23:31:59.446 # Configuration loaded
6376:X 04 Nov 2022 23:31:59.446 # systemd supervision requested, but NOTIFY_SOCKET not found
6376:X 04 Nov 2022 23:31:59.446 * Increased maximum number of open files to 10032 (it was originally set to 1024).
6376:X 04 Nov 2022 23:31:59.447 * Running mode=sentinel, port=26379.
6376:X 04 Nov 2022 23:31:59.455 # Sentinel ID is f505346a3640b4942963b26829e5d99a1fa6691e
6376:X 04 Nov 2022 23:31:59.455 # +monitor master mymaster 10.0.0.101 6379 quorum 2
6376:X 04 Nov 2022 23:31:59.457 * +slave slave 10.0.0.102:6379 10.0.0.102 6379 @ mymaster 10.0.0.101 6379
6376:X 04 Nov 2022 23:31:59.458 * +slave slave 10.0.0.103:6379 10.0.0.103 6379 @ mymaster 10.0.0.101 6379
6376:X 04 Nov 2022 23:32:08.989 * +sentinel sentinel 2f9b5e2cb38fc5b3db19417b9626f0ebcc3a7120 10.0.0.101 26379 @ mymaster 10.0.0.101 6379
6376:X 04 Nov 2022 23:32:12.716 * +sentinel sentinel b6aebc495fe5d6e81e57ffd5d4f5bef795b3819a 10.0.0.103 26379 @ mymaster 10.0.0.101 6379

#查看从节点日志

[root@slave1 ~]#tail -f /apps/redis/log/sentinel.log
6376:X 04 Nov 2022 23:31:59.446 # Configuration loaded
6376:X 04 Nov 2022 23:31:59.446 # systemd supervision requested, but NOTIFY_SOCKET not found
6376:X 04 Nov 2022 23:31:59.446 * Increased maximum number of open files to 10032 (it was originally set to 1024).
6376:X 04 Nov 2022 23:31:59.447 * Running mode=sentinel, port=26379.
6376:X 04 Nov 2022 23:31:59.455 # Sentinel ID is f505346a3640b4942963b26829e5d99a1fa6691e
6376:X 04 Nov 2022 23:31:59.455 # +monitor master mymaster 10.0.0.101 6379 quorum 2
6376:X 04 Nov 2022 23:31:59.457 * +slave slave 10.0.0.102:6379 10.0.0.102 6379 @ mymaster 10.0.0.101 6379
6376:X 04 Nov 2022 23:31:59.458 * +slave slave 10.0.0.103:6379 10.0.0.103 6379 @ mymaster 10.0.0.101 6379
6376:X 04 Nov 2022 23:32:08.989 * +sentinel sentinel 2f9b5e2cb38fc5b3db19417b9626f0ebcc3a7120 10.0.0.101 26379 @ mymaster 10.0.0.101 6379
6376:X 04 Nov 2022 23:32:12.716 * +sentinel sentinel b6aebc495fe5d6e81e57ffd5d4f5bef795b3819a 10.0.0.103 26379 @ mymaster 10.0.0.101 6379

1.3.5、当前sentinel状态

[root@slave1 ~]#redis-cli -p 26379
127.0.0.1:26379> INFO sentinel
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=mymaster,status=ok,address=10.0.0.101:6379,slaves=2,sentinels=3

1.3.6、验证数据是否同步

[root@master ~]#redis-cli -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:6379> set k1 v1
OK
127.0.0.1:6379> set k2 v2
OK
 
[root@slave1 ~]#redis-cli -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:6379> get k1
"v1"

[root@slave2 ~]#redis-cli -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:6379> get k1
"v1"

1.3.7、制造Redis Master节点故障故障转移并查看其过程

#停掉master节点的redis服务来模拟节点故障
[root@master ~]#killall redis-server
#这时候查看日志发现主节点已经变成了10.0.0.102机器
[root@master ~]#tail -f /apps/redis/log/sentinel.log
800:X 06 Nov 2022 14:36:38.502 # -sdown sentinel b6aebc495fe5d6e81e57ffd5d4f5bef795b3819a 10.0.0.103 26379 @ mymaster 10.0.0.101 6379
800:X 06 Nov 2022 14:36:38.604 # -sdown slave 10.0.0.103:6379 10.0.0.103 6379 @ mymaster 10.0.0.101 6379
800:X 06 Nov 2022 15:21:26.435 # +sdown master mymaster 10.0.0.101 6379
800:X 06 Nov 2022 15:21:26.500 # +new-epoch 1
800:X 06 Nov 2022 15:21:26.501 # +vote-for-leader f505346a3640b4942963b26829e5d99a1fa6691e 1
800:X 06 Nov 2022 15:21:27.019 # +config-update-from sentinel f505346a3640b4942963b26829e5d99a1fa6691e 10.0.0.102 26379 @ mymaster 10.0.0.101 6379
800:X 06 Nov 2022 15:21:27.019 # +switch-master mymaster 10.0.0.101 6379 10.0.0.102 6379
800:X 06 Nov 2022 15:21:27.019 * +slave slave 10.0.0.103:6379 10.0.0.103 6379 @ mymaster 10.0.0.102 6379
800:X 06 Nov 2022 15:21:27.019 * +slave slave 10.0.0.101:6379 10.0.0.101 6379 @ mymaster 10.0.0.102 6379
800:X 06 Nov 2022 15:21:30.032 # +sdown slave 10.0.0.101:6379 10.0.0.101 6379 @ mymaster 10.0.0.102 6379
#查看各节点sentinel信息
[root@master ~]#redis-cli -p 26379
127.0.0.1:26379> INFO sentinel
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=mymaster,status=ok,address=10.0.0.102:6379,slaves=2,sentinels=3

#故障转移后的sentinel配置文件会被自动修改

[root@slave1 ~]#grep "^[a-Z]" /apps/redis/etc/redis-sentinel.conf
bind 0.0.0.0
port 26379
daemonize no
pidfile "/apps/redis/run/redis-sentinel.pid"
logfile "/apps/redis/log/sentinel.log"
dir "/tmp"
sentinel myid f505346a3640b4942963b26829e5d99a1fa6691e
sentinel deny-scripts-reconfig yes
sentinel monitor mymaster 10.0.0.102 6379 2         #这里发现主节点已经变成10.0.0.102　　
sentinel down-after-milliseconds mymaster 3000
sentinel auth-pass mymaster 123456
sentinel config-epoch mymaster 1
protected-mode no
supervised systemd
sentinel leader-epoch mymaster 1
sentinel known-replica mymaster 10.0.0.101 6379
sentinel known-replica mymaster 10.0.0.103 6379
sentinel known-sentinel mymaster 10.0.0.103 26379 b6aebc495fe5d6e81e57ffd5d4f5bef795b3819a
sentinel known-sentinel mymaster 10.0.0.101 26379 2f9b5e2cb38fc5b3db19417b9626f0ebcc3a7120
sentinel current-epoch 1

#进入redis查看现在info replication信息

[root@slave1 ~]#redis-cli -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=10.0.0.103,port=6379,state=online,offset=905513,lag=0
master_replid:2824cc99085826d751e848cd7b04c9f4abc398cb
master_replid2:488fa34f2c093be4886741a38323dd1a0ecf9e03
master_repl_offset:905648
second_repl_offset:537903
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:905648

[root@slave2 ~]#redis-cli -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe .
127.0.0.1:6379> INFO replication
# Replication
role:slave
master_host:10.0.0.102
master_port:6379
master_link_status:up
master_last_io_seconds_ago:0
master_sync_in_progress:0
slave_repl_offset:901300
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:2824cc99085826d751e848cd7b04c9f4abc398cb
master_replid2:488fa34f2c093be4886741a38323dd1a0ecf9e03
master_repl_offset:901300
second_repl_offset:537903
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:564
repl_backlog_histlen:900737

1.3.8、查看选举新主的过程日志监测

[root@slave2 ~]#tail -f /apps/redis/log/sentinel.log
798:X 06 Nov 2022 15:21:26.509 # +sdown master mymaster 10.0.0.101 6379
798:X 06 Nov 2022 15:21:26.534 # +new-epoch 1
798:X 06 Nov 2022 15:21:26.535 # +vote-for-leader f505346a3640b4942963b26829e5d99a1fa6691e 1
798:X 06 Nov 2022 15:21:26.609 # +odown master mymaster 10.0.0.101 6379 #quorum 3/2
798:X 06 Nov 2022 15:21:26.609 # Next failover delay: I will not start a failover before Sun Nov  6 15:27:27 2022
798:X 06 Nov 2022 15:21:27.053 # +config-update-from sentinel f505346a3640b4942963b26829e5d99a1fa6691e 10.0.0.102 26379 @ mymaster 10.0.0.101 6379
798:X 06 Nov 2022 15:21:27.053 # +switch-master mymaster 10.0.0.101 6379 10.0.0.102 6379
798:X 06 Nov 2022 15:21:27.053 * +slave slave 10.0.0.103:6379 10.0.0.103 6379 @ mymaster 10.0.0.102 6379
798:X 06 Nov 2022 15:21:27.053 * +slave slave 10.0.0.101:6379 10.0.0.101 6379 @ mymaster 10.0.0.102 6379
798:X 06 Nov 2022 15:21:30.094 # +sdown slave 10.0.0.101:6379 10.0.0.101 6379 @ mymaster 10.0.0.102 6379

1.3.9、恢复故障的原master重新加入redis集群

#重新其他原master的redis服务，会发现会自动指向新的master
[root@master ~]#systemctl restart redis
#在原master上观察状态
[root@master ~]#redis-cli  -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:6379> info replication
# Replication
role:slave
master_host:10.0.0.102
master_port:6379
master_link_status:up
master_last_io_seconds_ago:1
master_sync_in_progress:0
slave_repl_offset:942522
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:2824cc99085826d751e848cd7b04c9f4abc398cb
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:942522
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:936383
repl_backlog_histlen:6140

[root@master ~]#redis-cli -p 26379
127.0.0.1:26379> INFO sentinel
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=mymaster,status=ok,address=10.0.0.102:6379,slaves=2,sentinels=3

#再回到新master节点观察状态
[root@slave1 ~]#redis-cli -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:2
slave0:ip=10.0.0.103,port=6379,state=online,offset=976915,lag=0
slave1:ip=10.0.0.101,port=6379,state=online,offset=977185,lag=0
master_replid:2824cc99085826d751e848cd7b04c9f4abc398cb
master_replid2:488fa34f2c093be4886741a38323dd1a0ecf9e03
master_repl_offset:977185
second_repl_offset:537903
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:977185

#在新主查看一下日志
[root@slave1 ~]#tail -f /apps/redis/log/sentinel.log
786:X 06 Nov 2022 15:21:27.030 * +slave-reconf-sent slave 10.0.0.103:6379 10.0.0.103 6379 @ mymaster 10.0.0.101 6379
786:X 06 Nov 2022 15:21:27.580 * +slave-reconf-inprog slave 10.0.0.103:6379 10.0.0.103 6379 @ mymaster 10.0.0.101 6379
786:X 06 Nov 2022 15:21:27.639 # -odown master mymaster 10.0.0.101 6379
786:X 06 Nov 2022 15:21:28.621 * +slave-reconf-done slave 10.0.0.103:6379 10.0.0.103 6379 @ mymaster 10.0.0.101 6379
786:X 06 Nov 2022 15:21:28.672 # +failover-end master mymaster 10.0.0.101 6379
786:X 06 Nov 2022 15:21:28.672 # +switch-master mymaster 10.0.0.101 6379 10.0.0.102 6379
786:X 06 Nov 2022 15:21:28.672 * +slave slave 10.0.0.103:6379 10.0.0.103 6379 @ mymaster 10.0.0.102 6379
786:X 06 Nov 2022 15:21:28.672 * +slave slave 10.0.0.101:6379 10.0.0.101 6379 @ mymaster 10.0.0.102 6379
786:X 06 Nov 2022 15:21:31.738 # +sdown slave 10.0.0.101:6379 10.0.0.101 6379 @ mymaster 10.0.0.102 6379
786:X 06 Nov 2022 15:54:28.253 # -sdown slave 10.0.0.101:6379 10.0.0.101 6379 @ mymaster 10.0.0.102 6379

加载全部内容