背景
部署的redis版本为4.0.10。集群为三主三备。由于只有三台主机，因此每台主机上部署2个节点。正式环境请使用至少6台主机。
为了方便管理，使用docker形式进行部署。
部署Redis单实例
1. 安装docker
用yum离线方式安装docker
---
- name: copy docker rpm package
unarchive:
src: /home/monitor/src/docker/docker-ce-18.09.3.zip
dest: /root
- name: install package
shell: |
yum install /root/containerd.io-1.2.4-3.1.el7.x86_64.rpm
yum install /root/container-selinux-2.9-4.el7.noarch.rpm
yum install /root/docker-ce-18.09.3-3.el7.x86_64.rpm
yum install /root/docker-ce-cli-18.09.3-3.el7.x86_64.rpm
- name: systemctl enable
service:
name: docker
enabled: yes
- name: systemctl start
service:
name: docker
state: started
2. 安装redis实例
- 创建数据目录、配置文件目录，日志文件，用于挂载docker
- 拷贝redis配置文件到配置目录下
- 需要映射redis服务端口和集群通信端口，集群通信端口一般为服务端口+10000，这里我们配置的节点信息如下：
- master1: node1-ip:6379 16379
- master2: node2-ip:6379 16379
- master3: node3-ip:6379 16379
- slave1: node1-ip:6378 16378
- slave2: node1-ip:6378 16378
- slave3: node1-ip:6378 16378
redis.conf配置：
################################GENERAL#####################################
daemonize no
pidfile /var/redis.pid
port 6379
bind 0.0.0.0
timeout 300
tcp-keepalive 60
loglevel verbose
logfile /var/redis.log
databases 16
#requirepass paic@123
################################SNAPSHOTTING(rdb)################################
save 900 1
save 300 10
save 60 10000
rdbcompression yes
dbfilename dump.rdb
# 数据目录
dir /data
maxclients 4096
############################## APPEND ONLY MODE(aof) ###############################
# 数据持久化
appendonly yes
appendfsync always
no-appendfsync-on-rewrite no
################################ REDIS CLUSTER###############################
# 开启集群模式
cluster-enabled yes
# 集群配置文件
cluster-config-file nodes-6379.conf
cluster-node-timeout 15000
cluster-require-full-coverage yes
---
- name: mkdir /data/data02/redis
file:
dest: '{{ item }}'
state: directory
mode: 0755
owner: root
recurse: true
with_items:
- /data/data02/redis/conf
- /data/data02/redis/data
- name: touch redis.log
file:
dest: /data/data02/redis/redis.log
state: touch
mode: 0644
- name: copy redis.conf
copy:
dest: /data/data02/redis/conf/redis.conf
src: /home/monitor/redis/redis.conf
mode: '0644'
- name: start2
shell: docker run -it --name redis-1 -v /data/data02/redis/redis.log:/var/redis.log -v /data/data02/redis/conf/redis.conf:/etc/redis/redis.conf -v /data/data02/redis/data:/data -p 6378:6379 -p 16378:16379 -tid telredis:1.0.0 redis-server /etc/redis/redis.conf
- name: start1
shell: docker run -it --name redis -v /data/data01/redis/redis.log:/var/redis.log -v /data/data01/redis/conf/redis.conf:/etc/redis/redis.conf -v /data/data01/redis/data:/data -p 6379:6379 -p 16379:16379 -tid telredis:1.0.0 redis-server /etc/redis/redis.conf
3. 配置集群模式
redis4.0.10版本的集群模式依赖于redis-trib.rb来实现。redis-trib.rb需要gem的redis模块，而安装gem则需要ruby环境。
3.1. 离线安装集群环境
通过yum info可以查看yum源支持的ruby版本为2.0.0，支持的gem版本为2.0.0。而gem的redis4.0.1模块需要至少gem2.2.3版本以上才能支持。
因此我们需要使用离线的方式进行安装。
3.1.1 下载源码包
ruby-2.2.3: https://cache.ruby-lang.org/pub/ruby/2.2/ruby-2.2.3.tar.gz
rubygems-2.2.4: https://rubygems.org/rubygems/rubygems-2.2.4.tgz
redis-4.0.1.gem: https://rubygems.org/downloads/redis-4.0.1.gem
redis-4.0.10: http://download.redis.io/releases/redis-4.0.11.tar.gz
3.1.2 编译安装ruby
tar zxvf ruby-2.2.3.tar.gz
cd ruby-2.2.3/
#创建安装路径
mkdir -p /usr/local/ruby
./configure --prefix=/usr/local/ruby
make && make install
#创建软连接
ln -s /usr/local/ruby/bin/ruby /usr/local/bin/rub
#查看版本信息
ruby -v
3.1.3 编译安装gem
unzip rubygems-2.2.4.tgz
cd rubygems-2.2.4
ruby setup.rb
3.1.4 安装gem redis模块
gem install -l redis-4.0.1.gem
3.2. 使用redis-trib.rb创建集群
因为redis的docker镜像没有redis-trib.rb，需要从源码包中获取。
tar zxvf redis-4.0.11.tar.gz
cd redis-4.0.11
./src/redis-trib.rb create --replicas 1 \
30.105.16.132:6379 \
30.105.16.131:6379 \
30.105.16.130:6379 \
30.105.16.132:6378 \
30.105.16.131:6378 \
30.105.16.130:6378
结果：
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
...
...
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
集群创建成功
3.3 测试集群模式
[root@CNSH431181 redis-4.0.11]# /opt/module/redis-4.0.10/bin/redis-cli -c -h 30.105.16.132 -p 6379
30.105.16.132:6379> ping
PONG
30.105.16.132:6379> set xpp sb
OK
30.105.16.132:6379> exit
You have mail in /var/spool/mail/root
[root@CNSH431181 redis-4.0.11]# /opt/module/redis-4.0.10/bin/redis-cli -c -h 30.105.16.132 -p 6379
30.105.16.132:6379> KEYS *
1) "xpp"
30.105.16.132:6379> get xpp
"sb"
30.105.16.132:6379>
[root@CNSH431181 redis-4.0.11]# /opt/module/redis-4.0.10/bin/redis-cli -c -h 30.105.16.132 -p 6378
30.105.16.132:6378> KEYS *
(empty list or set)
30.105.16.132:6378> get xpp
-> Redirected to slot [1895] located at 30.105.16.132:6379
"sb"
30.105.16.132:6379>
4. 集群安全配置
上述集群部署验证完成后，我们还需要对节点、集群设置密码。以提高安全性。
4.1 设置密码
在redis.conf文件中添加
masterauth yourpasswd
requirepass yourpasswd
4.2 重启集群
docker restart redis
4.3 修改redis-trib.rb脚本
其实到步骤4.2,集群通过密码鉴权已经建立。这一步主要是我们需要去验证、观察集群是否正常
#修改前
@r = Redis.new(:host => @info[:host], :port => @info[:port], :timeout => 60)
#修改后
@r = Redis.new(:host => @info[:host], :port => @info[:port], :timeout => 60, :password => yourpasswd)
4.4 验证
[root@CNSH431181 src]# /opt/module/redis-4.0.10/bin/redis-cli -c -p 6379 -h 30.105.16.132
30.105.16.132:6379> ping
(error) NOAUTH Authentication required.
30.105.16.132:6379> AUTH
(error) ERR wrong number of arguments for 'auth' command
30.105.16.132:6379> AUTH passwd12
OK
30.105.16.132:6379> ping
PONG
30.105.16.132:6379> exit
[root@CNSH431181 src]# ./redis-trib.rb check 30.105.16.132:6378
>>> Performing Cluster Check (using node 30.105.16.132:6378)
S: 4458d6ec81184efaa1817cecdb70c717103b3d9b 30.105.16.132:6378
slots: (0 slots) slave
replicates 1ceaeb829b441f39b0a20b3e6367b8c9188eb2c2
M: a7f2414fe27d293f37371bab864e882b759f514c 30.105.16.131:6379
slots:5461-10922 (5462 slots) master
0 additional replica(s)
M: 80fc03f5dcc7ee4785831ab1bffd5131aa3ab7d6 30.105.16.132:6379
slots:0-5460 (5461 slots) master
0 additional replica(s)
M: 1ceaeb829b441f39b0a20b3e6367b8c9188eb2c2 30.105.16.130:6379
slots:10923-16383 (5461 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
