consul历史版本地址
https://releases.hashicorp.com/consul
或者
https://www.consul.io/downloads.html
规划环境(生产环境consul server部署3个或者5个):
consul server:192.168.8.115,192.168.8.116,192.168.8.117
consul client:192.168.8.118 192.168.8.119 192.168.8.121
DNS server:192.168.8.120
192.168.8.115 consul-server-01
192.168.8.116 consul-server-02
192.168.8.117 consul-server-03
192.168.8.118 consul-client-01
192.168.8.119 consul-client-02
192.168.8.121 consul-client-03
关闭防火墙
chkconfig iptables off
consul的安装非常容易,从 官网 这里下载以后,解压即可使用,就是一个二进制文件,其他的都没有了。文件下载以后解压放到/usr/local/bin。就可以使用了。不依赖任何东西。上面的4台服务器都安装。
6台机器都创建目录,分别是放配置文件,以及存放数据的。以及存放mysql的健康检查脚本
mkdir /etc/consul.d/ -p && mkdir /data/consul/ -p && mkdir /data/consul/shell -p
然后把相关配置参数写入配置文件,其实也可以不用写,直接跟在命令后面就行,那样不方便管理。
一、建立consul server集群
consul server(192.168.8.115,192.168.8.116,192.168.8.117)配置文件(具体参数的意思请查询官网或者文章给的参考链接):
[root@consul-server-01 consul.d]# cat /etc/consul.d/server.json
{
"data_dir": "/data/consul",
"datacenter": "dc1",
"log_level": "INFO",
"server": true,
"bootstrap_expect": 3,
"bind_addr": "192.168.8.115",
"client_addr" : "0.0.0.0"
}
[root@consul-server-02 consul.d]# cat /etc/consul.d/server.json
{
"data_dir": "/data/consul",
"datacenter": "dc1",
"log_level": "INFO",
"server": true,
"bootstrap_expect": 3,
"bind_addr": "192.168.8.116",
"client_addr" : "0.0.0.0",
"ui":true
}
[root@consul-server-03 consul.d]# cat /etc/consul.d/server.json
{
"data_dir": "/data/consul",
"datacenter": "dc1",
"log_level": "INFO",
"server": true,
"bootstrap_expect": 3,
"bind_addr": "192.168.8.117",
"client_addr" : "0.0.0.0"
}
启动consul server:三台server分别启动
[root@consul-server-01 consul.d]# nohup consul agent -config-dir=/etc/consul.d/ > /data/consul/consul.log &
[root@consul-server-02 consul.d]# nohup consul agent -config-dir=/etc/consul.d/ > /data/consul/consul.log &
nohup consul agent -config-dir=/etc/consul.d/ > /data/consul/consul.log &
[root@consul-server-01 consul.d]# consul join 192.168.8.116
Successfully joined cluster by contacting 1 nodes.
[root@consul-server-03 consul.d]# consul join 192.168.8.116
Successfully joined cluster by contacting 1 nodes.
将115,117加入116
查看集群成员
[root@consul-server-02 consul.d]# consul members
Node Address Status Type Build Protocol DC Segment
consul-server-01 192.168.8.115:8301 alive server 1.0.6 2 dc1 <all>
consul-server-02 192.168.8.116:8301 alive server 1.0.6 2 dc1 <all>
consul-server-03 192.168.8.117:8301 alive server 1.0.6 2 dc1 <all>
登录前端管理平台查看
http://192.168.8.116:8500/ui/#/dc1/services
二、启动consul client,并加入集群
consul client(192.168.8.118,192.168.8.119,192.168.8.121)
[root@consul-client-01 consul.d]# cat /etc/consul.d/client.json
{
"data_dir": "/data/consul",
"enable_script_checks": true,
"bind_addr": "192.168.8.118",
"retry_join": ["192.168.8.116"],
"retry_interval": "30s",
"rejoin_after_leave": true,
"start_join": ["192.168.8.116"]
}
3台服务器的配置文件差异不大,唯一有区别的就是bind_addr地方,自行修改为你自己服务器的ip。我测试环境是虚拟机,有多快网卡,所以必须指定,否则可以绑定0.0.0.0。
启动client
[root@consul-client-01 consul.d]# nohup consul agent -config-dir=/etc/consul.d > /data/consul/consul.log &
[root@consul-client-02 consul.d]# nohup consul agent -config-dir=/etc/consul.d/ >/data/consul/consul.log &
[root@consul-client-03 consul.d]# nohup consul agent -config-dir=/etc/consul.d/ >/data/consul/consul.log &
查看集群成员
[root@consul-client-03 consul.d]# consul members
Node Address Status Type Build Protocol DC Segment
consul-server-01 192.168.8.115:8301 alive server 1.0.6 2 dc1 <all>
consul-server-02 192.168.8.116:8301 alive server 1.0.6 2 dc1 <all>
consul-server-03 192.168.8.117:8301 alive server 1.0.6 2 dc1 <all>
consul-client-01 192.168.8.118:8301 alive client 1.0.6 2 dc1 <default>
consul-client-02 192.168.8.119:8301 alive client 1.0.6 2 dc1 <default>
consul-client-03 192.168.8.121:8301 alive client 1.0.6 2 dc1 <default>
集群成员状态检查
[root@consul03 ~]# consul operator raft list-peers
Node ID Address State Voter RaftProtocol
consul03 90d0adda-0ed7-86cf-d175-4048bee39ecb 192.168.8.123:8300 follower true 3
consul01 e9374e04-1bc1-7dec-6944-2938a16589d0 192.168.8.121:8300 follower true 3
consul02 69276c00-e3fa-89f6-2c19-3441bfafb7e6 192.168.8.122:8300 leader true 3
三、注册服务
consul使用场景(mysql mha)
1、MHA负责主从切换和failover,当主从切换角色变化之后,consul服务发现能识别新master,DNS解析后,IP会定位到新master上,slave同理。
2、应用端访问数据库一般配置写域名即可(读写主库),若要读写分离,需要在客户端驱动里配置读写分离,分别配置写域名和读域名。
Consul 服务定义(mysql)
上面已经搭建好了consul集群,server是192.168.8.115--117 client是118、119、121. 那么118我们就拿来当master,119,121拿来当slave。下面定义服务(118,119,121都要存在)
除了address要修改为对应的服务器地址,其他一样。
[root@db-server-01 shell]# cd /etc/consul.d/
[root@db-server-01 consul.d]# ll
总用量 12
-rw-r--r--. 1 root root 224 3月 2 10:00 client.json
-rw-r--r--. 1 root root 318 3月 8 03:25 r-3306-mysql-test.json
-rw-r--r--. 1 root root 321 3月 8 03:21 w-3306-mysql-test.json
#更新服务(三台都要更新)
[root@consul-client-01 shell]# consul reload
Configuration reload triggered
服务更新后,新建的两个服务会进行注册,登录管理系统,查看服务注册情况,注册通过状态为passing,未通过为critical
则consul-client-01节点上注册的两个服务:
Service 'r-test' critical
Service 'w-test' passing
每个agent都注册后, 对应有两个域名:
w-test.service.consul(对应唯一的一个master IP)
r-test.service.consul(对应两个slave IP, 客户端请求时, 随机分配一个)
查询服务
[root@consul-client-03 shell]# dig @192.168.8.116 -p 8600 r-test.service.consul
; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.17.rc1.el6_4.6 <<>> @192.168.8.116 -p 8600 r-test.service.consul
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1244
;; flags: qr aa rd; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available
;; QUESTION SECTION:
;r-test.service.consul. IN A
;; ANSWER SECTION:
r-test.service.consul. 0 IN A 192.168.8.119
r-test.service.consul. 0 IN A 192.168.8.121
;; Query time: 1 msec
;; SERVER: 192.168.8.116#8600(192.168.8.116)
;; WHEN: Fri Mar 9 16:07:55 2018
;; MSG SIZE rcvd: 71
[root@consul-client-03 shell]# dig @192.168.8.116 -p 8600 w-test.service.consul
; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.17.rc1.el6_4.6 <<>> @192.168.8.116 -p 8600 w-test.service.consul
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 10574
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available
;; QUESTION SECTION:
;w-test.service.consul. IN A
;; ANSWER SECTION:
w-test.service.consul. 0 IN A 192.168.8.118
;; Query time: 1 msec
;; SERVER: 192.168.8.116#8600(192.168.8.116)
;; WHEN: Fri Mar 9 16:08:38 2018
;; MSG SIZE rcvd: 55
四、App端配置域名服务器IP来解析consul后缀的域名,DNS解析及跳转
有三个方案:
- 原内网dns服务器,做域名转发,consul后缀的,都转到consul server上(我们线上是采用这个)
- dns全部跳到consul DNS服务器上,非consul后缀的,使用 recursors 属性跳转到原DNS服务器上
- dnsmaq 转: server=/consul/10.16.X.X#8600 解析consul后缀的
我们内网dns是用的bind,对于bind的如何做域名转发consul官网也有例子:https://www.consul.io/docs/guides/forwarding.html
域名服务器为192.168.8.120
域名转发
[root@localhost named]# cat /etc/named.conf
options {
listen-on port 53 { any; };
listen-on-v6 port 53 { any; };
directory "/var/named";
dump-file "/var/named/data/cache_dump.db";
statistics-file "/var/named/data/named_stats.txt";
memstatistics-file "/var/named/data/named_mem_stats.txt";
allow-query { any; };
recursion yes;
dnssec-enable no;
dnssec-validation no;
/* Path to ISC DLV key */
bindkeys-file "/etc/named.iscdlv.key";
managed-keys-directory "/var/named/dynamic";
};
include "/etc/named/consul.conf";
[root@localhost named]# cat /etc/named/consul.conf
zone "consul" IN {
type forward;
forward only;
forwarders { 192.168.8.116 port 8600;
192.168.8.115 port 8600;
192.168.8.117 port 8600;
};
};
consul后缀的,都转到consul server上
测试
[root@localhost named]# cat /etc/resolv.conf
; generated by /sbin/dhclient-script
#nameserver 219.141.136.10
#nameserver 8.8.8.8
#search localhost
nameserver 192.168.8.120
[root@localhost named]# nslookup
> r-3306-mysql-test.service.consul
Server: 192.168.8.120
Address: 192.168.8.120#53
Non-authoritative answer:
Name: r-3306-mysql-test.service.consul
Address: 192.168.8.119
Name: r-3306-mysql-test.service.consul
Address: 192.168.8.118
> w-3306-mysql-test.service.consul
Server: 192.168.8.120
Address: 192.168.8.120#53
Non-authoritative answer:
Name: w-3306-mysql-test.service.consul
Address: 192.168.8.117