consul 服务发现搭建简介

consul历史版本地址

https://releases.hashicorp.com/consul
或者
https://www.consul.io/downloads.html

规划环境(生产环境consul server部署3个或者5个):
consul server:192.168.8.115,192.168.8.116,192.168.8.117
consul client:192.168.8.118 192.168.8.119 192.168.8.121
DNS server:192.168.8.120

192.168.8.115 consul-server-01
192.168.8.116 consul-server-02
192.168.8.117 consul-server-03

192.168.8.118 consul-client-01
192.168.8.119 consul-client-02
192.168.8.121 consul-client-03

关闭防火墙

chkconfig iptables off

consul的安装非常容易,从 官网 这里下载以后,解压即可使用,就是一个二进制文件,其他的都没有了。文件下载以后解压放到/usr/local/bin。就可以使用了。不依赖任何东西。上面的4台服务器都安装。

6台机器都创建目录,分别是放配置文件,以及存放数据的。以及存放mysql的健康检查脚本

mkdir /etc/consul.d/ -p && mkdir /data/consul/ -p && mkdir /data/consul/shell -p

然后把相关配置参数写入配置文件,其实也可以不用写,直接跟在命令后面就行,那样不方便管理。

一、建立consul server集群

consul server(192.168.8.115,192.168.8.116,192.168.8.117)配置文件(具体参数的意思请查询官网或者文章给的参考链接):
[root@consul-server-01 consul.d]# cat /etc/consul.d/server.json 
{
  "data_dir": "/data/consul",
  "datacenter": "dc1",
  "log_level": "INFO",
  "server": true,
  "bootstrap_expect": 3,
  "bind_addr": "192.168.8.115",
  "client_addr" : "0.0.0.0"
}
[root@consul-server-02 consul.d]# cat /etc/consul.d/server.json 
{
  "data_dir": "/data/consul",
  "datacenter": "dc1",
  "log_level": "INFO",
  "server": true,
  "bootstrap_expect": 3,
  "bind_addr": "192.168.8.116",
  "client_addr" : "0.0.0.0",
  "ui":true  
}
[root@consul-server-03 consul.d]# cat /etc/consul.d/server.json 
{
  "data_dir": "/data/consul",
  "datacenter": "dc1",
  "log_level": "INFO",
  "server": true,
  "bootstrap_expect": 3,
  "bind_addr": "192.168.8.117",
  "client_addr" : "0.0.0.0"
}

启动consul server:三台server分别启动

[root@consul-server-01 consul.d]# nohup consul agent -config-dir=/etc/consul.d/ > /data/consul/consul.log &
[root@consul-server-02 consul.d]# nohup consul agent -config-dir=/etc/consul.d/ > /data/consul/consul.log  &
 nohup consul agent -config-dir=/etc/consul.d/ > /data/consul/consul.log  &

[root@consul-server-01 consul.d]# consul join 192.168.8.116
Successfully joined cluster by contacting 1 nodes.

[root@consul-server-03 consul.d]# consul join 192.168.8.116
Successfully joined cluster by contacting 1 nodes.

将115,117加入116

查看集群成员

[root@consul-server-02 consul.d]# consul members
Node              Address             Status  Type    Build  Protocol  DC   Segment
consul-server-01  192.168.8.115:8301  alive   server  1.0.6  2         dc1  <all>
consul-server-02  192.168.8.116:8301  alive   server  1.0.6  2         dc1  <all>
consul-server-03  192.168.8.117:8301  alive   server  1.0.6  2         dc1  <all>

登录前端管理平台查看

http://192.168.8.116:8500/ui/#/dc1/services

二、启动consul client,并加入集群

consul client(192.168.8.118,192.168.8.119,192.168.8.121)

[root@consul-client-01 consul.d]# cat /etc/consul.d/client.json 
{
  "data_dir": "/data/consul",
  "enable_script_checks": true,
  "bind_addr": "192.168.8.118",
  "retry_join": ["192.168.8.116"],
  "retry_interval": "30s",
  "rejoin_after_leave": true,
  "start_join": ["192.168.8.116"]
}

3台服务器的配置文件差异不大,唯一有区别的就是bind_addr地方,自行修改为你自己服务器的ip。我测试环境是虚拟机,有多快网卡,所以必须指定,否则可以绑定0.0.0.0。

启动client

[root@consul-client-01 consul.d]# nohup consul agent -config-dir=/etc/consul.d > /data/consul/consul.log &
[root@consul-client-02 consul.d]# nohup consul agent -config-dir=/etc/consul.d/ >/data/consul/consul.log &
[root@consul-client-03 consul.d]# nohup consul agent -config-dir=/etc/consul.d/ >/data/consul/consul.log &

查看集群成员

[root@consul-client-03 consul.d]# consul members
Node              Address             Status  Type    Build  Protocol  DC   Segment
consul-server-01  192.168.8.115:8301  alive   server  1.0.6  2         dc1  <all>
consul-server-02  192.168.8.116:8301  alive   server  1.0.6  2         dc1  <all>
consul-server-03  192.168.8.117:8301  alive   server  1.0.6  2         dc1  <all>
consul-client-01  192.168.8.118:8301  alive   client  1.0.6  2         dc1  <default>
consul-client-02  192.168.8.119:8301  alive   client  1.0.6  2         dc1  <default>
consul-client-03  192.168.8.121:8301  alive   client  1.0.6  2         dc1  <default>

集群成员状态检查

[root@consul03 ~]# consul operator raft list-peers
Node      ID                                    Address             State     Voter  RaftProtocol
consul03  90d0adda-0ed7-86cf-d175-4048bee39ecb  192.168.8.123:8300  follower  true   3
consul01  e9374e04-1bc1-7dec-6944-2938a16589d0  192.168.8.121:8300  follower  true   3
consul02  69276c00-e3fa-89f6-2c19-3441bfafb7e6  192.168.8.122:8300  leader    true   3

三、注册服务
consul使用场景(mysql mha)
1、MHA负责主从切换和failover,当主从切换角色变化之后,consul服务发现能识别新master,DNS解析后,IP会定位到新master上,slave同理。
2、应用端访问数据库一般配置写域名即可(读写主库),若要读写分离,需要在客户端驱动里配置读写分离,分别配置写域名和读域名。

Consul 服务定义(mysql)
上面已经搭建好了consul集群,server是192.168.8.115--117 client是118、119、121. 那么118我们就拿来当master,119,121拿来当slave。下面定义服务(118,119,121都要存在)
除了address要修改为对应的服务器地址,其他一样。

[root@db-server-01 shell]# cd /etc/consul.d/
[root@db-server-01 consul.d]# ll
总用量 12
-rw-r--r--. 1 root root 224 3月   2 10:00 client.json
-rw-r--r--. 1 root root 318 3月   8 03:25 r-3306-mysql-test.json
-rw-r--r--. 1 root root 321 3月   8 03:21 w-3306-mysql-test.json
#更新服务(三台都要更新)
[root@consul-client-01 shell]# consul reload
Configuration reload triggered

服务更新后,新建的两个服务会进行注册,登录管理系统,查看服务注册情况,注册通过状态为passing,未通过为critical
则consul-client-01节点上注册的两个服务:

Service 'r-test'  critical
Service 'w-test'  passing 

每个agent都注册后, 对应有两个域名:
w-test.service.consul(对应唯一的一个master IP)
r-test.service.consul(对应两个slave IP, 客户端请求时, 随机分配一个)

查询服务

[root@consul-client-03 shell]# dig @192.168.8.116 -p 8600 r-test.service.consul
; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.17.rc1.el6_4.6 <<>> @192.168.8.116 -p 8600 r-test.service.consul
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1244
;; flags: qr aa rd; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;r-test.service.consul.         IN      A

;; ANSWER SECTION:
r-test.service.consul.  0       IN      A       192.168.8.119
r-test.service.consul.  0       IN      A       192.168.8.121

;; Query time: 1 msec
;; SERVER: 192.168.8.116#8600(192.168.8.116)
;; WHEN: Fri Mar  9 16:07:55 2018
;; MSG SIZE  rcvd: 71

[root@consul-client-03 shell]# dig @192.168.8.116 -p 8600 w-test.service.consul

; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.17.rc1.el6_4.6 <<>> @192.168.8.116 -p 8600 w-test.service.consul
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 10574
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;w-test.service.consul.         IN      A

;; ANSWER SECTION:
w-test.service.consul.  0       IN      A       192.168.8.118

;; Query time: 1 msec
;; SERVER: 192.168.8.116#8600(192.168.8.116)
;; WHEN: Fri Mar  9 16:08:38 2018
;; MSG SIZE  rcvd: 55

四、App端配置域名服务器IP来解析consul后缀的域名,DNS解析及跳转

有三个方案:

  1. 原内网dns服务器,做域名转发,consul后缀的,都转到consul server上(我们线上是采用这个)
  2. dns全部跳到consul DNS服务器上,非consul后缀的,使用 recursors 属性跳转到原DNS服务器上
  3. dnsmaq 转: server=/consul/10.16.X.X#8600 解析consul后缀的

我们内网dns是用的bind,对于bind的如何做域名转发consul官网也有例子:https://www.consul.io/docs/guides/forwarding.html

域名服务器为192.168.8.120
域名转发

[root@localhost named]# cat /etc/named.conf 
options {
  listen-on port 53 { any; };
  listen-on-v6 port 53 { any; };
  directory       "/var/named";
  dump-file       "/var/named/data/cache_dump.db";
  statistics-file "/var/named/data/named_stats.txt";
  memstatistics-file "/var/named/data/named_mem_stats.txt";
  allow-query     { any; };
  recursion yes;

  dnssec-enable no;
  dnssec-validation no;

  /* Path to ISC DLV key */
  bindkeys-file "/etc/named.iscdlv.key";

  managed-keys-directory "/var/named/dynamic";
};

include "/etc/named/consul.conf";

[root@localhost named]# cat /etc/named/consul.conf 
zone "consul" IN {
  type forward;
  forward only;
  forwarders { 192.168.8.116 port 8600; 
               192.168.8.115 port 8600;
               192.168.8.117 port 8600;
             };
};

consul后缀的,都转到consul server上

测试

[root@localhost named]# cat /etc/resolv.conf 
; generated by /sbin/dhclient-script
#nameserver 219.141.136.10
#nameserver 8.8.8.8
#search localhost
nameserver 192.168.8.120

[root@localhost named]# nslookup 
> r-3306-mysql-test.service.consul
Server:         192.168.8.120
Address:        192.168.8.120#53

Non-authoritative answer:
Name:   r-3306-mysql-test.service.consul
Address: 192.168.8.119
Name:   r-3306-mysql-test.service.consul
Address: 192.168.8.118
> w-3306-mysql-test.service.consul
Server:         192.168.8.120
Address:        192.168.8.120#53

Non-authoritative answer:
Name:   w-3306-mysql-test.service.consul
Address: 192.168.8.117

留下评论

您的邮箱地址不会被公开。 必填项已用 * 标注