Um Guia Prático para o Cônsul da HashiCorp - Parte 1

Esta é a parte 1 de um Guia Prático de 2 partes para o Cônsul da HashiCorp. Esta parte é focada principalmente em entender os problemas que a Consul resolve e como os resolve. A segunda parte é mais focada na aplicação prática do Consul em um exemplo da vida real e será publicada na próxima semana. Vamos começar.

Que tal configurar uma malha de serviço identificável, personalizável e segura com uma ferramenta?

E se disséssemos que essa ferramenta é independente de plataforma e está pronta para ser executada na nuvem?

E vem como um único arquivo binário.

Isso tudo é verdade. A ferramenta da qual estamos falando é o Cônsul da HashiCorp .

O Consul fornece descoberta de serviço , verificação de integridade , balanceamento de carga , gráfico de serviço , aplicação de identidade com TLS e gerenciamento de configuração de serviço distribuído .

Consul , .

:

. , , .

("") — A, B, C D

("") — A, B, C D

HashiCorp Consul – , , , , , , TLS . Consul control plane service mesh.

HashiCorp Consul ,

HashiCorp Consul 2014 , .

, Consul HashiCorp .

, . , . , .

- (SOA)

, . , .

, .

, HashiCorp Consul, . -, (, A), ( B), ( ) ( D).

, — , — .

. .

: , , A, , .

production.

, , D.

. .

, .

, :

, SOA-

:

A SOA-

, . , .

. . . , . .

, , .

, , . .

, Consul

, . — . , .

, . , , .

A , B. , A B, B?

A

A , B. , . , , , .

, , . .

,

IP-, .

, . IP- , .

, . , . .

. , , . , .

. A B, A B, B. B .

.

. , . , . .

Consul

Consul — .

Consul , . . .

Consul A

A B, B, B. , .

Consul (health-checks) . (health-check), . , , .

, , Consul . , .

Consul .

. Consul , , .

.

, Consul

, , , - YAML, XML JSON. , .

,

, . , .

, . .

, . , .

• . .

  
  
  
  
  
  
  

• , , , .

  
  
  
  
  
  
  

Consul — .

Consul’s KV

Consul . , .

. . .

, Consul

, .

. , .

— . .

— , . .

, . , .

.

. , .

• , - . , .

  
  
  
  
  
  
  

• , , .

  
  
  
  
  
  
  

• , .

  
  
  
  
  
  
  

• - . , , , .

  
  
  
  
  
  
  

SOA

• . , , , , .

Consul TLS.

Consul

Consul , , . Consul Consul Connect.

Consul Connect , , . , , A B, B C.

, IP-. . . , , IP- . .

Consul Connect , TLS. TLS, .

. TLS . .

Consul TLS , . Sidecar. - - .

. , .

Consul

Consul — .

Consul . , .

, Consul, Consul. Consul.

Consul , . /.

Consul — , .

Consul , HashiCorp 3-5 , . Consul. , .

. HashiCorp .

Consul . Consul .

Consul ,

, Consul Consul. Consul Consul.

Consul

Consul , .

, Consul , - HashiCorps Consul GitHub.

Consul .

Consul — Consul.

Consul

Consul .

$ wget https://releases.hashicorp.com/consul/1.4.3/consul_1.4.3_linux_amd64.zip -O consul.zip

--2019-03-10 00:14:07--  https://releases.hashicorp.com/consul/1.4.3/consul_1.4.3_linux_amd64.zip
Resolving releases.hashicorp.com (releases.hashicorp.com)... 151.101.37.183, 2a04:4e42:9::439
Connecting to releases.hashicorp.com (releases.hashicorp.com)|151.101.37.183|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 34777003 (33M) [application/zip]
Saving to: ‘consul.zip’

consul.zip             100%[============================>]  33.17M  4.46MB/s    in 9.2s    

2019-03-10 00:14:17 (3.60 MB/s) - ‘consul.zip’ saved [34777003/34777003]
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

zip-.

$ unzip consul.zip

Archive:  consul.zip
  inflating: consul
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

PATH.

$ export PATH="$PATH:/path/to/consul"
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

Consul

, .

$ consul agent -dev

==> Starting Consul agent...
==> Consul agent running!
           Version: 'v1.4.2'
           Node ID: 'ef46ebb7-3496-346f-f67a-30117cfec0ad'
         Node name: 'devcube'
        Datacenter: 'dc1' (Segment: '<all>')
            Server: true (Bootstrap: false)
       Client Addr: [127.0.0.1] (HTTP: 8500, HTTPS: -1, gRPC: 8502, DNS: 8600)
      Cluster Addr: 127.0.0.1 (LAN: 8301, WAN: 8302)
           Encrypt: Gossip: false, TLS-Outgoing: false, TLS-Incoming: false

==> Log data will now stream in as it occurs:

    2019/03/04 00:38:01 [DEBUG] agent: Using random ID "ef46ebb7-3496-346f-f67a-30117cfec0ad" as node ID
    2019/03/04 00:38:01 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:ef46ebb7-3496-346f-f67a-30117cfec0ad Address:127.0.0.1:8300}]
    2019/03/04 00:38:01 [INFO] raft: Node at 127.0.0.1:8300 [Follower] entering Follower state (Leader: "")
    2019/03/04 00:38:01 [INFO] serf: EventMemberJoin: devcube.dc1 127.0.0.1
    2019/03/04 00:38:01 [INFO] serf: EventMemberJoin: devcube 127.0.0.1
    2019/03/04 00:38:01 [INFO] consul: Adding LAN server devcube (Addr: tcp/127.0.0.1:8300) (DC: dc1)
    2019/03/04 00:38:01 [INFO] consul: Handled member-join event for server "devcube.dc1" in area "wan"
    2019/03/04 00:38:01 [DEBUG] agent/proxy: managed Connect proxy manager started
    2019/03/04 00:38:01 [WARN] raft: Heartbeat timeout from "" reached, starting election
    2019/03/04 00:38:01 [INFO] raft: Node at 127.0.0.1:8300 [Candidate] entering Candidate state in term 2
    2019/03/04 00:38:01 [DEBUG] raft: Votes needed: 1
    2019/03/04 00:38:01 [DEBUG] raft: Vote granted from ef46ebb7-3496-346f-f67a-30117cfec0ad in term 2. Tally: 1
    2019/03/04 00:38:01 [INFO] raft: Election won. Tally: 1
    2019/03/04 00:38:01 [INFO] raft: Node at 127.0.0.1:8300 [Leader] entering Leader state
    2019/03/04 00:38:01 [INFO] consul: cluster leadership acquired
    2019/03/04 00:38:01 [INFO] consul: New leader elected: devcube
    2019/03/04 00:38:01 [INFO] agent: Started DNS server 127.0.0.1:8600 (tcp)
    2019/03/04 00:38:01 [INFO] agent: Started DNS server 127.0.0.1:8600 (udp)
    2019/03/04 00:38:01 [INFO] agent: Started HTTP server on 127.0.0.1:8500 (tcp)
    2019/03/04 00:38:01 [INFO] agent: Started gRPC server on 127.0.0.1:8502 (tcp)
    2019/03/04 00:38:01 [INFO] agent: started state syncer
    2019/03/04 00:38:01 [INFO] connect: initialized primary datacenter CA with provider "consul"
    2019/03/04 00:38:01 [DEBUG] consul: Skipping self join check for "devcube" since the cluster is too small
    2019/03/04 00:38:01 [INFO] consul: member 'devcube' joined, marking health alive
    2019/03/04 00:38:01 [DEBUG] agent: Skipping remote check "serfHealth" since it is managed automatically
    2019/03/04 00:38:01 [INFO] agent: Synced node info
    2019/03/04 00:38:01 [DEBUG] agent: Node info in sync
    2019/03/04 00:38:01 [DEBUG] agent: Skipping remote check "serfHealth" since it is managed automatically
    2019/03/04 00:38:01 [DEBUG] agent: Node info in sync
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

.

Consul

, Consul.

$ consul members

Node     Address         Status  Type    Build  Protocol  DC   Segment
devcube  127.0.0.1:8301  alive   server  1.4.0  2         dc1  <all>
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

, , . , Consul.

gossip .

Consul HTTP API

Consul HTTP API, .

$ curl localhost:8500/v1/catalog/nodes

[
    {
        "ID": "ef46ebb7-3496-346f-f67a-30117cfec0ad",
        "Node": "devcube",
        "Address": "127.0.0.1",
        "Datacenter": "dc1",
        "TaggedAddresses": {
            "lan": "127.0.0.1",
            "wan": "127.0.0.1"
        },
        "Meta": {
            "consul-network-segment": ""
        },
        "CreateIndex": 9,
        "ModifyIndex": 10
    }
]
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

DNS Consul

Consul DNS- . DNS 8600. .

$ dig @127.0.0.1 -p 8600 devcube.node.consul

; <<>> DiG 9.11.3-1ubuntu1.5-Ubuntu <<>> @127.0.0.1 -p 8600 devcube.node.consul
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 42215
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 2
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;devcube.node.consul.       IN  A

;; ANSWER SECTION:
devcube.node.consul.    0   IN  A   127.0.0.1

;; ADDITIONAL SECTION:
devcube.node.consul.    0   IN  TXT "consul-network-segment="

;; Query time: 19 msec
;; SERVER: 127.0.0.1#8600(127.0.0.1)
;; WHEN: Mon Mar 04 00:45:44 IST 2019
;; MSG SIZE  rcvd: 100
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

Consul , HTTP API.

() Consul

() — . .

, consul.d — ‘.d’ , , consul.

$ mkdir ./consul.d
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

- Django, 80 .

$ echo '{"service": {"name": "web", "tags": ["django"], "port": 80}}' \
    > ./consul.d/web.json
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

- , .

$ consul agent -dev -config-dir=./consul.d

==> Starting Consul agent...
==> Consul agent running!
           Version: 'v1.4.2'
           Node ID: '810f4804-dbce-03b1-056a-a81269ca90c1'
         Node name: 'devcube'
        Datacenter: 'dc1' (Segment: '<all>')
            Server: true (Bootstrap: false)
       Client Addr: [127.0.0.1] (HTTP: 8500, HTTPS: -1, gRPC: 8502, DNS: 8600)
      Cluster Addr: 127.0.0.1 (LAN: 8301, WAN: 8302)
           Encrypt: Gossip: false, TLS-Outgoing: false, TLS-Incoming: false

==> Log data will now stream in as it occurs:

    2019/03/04 00:55:28 [DEBUG] agent: Using random ID "810f4804-dbce-03b1-056a-a81269ca90c1" as node ID
    2019/03/04 00:55:28 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:810f4804-dbce-03b1-056a-a81269ca90c1 Address:127.0.0.1:8300}]
    2019/03/04 00:55:28 [INFO] raft: Node at 127.0.0.1:8300 [Follower] entering Follower state (Leader: "")
    2019/03/04 00:55:28 [INFO] serf: EventMemberJoin: devcube.dc1 127.0.0.1
    2019/03/04 00:55:28 [INFO] serf: EventMemberJoin: devcube 127.0.0.1
    2019/03/04 00:55:28 [INFO] consul: Adding LAN server devcube (Addr: tcp/127.0.0.1:8300) (DC: dc1)
    2019/03/04 00:55:28 [DEBUG] agent/proxy: managed Connect proxy manager started
    2019/03/04 00:55:28 [INFO] consul: Handled member-join event for server "devcube.dc1" in area "wan"
    2019/03/04 00:55:28 [INFO] agent: Started DNS server 127.0.0.1:8600 (udp)
    2019/03/04 00:55:28 [INFO] agent: Started DNS server 127.0.0.1:8600 (tcp)
    2019/03/04 00:55:28 [INFO] agent: Started HTTP server on 127.0.0.1:8500 (tcp)
    2019/03/04 00:55:28 [INFO] agent: started state syncer
    2019/03/04 00:55:28 [INFO] agent: Started gRPC server on 127.0.0.1:8502 (tcp)
    2019/03/04 00:55:28 [WARN] raft: Heartbeat timeout from "" reached, starting election
    2019/03/04 00:55:28 [INFO] raft: Node at 127.0.0.1:8300 [Candidate] entering Candidate state in term 2
    2019/03/04 00:55:28 [DEBUG] raft: Votes needed: 1
    2019/03/04 00:55:28 [DEBUG] raft: Vote granted from 810f4804-dbce-03b1-056a-a81269ca90c1 in term 2. Tally: 1
    2019/03/04 00:55:28 [INFO] raft: Election won. Tally: 1
    2019/03/04 00:55:28 [INFO] raft: Node at 127.0.0.1:8300 [Leader] entering Leader state
    2019/03/04 00:55:28 [INFO] consul: cluster leadership acquired
    2019/03/04 00:55:28 [INFO] consul: New leader elected: devcube
    2019/03/04 00:55:28 [INFO] connect: initialized primary datacenter CA with provider "consul"
    2019/03/04 00:55:28 [DEBUG] consul: Skipping self join check for "devcube" since the cluster is too small
    2019/03/04 00:55:28 [INFO] consul: member 'devcube' joined, marking health alive
    2019/03/04 00:55:28 [DEBUG] agent: Skipping remote check "serfHealth" since it is managed automatically
    2019/03/04 00:55:28 [INFO] agent: Synced service "web"
    2019/03/04 00:55:28 [DEBUG] agent: Node info in sync
    2019/03/04 00:55:29 [DEBUG] agent: Skipping remote check "serfHealth" since it is managed automatically
    2019/03/04 00:55:29 [DEBUG] agent: Service "web" in sync
    2019/03/04 00:55:29 [DEBUG] agent: Node info in sync
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

— , web



. Consul . .

DNS Consul

DNS, . :

$ dig @127.0.0.1 -p 8600 web.service.consul

; <<>> DiG 9.11.3-1ubuntu1.5-Ubuntu <<>> @127.0.0.1 -p 8600 web.service.consul
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 51488
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 2
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;web.service.consul.        IN  A

;; ANSWER SECTION:
web.service.consul. 0   IN  A   127.0.0.1

;; ADDITIONAL SECTION:
web.service.consul. 0   IN  TXT "consul-network-segment="

;; Query time: 0 msec
;; SERVER: 127.0.0.1#8600(127.0.0.1)
;; WHEN: Mon Mar 04 00:59:32 IST 2019
;; MSG SIZE  rcvd: 99
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

DNS , , .

$ dig @127.0.0.1 -p 8600 web.service.consul SRV

; <<>> DiG 9.11.3-1ubuntu1.5-Ubuntu <<>> @127.0.0.1 -p 8600 web.service.consul SRV
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 712
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 3
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;web.service.consul.        IN  SRV

;; ANSWER SECTION:
web.service.consul. 0   IN  SRV 1 1 80 devcube.node.dc1.consul.

;; ADDITIONAL SECTION:
devcube.node.dc1.consul. 0  IN  A   127.0.0.1
devcube.node.dc1.consul. 0  IN  TXT "consul-network-segment="

;; Query time: 0 msec
;; SERVER: 127.0.0.1#8600(127.0.0.1)
;; WHEN: Mon Mar 04 00:59:43 IST 2019
;; MSG SIZE  rcvd: 142
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

TAG, , :

$ dig @127.0.0.1 -p 8600 django.web.service.consul

; <<>> DiG 9.11.3-1ubuntu1.5-Ubuntu <<>> @127.0.0.1 -p 8600 django.web.service.consul
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 12278
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 2
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;django.web.service.consul. IN  A

;; ANSWER SECTION:
django.web.service.consul. 0    IN  A   127.0.0.1

;; ADDITIONAL SECTION:
django.web.service.consul. 0    IN  TXT "consul-network-segment="

;; Query time: 0 msec
;; SERVER: 127.0.0.1#8600(127.0.0.1)
;; WHEN: Mon Mar 04 01:01:17 IST 2019
;; MSG SIZE  rcvd: 106
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

Consul HTTP API

HTTP API:

$ curl http://localhost:8500/v1/catalog/service/web

[
    {
        "ID": "810f4804-dbce-03b1-056a-a81269ca90c1",
        "Node": "devcube",
        "Address": "127.0.0.1",
        "Datacenter": "dc1",
        "TaggedAddresses": {
            "lan": "127.0.0.1",
            "wan": "127.0.0.1"
        },
        "NodeMeta": {
            "consul-network-segment": ""
        },
        "ServiceKind": "",
        "ServiceID": "web",
        "ServiceName": "web",
        "ServiceTags": [
            "django"
        ],
        "ServiceAddress": "",
        "ServiceWeights": {
            "Passing": 1,
            "Warning": 1
        },
        "ServiceMeta": {},
        "ServicePort": 80,
        "ServiceEnableTagOverride": false,
        "ServiceProxyDestination": "",
        "ServiceProxy": {},
        "ServiceConnect": {},
        "CreateIndex": 10,
        "ModifyIndex": 10
    }
]
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

HTTP API:

$ curl http://localhost:8500/v1/catalog/service/web?passing

[
    {
        "ID": "810f4804-dbce-03b1-056a-a81269ca90c1",
        "Node": "devcube",
        "Address": "127.0.0.1",
        "Datacenter": "dc1",
        "TaggedAddresses": {
            "lan": "127.0.0.1",
            "wan": "127.0.0.1"
        },
        "NodeMeta": {
            "consul-network-segment": ""
        },
        "ServiceKind": "",
        "ServiceID": "web",
        "ServiceName": "web",
        "ServiceTags": [
            "django"
        ],
        "ServiceAddress": "",
        "ServiceWeights": {
            "Passing": 1,
            "Warning": 1
        },
        "ServiceMeta": {},
        "ServicePort": 80,
        "ServiceEnableTagOverride": false,
        "ServiceProxyDestination": "",
        "ServiceProxy": {},
        "ServiceConnect": {},
        "CreateIndex": 10,
        "ModifyIndex": 10
    }
]
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

Consul

Consul, .

. SIGHUP , Consul, SIGHUP , HTTP API, , .

$ ps aux | grep [c]onsul

pranav   21289  2.4  0.3 177012 54924 pts/2    Sl+  00:55   0:22 consul agent -dev -config-dir=./consul.d
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

SIGHUP 21289

$ kill -SIGHUP 21289
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

Consul

$ consul reload
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

.

Consul.

...
    2019/03/04 01:10:46 [INFO] agent: Caught signal:  hangup
    2019/03/04 01:10:46 [INFO] agent: Reloading configuration...
    2019/03/04 01:10:46 [DEBUG] agent: removed service "web"
    2019/03/04 01:10:46 [INFO] agent: Synced service "web"
    2019/03/04 01:10:46 [DEBUG] agent: Node info in sync
...
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

- Consul.

Consul - . 8500.

http://localhost:8500. .

Consul , Consul -.

(Exploring) - Consul

, .

(Exploring) - Consul

, .

, - Consul.

, Consul Web UI , Consul.

Consul Zookeeper, doozerd etcd?

Consul , (health-check), -, -.

Zookeeper, doozerd etcd . - , , .

, Consul, , .

, /.

. Consul , .

. , . heartbeats TTL. health checks -. TTL.

Zookeeper, Consul -, , gossip. ( localhost), , .

, Consul HTTP DNS , . , .

- Consul Consul .

HashiCorp Consul

HashiCorp Consul.

Consul HashiCorp:

Consul Template — Consul. , HashiCorp Consul Vault. Consul consul-template.

Envconsul — Consul. Envconsul , HashiCorp Consul Vault.

Consul Replicate — Consul cross-DC KV . Consul consul-replicate.

Consul Migrate — Consul Consul 0.5.1+.

Consul , .

Confd — etcd consul.

Fabio — Fabio — , HTTP(S) TCP , Consul. consul, , Fabio . .

Registrator — Docker . Docker, .

Hashi-UI — HashiCorp Consul & Nomad.

Git2consul — git Consul KVs. git2consul git Consul KVs. , git , , Consul- .

Spring-cloud-consul — Consul Spring Boot Spring Spring. Consul.

Crypt — etcd consul.

Mesos-Consul — Mesos to Consul bridge . Mesos-Consul / , Mesos.

Consul-cli — Consul HTTP API.

. — . HashiCorp Consul , .

, Consul, , .

, , HashiCorp Consul .

, Consul. , . , , .

, - , .

HashiCorp Consul GitHub.

HashiCorp Consul Guides and Code

, .

HashiCorp Consul

PS Vou acrescentar que você também pode instalar o cônsul a partir do repositório de pacotes: https://www.hashicorp.com/blog/announcing-the-hashicorp-linux-repository