elasticsearch集群搭建


Elasticsearch 是用 Java 编写的,它的内部使用 Lucene 做索引与搜索,但是它的目的是使全文检索变得简单, 通过隐藏 Lucene 的复杂性,取而代之的提供一套简单一致的 RESTful API。 现有两部机器:192.168.31.86,192.168.31.87

1、注意事项

集群需要在同一网段

2、配置elasticsearch.yml

cluster.name :配置es的集群名称,不同的集群用名字来区分, 如果在同一网段下有多个集群,就可以用这个属性来区分不同的集群。

node.name:节点名称,一部服务器一个节点,分别为node-1,node-2,node-3 。

http.port :服务端口,一般默认是9200,如果在同一部服务器上面配置多个elasticsearch服务,则端口需要不同

discovery.zen.ping.unicast.hosts:自动检索可用节点,设置了这个参数才会自动检测可用的节点。 同一集群名下,elasticsearch会自动去发现其他的节点。

具体配置:

192.168.31.86: cluster.name: mycluster node.name: node-1 指定了该节点可能成为 master 节点,还可以是数据节点 node.master: true node.data: true network.host: 192.168.31.86 http.port: 9200 transport.tcp.port: 9300 discovery.zen.ping.unicast.hosts: ["192.168.31.86", "192.168.31.87"]

192.168.31.87: cluster.name: mycluster node.name: node-2 指定了该节点可能成为 master 节点,还可以是数据节点 node.master: true node.data: true network.host: 192.168.31.87 http.port: 9200 transport.tcp.port: 9300 discovery.zen.ping.unicast.hosts: ["192.168.31.86", "192.168.31.87"]

然后分别启动两个节点的ES服务。

检查两个节点的情况:

192.168.31.86: [zsz@zsz elasticsearch-2.3.2]$ curl '192.168.31.86:9200/_cat/indices?v' health status index pri rep docs.count docs.deleted store.size pri.store.size green open myindex 5 1 163 0 294kb 147kb green open customer 5 1 0 0 1.5kb 795b

192.168.31.87: [zsz@VS-SUPSV-TEST1 elasticsearch-2.3.2]$ curl '192.168.31.87:9200/_cat/indices?v' health status index pri rep docs.count docs.deleted store.size pri.store.size green open myindex 5 1 163 0 294kb 147kb green open customer 5 1 0 0 1.5kb 795b

发现是GREEN的绿色健康状态,说明备份被分配完成,可用。集群配置完成。

测试集群中两个节点的查询情况:

[zsz@zsz ~]$ curl '192.168.31.87:9200/myindex/_search?q=ghdfdfgddfg' {"took":13,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":1,"max_score":0.49305484,"hits":[{"_index":"myindex","_type":"mytype","id":"AVeoqjX-bv4eY7Nx-xL","_score":0.49305484,"_source":{"pk":14,"user_pk":10,"media_pk":2,"topic_pk":0,"access_source_pk":0,"p_channel_pk":0,"c_channel_pk":0,"title":"ghdfdfgddfg","cover_url":{"S":["http://ochz1y8tu.bkt.clouddn.com/lg-6Q2Cy2yjF_SqLSo5YaKJKx2E520160829102601_2.jpg"],"L":["http://ochz1y8tu.bkt.clouddn.com/lg-6Q2Cy2yjF_SqLSo5YaKJKx2E520160829102601_1.jpg"]},"summary":""好奇号"发回火星最新影像 地貌酷似地球","read_count":0,"like_count":0,"share_count":0,"favorites_count":0,"check_status":3,"release_status":1,"isurgent":1,"created_at":"2016-08-15T14:07:19.000+08:00","updated_at":"2016-10-04T10:11:56.000+08:00"}}]}}

[zsz@zsz ~]$ curl '192.168.31.86:9200/myindex/_search?q=ghdfdfgddfg' {"took":5,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":1,"max_score":0.49305484,"hits":[{"_index":"myindex","_type":"mytype","id":"AVeoqjX-bv4eY7Nx-xL","_score":0.49305484,"_source":{"pk":14,"user_pk":10,"media_pk":2,"topic_pk":0,"access_source_pk":0,"p_channel_pk":0,"c_channel_pk":0,"title":"ghdfdfgddfg","cover_url":{"S":["http://ochz1y8tu.bkt.clouddn.com/lg-6Q2Cy2yjF_SqLSo5YaKJKx2E520160829102601_2.jpg"],"L":["http://ochz1y8tu.bkt.clouddn.com/lg-6Q2Cy2yjF_SqLSo5YaKJKx2E520160829102601_1.jpg"]},"summary":""好奇号"发回火星最新影像 地貌酷似地球","read_count":0,"like_count":0,"share_count":0,"favorites_count":0,"check_status":3,"release_status":1,"isurgent":1,"created_at":"2016-08-15T14:07:19.000+08:00","updated_at":"2016-10-04T10:11:56.000+08:00"}}]}}

测试192.168.31.86节点对数据修改后,两个节点的数据是否同步

[zsz@zsz ~]$ curl -XPOST '192.168.31.86:9200/myindex/mytype/AVeoqjX_-bv4eY7Nx-xL/_update?pretty' -d '{
"doc": { "summary ": "TEST UPDATE DATA" } }' { "_index" : "myindex", "type" : "AVeoqjX-bv4eY7Nx-xL", "_id" : "_update", "_version" : 1, "_shards" : { "total" : 2, "successful" : 2, "failed" : 0 }, "created" : true }

[zsz@zsz ~]$ curl '192.168.31.86:9200/myindex/_search?q=ghdfdfgddfg' {"took":5,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":1,"max_score":0.44561866,"hits":[{"_index":"myindex","_type":"mytype","id":"AVeoqjX-bv4eY7Nx-xL","_score":0.44561866,"_source":{"pk":14,"user_pk":10,"media_pk":2,"topic_pk":0,"access_source_pk":0,"p_channel_pk":0,"c_channel_pk":0,"title":"ghdfdfgddfg","cover_url":{"S":["http://ochz1y8tu.bkt.clouddn.com/lg-6Q2Cy2yjF_SqLSo5YaKJKx2E520160829102601_2.jpg"],"L":["http://ochz1y8tu.bkt.clouddn.com/lg-6Q2Cy2yjF_SqLSo5YaKJKx2E520160829102601_1.jpg"]},"summary":""好奇号"发回火星最新影像 地貌酷似地球","read_count":0,"like_count":0,"share_count":0,"favorites_count":0,"check_status":3,"release_status":1,"isurgent":1,"created_at":"2016-08-15T14:07:19.000+08:00","updated_at":"2016-10-04T10:11:56.000+08:00","summary ":"TEST UPDATE DATA"}}]}}

[zsz@zsz ~]$ curl '192.168.31.87:9200/myindex/_search?q=ghdfdfgddfg' {"took":2,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":1,"max_score":0.44561866,"hits":[{"_index":"myindex","_type":"mytype","id":"AVeoqjX-bv4eY7Nx-xL","_score":0.44561866,"_source":{"pk":14,"user_pk":10,"media_pk":2,"topic_pk":0,"access_source_pk":0,"p_channel_pk":0,"c_channel_pk":0,"title":"ghdfdfgddfg","cover_url":{"S":["http://ochz1y8tu.bkt.clouddn.com/lg-6Q2Cy2yjF_SqLSo5YaKJKx2E520160829102601_2.jpg"],"L":["http://ochz1y8tu.bkt.clouddn.com/lg-6Q2Cy2yjF_SqLSo5YaKJKx2E520160829102601_1.jpg"]},"summary":""好奇号"发回火星最新影像 地貌酷似地球","read_count":0,"like_count":0,"share_count":0,"favorites_count":0,"check_status":3,"release_status":1,"isurgent":1,"created_at":"2016-08-15T14:07:19.000+08:00","updated_at":"2016-10-04T10:11:56.000+08:00","summary ":"TEST UPDATE DATA"}}]}}

在192.168.31.86机器上更新的数据,分别查询192.168.31.86和192.168.31.87,发觉数据是同步了的。elasticsearch集群自我完成了实时同步。

常见问题:

1、问题报错:no rute to host

解决方案:由于端口没开启,处理办法 (1)关闭服务器防火墙,不推荐。 (2)开放端口: /sbin/iptables -I INPUT -p tcp --dport 9200 -j ACCEPT /sbin/iptables -I INPUT -p tcp --dport 9300 -j ACCEPT /etc/rc.d/init.d/iptables save /etc/rc.d/init.d/iptables restart /etc/init.d/iptables status

2、max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]

每个进程最大同时打开文件数太小,可通过下面2个命令查看当前数量

ulimit -Hn ulimit -Sn   修改/etc/security/limits.conf文件,增加配置,用户退出后重新登录生效

*               soft    nofile          65536
*               hard    nofile          65536

3、max number of threads [3818] for user [es] is too low, increase to at least [4096]

问题同上,最大线程个数太低。修改配置文件/etc/security/limits.conf,增加配置

*               soft    nproc           4096
*               hard    nproc           4096

可通过命令查看

ulimit -Hu ulimit -Su

4、max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

修改/etc/sysctl.conf文件,增加配置vm.max_map_count=262144

vi /etc/sysctl.conf sysctl -p   执行命令sysctl -p生效

5、Exception in thread "main" java.nio.file.AccessDeniedException: /usr/local/elasticsearch/elasticsearch-6.2.2-1/config/jvm.options

elasticsearch用户没有该文件夹的权限,执行命令

chown -R es:es /usr/local/elasticsearch/

详细配置

https://www.cnblogs.com/zlslch/p/6419948.html

入门级案例

image