ElasticSearch安装go-mysql-elasticsearch同步mysql的数据

OStack程序员社区-中国程序员成长平台 › 门户 › 编程› Go›Go教程

原作者: [db:作者] 来自: [db:来源] 收藏邀请

一、首先在Centos6.5上安装 go 语言环境

　　下载Golang语言包：https://studygolang.com/dl

1 [hoojjack@localhost src]$ ls
2 apache-maven-3.5.0  go  go1.6.2.linux-amd64.tar.gz

二、解压.tar.gz安装包。

三、配置安装环境。

[hoojjack@localhost etc]$ vim profile
export GOPATH=/usr/local/src/go/work
export GOROOT=/usr/local/src/go
export PATH=$GOPATH/bin:$PATH

GOARCH（指定系统环境，i386表示x86，amd64表示x64）：amd64

GOROOT：/usr/local/src/go（go的解压路径）

GOBIN：%GOROOT%/bin（exe执行文件路径）

GOOS：（go运行的系统）

GOPATH：/usr/local/src/go/work（用于存放Go语言Package的目录，这个目录不能在Go的安装目录中，放了好像也没什么事，至少我目前没事 ^_^）

我们需要添加的环境变量分别是go的安装路径GOROOT：/usr/local/src/go，PATH：$GOROOT/bin:$PATH，go的工作路径GOPATH:/usr/local/src/go/work

四、配置go环境后配置go-mysql-elasticsearch

首先确保环境中安装了git

安装依赖包：

sudo yum install gettext-devel openssl-devel perl-CPAN perl-devel zlib-devel

也可以直接用yum命令安装：

yum install git

然后：

获取安装包：go get github.com/siddontang/go-mysql-elasticsearch

安装：

cd $GOPATH/src/github.com/siddontang/go-mysql-elasticsearch
make

1 [hoojjack@localhost etc]$ cd /usr/local/src/go/src/github.com/siddontang/go-mysql-elasticsearch
2 [hoojjack@localhost go-mysql-elasticsearch]$ ls
3 bin  cmd  elastic  etc  Godeps  LICENSE  Makefile  README.md  river  var
4 [hoojjack@localhost go-mysql-elasticsearch]$

五、修改配置文件

 1 # MySQL address, user and password
 2 # user must have replication privilege in MySQL.
 3 my_addr = "127.0.0.1:3306"     //需要同步的mysql基本设置
 4 my_user = "root"
 5 my_pass = "root"
 6 
 7 # Elasticsearch address
 8 es_addr = "127.0.0.1:9200"     //本地elasticsearch配置
 9 
10 # Path to store data, like master.info, and dump MySQL data 
11 data_dir = "./var"             //数据存储的url
12  //以下配置保存默认不变
13 # Inner Http status address
14 stat_addr = "127.0.0.1:12800"
15 
16 # pseudo server id like a slave 
17 server_id = 1001
18 
19 # mysql or mariadb
20 flavor = "mysql"
21 # mysqldump execution path
22 mysqldump = "mysqldump"
23 
24 # MySQL data source
25 [[source]]
26 schema = "hoojjack"             //elasticsearch 与 mysql 同步时对应的数据库名称
27 
28 # Only below tables will be synced into Elasticsearch.
29 # "test_river_[0-9]{4}" is a wildcard table format, you can use it if you have many sub tables, like table_0000 - table_1023
30 # I don't think it is necessary to sync all tables in a database.
31 tables = ["test_river", "test_river_[0-9]{4}"]    //支持通配符，可以指定只复制hoojjack数据库中指定的表数据
32 
33 # Below is for special rule mapping
34 [[rule]]
35 schema = "hoojjack"    //数据库名称
36 table = "test_river"   //表名称
37 index = "river"        //对应的索引名称
38 # title is MySQL test_river field name, es_title is the customized name in Elasticsearch
39   [rule.field]
40 # This will map column title to elastic search my_title
41   title="es_title"     //将可以将mysql的某个属性对应指向elasticsearch的某个field, 如test_river的titile属性对应es_title
42 # This will map column tags to elastic search my_tags and use array type
43   tags="my_tags,list"
44 # This will map column keywords to elastic search keywords and use array type
45   keywords=",list"
46 
47 # wildcard table rule, the wildcard table must be in source tables 
48   [[rule]]
49   schema = "hoojjack"
50   table = "test_river_[0-9]{4}"
51   index = "river"
52   type = "river"
53 
54 # title is MySQL test_river field name, es_title is the customized name in Elasticsearch
55   [[rule.fields]]
56   mysql = "title"
57   elastic = "es_title"

六、执行go-mysql-elasticsearch同步，首先进入 go-mysql-elasticsearch 路径，然后执行： ./bin/go-mysql-elasticsearch -config=./etc/river.toml

1 cd $GOPATH/src/github.com/siddontang/go-mysql-elasticsearch 
2 ./bin/go-mysql-elasticsearch -config=./etc/river.toml

效果如下：

mysql> select * from test_river;
+----+-----------+-----------------+--------+
| id | name      | title           | number |
+----+-----------+-----------------+--------+
|  1 | hoojjack  | wo shi yi bing  |      1 |
|  2 | xunchuan  | ni shi er ren   |      2 |

七、出现的问题：

1、mysqldump: command not found

mysql的mysqldump命令没有找到，解决办法：创建软连接，查看mysqldump是在哪个路径下：find / -name mysqldump 会获得路径，然后创建软连接：ln -s /usr/local/mysql/bin/mysqldump /usr/bin

这是由于系统默认会查找/usr/bin下的命令，如果这个命令不在这个目录下，当然会找不到命令

如果没有安装mysql，则可以利用yum安装mysql5.6

1）检查系统是否安装其他版本的MYSQL数据

#yum list installed | grep mysql
#yum -y remove mysql-libs.x86_64

2）安装及配置

# wget http://repo.mysql.com/mysql-community-release-el6-5.noarch.rpm
# rpm -ivh mysql-community-release-el6-5.noarch.rpm
# yum repolist all | grep mysql

3）安装MYSQL数据库