在线时间:8:00-16:00
迪恩网络APP
随时随地掌握行业动态
扫描二维码
关注迪恩网络微信公众号
本文首发http://f.dataguru.cn/thread-217843-1-1.html。
一、下载安装thrift 下载thrift-0.9.1.tar.gz安装包 使用tar -xvf thrift-0.9.1.tar.gz解压该包 cd thrift-0.9.1 按照官方文档上的instal thrift 下载相应依赖包,地址为: http://thrift.apache.org/docs/install/ubuntu/ 执行如下命令 : sudo apt-get install libboost-dev libboost-test-dev libboost-program-options-dev libevent-dev automake libtool flex bison pkg-config g++ libssl-dev
增加perl的依赖包: sudo apt-get install libbit-vector-perl
在~/.bashrc中添加新的环境变量(为以后好编译Perl) export THRIFT_HOME=/home/grid/thrift-0.9.1
对thrift进行安装 ./configure
grid@h01:~/thrift-0.9.1$ ./configure checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for a thread-safe mkdir -p... /bin/mkdir -p checking for gawk... no checking for mawk... mawk checking whether make sets $(MAKE)... yes checking how to create a ustar tar archive... gnutar checking for pkg-config... /usr/bin/pkg-config checking pkg-config is at least version 0.9.0... yes checking for gcc... Gcc ....
thrift 0.9.1
Building C++ Library ......... : yes Building C (GLib) Library .... : yes Building Java Library ........ : no Building C# Library .......... : no Building Python Library ...... : yes Building Ruby Library ........ : no Building Haskell Library ..... : no Building Perl Library ........ : yes Building PHP Library ......... : no Building Erlang Library ...... : no Building Go Library .......... : no Building D Library ........... : no
C++ Library: Build TZlibTransport ...... : yes Build TNonblockingServer .. : yes Build TQTcpServer (Qt) .... : no
Python Library: Using Python .............. : /usr/bin/python
Perl Library: Using Perl ................ : /usr/bin/perl
If something is missing that you think should be present, please skim the output of configure to find the missing component. Details are present in config.log. grid@h01:~/thrift-0.9.1$ make&make install [1] 41403 make all-recursive Making install in compiler/cpp make[1]: Entering directory `/home/grid/thrift-0.9.1' Making all in compiler/cpp make[2]: Entering directory `/home/grid/thrift-0.9.1/compiler/cpp' make[1]: Entering directory `/home/grid/thrift-0.9.1/compiler/cpp' make install-am make all-am make[3]: Entering directory `/home/grid/thrift-0.9.1/compiler/cpp' make[2]: Entering directory `/home/grid/thrift-0.9.1/compiler/cpp' g++ -DHAVE_CONFIG_H -I. -I../.. -I../../lib/cpp/src/thrift -I./src -Wall -g -O2 -MT thrift-main.o -MD -MP -MF .deps/thrift-main.Tpo -c -o thrift-main.o `test -f 'src/main.cc' || echo './'`src/main.cc g++ -DHAVE_CONFIG_H -I. -I../.. -I../../lib/cpp/src/thrift -I./src -Wall -g -O2 -MT thrift-main.o -MD -MP -MF .deps/thrift-main.Tpo -c -o thrift-main.o `test -f 'src/main.cc' || echo './'`src/main.cc mv -f .deps/thrift-main.Tpo .deps/thrift-main.Po mv -f .deps/thrift-main.Tpo .deps/thrift-main.Po g++ -DHAVE_CONFIG_H -I. -I../.. -I../../lib/cpp/src/thrift -I./src -Wall -g -O2 -MT thrift-t_c_glib_generator.o -MD -MP -MF .deps/thrift-t_c_glib_generator.Tpo -c -o thrift-t_c_glib_generator.o `test -f 'src/generate/t_c_glib_generator.cc' || echo './'`src/generate/t_c_glib_generator.cc mv: cannot stat `.deps/thrift-main.Tpo': No such file or directory make[3]: *** [thrift-main.o] Error 1 make[3]: Leaving directory `/home/grid/thrift-0.9.1/compiler/cpp' make[2]: *** [all] Error 2 make[2]: Leaving directory `/home/grid/thrift-0.9.1/compiler/cpp' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/home/grid/thrift-0.9.1' make: *** [all] Error 2 mv -f .deps/thrift-t_c_glib_generator.Tpo .deps/thrift-t_c_glib_generator.Po g++ -DHAVE_CONFIG_H -I. -I../.. -I../../lib/cpp/src/thrift -I./src -Wall -g -O2 -MT thrift-t_cpp_generator.o -MD -MP -MF .deps/thrift-t_cpp_generator.Tpo -c -o thrift-t_cpp_generator.o `test -f 'src/generate/t_cpp_generator.cc' || echo './'`src/generate/t_cpp_generator.cc mv -f .deps/thrift-t_cpp_generator.Tpo .deps/thrift-t_cpp_generator.Po g++ -DHAVE_CONFIG_H -I. -I../.. -I../../lib/cpp/src/thrift -I./src -Wall -g -O2 -MT thrift-t_java_generator.o -MD -MP -MF .deps/thrift-t_java_generator.Tpo -c -o thrift-t_java_generator.o `test -f 'src/generate/t_java_generator.cc' || echo './'`src/generate/t_java_generator.cc mv -f .deps/thrift-t_java_generator.Tpo .deps/thrift-t_java_generator.Po g++ -DHAVE_CONFIG_H -I. -I../.. -I../../lib/cpp/src/thrift -I./src -Wall -g -O2 -MT thrift-t_as3_generator.o -MD -MP -MF .deps/thrift-t_as3_generator.Tpo -c -o thrift-t_as3_generator.o `test -f 'src/generate/t_as3_generator.cc' || echo './'`src/generate/t_as3_generator.cc mv -f .deps/thrift-t_as3_generator.Tpo .deps/thrift-t_as3_generator.Po g++ -DHAVE_CONFIG_H -I. -I../.. -I../../lib/cpp/src/thrift -I./src -Wall -g -O2 -MT thrift-t_csharp_generator.o -MD -MP -MF .deps/thrift-t_csharp_generator.Tpo -c -o thrift-t_csharp_generator.o `test -f 'src/generate/t_csharp_generator.cc' || echo './'`src/generate/t_csharp_generator.cc mv -f .deps/thrift-t_csharp_generator.Tpo .deps/thrift-t_csharp_generator.Po g++ -DHAVE_CONFIG_H -I. -I../.. -I../../lib/cpp/src/thrift -I./src -Wall -g -O2 -MT thrift-t_py_generator.o -MD -MP -MF .deps/thrift-t_py_generator.Tpo -c -o thrift-t_py_generator.o `test -f 'src/generate/t_py_generator.cc' || echo './'`src/generate/t_py_generator.cc mv -f .deps/thrift-t_py_generator.Tpo .deps/thrift-t_py_generator.Po g++ -DHAVE_CONFIG_H -I. -I../.. -I../../lib/cpp/src/thrift -I./src -Wall -g -O2 -MT thrift-t_rb_generator.o -MD -MP -MF .deps/thrift-t_rb_generator.Tpo -c -o thrift-t_rb_generator.o `test -f 'src/generate/t_rb_generator.cc' || echo './'`src/generate/t_rb_generator.cc mv -f .deps/thrift-t_rb_generator.Tpo .deps/thrift-t_rb_generator.Po g++ -DHAVE_CONFIG_H -I. -I../.. -I../../lib/cpp/src/thrift -I./src -Wall -g -O2 -MT thrift-t_perl_generator.o -MD -MP -MF .deps/thrift-t_perl_generator.Tpo -c -o thrift-t_perl_generator.o `test -f 'src/generate/t_perl_generator.cc' || echo './'`src/generate/t_perl_generator.cc mv -f .deps/thrift-t_perl_generator.Tpo .deps/thrift-t_perl_generator.Po g++ -DHAVE_CONFIG_H -I. -I../.. -I../../lib/cpp/src/thrift -I./src -Wall -g -O2 -MT thrift-t_php_generator.o -MD -MP -MF .deps/thrift-t_php_generator.Tpo -c -o thrift-t_php_generator.o `test -f 'src/generate/t_php_generator.cc' || echo './'`src/generate/t_php_generator.cc mv -f .deps/thrift-t_php_generator.Tpo .deps/thrift-t_php_generator.Po g++ -DHAVE_CONFIG_H -I. -I../.. -I../../lib/cpp/src/thrift -I./src -Wall -g -O2 -MT thrift-t_erl_generator.o -MD -MP -MF .deps/thrift-t_erl_generator.Tpo -c -o thrift-t_erl_generator.o `test -f 'src/generate/t_erl_generator.cc' || echo './'`src/generate/t_erl_generator.cc mv -f .deps/thrift-t_erl_generator.Tpo .deps/thrift-t_erl_generator.Po g++ -DHAVE_CONFIG_H -I. -I../.. -I../../lib/cpp/src/thrift -I./src -Wall -g -O2 -MT thrift-t_cocoa_generator.o -MD -MP -MF .deps/thrift-t_cocoa_generator.Tpo -c -o thrift-t_cocoa_generator.o `test -f 'src/generate/t_cocoa_generator.cc' || echo './'`src/generate/t_cocoa_generator.cc mv -f .deps/thrift-t_cocoa_generator.Tpo .deps/thrift-t_cocoa_generator.Po g++ -DHAVE_CONFIG_H -I. -I../.. -I../../lib/cpp/src/thrift -I./src -Wall -g -O2 -MT thrift-t_st_generator.o -MD -MP -MF .deps/thrift-t_st_generator.Tpo -c -o thrift-t_st_generator.o `test -f 'src/generate/t_st_generator.cc' || echo './'`src/generate/t_st_generator.cc mv -f .deps/thrift-t_st_generator.Tpo .deps/thrift-t_st_generator.Po g++ -DHAVE_CONFIG_H -I. -I../.. -I../../lib/cpp/src/thrift -I./src -Wall -g -O2 -MT thrift-t_ocaml_generator.o -MD -MP -MF .deps/thrift-t_ocaml_generator.Tpo -c -o thrift-t_ocaml_generator.o `test -f 'src/generate/t_ocaml_generator.cc' || echo './'`src/generate/t_ocaml_generator.cc mv -f .deps/thrift-t_ocaml_generator.Tpo .deps/thrift-t_ocaml_generator.Po g++ -DHAVE_CONFIG_H -I. -I../.. -I../../lib/cpp/src/thrift -I./src -Wall -g -O2 -MT thrift-t_hs_generator.o -MD -MP -MF .deps/thrift-t_hs_generator.Tpo -c -o thrift-t_hs_generator.o `test -f 'src/generate/t_hs_generator.cc' || echo './'`src/generate/t_hs_generator.cc mv -f .deps/thrift-t_hs_generator.Tpo .deps/thrift-t_hs_generator.Po g++ -DHAVE_CONFIG_H -I. -I../.. -I../../lib/cpp/src/thrift -I./src -Wall -g -O2 -MT thrift-t_js_generator.o -MD -MP -MF .deps/thrift-t_js_generator.Tpo -c -o thrift-t_js_generator.o `test -f 'src/generate/t_js_generator.cc' || echo './'`src/generate/t_js_generator.cc mv -f .deps/thrift-t_js_generator.Tpo .deps/thrift-t_js_generator.Po g++ -DHAVE_CONFIG_H -I. -I../.. -I../../lib/cpp/src/thrift -I./src -Wall -g -O2 -MT thrift-t_javame_generator.o -MD -MP -MF .deps/thrift-t_javame_generator.Tpo -c -o thrift-t_javame_generator.o `test -f 'src/generate/t_javame_generator.cc' || echo './'`src/generate/t_javame_generator.cc mv -f .deps/thrift-t_javame_generator.Tpo .deps/thrift-t_javame_generator.Po g++ -DHAVE_CONFIG_H -I. -I../.. -I../../lib/cpp/src/thrift -I./src -Wall -g -O2 -MT thrift-t_delphi_generator.o -MD -MP -MF .deps/thrift-t_delphi_generator.Tpo -c -o thrift-t_delphi_generator.o `test -f 'src/generate/t_delphi_generator.cc' || echo './'`src/generate/t_delphi_generator.cc mv -f .deps/thrift-t_delphi_generator.Tpo .deps/thrift-t_delphi_generator.Po g++ -DHAVE_CONFIG_H -I. -I../.. -I../../lib/cpp/src/thrift -I./src -Wall -g -O2 -MT thrift-t_go_generator.o -MD -MP -MF .deps/thrift-t_go_generator.Tpo -c -o thrift-t_go_generator.o `test -f 'src/generate/t_go_generator.cc' || echo './'`src/generate/t_go_generator.cc mv -f .deps/thrift-t_go_generator.Tpo .deps/thrift-t_go_generator.Po g++ -DHAVE_CONFIG_H -I. -I../.. -I../../lib/cpp/src/thrift -I./src -Wall -g -O2 -MT thrift-t_d_generator.o -MD -MP -MF .deps/thrift-t_d_generator.Tpo -c -o thrift-t_d_generator.o `test -f 'src/generate/t_d_generator.cc' || echo './'`src/generate/t_d_generator.cc mv -f .deps/thrift-t_d_generator.Tpo .deps/thrift-t_d_generator.Po /bin/bash ../../libtool --tag=CXX --mode=link g++ -Wall -g -O2 -o thrift thrift-main.o thrift-md5.o thrift-t_generator.o thrift-parse.o thrift-t_c_glib_generator.o thrift-t_cpp_generator.o thrift-t_java_generator.o thrift-t_as3_generator.o thrift-t_csharp_generator.o thrift-t_py_generator.o thrift-t_rb_generator.o thrift-t_perl_generator.o thrift-t_php_generator.o thrift-t_erl_generator.o thrift-t_cocoa_generator.o thrift-t_st_generator.o thrift-t_ocaml_generator.o thrift-t_hs_generator.o thrift-t_xsd_generator.o thrift-t_html_generator.o thrift-t_js_generator.o thrift-t_javame_generator.o thrift-t_delphi_generator.o thrift-t_go_generator.o thrift-t_gv_generator.o thrift-t_d_generator.o -lfl libparse.a libtool: link: g++ -Wall -g -O2 -o thrift thrift-main.o thrift-md5.o thrift-t_generator.o thrift-parse.o thrift-t_c_glib_generator.o thrift-t_cpp_generator.o thrift-t_java_generator.o thrift-t_as3_generator.o thrift-t_csharp_generator.o thrift-t_py_generator.o thrift-t_rb_generator.o thrift-t_perl_generator.o thrift-t_php_generator.o thrift-t_erl_generator.o thrift-t_cocoa_generator.o thrift-t_st_generator.o thrift-t_ocaml_generator.o thrift-t_hs_generator.o thrift-t_xsd_generator.o thrift-t_html_generator.o thrift-t_js_generator.o thrift-t_javame_generator.o thrift-t_delphi_generator.o thrift-t_go_generator.o thrift-t_gv_generator.o thrift-t_d_generator.o -lfl libparse.a make[3]: Entering directory `/home/grid/thrift-0.9.1/compiler/cpp' test -z "/usr/local/bin" || /bin/mkdir -p "/usr/local/bin" /bin/bash ../../libtool --mode=install /usr/bin/install -c thrift '/usr/local/bin' libtool: install: /usr/bin/install -c thrift /usr/local/bin/thrift make[3]: Nothing to be done for `install-data-am'. make[3]: Leaving directory `/home/grid/thrift-0.9.1/compiler/cpp' make[2]: Leaving directory `/home/grid/thrift-0.9.1/compiler/cpp' make[1]: Leaving directory `/home/grid/thrift-0.9.1/compiler/cpp' Making install in lib make[1]: Entering directory `/home/grid/thrift-0.9.1/lib' Making install in cpp make[2]: Entering directory `/home/grid/thrift-0.9.1/lib/cpp' Making install in . make[3]: Entering directory `/home/grid/thrift-0.9.1/lib/cpp' make[4]: Entering directory `/home/grid/thrift-0.9.1/lib/cpp' test -z "/usr/local/lib" || /bin/mkdir -p "/usr/local/lib" /bin/bash ../../libtool --mode=install /usr/bin/install -c libthrift.la libthriftnb.la libthriftz.la '/usr/local/lib' libtool: install: /usr/bin/install -c .libs/libthrift-0.9.1.so /usr/local/lib/libthrift-0.9.1.so /usr/bin/install: cannot remove `/usr/local/lib/libthrift-0.9.1.so': Permission denied make[4]: *** [install-libLTLIBRARIES] Error 1 make[4]: Leaving directory `/home/grid/thrift-0.9.1/lib/cpp' make[3]: *** [install-am] Error 2 make[3]: Leaving directory `/home/grid/thrift-0.9.1/lib/cpp' make[2]: *** [install-recursive] Error 1 make[2]: Leaving directory `/home/grid/thrift-0.9.1/lib/cpp' make[1]: *** [install-recursive] Error 1 make[1]: Leaving directory `/home/grid/thrift-0.9.1/lib' make: *** [install-recursive] Error 1 [1]+ Exit 2 make grid@h01:~$ thrift -version Thrift version 0.9.1
这个过程中出现了一些目录没有找到和权限不足的情况,有同学说是没有执行安装目录,尝试过在.configure 后加./configure --prefix=/home/grid/thrift也没有生效,欢迎朋友指教怎么设置参数规避这个问题。
二、用Perl进行连接Hbase 生成先关文件: thrift --gen perl \ /home/grid/hbase-0.94.13/src/main/resources/org/apache/hadoop/hbase/thrift/Hbase.thrift 可以看到生成了相应的Perl包 grid@h01:~/thrift-0.9.1/gen-perl/Hbase$ ls Constants.pm Hbase.pm Types.pm 接着设置环境变量(这里表示Hbase Perl环境变量的意思,哈哈) Export HP_HOME=~/thrift-0.9.1/gen-perl/
启动hadoop和hbase/ grid@h01:~/wangn$ jps 7622 HQuorumPeer 3612 DataNode 4196 TaskTracker 3341 NameNode 35516 HMaster 35785 HRegionServer 3957 JobTracker 42656 ThriftServer 49545 Jps 3856 SecondaryNameNode
基于这个目录里的Demo来完成实验,这里面的例子相对来说应该是最全的,是很好的学习材料。11 grid@h01:~/hbase-0.94.13/src/examples/thrift$ ls DemoClient.cpp DemoClient.java DemoClient.php DemoClient.pl DemoClient.py DemoClient.rb Makefile README.txt Cp DemoClient.pl hclient.pl
在hclient.pl中添加红色部分。 use strict; use warnings;
use Env qw(HP_HOME); use Env qw(THRIFT_HOME);
use lib "$HP_HOME/lib"; use lib "$THRIFT_HOME/lib/perl/lib";
use Thrift::Socket; use Thrift::BufferedTransport; use Thrift::BinaryProtocol; use Hbase::Hbase; use Data:umper;
执行perl hclient.pl 此时有可能会得到一个: Base class package "Class::Accessor" is empty.的异常,说明缺少依赖的包。 下载 :http://search.cpan.org/~kasei/Class-Accessor-0.34/lib/Class/Accessor.pm 将其中的lib中的Class目录放 在@INC的目录中,此时编译应该没有问题了。
此时有可能会得到如下错误: column families in {demo_table}: column: {entry:}, maxVer: {10} column: {unused:}, maxVer: {3} Can't use string ("shouldn't get here! at hclient.p"...) as a HASH ref while "strict refs" in use at hclient.pl line 166. 表示作者的的一个假设没有成立,可能和hbase的版本有关。 我们注释162行:#die ("shouldn't get here!";
从新执行 perl hclient.pl perl hclient.pl scanning tables... found {demo_table} disabling table: {demo_table} deleting table: {demo_table} found {member} found {person} found {test01} creating table: {demo_table}
。。。。 会列出我们hbase里的表和创建新表插入数据,扫描数据等操作。 |
请发表评论