• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    公众号

对象存储 - Swift 原理 及 Swift+keystone+dashboard 架构搭建

原作者: [db:作者] 来自: [db:来源] 收藏 邀请

1. 原理介绍

   Swift 架构、原理及功能: http://www.cnblogs.com/sammyliu/p/4955241.html

 总结的很详细也很全面,受益匪浅,感谢分享。

2. keystone + swift + dashboard 安装和配置

2.1 基础环境介绍

    vmware 11.0
    系统:rhel7.2
    openstack版本:openstack-Mitaka

 

   swift-controller:    192.168.0.11
    swift-object1:      192.168.0.51
    swift-object2:      192.168.0.52

 

注意:这里的第一张网卡nat作为内网网卡,第二张网卡桥接作为外网网卡(使用一张桥接网卡可实现,这里是为了ip规范使用nat网络)

/etc/hosts
192.168.0.11    controller
192.168.0.51    object1
192.168.0.52    object2

时间同步,三节点都执行:

# ntpdate tiger.sina.com.cn

2.2 安装初始化

swift-controller配置:

# yum install python-openstackclient mariadb mariadb-server python2-PyMySQL rabbitmq-server memcached python-memcached -y

配置数据库

# vim /etc/my.cnf.d/openstack.cnf

[mysqld]
bind-address = 192.168.0.11
default-storage-engine = innodb
innodb_file_per_table
max_connections = 4096
collation-server = utf8_general_ci
character-set-server = utf8

启动及初始化数据库

# systemctl enable mariadb ; systemctl start mariadb
# mysql_secure_installation        # 密码 123456

启动消息队列 rabbitmq服务

# systemctl enable rabbitmq-server ; systemctl start rabbitmq-server

添加用户openstack的用户并赋权

# rabbitmqctl add_user openstack openstack
# rabbitmqctl set_permissions openstack ".*" ".*" ".*"

启动memcache服务

# systemctl enable memcached ; systemctl start memcached

查看服务启动情况

3306: mariadb
11211:memcache
2567:rabbitmq

# netstat -ntplu | egrep "3306|11211|2567"
tcp        0      0 0.0.0.0:25672           0.0.0.0:*               LISTEN      20546/beam.smp      
tcp        0      0 192.168.0.11:3306       0.0.0.0:*               LISTEN      20412/mysqld        
tcp        0      0 127.0.0.1:11211         0.0.0.0:*               LISTEN      21084/memcached     
tcp6       0      0 ::1:11211               :::*                    LISTEN      21084/memcached     
udp        0      0 127.0.0.1:11211         0.0.0.0:*                           21084/memcached     
udp6       0      0 ::1:11211               :::*                                21084/memcached    

2.3 keystone安装配置

创建keystone数据库并授权

# mysql -p123456
MariaDB [(none)]> create database keystone;
MariaDB [(none)]> GRANT ALL PRIVILEGES ON keystone.* TO \'keystone\'@\'localhost\' IDENTIFIED BY \'keystone\';
MariaDB [(none)]> GRANT ALL PRIVILEGES ON keystone.* TO \'keystone\'@\'%\' IDENTIFIED BY \'keystone\';

安装程序包

# yum install openstack-keystone httpd mod_wsgi -y

配置keystone.conf

# vim /etc/keystone/keystone.conf

[DEFAULT]
...
admin_token = 2b64e54cdce5900a22f8
...
[database]
...
connection = mysql+pymysql://keystone:keystone@controller/keystone
...
[token]
...
provider = fernet
...

初始化 keystone 数据库

# su -s /bin/sh -c "keystone-manage db_sync" keystone

初始化Fernet keys:

# keystone-manage fernet_setup --keystone-user keystone --keystone-group keystone

配置apache

# vim /etc/httpd/conf/httpd.conf
...
ServerName controller
...

配置openstack使用的虚拟主机:

# vim /etc/httpd/conf.d/wsgi-keystone.conf

Listen 5000
Listen 35357

<VirtualHost *:5000>
    WSGIDaemonProcess keystone-public processes=5 threads=1 user=keystone group=keystone display-name=%{GROUP}
    WSGIProcessGroup keystone-public
    WSGIScriptAlias / /usr/bin/keystone-wsgi-public
    WSGIApplicationGroup %{GLOBAL}
    WSGIPassAuthorization On
    ErrorLogFormat "%{cu}t %M"
    ErrorLog /var/log/httpd/keystone-error.log
    CustomLog /var/log/httpd/keystone-access.log combined

    <Directory /usr/bin>
        Require all granted
    </Directory>
</VirtualHost>

<VirtualHost *:35357>
    WSGIDaemonProcess keystone-admin processes=5 threads=1 user=keystone group=keystone display-name=%{GROUP}
    WSGIProcessGroup keystone-admin
    WSGIScriptAlias / /usr/bin/keystone-wsgi-admin
    WSGIApplicationGroup %{GLOBAL}
    WSGIPassAuthorization On
    ErrorLogFormat "%{cu}t %M"
    ErrorLog /var/log/httpd/keystone-error.log
    CustomLog /var/log/httpd/keystone-access.log combined

    <Directory /usr/bin>
        Require all granted
    </Directory>
</VirtualHost>

配置完成,启动服务

# systemctl enable httpd ; systemctl start httpd

配置认证令牌、端点URL、api版本

# export OS_TOKEN=2b64e54cdce5900a22f8
# export OS_URL=http://controller:35357/v3
# export OS_IDENTITY_API_VERSION=3

(1)创建keystone服务

# openstack service create --name keystone --description "OpenStack Identity" identity

(2)创建api端点

# openstack endpoint create --region RegionOne identity public http://controller:5000/v3
# openstack endpoint create --region RegionOne identity internal http://controller:5000/v3
# openstack endpoint create --region RegionOne identity admin http://controller:35357/v3

(3)创建域

# openstack domain create --description "Default Domain" default

(4)创建项目

# openstack project create --domain default   --description "Admin Project" admin

(5)创建用户(admin密码:admin)

# openstack user create --domain default   --password-prompt admin

(6)创建角色

# openstack role create admin

(7)添加角色到项目和用户上

# openstack role add --project admin --user admin admin

(8)创建 service 项目

# openstack project create --domain default   --description "Service Project" service

验证:

# unset OS_TOKEN OS_URL
# openstack --os-auth-url http://controller:35357/v3 \
--os-project-domain-name default --os-user-domain-name default \
--os-project-name admin --os-username admin token issue
# 密码上面设置: admin

 

创建认证脚本

# vim admin-openrc

export OS_PROJECT_DOMAIN_NAME=default
export OS_USER_DOMAIN_NAME=default
export OS_PROJECT_NAME=admin
export OS_USERNAME=admin
export OS_PASSWORD=admin
export OS_AUTH_URL=http://controller:35357/v3
export OS_IDENTITY_API_VERSION=3
export OS_IMAGE_API_VERSION=2

测试认证

# . admin-openrc
# openstack token issue

2.4 对象存储 Swift 安装和配置

2.4.1 swift-controller 配置
# . admin-openrc

(1)创建 swift用户,给swift用户添加admin角色

# openstack user create --domain default --password-prompt swift
密码:swift
# openstack role add --project service --user swift admin

(2)创建swift服务

# openstack service create --name swift   --description "OpenStack Object Storage" object-store

(3)创建对象存储服务 API 端点

# openstack endpoint create --region RegionOne   object-store public http://controller:8080/v1/AUTH_%\(tenant_id\)s
# openstack endpoint create --region RegionOne   object-store internal http://controller:8080/v1/AUTH_%\(tenant_id\)s
# openstack endpoint create --region RegionOne   object-store admin http://controller:8080/v1

安装 swift 程序包

yum install openstack-swift-proxy python-swiftclient python-keystoneclient python-keystonemiddleware memcached -y

获取代理服务的配置文件:

# curl -o /etc/swift/proxy-server.conf https://git.openstack.org/cgit/openstack/swift/plain/etc/proxy-server.conf-sample

 

修改如下内容:

# vim /etc/swift/proxy-server.conf
[DEFAULT]
bind_port = 8080
user = swift
swift_dir = /etc/swift
...
[pipeline:main]
pipeline = catch_errors gatekeeper healthcheck proxy-logging cache container_sync bulk ratelimit authtoken keystoneauth container-quotas account-quotas slo dlo versioned_writes proxy-logging proxy-server
...
[app:proxy-server]
use = egg:swift#proxy
account_autocreate = True
...
[filter:keystoneauth]
use = egg:swift#keystoneauth
operator_roles = admin,user
...
[filter:authtoken]
paste.filter_factory = keystonemiddleware.auth_token:filter_factory
auth_uri = http://controller:5000
auth_url = http://controller:35357
memcached_servers = 127.0.0.1:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = swift
password = swift
delay_auth_decision = True
...
[filter:cache]
use = egg:swift#memcache
memcache_servers = 127.0.0.1:11211

 

2.4.2 swift-object 配置

以下操作在 object1 和 object2 上执行

    swift-object1:      192.168.0.51
    swift-object2:      192.168.0.52

(1)安装支持工具包

# yum install xfsprogs rsync openstack-swift-account openstack-swift-container openstack-swift-object -y

(2)使用 xfs 格式化磁盘

# mkfs.xfs /dev/sdb
# mkfs.xfs /dev/sdc
# mkfs.xfs /dev/sdd

(3)创建挂载点

# mkdir -pv /srv/node/sd{b,c,d}

(4)编辑 /etc/fstab 添加挂载内容

# vim /etc/fstab
...
/dev/sdb /srv/node/sdb xfs noatime,nodiratime,nobarrier,logbufs=8 0 2
/dev/sdc /srv/node/sdc xfs noatime,nodiratime,nobarrier,logbufs=8 0 2
/dev/sdd /srv/node/sdd xfs noatime,nodiratime,nobarrier,logbufs=8 0 2
# mount -a

(5)创建并编写 /etc/rsyncd.conf 文件

# vim /etc/rsyncd.conf
uid = swift
gid = swift
log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
address = 192.168.0.51        # object1为 192.168.0.51 object2为 192.168.0.52

[account]
max connections = 2
path = /srv/node/
read only = False
lock file = /var/lock/account.lock

[container]
max connections = 2
path = /srv/node/
read only = False
lock file = /var/lock/container.lock

[object]
max connections = 2
path = /srv/node/
read only = False
lock file = /var/lock/object.lock

(6)启动 rsyncd 服务并开机启动

# systemctl enable rsyncd.service ; systemctl start rsyncd.service

从仓库下载account、container、object、swift配置文件

# curl -o /etc/swift/account-server.conf https://git.openstack.org/cgit/openstack/swift/plain/etc/account-server.conf-sample
# curl -o /etc/swift/container-server.conf https://git.openstack.org/cgit/openstack/swift/plain/etc/container-server.conf-sample
# curl -o /etc/swift/object-server.conf https://git.openstack.org/cgit/openstack/swift/plain/etc/object-server.conf-sample
# curl -o /etc/swift/swift.conf https://git.openstack.org/cgit/openstack/swift/plain/etc/swift.conf-sample

(1)配置 /etc/swift/account-server.conf

# vim /etc/swift/account-server.conf

[DEFAULT]
bind_ip = 192.168.0.51        # object1为 192.168.0.51 object2为 192.168.0.52
bind_port = 6002
user = swift
swift_dir = /etc/swift
devices = /srv/node
mount_check = True
...
[pipeline:main]
pipeline = healthcheck recon account-server
...
[filter:recon]
use = egg:swift#recon
...
recon_cache_path = /var/cache/swift

(2)配置 /etc/swift/container-server.conf

# vim /etc/swift/container-server.conf
[DEFAULT]
bind_ip = 192.168.0.51        # object1为 192.168.0.51 object2为 192.168.0.52
bind_port = 6001
user = swift
swift_dir = /etc/swift
devices = /srv/node
mount_check = True
...
[pipeline:main]
pipeline = healthcheck recon container-server
...
[filter:recon]
use = egg:swift#recon
...
recon_cache_path = /var/cache/swift

(3)配置 /etc/swift/object-server.conf

# vim /etc/swift/object-server.conf
[DEFAULT]
bind_ip = 192.168.0.51        # object1为 192.168.0.51 object2为 192.168.0.52
bind_port = 6000
user = swift
swift_dir = /etc/swift
devices = /srv/node
mount_check = True
...
[pipeline:main]
pipeline = healthcheck recon object-server
...
[filter:recon]
use = egg:swift#recon
recon_cache_path = /var/cache/swift
recon_lock_path = /var/lock

确认挂载点目录结构是否有合适的所有权

# chown -R swift:swift /srv/node
# mkdir -p /var/cache/swift
# chown -R root:swift /var/cache/swift
# chmod -R 775 /var/cache/swift
2.4.3 创建,分发并初始化rings

以下操作在 swift-container 上执行

    swift-controller: 192.168.0.11

创建账户ring

切换到 /etc/swift 目录。

# cd /etc/swift/

创建account ring 文件

(1)创建基本 account.builder 文件

# swift-ring-builder account.builder create 10 3 1

(2)添加每个节点到 ring 中:

object1: 192.168.0.51
# swift-ring-builder account.builder add --region 1 --zone 1 --ip 192.168.0.51 --port 6002 --device sdb --weight 100
# swift-ring-builder account.builder add --region 1 --zone 1 --ip 192.168.0.51 --port 6002 --device sdc --weight 100
# swift-ring-builder account.builder add --region 1 --zone 1 --ip 192.168.0.51 --port 6002 --device sdd --weight 100
object2: 192.168.0.52
# swift-ring-builder account.builder add --region 1 --zone 2 --ip 192.168.0.52 --port 6002 --device sdb --weight 100
# swift-ring-builder account.builder add --region 1 --zone 2 --ip 192.168.0.52 --port 6002 --device sdc --weight 100
# swift-ring-builder account.builder add --region 1 --zone 2 --ip 192.168.0.52 --port 6002 --device sdd --weight 100

(3)验证 ring 内容

# swift-ring-builder account.builder

(4)平衡 ring

# swift-ring-builder account.builder rebalance

创建container ring 文件

(1)创建基本 container.builder 文件

# swift-ring-builder container.builder create 10 3 1

(2)添加每个节点到 ring 中:

object1: 192.168.0.51
# swift-ring-builder container.builder add --region 1 --zone 1 --ip 192.168.0.51 --port 6001 --device sdb --weight 100
# swift-ring-builder container.builder add --region 1 --zone 1 --ip 192.168.0.51 --port 6001 --device sdc --weight 100
# swift-ring-builder container.builder add --region 1 --zone 1 --ip 192.168.0.51 --port 6001 --device sdd --weight 100
object2: 192.168.0.52
# swift-ring-builder container.builder add --region 1 --zone 2 --ip 192.168.0.52 --port 6001 --device sdb --weight 100
# swift-ring-builder container.builder add --region 1 --zone 2 --ip 192.168.0.52 --port 6001 --device sdc --weight 100
# swift-ring-builder container.builder add --region 1 --zone 2 --ip 192.168.0.52 --port 6001 --device sdd --weight 100

(3)验证 ring 内容

# swift-ring-builder container.builder

(4)平衡 ring

# swift-ring-builder container.builder rebalance

创建 object ring 文件

(1)创建基本 container.builder 文件

# swift-ring-builder object.builder create 10 3 1

(2)添加每个节点到 ring 中:

# swift-ring-builder object.builder add --region 1 --zone 1 --ip 192.168.0.51 --port 6000 --device sdb --weight 100
# swift-ring-builder object.builder add --region 1 --zone 1 --ip 192.168.0.51 --port 6000 --device sdc --weight 100
# swift-ring-builder object.builder add --region 1 --zone 1 --ip 192.168.0.51 --port 6000 --device sdd --weight 100
object2: 192.168.0.52
# swift-ring-builder object.builder add --region 1 --zone 2 --ip 192.168.0.52 --port 6000 --device sdb --weight 100
# swift-ring-builder object.builder add --region 1 --zone 2 --ip 192.168.0.52 --port 6000 --device sdc --weight 100
# swift-ring-builder object.builder add --region 1 --zone 2 --ip 192.168.0.52 --port 6000 --device sdd --weight 100

(3)验证 ring 内容

# swift-ring-builder object.builder

(4)平衡 ring

# swift-ring-builder object.builder rebalance

将ring配置文件拷贝到每个存储节点,这里也就是 object1 和 object2

# scp *.ring.gz object1:/etc/swift/
# scp *.ring.gz object2:/etc/swift/

配置 /etc/swift/swift.conf 文件

# vim /etc/swift/swift.conf
[swift-hash]
...
swift_hash_path_suffix = mickey
swift_hash_path_prefix = minnie
...

复制 /etc/swift/swift.conf 到其他object节点和代理节点,这里也就是 object1 和 object2

# scp swift.conf object1:/etc/swift/
# scp swift.conf object2:/etc/swift/

确认三个节点配置文件权限,在所有节点执行;

# chown -R root:swift /etc/swift
2.4.4 启动 swift 服务

(1)swift-controller 节点启动服务

# systemctl enable openstack-swift-proxy.service memcached.service; systemctl start openstack-swift-proxy.service memcached.service

问题1:在启动 openstack-swift-proxy.service 服务后,查看启动信息

# systemctl status -l  openstack-swift-proxy.service

Jan 22 22:08:30 controller liberasurecode[15717]: liberasurecode_backend_open: dynamic linking error libisal.so.2: cannot open shared object file: No such file or directory
Jan 22 22:08:30 controller liberasurecode[15717]: liberasurecode_backend_open: dynamic linking error libshss.so.1: cannot open shared object file: No such file or directory

缺少库文件,需要编译安装:

# yum install gcc gcc-c++ make automake autoconf libtool yasm -y
# cd /usr/local/src/
# wget https://codeload.github.com/01org/isa-l/zip/master
# unzip isa-l-master.zip
# ./autogen.sh
# ./configure --prefix=/usr --libdir=/usr/lib64
# make -j 2 && make install
# systemctl restart openstack-swift-proxy.service
# systemctl status -l openstack-swift-proxy.service
Jan 22 22:16:55 controller liberasurecode[23647]: liberasurecode_backend_open: dynamic linking error libJerasure.so.2: cannot open shared object file: No such file or directory
Jan 22 22:16:55 controller liberasurecode[23647]: liberasurecode_backend_open: dynamic linking error libshss.so.1: cannot open shared object file: No such file or directory

liberasurecode-1.1.0.tar.gz下载地址:   https://bitbucket.org/tsg-/liberasurecode/downloads/

升级liberasurecode版本

# tar xf liberasurecode-1.1.0.tar.gz
# ./autogen.sh
# ./configure
# make -j 2 && make install
强制卸载低版本
# rpm -e --nodeps  liberasurecode
# systemctl restart openstack-swift-proxy.service
# systemctl status -l  openstack-swift-proxy.service 

再无报错信息

(2)swift-object 节点启动服务

    swift-object1:      192.168.0.51
    swift-object2:      192.168.0.52

升级liberasurecode版本,安装libisal
python-six 包必须安装,否则服务启动失败

python-six 包必须安装,否则服务启动失败
# yum install gcc gcc-c++ make automake autoconf libtool yasm python-six -y
# cd /usr/local/src/
# wget https://codeload.github.com/01org/isa-l/zip/master
# unzip isa-l-master.zip
# cd isa-l-master
# ./autogen.sh
# ./configure --prefix=/usr --libdir=/usr/lib64
# make -j 2 && make install
# tar xf liberasurecode-1.1.0.tar.gz
# cd liberasurecode-1.1.0
# ./autogen.sh
# ./configure
# make -j 2 && make install
强制卸载低版本
# rpm -e --nodeps  liberasurecode

启动服务

# systemctl enable openstack-swift-account.service openstack-swift-account-auditor.service \
  openstack-swift-account-reaper.service openstack-swift-account-replicator.service
# systemctl start openstack-swift-account.service openstack-swift-account-auditor.service \
  openstack-swift-account-reaper.service openstack-swift-account-replicator.service
# systemctl enable openstack-swift-container.service \
  openstack-swift-container-auditor.service openstack-swift-container-replicator.service \
  openstack-swift-container-updater.service
# systemctl start openstack-swift-container.service \
  openstack-swift-container-auditor.service openstack-swift-container-replicator.service \
  openstack-swift-container-updater.service
# systemctl enable openstack-swift-object.service openstack-swift-object-auditor.service \
  openstack-swift-object-replicator.service openstack-swift-object-updater.service
# systemctl start openstack-swift-object.service openstack-swift-object-auditor.service \
  openstack-swift-object-replicator.service openstack-swift-object-updater.service

问题2:启动 openstack-swift-object-replicator.service 报错

Jan 22 22:40:09 object2 object-replicator[25374]: ERROR creating /srv/node/sdb/objects: #012Traceback (most recent call last):#012...
Jan 22 22:40:09 object2 object-replicator[25374]: ERROR creating /srv/node/sdc/objects: #012Traceback (most recent call last):#012...
Jan 22 22:40:09 object2 object-replicator[25374]: ERROR creating /srv/node/sdd/objects: #012Traceback (most recent call last):#012...

检查 /srv/node/ 权限

# ll -d /srv/node/
drwxr-xr-x 5 root root 36 Jan 22 21:24 /srv/node/

# chown -R swift:swift /srv/node
# mkdir -p /var/cache/swift
# chown -R root:swift /var/cache/swift
# chmod -R 775 /var/cache/swift

重启 openstack-swift-object-replicator.service 恢复正常

2.4.5 验证 swift 服务

在 swift-controller 节点执行:

# . admin-openrc
# swift stat
        Account: AUTH_3b6f963488db4af49e4e0c0d095dd6cf
     Containers: 0
        Objects: 0
          Bytes: 0
X-Put-Timestamp: 1516632330.04818
    X-Timestamp: 1516632330.04818
     X-Trans-Id: tx5fa7f5e817714cbca5cca-005a65f909
   Content-Type: text/plain; charset=utf-

创建一个测试文件,上传该测试文件到 container1 容器中

创建一个测试文件
# touch testfile

上传该测试文件到 container1 容器中
# swift upload container1 testfile 
testfile

查看该文件
# swift list container1 
testfile

下载该文件
# swift download container1 testfile 
testfile [auth 0.343s, headers 0.625s, total 0.626s, 0.000 MB/s]

 

swift 服务测试成功

2.5 安装 dashboard 服务

在 swift-controller 节点执行:

    swift-controller: 192.168.0.11
# yum install openstack-dashboard -y

编辑配置文件 /etc/openstack-dashboard/local_settings

# vim /etc/openstack-dashboard/local_settings
修改以下内容
...
OPENSTACK_HOST = "controller"
...
ALLOWED_HOSTS = [\'*\', ]
...
SESSION_ENGINE = \'django.contrib.sessions.backends.cache\'
CACHES = {
    \'default\': {
         \'BACKEND\': \'django.core.cache.backends.memcached.MemcachedCache\',
         \'LOCATION\': \'127.0.0.1:11211\',
    }
}
...
OPENSTACK_KEYSTONE_URL = "http://%s:5000/v3" % OPENSTACK_HOST
...
OPENSTACK_KEYSTONE_MULTIDOMAIN_SUPPORT = True
...
OPENSTACK_API_VERSIONS = {
    "identity": 3,
    "image": 2,
    "volume": 2,
}
...
OPENSTACK_KEYSTONE_DEFAULT_DOMAIN = "default"
...
OPENSTACK_KEYSTONE_DEFAULT_ROLE = "user"
...
TIME_ZONE = "Asia/Shanghai"

配置完成,重启服务

# systemctl restart httpd.service memcached.service

浏览器登录:
    域:         default
    用户名:  admin
    密码:      admin

 


鲜花

握手

雷人

路过

鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
上一篇:
Swift基础使用方法(Swift开发之中的一个)发布时间:2022-07-13
下一篇:
Swift - Spritekit 实现游戏摇杆发布时间:2022-07-13
热门推荐
热门话题
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服(服务时间 9:00~18:00)

在线QQ客服
地址:深圳市南山区西丽大学城创智工业园
电邮:jeky_zhao#qq.com
移动电话:139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap