1. 原理介绍
Swift 架构、原理及功能： http://www.cnblogs.com/sammyliu/p/4955241.html
总结的很详细也很全面，受益匪浅，感谢分享。
2. keystone + swift + dashboard 安装和配置
2.1 基础环境介绍
vmware 11.0
系统：rhel7.2
openstack版本：openstack-Mitaka
swift-controller: 192.168.0.11
swift-object1: 192.168.0.51
swift-object2: 192.168.0.52
注意：这里的第一张网卡nat作为内网网卡，第二张网卡桥接作为外网网卡（使用一张桥接网卡可实现，这里是为了ip规范使用nat网络）
/etc/hosts 192.168.0.11 controller 192.168.0.51 object1 192.168.0.52 object2
时间同步，三节点都执行：
# ntpdate tiger.sina.com.cn
2.2 安装初始化
swift-controller配置：
# yum install python-openstackclient mariadb mariadb-server python2-PyMySQL rabbitmq-server memcached python-memcached -y
配置数据库
# vim /etc/my.cnf.d/openstack.cnf [mysqld] bind-address = 192.168.0.11 default-storage-engine = innodb innodb_file_per_table max_connections = 4096 collation-server = utf8_general_ci character-set-server = utf8
启动及初始化数据库
# systemctl enable mariadb ; systemctl start mariadb # mysql_secure_installation # 密码 123456
启动消息队列 rabbitmq服务
# systemctl enable rabbitmq-server ; systemctl start rabbitmq-server
添加用户openstack的用户并赋权
# rabbitmqctl add_user openstack openstack # rabbitmqctl set_permissions openstack ".*" ".*" ".*"
启动memcache服务
# systemctl enable memcached ; systemctl start memcached
查看服务启动情况
3306: mariadb 11211：memcache 2567：rabbitmq # netstat -ntplu | egrep "3306|11211|2567" tcp 0 0 0.0.0.0:25672 0.0.0.0:* LISTEN 20546/beam.smp tcp 0 0 192.168.0.11:3306 0.0.0.0:* LISTEN 20412/mysqld tcp 0 0 127.0.0.1:11211 0.0.0.0:* LISTEN 21084/memcached tcp6 0 0 ::1:11211 :::* LISTEN 21084/memcached udp 0 0 127.0.0.1:11211 0.0.0.0:* 21084/memcached udp6 0 0 ::1:11211 :::* 21084/memcached
2.3 keystone安装配置
创建keystone数据库并授权
# mysql -p123456 MariaDB [(none)]> create database keystone; MariaDB [(none)]> GRANT ALL PRIVILEGES ON keystone.* TO \'keystone\'@\'localhost\' IDENTIFIED BY \'keystone\'; MariaDB [(none)]> GRANT ALL PRIVILEGES ON keystone.* TO \'keystone\'@\'%\' IDENTIFIED BY \'keystone\';
安装程序包
# yum install openstack-keystone httpd mod_wsgi -y
配置keystone.conf
# vim /etc/keystone/keystone.conf [DEFAULT] ... admin_token = 2b64e54cdce5900a22f8 ... [database] ... connection = mysql+pymysql://keystone:keystone@controller/keystone ... [token] ... provider = fernet ...
初始化 keystone 数据库
# su -s /bin/sh -c "keystone-manage db_sync" keystone
初始化Fernet keys：
# keystone-manage fernet_setup --keystone-user keystone --keystone-group keystone
配置apache
# vim /etc/httpd/conf/httpd.conf
...
ServerName controller
...
配置openstack使用的虚拟主机：
# vim /etc/httpd/conf.d/wsgi-keystone.conf Listen 5000 Listen 35357 <VirtualHost *:5000> WSGIDaemonProcess keystone-public processes=5 threads=1 user=keystone group=keystone display-name=%{GROUP} WSGIProcessGroup keystone-public WSGIScriptAlias / /usr/bin/keystone-wsgi-public WSGIApplicationGroup %{GLOBAL} WSGIPassAuthorization On ErrorLogFormat "%{cu}t %M" ErrorLog /var/log/httpd/keystone-error.log CustomLog /var/log/httpd/keystone-access.log combined <Directory /usr/bin> Require all granted </Directory> </VirtualHost> <VirtualHost *:35357> WSGIDaemonProcess keystone-admin processes=5 threads=1 user=keystone group=keystone display-name=%{GROUP} WSGIProcessGroup keystone-admin WSGIScriptAlias / /usr/bin/keystone-wsgi-admin WSGIApplicationGroup %{GLOBAL} WSGIPassAuthorization On ErrorLogFormat "%{cu}t %M" ErrorLog /var/log/httpd/keystone-error.log CustomLog /var/log/httpd/keystone-access.log combined <Directory /usr/bin> Require all granted </Directory> </VirtualHost>
配置完成，启动服务
# systemctl enable httpd ; systemctl start httpd
配置认证令牌、端点URL、api版本
# export OS_TOKEN=2b64e54cdce5900a22f8 # export OS_URL=http://controller:35357/v3 # export OS_IDENTITY_API_VERSION=3
（1）创建keystone服务
# openstack service create --name keystone --description "OpenStack Identity" identity
（2）创建api端点
# openstack endpoint create --region RegionOne identity public http://controller:5000/v3 # openstack endpoint create --region RegionOne identity internal http://controller:5000/v3 # openstack endpoint create --region RegionOne identity admin http://controller:35357/v3
（3）创建域
# openstack domain create --description "Default Domain" default
（4）创建项目
# openstack project create --domain default --description "Admin Project" admin
（5）创建用户（admin密码：admin）
# openstack user create --domain default --password-prompt admin
（6）创建角色
# openstack role create admin
（7）添加角色到项目和用户上
# openstack role add --project admin --user admin admin
（8）创建 service 项目
# openstack project create --domain default --description "Service Project" service
验证：
# unset OS_TOKEN OS_URL # openstack --os-auth-url http://controller:35357/v3 \ --os-project-domain-name default --os-user-domain-name default \ --os-project-name admin --os-username admin token issue # 密码上面设置： admin
创建认证脚本
# vim admin-openrc export OS_PROJECT_DOMAIN_NAME=default export OS_USER_DOMAIN_NAME=default export OS_PROJECT_NAME=admin export OS_USERNAME=admin export OS_PASSWORD=admin export OS_AUTH_URL=http://controller:35357/v3 export OS_IDENTITY_API_VERSION=3 export OS_IMAGE_API_VERSION=2
测试认证
# . admin-openrc
# openstack token issue
2.4 对象存储 Swift 安装和配置
2.4.1 swift-controller 配置
# . admin-openrc
（1）创建 swift用户，给swift用户添加admin角色
# openstack user create --domain default --password-prompt swift
密码：swift
# openstack role add --project service --user swift admin
（2）创建swift服务
# openstack service create --name swift --description "OpenStack Object Storage" object-store
（3）创建对象存储服务 API 端点
# openstack endpoint create --region RegionOne object-store public http://controller:8080/v1/AUTH_%\(tenant_id\)s # openstack endpoint create --region RegionOne object-store internal http://controller:8080/v1/AUTH_%\(tenant_id\)s # openstack endpoint create --region RegionOne object-store admin http://controller:8080/v1
安装 swift 程序包
yum install openstack-swift-proxy python-swiftclient python-keystoneclient python-keystonemiddleware memcached -y
获取代理服务的配置文件：
# curl -o /etc/swift/proxy-server.conf https://git.openstack.org/cgit/openstack/swift/plain/etc/proxy-server.conf-sample
修改如下内容：
# vim /etc/swift/proxy-server.conf [DEFAULT] bind_port = 8080 user = swift swift_dir = /etc/swift ... [pipeline:main] pipeline = catch_errors gatekeeper healthcheck proxy-logging cache container_sync bulk ratelimit authtoken keystoneauth container-quotas account-quotas slo dlo versioned_writes proxy-logging proxy-server ... [app:proxy-server] use = egg:swift#proxy account_autocreate = True ... [filter:keystoneauth] use = egg:swift#keystoneauth operator_roles = admin,user ... [filter:authtoken] paste.filter_factory = keystonemiddleware.auth_token:filter_factory auth_uri = http://controller:5000 auth_url = http://controller:35357 memcached_servers = 127.0.0.1:11211 auth_type = password project_domain_name = default user_domain_name = default project_name = service username = swift password = swift delay_auth_decision = True ... [filter:cache] use = egg:swift#memcache memcache_servers = 127.0.0.1:11211
2.4.2 swift-object 配置
以下操作在 object1 和 object2 上执行
swift-object1: 192.168.0.51 swift-object2: 192.168.0.52
（1）安装支持工具包
# yum install xfsprogs rsync openstack-swift-account openstack-swift-container openstack-swift-object -y
（2）使用 xfs 格式化磁盘
# mkfs.xfs /dev/sdb # mkfs.xfs /dev/sdc # mkfs.xfs /dev/sdd
（3）创建挂载点
# mkdir -pv /srv/node/sd{b,c,d}
（4）编辑 /etc/fstab 添加挂载内容
# vim /etc/fstab ... /dev/sdb /srv/node/sdb xfs noatime,nodiratime,nobarrier,logbufs=8 0 2 /dev/sdc /srv/node/sdc xfs noatime,nodiratime,nobarrier,logbufs=8 0 2 /dev/sdd /srv/node/sdd xfs noatime,nodiratime,nobarrier,logbufs=8 0 2 # mount -a
（5）创建并编写 /etc/rsyncd.conf 文件
# vim /etc/rsyncd.conf uid = swift gid = swift log file = /var/log/rsyncd.log pid file = /var/run/rsyncd.pid address = 192.168.0.51 # object1为 192.168.0.51 object2为 192.168.0.52 [account] max connections = 2 path = /srv/node/ read only = False lock file = /var/lock/account.lock [container] max connections = 2 path = /srv/node/ read only = False lock file = /var/lock/container.lock [object] max connections = 2 path = /srv/node/ read only = False lock file = /var/lock/object.lock
（6）启动 rsyncd 服务并开机启动
# systemctl enable rsyncd.service ; systemctl start rsyncd.service
从仓库下载account、container、object、swift配置文件
# curl -o /etc/swift/account-server.conf https://git.openstack.org/cgit/openstack/swift/plain/etc/account-server.conf-sample # curl -o /etc/swift/container-server.conf https://git.openstack.org/cgit/openstack/swift/plain/etc/container-server.conf-sample # curl -o /etc/swift/object-server.conf https://git.openstack.org/cgit/openstack/swift/plain/etc/object-server.conf-sample # curl -o /etc/swift/swift.conf https://git.openstack.org/cgit/openstack/swift/plain/etc/swift.conf-sample
（1）配置 /etc/swift/account-server.conf
# vim /etc/swift/account-server.conf [DEFAULT] bind_ip = 192.168.0.51 # object1为 192.168.0.51 object2为 192.168.0.52 bind_port = 6002 user = swift swift_dir = /etc/swift devices = /srv/node mount_check = True ... [pipeline:main] pipeline = healthcheck recon account-server ... [filter:recon] use = egg:swift#recon ... recon_cache_path = /var/cache/swift
（2）配置 /etc/swift/container-server.conf
# vim /etc/swift/container-server.conf [DEFAULT] bind_ip = 192.168.0.51 # object1为 192.168.0.51 object2为 192.168.0.52 bind_port = 6001 user = swift swift_dir = /etc/swift devices = /srv/node mount_check = True ... [pipeline:main] pipeline = healthcheck recon container-server ... [filter:recon] use = egg:swift#recon ... recon_cache_path = /var/cache/swift
（3）配置 /etc/swift/object-server.conf
# vim /etc/swift/object-server.conf [DEFAULT] bind_ip = 192.168.0.51 # object1为 192.168.0.51 object2为 192.168.0.52 bind_port = 6000 user = swift swift_dir = /etc/swift devices = /srv/node mount_check = True ... [pipeline:main] pipeline = healthcheck recon object-server ... [filter:recon] use = egg:swift#recon recon_cache_path = /var/cache/swift recon_lock_path = /var/lock
确认挂载点目录结构是否有合适的所有权
# chown -R swift:swift /srv/node # mkdir -p /var/cache/swift # chown -R root:swift /var/cache/swift # chmod -R 775 /var/cache/swift
2.4.3 创建，分发并初始化rings
以下操作在 swift-container 上执行
swift-controller: 192.168.0.11
创建账户ring
切换到 /etc/swift 目录。
# cd /etc/swift/
创建account ring 文件
（1）创建基本 account.builder 文件
# swift-ring-builder account.builder create 10 3 1
（2）添加每个节点到 ring 中：
object1: 192.168.0.51 # swift-ring-builder account.builder add --region 1 --zone 1 --ip 192.168.0.51 --port 6002 --device sdb --weight 100 # swift-ring-builder account.builder add --region 1 --zone 1 --ip 192.168.0.51 --port 6002 --device sdc --weight 100 # swift-ring-builder account.builder add --region 1 --zone 1 --ip 192.168.0.51 --port 6002 --device sdd --weight 100 object2: 192.168.0.52 # swift-ring-builder account.builder add --region 1 --zone 2 --ip 192.168.0.52 --port 6002 --device sdb --weight 100 # swift-ring-builder account.builder add --region 1 --zone 2 --ip 192.168.0.52 --port 6002 --device sdc --weight 100 # swift-ring-builder account.builder add --region 1 --zone 2 --ip 192.168.0.52 --port 6002 --device sdd --weight 100
（3）验证 ring 内容
# swift-ring-builder account.builder
（4）平衡 ring
# swift-ring-builder account.builder rebalance
创建container ring 文件
（1）创建基本 container.builder 文件
# swift-ring-builder container.builder create 10 3 1
（2）添加每个节点到 ring 中：
object1: 192.168.0.51 # swift-ring-builder container.builder add --region 1 --zone 1 --ip 192.168.0.51 --port 6001 --device sdb --weight 100 # swift-ring-builder container.builder add --region 1 --zone 1 --ip 192.168.0.51 --port 6001 --device sdc --weight 100 # swift-ring-builder container.builder add --region 1 --zone 1 --ip 192.168.0.51 --port 6001 --device sdd --weight 100 object2: 192.168.0.52 # swift-ring-builder container.builder add --region 1 --zone 2 --ip 192.168.0.52 --port 6001 --device sdb --weight 100 # swift-ring-builder container.builder add --region 1 --zone 2 --ip 192.168.0.52 --port 6001 --device sdc --weight 100 # swift-ring-builder container.builder add --region 1 --zone 2 --ip 192.168.0.52 --port 6001 --device sdd --weight 100
（3）验证 ring 内容
# swift-ring-builder container.builder
（4）平衡 ring
# swift-ring-builder container.builder rebalance
创建 object ring 文件
（1）创建基本 container.builder 文件
# swift-ring-builder object.builder create 10 3 1
（2）添加每个节点到 ring 中：
# swift-ring-builder object.builder add --region 1 --zone 1 --ip 192.168.0.51 --port 6000 --device sdb --weight 100 # swift-ring-builder object.builder add --region 1 --zone 1 --ip 192.168.0.51 --port 6000 --device sdc --weight 100 # swift-ring-builder object.builder add --region 1 --zone 1 --ip 192.168.0.51 --port 6000 --device sdd --weight 100 object2: 192.168.0.52 # swift-ring-builder object.builder add --region 1 --zone 2 --ip 192.168.0.52 --port 6000 --device sdb --weight 100 # swift-ring-builder object.builder add --region 1 --zone 2 --ip 192.168.0.52 --port 6000 --device sdc --weight 100 # swift-ring-builder object.builder add --region 1 --zone 2 --ip 192.168.0.52 --port 6000 --device sdd --weight 100
（3）验证 ring 内容
# swift-ring-builder object.builder
（4）平衡 ring
# swift-ring-builder object.builder rebalance
将ring配置文件拷贝到每个存储节点，这里也就是 object1 和 object2
# scp *.ring.gz object1:/etc/swift/ # scp *.ring.gz object2:/etc/swift/
配置 /etc/swift/swift.conf 文件
# vim /etc/swift/swift.conf [swift-hash] ... swift_hash_path_suffix = mickey swift_hash_path_prefix = minnie ...
复制 /etc/swift/swift.conf 到其他object节点和代理节点，这里也就是 object1 和 object2
# scp swift.conf object1:/etc/swift/ # scp swift.conf object2:/etc/swift/
确认三个节点配置文件权限，在所有节点执行；
# chown -R root:swift /etc/swift
2.4.4 启动 swift 服务
（1）swift-controller 节点启动服务
# systemctl enable openstack-swift-proxy.service memcached.service; systemctl start openstack-swift-proxy.service memcached.service
问题1：在启动 openstack-swift-proxy.service 服务后，查看启动信息
# systemctl status -l openstack-swift-proxy.service Jan 22 22:08:30 controller liberasurecode[15717]: liberasurecode_backend_open: dynamic linking error libisal.so.2: cannot open shared object file: No such file or directory Jan 22 22:08:30 controller liberasurecode[15717]: liberasurecode_backend_open: dynamic linking error libshss.so.1: cannot open shared object file: No such file or directory
缺少库文件，需要编译安装：
# yum install gcc gcc-c++ make automake autoconf libtool yasm -y # cd /usr/local/src/ # wget https://codeload.github.com/01org/isa-l/zip/master # unzip isa-l-master.zip # ./autogen.sh # ./configure --prefix=/usr --libdir=/usr/lib64 # make -j 2 && make install # systemctl restart openstack-swift-proxy.service # systemctl status -l openstack-swift-proxy.service Jan 22 22:16:55 controller liberasurecode[23647]: liberasurecode_backend_open: dynamic linking error libJerasure.so.2: cannot open shared object file: No such file or directory Jan 22 22:16:55 controller liberasurecode[23647]: liberasurecode_backend_open: dynamic linking error libshss.so.1: cannot open shared object file: No such file or directory
liberasurecode-1.1.0.tar.gz下载地址： https://bitbucket.org/tsg-/liberasurecode/downloads/
升级liberasurecode版本
# tar xf liberasurecode-1.1.0.tar.gz # ./autogen.sh # ./configure # make -j 2 && make install 强制卸载低版本 # rpm -e --nodeps liberasurecode # systemctl restart openstack-swift-proxy.service # systemctl status -l openstack-swift-proxy.service
再无报错信息
（2）swift-object 节点启动服务
swift-object1: 192.168.0.51 swift-object2: 192.168.0.52
升级liberasurecode版本，安装libisal
python-six 包必须安装，否则服务启动失败
python-six 包必须安装，否则服务启动失败 # yum install gcc gcc-c++ make automake autoconf libtool yasm python-six -y # cd /usr/local/src/ # wget https://codeload.github.com/01org/isa-l/zip/master # unzip isa-l-master.zip # cd isa-l-master # ./autogen.sh # ./configure --prefix=/usr --libdir=/usr/lib64 # make -j 2 && make install # tar xf liberasurecode-1.1.0.tar.gz # cd liberasurecode-1.1.0 # ./autogen.sh # ./configure # make -j 2 && make install 强制卸载低版本 # rpm -e --nodeps liberasurecode
启动服务
# systemctl enable openstack-swift-account.service openstack-swift-account-auditor.service \ openstack-swift-account-reaper.service openstack-swift-account-replicator.service # systemctl start openstack-swift-account.service openstack-swift-account-auditor.service \ openstack-swift-account-reaper.service openstack-swift-account-replicator.service # systemctl enable openstack-swift-container.service \ openstack-swift-container-auditor.service openstack-swift-container-replicator.service \ openstack-swift-container-updater.service # systemctl start openstack-swift-container.service \ openstack-swift-container-auditor.service openstack-swift-container-replicator.service \ openstack-swift-container-updater.service # systemctl enable openstack-swift-object.service openstack-swift-object-auditor.service \ openstack-swift-object-replicator.service openstack-swift-object-updater.service # systemctl start openstack-swift-object.service openstack-swift-object-auditor.service \ openstack-swift-object-replicator.service openstack-swift-object-updater.service
问题2：启动 openstack-swift-object-replicator.service 报错
Jan 22 22:40:09 object2 object-replicator[25374]: ERROR creating /srv/node/sdb/objects: #012Traceback (most recent call last):#012... Jan 22 22:40:09 object2 object-replicator[25374]: ERROR creating /srv/node/sdc/objects: #012Traceback (most recent call last):#012... Jan 22 22:40:09 object2 object-replicator[25374]: ERROR creating /srv/node/sdd/objects: #012Traceback (most recent call last):#012...
检查 /srv/node/ 权限
# ll -d /srv/node/ drwxr-xr-x 5 root root 36 Jan 22 21:24 /srv/node/ # chown -R swift:swift /srv/node # mkdir -p /var/cache/swift # chown -R root:swift /var/cache/swift # chmod -R 775 /var/cache/swift
重启 openstack-swift-object-replicator.service 恢复正常
2.4.5 验证 swift 服务
在 swift-controller 节点执行：
# . admin-openrc # swift stat Account: AUTH_3b6f963488db4af49e4e0c0d095dd6cf Containers: 0 Objects: 0 Bytes: 0 X-Put-Timestamp: 1516632330.04818 X-Timestamp: 1516632330.04818 X-Trans-Id: tx5fa7f5e817714cbca5cca-005a65f909 Content-Type: text/plain; charset=utf-
创建一个测试文件，上传该测试文件到 container1 容器中
创建一个测试文件 # touch testfile 上传该测试文件到 container1 容器中 # swift upload container1 testfile testfile 查看该文件 # swift list container1 testfile 下载该文件 # swift download container1 testfile testfile [auth 0.343s, headers 0.625s, total 0.626s, 0.000 MB/s]
swift 服务测试成功
2.5 安装 dashboard 服务
在 swift-controller 节点执行：
swift-controller: 192.168.0.11
# yum install openstack-dashboard -y
编辑配置文件 /etc/openstack-dashboard/local_settings
# vim /etc/openstack-dashboard/local_settings 修改以下内容 ... OPENSTACK_HOST = "controller" ... ALLOWED_HOSTS = [\'*\', ] ... SESSION_ENGINE = \'django.contrib.sessions.backends.cache\' CACHES = { \'default\': { \'BACKEND\': \'django.core.cache.backends.memcached.MemcachedCache\', \'LOCATION\': \'127.0.0.1:11211\', } } ... OPENSTACK_KEYSTONE_URL = "http://%s:5000/v3" % OPENSTACK_HOST ... OPENSTACK_KEYSTONE_MULTIDOMAIN_SUPPORT = True ... OPENSTACK_API_VERSIONS = { "identity": 3, "image": 2, "volume": 2, } ... OPENSTACK_KEYSTONE_DEFAULT_DOMAIN = "default" ... OPENSTACK_KEYSTONE_DEFAULT_ROLE = "user" ... TIME_ZONE = "Asia/Shanghai"
配置完成，重启服务
# systemctl restart httpd.service memcached.service
浏览器登录：
域： default
用户名： admin
密码： admin
请发表评论