Swift Installation
http://docs.openstack.org/developer/swift/howto_installmultinode.html#config-proxy
http://docs.openstack.org/trunk/openstack-compute/install/apt/content/ch_installing-openstack-object-storage.html
Add Repository
先加入 Openstack Folsom 的 PPA 路徑
root@swift-proxy:~$ apt-get install -y python-software-properties root@swift-proxy:~$ add-apt-repository ppa:openstack-ubuntu-testing/folsom-trunk-testing root@swift-proxy:~$ add-apt-repository ppa:openstack-ubuntu-testing/folsom-deps-staging root@swift-proxy:~$ apt-get update && apt-get -y dist-upgrade
Basic Package Installation
在 proxy-server 及所有的 storage server 都要裝這些 package, 並且 swift.conf 內的 hash 值要相同
root@swift-proxy:~$ apt-get install swift openssh-server rsync memcached python-netifaces python-xattr python-memcache python-swiftclient root@swift-proxy:~$ mkdir -p /etc/swift root@swift-proxy:~$ chown -R swift:swift /etc/swift/ root@swift-proxy:~$ cat swift.conf [swift-hash] # random unique string that can never change (DO NOT LOSE). I’m using 03c9f48da2229770. # od -t x8 -N 8 -A n < /dev/random # The above command can be used to generate random a string. swift_hash_path_suffix = 34c486c41efd7f62 root@swift-storage:/etc/swift$ dpkg -l | grep swift ii python-swift 1.7.1+git201209042100~precise-0ubuntu1 distributed virtual object store - Python libraries ii python-swiftclient 1:1.2.0.6.a99a37f+git201210020230~precise-0ubuntu1 Client libary for Openstack Swift API. ii swift 1.7.1+git201209042100~precise-0ubuntu1 distributed virtual object store - common files
Storage Server Installation
root@swift-storage:~$ apt-get install swift-account swift-container swift-object xfsprogs root@swift-storage:~$ dpkg -l | grep swift ii swift-account 1.7.1+git201209042100~precise-0ubuntu1 distributed virtual object store - account server ii swift-container 1.7.1+git201209042100~precise-0ubuntu1 distributed virtual object store - container server ii swift-object 1.7.1+git201209042100~precise-0ubuntu1 distributed virtual object store - object server設定 partition
root@swift-storage:~$ mkfs.xfs -i size=1024 /dev/vda3
meta-data=/dev/vda3 isize=1024 agcount=4, agsize=2949120 blks
= sectsz=512 attr=2, projid32bit=0
data = bsize=4096 blocks=11796480, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0
log =internal log bsize=4096 blocks=5760, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
root@swift-storage:~$ echo "/dev/vda3 /srv/node/vda3 xfs noatime,nodiratime,nobarrier,logbufs=8 0 0" >> /etc/fstab
root@swift-storage:~$ mkdir -p /srv/node/vda3
root@swift-storage:~$ mount /srv/node/vda3
root@swift-storage:~$ chown -R swift:swift /srv/node
root@swift-storage:~$ df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/vda2 3094288 1592280 1344824 55% /
udev 4080360 4 4080356 1% /dev
tmpfs 1635780 252 1635528 1% /run
none 5120 0 5120 0% /run/lock
none 4089444 0 4089444 0% /run/shm
/dev/vda3 47162880 32976 47129904 1% /srv/node/vda3
設定rsync # 設定 /etc/rsyncd.conf , 記得 address 要改一下 root@swift:~$ cat /etc/rsyncd.conf uid = swift gid = swift log file = /var/log/rsyncd.log pid file = /var/run/rsyncd.pid address = 172.17.123.15 [account] max connections = 2 path = /srv/node/ read only = false lock file = /var/lock/account.lock [container] max connections = 2 path = /srv/node/ read only = false lock file = /var/lock/container.lock [object] max connections = 2 path = /srv/node/ read only = false lock file = /var/lock/object.lock # 把 RSYNC_ENABLE 改成 true root@swift:~$ head /etc/default/rsync # defaults file for rsync daemon mode # start rsync in daemon mode from init.d script? # only allowed values are "true", "false", and "inetd" # Use "inetd" if you want to start the rsyncd from inetd, # all this does is prevent the init.d script from printing a message # about not starting rsyncd (you still need to modify inetd's config yourself). RSYNC_ENABLE=true root@swift:~$ service rsync restart * Restarting rsync daemon rsync * rsync daemon not running, attempting to start. ...done.裝完後會在 /etc/swift 下看到三個 .conf 檔, account-server.conf、container-server.conf、object-server.conf, 但在裝的過程中, 發現他還需要一個 object-expirer.conf, 目前不確定這個檔案一定要裝, 但至少我加上去也沒有造成什麼問題, 然後 container-server.conf 預設的內容也有少, 後來我是直接去看 git 內的 container-server.conf-sample 才知道的.
account-server.conf 的內容
[DEFAULT] bind_ip = 0.0.0.0 workers = 2 [pipeline:main] pipeline = account-server [app:account-server] use = egg:swift#account [account-replicator] [account-auditor] [account-reaper]
container-server.conf 的內容
記得要多加 container-sync[DEFAULT] bind_ip = 0.0.0.0 workers = 2 [pipeline:main] pipeline = container-server [app:container-server] use = egg:swift#container [container-replicator] [container-updater] [container-auditor] [container-sync]
object-server.conf 的內容
[DEFAULT] bind_ip = 0.0.0.0 workers = 2 [pipeline:main] pipeline = object-server [app:object-server] use = egg:swift#object [object-replicator] [object-updater] [object-auditor]
container-expirer.conf 的內容
[DEFAULT] [object-expirer] interval = 300 [pipeline:main] pipeline = catch_errors cache proxy-server [app:proxy-server] use = egg:swift#proxy [filter:cache] use = egg:swift#memcache [filter:catch_errors] use = egg:swift#catch_errors
Proxy Server Installation
root@swift-proxy:~$ apt-get install swift-proxy memcached python-keystoneclient keystone root@swift-proxy:~$ dpkg -l | grep swift ii python-swift 1.7.1+git201209042100~precise-0ubuntu1 distributed virtual object store - Python libraries ii swift 1.7.1+git201209042100~precise-0ubuntu1 distributed virtual object store - common files ii swift-proxy 1.7.1+git201209042100~precise-0ubuntu1 distributed virtual object store - proxy server root@swift-proxy:~$ dpkg -l | grep keystone ii python-keystone 2012.2+git201209252030~precise-0ubuntu1 OpenStack identity service - Python library ii python-keystoneclient 1:0.1.3.19+git201210011900~precise-0ubuntu1 Client libary for Openstack Keystone API在 /etc/swift 下生成 certificate
root@swift-proxy:~$ cd /etc/swift/ root@swift-proxy:/etc/swift$ openssl req -new -x509 -nodes -out cert.crt -keyout cert.key Generating a 1024 bit RSA private key .....++++++ ..........................++++++ writing new private key to 'cert.key' ----- You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Country Name (2 letter code) [AU]:TW State or Province Name (full name) [Some-State]:Taiwan Locality Name (eg, city) []:Taipei Organization Name (eg, company) [Internet Widgits Pty Ltd]: Organizational Unit Name (eg, section) []: Common Name (e.g. server FQDN or YOUR name) []: Email Address []:設定 memcache
root@swift:~$ cat /etc/memcached.conf | grep -B 3 0.0.0.0 # Specify which IP address to listen on. The default is to listen on all IP addresses # This parameter is one of the only security measures that memcached has, so make sure # it's listening on a firewalled interface. -l 172.17.123.14 root@swift:~$ service memcached restart然後設定 proxy-server 的內容, 因為前一篇我在 keystone 內設定 swift 的 port 是 8080, 所以我在這裡也要設定成 8080. 另外 [filter:keystoneauth]、[filter:authtoken] 的內容, 我都直接參考 git 內的 sample 來設定, 按照官網上的設定方式似乎會有問題. 另外我會在 /etc/hosts 先加上 keystone 的 ip, 所以在設定檔內的 service_host 我填的是 hostname, 而不是 ip
[DEFAULT] bind_port = 8080 user = swift [pipeline:main] pipeline = catch_errors healthcheck cache authtoken keystoneauth proxy-server [app:proxy-server] use = egg:swift#proxy allow_account_management = true account_autocreate = true [filter:keystoneauth] use = egg:swift#keystoneauth #paste.filter_factory = keystone.middleware.swift_auth:filter_factory operator_roles = Member,admin,swiftoperator [filter:authtoken] paste.filter_factory = keystone.middleware.auth_token:filter_factory service_protocol = http service_port = 5000 service_host = keystone auth_port = 35357 auth_host = keystone auth_protocol = http admin_tenant_name = service admin_user = swift admin_password = password signing_dir = /etc/swift [filter:cache] use = egg:swift#memcache set log_name = cache #memcache_servers = 172.17.123.14:11211 [filter:catch_errors] use = egg:swift#catch_errors [filter:healthcheck] use = egg:swift#healthcheck [filter:proxy-logging] use = egg:swift#proxy_logging [filter:ratelimit] use = egg:swift#ratelimit設定 ring, 我的 storage server ip 為 172.17.123.15, 剛剛加入的 partition 是 vda3, 我把它設定到 z1 去, 如果要加入更多其它 partition, 就必須要多執行幾次 add 的動作, 而且要規劃好放在那一個 zone, 可以參考以下兩份文件.
http://cssoss.files.wordpress.com/2012/05/openstackbookv3-0_csscorp2.pdf
http://www.hastexo.com/resources/docs/installing-openstack-essex-20121-ubuntu-1204-precise-pangolin/appendix-c-setting-op-1
root@swift:~$ cd /etc/swift/
root@swift:/etc/swift$ swift-ring-builder account.builder create 18 3 1
root@swift:/etc/swift$ swift-ring-builder container.builder create 18 3 1
root@swift:/etc/swift$ swift-ring-builder object.builder create 18 3 1
root@swift:/etc/swift$ swift-ring-builder account.builder add z1-172.17.123.15:6002/vda3 100
Device z1-172.17.123.15:6002/vda3_"" with 100.0 weight got id 0
root@swift:/etc/swift$ swift-ring-builder container.builder add z1-172.17.123.15:6001/vda3 100
Device z1-172.17.123.15:6001/vda3_"" with 100.0 weight got id 0
root@swift:/etc/swift$ swift-ring-builder object.builder add z1-172.17.123.15:6000/vda3 100
Device z1-172.17.123.15:6000/vda3_"" with 100.0 weight got id 0
root@swift:/etc/swift$ swift-ring-builder account.builder
account.builder, build version 1
262144 partitions, 3 replicas, 1 zones, 1 devices, 100.00 balance
The minimum number of hours before a partition can be reassigned is 1
Devices: id zone ip address port name weight partitions balance meta
0 1 172.17.123.15 6002 vda3 100.00 0 -100.00
root@swift:/etc/swift$ swift-ring-builder container.builder
container.builder, build version 1
262144 partitions, 3 replicas, 1 zones, 1 devices, 100.00 balance
The minimum number of hours before a partition can be reassigned is 1
Devices: id zone ip address port name weight partitions balance meta
0 1 172.17.123.15 6001 vda3 100.00 0 -100.00
root@swift:/etc/swift$ swift-ring-builder object.builder
object.builder, build version 1
262144 partitions, 3 replicas, 1 zones, 1 devices, 100.00 balance
The minimum number of hours before a partition can be reassigned is 1
Devices: id zone ip address port name weight partitions balance meta
0 1 172.17.123.15 6000 vda3 100.00 0 -100.00
root@swift-proxy:/etc/swift$ swift-ring-builder account.builder rebalance
Reassigned 262144 (100.00%) partitions. Balance is now 0.00.
root@swift-proxy:/etc/swift$ swift-ring-builder container.builder rebalance
Reassigned 262144 (100.00%) partitions. Balance is now 0.00.
root@swift-proxy:/etc/swift$ swift-ring-builder object.builder rebalance
Reassigned 262144 (100.00%) partitions. Balance is now 0.00.
root@swift-proxy:/etc/swift$ chown -R swift:swift /etc/swift
做完 rebalance 之後會產生三個 ring.gz 檔案, 記得把這些檔案放到所有的 storage server 的 /etc/swift 下, 而且傳過去之後, 記得要把 owner 改成 swift:swift Startup Service
在 proxy server 下root@swift-proxy:~$ swift-init proxy start Starting proxy-server...(/etc/swift/proxy-server.conf)在 storage server 下
root@swift-storage:~$ swift-init all start Starting container-updater...(/etc/swift/container-server.conf) Starting account-auditor...(/etc/swift/account-server.conf) Starting object-replicator...(/etc/swift/object-server.conf) Unable to locate config for proxy-server Starting container-replicator...(/etc/swift/container-server.conf) Starting object-auditor...(/etc/swift/object-server.conf) Starting object-expirer...(/etc/swift/object-expirer.conf) Starting container-auditor...(/etc/swift/container-server.conf) Starting container-server...(/etc/swift/container-server.conf) Starting account-server...(/etc/swift/account-server.conf) Starting account-reaper...(/etc/swift/account-server.conf) Starting container-sync...(/etc/swift/container-server.conf) Starting account-replicator...(/etc/swift/account-server.conf) Starting object-updater...(/etc/swift/object-server.conf) Starting object-server...(/etc/swift/object-server.conf)
Test and Verify
編輯一個檔, 設定一些等下會用到的區域變數, 並且匯到 .bashrc 之中, 下次再進來就不需要重新設定root@swift-proxy:~$ cat novarc export OS_TENANT_NAME=admin export OS_USERNAME=admin export OS_PASSWORD=password export OS_AUTH_URL="http://keystone:5000/v2.0/" export SERVICE_ENDPOINT="http://keystone:35357/v2.0" export SERVICE_TOKEN=password root@swift-proxy:~$ source novarc root@swift-proxy:~$ echo "source novarc">>.bashrc另外在 keystone 的 endpoint-list 也請記得修改一下 ip, 改成 proxy server 的 ip, 不要再設成 127.0.0.1, 除非你是裝在同一台
root@swift-proxy:~$ keystone endpoint-list +----------------------------------+-----------+-----------------------------------------------+-----------------------------------------------+----------------------------------------+ | id | region | publicurl | internalurl | adminurl | +----------------------------------+-----------+-----------------------------------------------+-----------------------------------------------+----------------------------------------+ | 580b71d126804c5197b91c79fd74a330 | RegionOne | http://keystone:5000/v2.0 | http://keystone:5000/v2.0 | http://keystone:35357/v2.0 | | 6c788747593d475f831b6ff128bde995 | RegionOne | http://cinder:8776/v1/$(tenant_id)s | http://cinder:8776/v1/$(tenant_id)s | http://cinder:8776/v1/$(tenant_id)s | | 95e16e71a8f04ac68ae401df5284ce3e | RegionOne | http://swift-proxy:8080/v1/AUTH_$(tenant_id)s | http://swift-proxy:8080/v1/AUTH_$(tenant_id)s | http://swift-proxy:8080/v1 | +----------------------------------+-----------+-----------------------------------------------+-----------------------------------------------+----------------------------------------+測試的時間到了
http://docs.openstack.org/trunk/openstack-compute/install/apt/content/verify-swift-installation.html
root@swift-proxy:~$ swift list
root@swift-proxy:~$ swift post test
root@swift-proxy:~$ swift list
test
root@swift-proxy:~$ swift upload test /etc/motd
etc/motd
root@swift-proxy:~$ swift list test
etc/motd
root@swift-proxy:~$ swift stat
Account: AUTH_eefa301a6a424e7da3d582649ad0e59e
Containers: 1
Objects: 1
Bytes: 451
Accept-Ranges: bytes
X-Timestamp: 1349422137.92607
X-Trans-Id: tx251024dd19464f55b2945092b6f3678a
What is Swift?
Mirantis 有一篇介紹 Swift 及 Ceph, 但我覺得他對於 Swift 的簡介十分的易懂明瞭, 對於第一次接觸的人來說, 是個很棒的入門篇http://www.mirantis.com/blog/object-storage-openstack-cloud-swift-ceph/
再深入一點的文章, 這篇講解 Ring 的概念, 並且也點出了 Swift 的不足之處
http://julien.danjou.info/blog/2012/openstack-swift-consistency-analysis
http://www.mirantis.com/blog/under-the-hood-of-swift-the-ring/
這篇就比較純文字一點, 先看過前面幾篇再來看這篇會比較適合
http://programmerthoughts.com/openstack/swift-tech-overview/
他也從 CAP 理論的角度來探討 Swift 的特性是滿足 AP (Available + Partition Tolerance), 所以在 Consistency 方面會放鬆一些, 也就是偶而會拿到不一致的資料
Swift achieves high scalability by relaxing constraints on consistency. While swift provides read-your-writes consistency for new objects, listings and aggregate metadata (like usage information) may not be immediately accurate. Similarly, reading an object that has been overwritten with new data may return an older version of the object data. However, swift provides the ability for the client to request the most up-to-date version at the cost of request latency.
沒有留言:
張貼留言