Openstack Folsom Release
前幾天 Openstack Folsom 正式 ReleaseRelease Software Site
http://www.openstack.org/software/folsom/
Release Note
http://wiki.openstack.org/ReleaseNotes/Folsom
Architecture
http://ken.pepple.info/openstack/2012/09/25/openstack-folsom-architecture/
這次的 Release 多了 Quantum 及 Cinder 兩個 project. Quantum 是將 Open vSwitch 整合進 Openstack, 加強了原本 Openstack 對於網路虛擬化不足的地方. Cinder 則是將原來 nova-volume 拆出來變成一個獨立的 module, 我想原因不外乎背後有太多 Storage 的大廠想要整合進 Openstack, ex: nexenta、netapp... 所以把這塊的 API 獨立出來, 和各家整合才會順利.
PPA of openstack testing
root@ubuntu12:~$ apt-get install -y python-software-properties root@ubuntu12:~$ add-apt-repository ppa:openstack-ubuntu-testing/folsom-trunk-testing root@ubuntu12:~$ add-apt-repository ppa:openstack-ubuntu-testing/folsom-deps-staging root@ubuntu12:~$ apt-get update && apt-get -y dist-upgrade
PPA of Ubuntu Cloud
root@ubuntu12:~$ add-apt-repository ppa:ubuntu-cloud-archive/folsom-staging You are about to add the following PPA to your system: More info: https://launchpad.net/~ubuntu-cloud-archive/+archive/folsom-staging Press [ENTER] to continue or ctrl-c to cancel adding it gpg: keyring `/tmp/tmpdzKHU_/secring.gpg' created gpg: keyring `/tmp/tmpdzKHU_/pubring.gpg' created gpg: requesting key 9F68104E from hkp server keyserver.ubuntu.com gpg: /tmp/tmpdzKHU_/trustdb.gpg: trustdb created gpg: key 9F68104E: public key "Launchpad PPA for Ubuntu Cloud Archive Team" imported gpg: Total number processed: 1 gpg: imported: 1 (RSA: 1) OK
Openstack 的更新速度會比 Ubuntu Cloud 來的快, 以下的流程是以 Opentack 的 PPA 來進行
Installation
這次我先從 Cinder 下手, 並且目標是要把它和 Ceph 整合在一起. 原本想要單獨裝 Cinder 就好, 但研究 Cinder Source Code 後, 發現他目前不支援 noauth 的模式, 所以就一定要裝 Keystone, 所以整體來說重要的 Component 有: Cinder + Keystone + MySQL + CEPH首先先裝基本的套件, MySQL 及 RabbitMQ
MySQL Installation
root@ubuntu12:~$ apt-get -y install mysql-server python-mysqldb root@ubuntu12:~$ sed -i 's/127.0.0.1/0.0.0.0/g' /etc/mysql/my.cnf root@ubuntu12:~$ service mysql restart root@ubuntu12:~$ mysql -u root -ppassword Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 36 Server version: 5.5.24-0ubuntu0.12.04.1 (Ubuntu) Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> CREATE DATABASE cinder; Query OK, 1 row affected (0.01 sec) mysql> GRANT ALL PRIVILEGES ON cinder.* TO 'cinder'@'%' IDENTIFIED BY 'password'; Query OK, 0 rows affected (0.00 sec) mysql> GRANT ALL PRIVILEGES ON cinder.* TO 'cinder'@'localhost' IDENTIFIED BY 'password'; Query OK, 0 rows affected (0.00 sec) mysql> CREATE DATABASE keystone; Query OK, 1 row affected (0.00 sec) mysql> GRANT ALL PRIVILEGES ON keystone.* TO 'keystone'@'%' IDENTIFIED BY 'password'; Query OK, 0 rows affected (0.00 sec) mysql> GRANT ALL PRIVILEGES ON keystone.* TO 'keystone'@'localhost' IDENTIFIED BY 'password'; Query OK, 0 rows affected (0.00 sec) mysql> FLUSH PRIVILEGES; Query OK, 0 rows affected (0.00 sec)
RabbitMQ Installation
root@ubuntu12:~$ sudo apt-get -y install rabbitmq-server root@ubuntu12:~$ rabbitmqctl change_password guest password Changing password for user "guest" ... ...done.
Keystone Installation
root@ubuntu12:~$ apt-get -y install keystone python-keystone python-keystoneclient root@ubuntu12:~$ dpkg -l | grep keystone ii keystone 2012.2+git201209252030~precise-0ubuntu1 OpenStack identity service - Daemons ii python-keystone 2012.2+git201209252030~precise-0ubuntu1 OpenStack identity service - Python library ii python-keystoneclient 1:0.1.3.19+git201210011900~precise-0ubuntu1 Client libary for Openstack Keystone API # 修改 keyston.conf 的內容, 隨機取一個 admin_token root@ubuntu12:~$ cat /etc/keystone/keystone.conf [DEFAULT] # A "shared secret" between keystone and other openstack services admin_token = password # The IP address of the network interface to listen on bind_host = 0.0.0.0 # The port number which the public service listens on public_port = 5000 # The port number which the public admin listens on admin_port = 35357 # The port number which the OpenStack Compute service listens on compute_port = 8774 # === Logging Options === # Print debugging output verbose = True # Print more verbose output # (includes plaintext request logging, potentially including passwords) debug = True ... [sql] # The SQLAlchemy connection string used to connect to the database # connection = sqlite:////var/lib/keystone/keystone.db connection = mysql://keystone:password@localhost:3306/keystone # the timeout before idle sql connections are reaped idle_timeout = 200 ... root@ubuntu12:~$ service keystone restart root@ubuntu12:~$ keystone-manage db_sync # 編輯一個檔, 設定一些等下會用到的區域變數, 並且匯到 .bashrc 之中, 下次再進來就不需要重新設定 root@ubuntu12:~# cat novarc export OS_TENANT_NAME=admin export OS_USERNAME=admin export OS_PASSWORD=password export OS_AUTH_URL="http://localhost:5000/v2.0/" export SERVICE_ENDPOINT="http://localhost:35357/v2.0" export SERVICE_TOKEN=password root@ubuntu12:~$ source novarc root@ubuntu12:~$ echo "source novarc">>.bashrc
接下來下載兩個網路上別人提供好的 script, 裡面預設的 token 也是 "password", 如果你要使用別的 token, 請記得要修改 script 的內容
root@ubuntu12:~$ wget https://raw.github.com/EmilienM/openstack-folsom-guide/master/scripts/keystone-data.sh root@ubuntu12:~$ wget https://raw.github.com/EmilienM/openstack-folsom-guide/master/scripts/keystone-endpoints.sh root@ubuntu12:~$ chmod a+x *.sh # 記得把 keystone-endpoints.sh 內的 MASTER 改成你自己的 ip root@ubuntu12:~$ less keystone-endpoints.sh .... # other definitions MASTER="172.17.123.13" .... root@ubuntu12:~$ ./keystone-data.sh root@ubuntu12:~$ ./keystone-endpoints.sh +-------------+----------------------------------+ | Property | Value | +-------------+----------------------------------+ | description | OpenStack Compute Service | | id | 597fff05550043efb530ab05fa85d818 | | name | nova | | type | compute | +-------------+----------------------------------+ +-------------+----------------------------------+ | Property | Value | +-------------+----------------------------------+ | description | OpenStack Volume Service | | id | 27e3539fea104d159fcc7ec9766ac8b3 | | name | cinder | | type | volume | +-------------+----------------------------------+ +-------------+----------------------------------+ | Property | Value | +-------------+----------------------------------+ | description | OpenStack Image Service | | id | b7047156ef81464b8c6754fc7994ecea | | name | glance | | type | image | +-------------+----------------------------------+ +-------------+----------------------------------+ | Property | Value | +-------------+----------------------------------+ | description | OpenStack Storage Service | | id | 86e08436f65944be8ab0e23657a9d3e2 | | name | swift | | type | object-store | +-------------+----------------------------------+ +-------------+----------------------------------+ | Property | Value | +-------------+----------------------------------+ | description | OpenStack Identity | | id | 4ec3ab138f2a4bd2800b5a4a5d407ef1 | | name | keystone | | type | identity | +-------------+----------------------------------+ +-------------+----------------------------------+ | Property | Value | +-------------+----------------------------------+ | description | OpenStack EC2 service | | id | 15bb5a5a0c3d446e8b3bfa293df3b10e | | name | ec2 | | type | ec2 | +-------------+----------------------------------+ +-------------+----------------------------------+ | Property | Value | +-------------+----------------------------------+ | description | OpenStack Networking service | | id | cacfce49cb2141a1ac48b3e31cef5c01 | | name | quantum | | type | network | +-------------+----------------------------------+ +-------------+--------------------------------------------+ | Property | Value | +-------------+--------------------------------------------+ | adminurl | http://172.17.123.13:8774/v2/$(tenant_id)s | | id | 090f849da5a94dcb817aa340b39eb83c | | internalurl | http://172.17.123.13:8774/v2/$(tenant_id)s | | publicurl | http://172.17.123.13:8774/v2/$(tenant_id)s | | region | RegionOne | | service_id | 597fff05550043efb530ab05fa85d818 | +-------------+--------------------------------------------+ +-------------+--------------------------------------------+ | Property | Value | +-------------+--------------------------------------------+ | adminurl | http://172.17.123.13:8776/v1/$(tenant_id)s | | id | 6363131b18974040a3dd1276ddc2c72e | | internalurl | http://172.17.123.13:8776/v1/$(tenant_id)s | | publicurl | http://172.17.123.13:8776/v1/$(tenant_id)s | | region | RegionOne | | service_id | 27e3539fea104d159fcc7ec9766ac8b3 | +-------------+--------------------------------------------+ +-------------+----------------------------------+ | Property | Value | +-------------+----------------------------------+ | adminurl | http://172.17.123.13:9292/v2 | | id | 1db94d180acb4d64adab45c3116812cb | | internalurl | http://172.17.123.13:9292/v2 | | publicurl | http://172.17.123.13:9292/v2 | | region | RegionOne | | service_id | b7047156ef81464b8c6754fc7994ecea | +-------------+----------------------------------+ +-------------+-------------------------------------------------+ | Property | Value | +-------------+-------------------------------------------------+ | adminurl | http://172.17.123.13:8080/v1 | | id | b9006b5f4fc64e4d854c171ea157b7b4 | | internalurl | http://172.17.123.13:8080/v1/AUTH_$(tenant_id)s | | publicurl | http://172.17.123.13:8080/v1/AUTH_$(tenant_id)s | | region | RegionOne | | service_id | 86e08436f65944be8ab0e23657a9d3e2 | +-------------+-------------------------------------------------+ +-------------+----------------------------------+ | Property | Value | +-------------+----------------------------------+ | adminurl | http://172.17.123.13:35357/v2.0 | | id | faf6ca350f4842799b3801d0b7571a59 | | internalurl | http://172.17.123.13:5000/v2.0 | | publicurl | http://172.17.123.13:5000/v2.0 | | region | RegionOne | | service_id | 4ec3ab138f2a4bd2800b5a4a5d407ef1 | +-------------+----------------------------------+ +-------------+------------------------------------------+ | Property | Value | +-------------+------------------------------------------+ | adminurl | http://172.17.123.13:8773/services/Admin | | id | da745295e7ef419682d45516456047c5 | | internalurl | http://172.17.123.13:8773/services/Cloud | | publicurl | http://172.17.123.13:8773/services/Cloud | | region | RegionOne | | service_id | 15bb5a5a0c3d446e8b3bfa293df3b10e | +-------------+------------------------------------------+ +-------------+----------------------------------+ | Property | Value | +-------------+----------------------------------+ | adminurl | http://172.17.123.13:9696/ | | id | 8b570092f8614cba98fe227de6e65e27 | | internalurl | http://172.17.123.13:9696/ | | publicurl | http://172.17.123.13:9696/ | | region | RegionOne | | service_id | cacfce49cb2141a1ac48b3e31cef5c01 | +-------------+----------------------------------+
安裝好之後驗証一下
root@ubuntu12:~$ keystone endpoint-list
+----------------------------------+-----------+-------------------------------------------------+-------------------------------------------------+--------------------------------------------+
| id | region | publicurl | internalurl | adminurl |
+----------------------------------+-----------+-------------------------------------------------+-------------------------------------------------+--------------------------------------------+
| 090f849da5a94dcb817aa340b39eb83c | RegionOne | http://172.17.123.13:8774/v2/$(tenant_id)s | http://172.17.123.13:8774/v2/$(tenant_id)s | http://172.17.123.13:8774/v2/$(tenant_id)s |
| 1db94d180acb4d64adab45c3116812cb | RegionOne | http://172.17.123.13:9292/v2 | http://172.17.123.13:9292/v2 | http://172.17.123.13:9292/v2 |
| 6363131b18974040a3dd1276ddc2c72e | RegionOne | http://172.17.123.13:8776/v1/$(tenant_id)s | http://172.17.123.13:8776/v1/$(tenant_id)s | http://172.17.123.13:8776/v1/$(tenant_id)s |
| 8b570092f8614cba98fe227de6e65e27 | RegionOne | http://172.17.123.13:9696/ | http://172.17.123.13:9696/ | http://172.17.123.13:9696/ |
| b9006b5f4fc64e4d854c171ea157b7b4 | RegionOne | http://172.17.123.13:8080/v1/AUTH_$(tenant_id)s | http://172.17.123.13:8080/v1/AUTH_$(tenant_id)s | http://172.17.123.13:8080/v1 |
| da745295e7ef419682d45516456047c5 | RegionOne | http://172.17.123.13:8773/services/Cloud | http://172.17.123.13:8773/services/Cloud | http://172.17.123.13:8773/services/Admin |
| faf6ca350f4842799b3801d0b7571a59 | RegionOne | http://172.17.123.13:5000/v2.0 | http://172.17.123.13:5000/v2.0 | http://172.17.123.13:35357/v2.0 |
+----------------------------------+-----------+-------------------------------------------------+-------------------------------------------------+--------------------------------------------+
root@ubuntu12:~$ sudo apt-get install -y curl openssl
root@ubuntu12:~$ curl -d '{"auth": {"tenantName": "admin", "passwordCredentials":{"username": "admin", "password": "pass
word"}}}' -H "Content-type: application/json" http://172.17.123.13:35357/v2.0/tokens | python -mjson.tool
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 2946 0 2844 100 102 18698 670 --:--:-- --:--:-- --:--:-- 18834
{
"access": {
"metadata": {
"is_admin": 0,
"roles": [
"5660593decab401bbc8f13aa8b19dc23",
"833b1617cb404466ba8546c8194f8ad6",
"7ec43588e3a14b69ae2278334bee1423"
]
},
"serviceCatalog": [
{
"endpoints": [
{
"adminURL": "http://172.17.123.13:8774/v2/4b310104ee3345fd988fe16dd2f1f79d",
"id": "090f849da5a94dcb817aa340b39eb83c",
"internalURL": "http://172.17.123.13:8774/v2/4b310104ee3345fd988fe16dd2f1f79d",
"publicURL": "http://172.17.123.13:8774/v2/4b310104ee3345fd988fe16dd2f1f79d",
"region": "RegionOne"
}
],
"endpoints_links": [],
"name": "nova",
"type": "compute"
},
{
"endpoints": [
{
"adminURL": "http://172.17.123.13:9696/",
"id": "8b570092f8614cba98fe227de6e65e27",
"internalURL": "http://172.17.123.13:9696/",
"publicURL": "http://172.17.123.13:9696/",
"region": "RegionOne"
}
],
"endpoints_links": [],
"name": "quantum",
"type": "network"
},
{
"endpoints": [
{
"adminURL": "http://172.17.123.13:9292/v2",
"id": "1db94d180acb4d64adab45c3116812cb",
"internalURL": "http://172.17.123.13:9292/v2",
"publicURL": "http://172.17.123.13:9292/v2",
"region": "RegionOne"
}
],
"endpoints_links": [],
"name": "glance",
"type": "image"
},
{
"endpoints": [
{
"adminURL": "http://172.17.123.13:8776/v1/4b310104ee3345fd988fe16dd2f1f79d",
"id": "6363131b18974040a3dd1276ddc2c72e",
"internalURL": "http://172.17.123.13:8776/v1/4b310104ee3345fd988fe16dd2f1f79d",
"publicURL": "http://172.17.123.13:8776/v1/4b310104ee3345fd988fe16dd2f1f79d",
"region": "RegionOne"
}
],
"endpoints_links": [],
"name": "cinder",
"type": "volume"
},
{
"endpoints": [
{
"adminURL": "http://172.17.123.13:8773/services/Admin",
"id": "da745295e7ef419682d45516456047c5",
"internalURL": "http://172.17.123.13:8773/services/Cloud",
"publicURL": "http://172.17.123.13:8773/services/Cloud",
"region": "RegionOne"
}
],
"endpoints_links": [],
"name": "ec2",
"type": "ec2"
},
{
"endpoints": [
{
"adminURL": "http://172.17.123.13:8080/v1",
"id": "b9006b5f4fc64e4d854c171ea157b7b4",
"internalURL": "http://172.17.123.13:8080/v1/AUTH_4b310104ee3345fd988fe16dd2f1f79d",
"publicURL": "http://172.17.123.13:8080/v1/AUTH_4b310104ee3345fd988fe16dd2f1f79d",
"region": "RegionOne"
}
],
"endpoints_links": [],
"name": "swift",
"type": "object-store"
},
{
"endpoints": [
{
"adminURL": "http://172.17.123.13:35357/v2.0",
"id": "faf6ca350f4842799b3801d0b7571a59",
"internalURL": "http://172.17.123.13:5000/v2.0",
"publicURL": "http://172.17.123.13:5000/v2.0",
"region": "RegionOne"
}
],
"endpoints_links": [],
"name": "keystone",
"type": "identity"
}
],
"token": {
"expires": "2012-10-03T06:55:20Z",
"id": "5f2797bdf8fd4380bfe919a05b01772e",
"tenant": {
"description": null,
"enabled": true,
"id": "4b310104ee3345fd988fe16dd2f1f79d",
"name": "admin"
}
},
"user": {
"id": "c7f17dfd798242fc9065afd2ea251a6d",
"name": "admin",
"roles": [
{
"name": "admin"
},
{
"name": "KeystoneAdmin"
},
{
"name": "KeystoneServiceAdmin"
}
],
"roles_links": [],
"username": "admin"
}
}
}
CEPH Installation
root@ubuntu12:~$ wget -q -O - https://raw.github.com/ceph/ceph/master/keys/release.asc | apt-key add -
OK
# 手動增加一個 ceph.list 在 /etc/apt/sources.list.d 下
root@ubuntu12:/etc/apt/sources.list.d$ cat ceph.list
deb http://ceph.newdream.net/debian/ precise main
deb-src http://ceph.newdream.net/debian/ precise main
root@ubuntu12:~$ apt-get update
root@ubuntu12:~$ apt-get install -y ceph python-ceph
root@ubuntu12:~$ dpkg -l | grep ceph
ii ceph 0.48.2argonaut-1precise distributed storage and file system
ii ceph-common 0.48.2argonaut-1precise common utilities to mount and interact with a ceph storage cluster
ii ceph-fs-common 0.48.2argonaut-1precise common utilities to mount and interact with a ceph file system
ii ceph-fuse 0.48.2argonaut-1precise FUSE-based client for the Ceph distributed file system
ii ceph-mds 0.48.2argonaut-1precise metadata server for the ceph distributed file system
ii libcephfs1 0.48.2argonaut-1precise Ceph distributed file system client library
ii python-ceph 0.48.2argonaut-1precise Python libraries for the Ceph distributed filesystem
# 安裝好之後, 就把你的 ceph cluster 的設定檔 copy 到 /etc/ceph 下, 正常就可以使用
# 至於怎麼安裝 ceph cluster 就請到 ceph 的官網去看囉~
root@ubuntu12:~$ ceph -s
health HEALTH_OK
monmap e1: 3 mons at {wistor-003=172.17.123.92:6789/0,wistor-006=172.17.123.94:6789/0,wistor-007=172.17.123.95:6789/0}, election epoch 10, quorum 0,1,2 wistor-003,wistor-006,wistor-007
osdmap e24: 23 osds: 23 up, 23 in
pgmap v2242: 4416 pgs: 4416 active+clean; 8362 MB data, 156 GB used, 19850 GB / 21077 GB avail
mdsmap e1: 0/0/1 up
如果只是要和一般的 iscsi server 整合, 可以參考官網, 大致上就是把一個 block device (ex:/dev/sdb) 丟到 LVM 去管理, 然後記得要建一個 cinder-volume
http://docs.openstack.org/trunk/openstack-compute/install/apt/content/osfolubuntu-cinder.html
Cinder Installation
root@ubuntu12:~$ apt-get install -y cinder-api cinder-scheduler cinder-volume iscsitarget open-iscsi iscsitarget-dkms python-cinderclient tgt root@ubuntu12:~$ dpkg -l | grep cinder ii cinder-api 2012.2+git201209252100~precise-0ubuntu1 Cinder storage service - api server ii cinder-common 2012.2+git201209252100~precise-0ubuntu1 Cinder starage service - common files ii cinder-scheduler 2012.2+git201209252100~precise-0ubuntu1 Cinder storage service - api server ii cinder-volume 2012.2+git201209252100~precise-0ubuntu1 Cinder storage service - api server ii python-cinder 2012.2+git201209252100~precise-0ubuntu1 Cinder python libraries ii python-cinderclient 1:0.2.26+git201209201100~precise-0ubuntu1 python bindings to the OpenStack Volume API
修改 /etc/cinder/api_paste.ini
[filter:authtoken] paste.filter_factory = keystone.middleware.auth_token:filter_factory service_protocol = http service_host = 127.0.0.1 service_port = 5000 auth_host = 127.0.0.1 auth_port = 35357 auth_protocol = http # 修改這三行 admin_tenant_name = service admin_user = cinder admin_password = password
修改 /etc/cinder/api_paste.ini, 改成使用 mysql 及 RBD driver, 並且把 rabbitmq 的 password 修改一下
[DEFAULT] rootwrap_config = /etc/cinder/rootwrap.conf api_paste_confg = /etc/cinder/api-paste.ini iscsi_helper = tgtadm volume_name_template = volume-%s volume_group = cinder-volumes verbose = True auth_strategy = keystone state_path = /var/lib/cinder rabbit_password = password sql_connection = mysql://cinder:password@localhost:3306/cinder volume_driver=cinder.volume.driver.RBDDriver
root@ubuntu12:~$ cinder-manage db sync
root@ubuntu12:~$ service cinder-api restart
root@ubuntu12:~$ service cinder-scheduler restart
root@ubuntu12:~$ service cinder-volume restart
root@ubuntu12:~$ cinder create --display_name test 1
+---------------------+--------------------------------------+
| Property | Value |
+---------------------+--------------------------------------+
| attachments | [] |
| availability_zone | nova |
| created_at | 2012-10-02T07:14:34.815546 |
| display_description | None |
| display_name | test |
| id | e7e83b13-761e-40e3-8b4c-415126404e40 |
| metadata | {} |
| size | 1 |
| snapshot_id | None |
| status | creating |
| volume_type | None |
+---------------------+--------------------------------------+
root@ubuntu12:~$ cinder list
+--------------------------------------+-----------+--------------+------+-------------+-------------+
| ID | Status | Display Name | Size | Volume Type | Attached to |
+--------------------------------------+-----------+--------------+------+-------------+-------------+
| e7e83b13-761e-40e3-8b4c-415126404e40 | available | test | 1 | None | |
+--------------------------------------+-----------+--------------+------+-------------+-------------+
root@ubuntu12:~$ rbd list
volume-e7e83b13-761e-40e3-8b4c-415126404e40
Reference
Folsom Announcementhttp://lists.openstack.org/pipermail/openstack-announce/2012-September/000035.html
Folsom: How it was made
http://blog.bitergia.com/2012/09/27/how-the-new-release-of-openstack-was-built/
Cinder Installation Document
http://docs.openstack.org/trunk/openstack-compute/install/apt/content/osfolubuntu-cinder.htmlhttps://github.com/EmilienM/openstack-folsom-guide/blob/master/doc/out/pdf/openstack-folsom-guide.pdf
Cinder Developer document
http://docs.openstack.org/developer/cinder/
Cinder Source Code
https://github.com/openstack/cinder.git
https://github.com/openstack/python-cinderclient
The Top 3 New Swift Features in OpenStack Folsom
http://swiftstack.com/blog/2012/09/27/top-three-swift-features-in-openstack-folsom/
沒有留言:
張貼留言