Automatic Deployment of virtual environments on Open Telekom Cloud
Kurt Garloff
OpenStack Cloud Architect
kurt.garloff@t-systems.com
Deutsche OpenStack Tage 21./22.6.2016, Köln
Why public cloud?
Classical Deployment
Cloud Installation
And the result is ...
What went wrong?
Culture
- Manual approvals
Technology
- Manual installations
Automation
Automating virtual Infrastructure
- DIY REST (curl, jq)
- libcloud
- heat
- scalr
- terraform
- cliqr
- bosh
- juju
- ansible
- ...
OpenStack REST API
cloud-init
- controlled by meta-data src
- run upon first boot
- user accounts, ssh keys
- hostname, timezone
- networks, mounts
- package updates & installation
- arbitrary scripts
- phone home
- OTC: Inject files into /etc/cloud/cloud.cfd.d/
Config Mgmt Tools
- puppet, chef, ansible, saltstack
- run repeatedly to ensure compliance or changes
- configuration properties
- software packages
- can control infra as well
Application Management
OTC hierarchy
Open Telekom Cloud
API
Text
Getting the parameters
Environment ~/.ostackrc
# Adapt the first two lines ...
export OS_PASSWORD="XXXXX"
export OS_USERNAME="14610698 OTC00000000001000000205"
export OS_TENANT_NAME=eu-de
export OS_PROJECT_NAME=eu-de
export OS_USER_DOMAIN_NAME="${OS_USERNAME##* }"
export OS_AUTH_URL=https://iam.eu-de.otc.t-systems.com:443/v3
export OS_PROJECT_DOMAIN_NAME=
export OS_IDENTITY_API_VERSION=3
export OS_VOLUME_API_VERSION=2
export OS_IMAGE_API_VERSION=2
export OS_ENDPOINT_TYPE=publicURL
export NOVA_ENDPOINT_TYPE=publicURL
export CINDER_ENDPOINT_TYPE=publicURL
# no longer needed
#export OS_CACERT=/etc/ssl/OTC-API-GW-CA-Bundle.pem
Template preinstalled in our images
Example using otc-tools
Simple shell/curl/jq code
github https://github.com/OpenTelekomCloud/, OBS home:garloff:OTC
To be superceded by python-otcclient
garloff@ImgFact-JumpHost2:~ [0]$ otc ecs list
04213851-9ce9-4fbe-b2eb-42f950822e1c ecs-8430
3902c40e-8f63-49e7-9a46-3147834c9a7d test-os421-longssh
garloff@ImgFact-JumpHost2:~ [1]$ otc vpc list
b4690bfa-6d61-40b4-86b5-e8ab903ef11e vpc-32 OK 192.168.32.0/20
garloff@ImgFact-JumpHost2:~ [0]$ otc subnet list
[
{
"id": "7cbe489c-cf80-4f40-a117-a8946714a001",
"name": "subnet-33",
"cidr": "192.168.33.0/24",
"status": "ACTIVE",
"vpc_id": "b4690bfa-6d61-40b4-86b5-e8ab903ef11e",
"gateway_ip": "192.168.33.1",
"dhcp_enable": true,
"primary_dns": "100.125.4.25",
"secondary_dns": "217.150.148.148",
"availability_zone": "eu-de-02"
},
{
"id": "c095a6be-0d22-4393-8e46-05660e32da3f",
"name": "subnet-32",
"cidr": "192.168.32.0/24",
"status": "ACTIVE",
"vpc_id": "b4690bfa-6d61-40b4-86b5-e8ab903ef11e",
"gateway_ip": "192.168.32.1",
"dhcp_enable": true,
"primary_dns": "100.125.4.25",
"secondary_dns": "195.244.235.14",
"availability_zone": "eu-de-01"
}
]
Creating a VM with otc-tools
otc ecs create \
--wait --instance-type computev1-1 --instance-name TEST_VM \
--image-name Standard_openSUSE_42.1_JeOS_latest --subnet-name subnet-32 \
--vpc-name vpc-32 --security-group-name sg-ssh --admin-pass "Cloud.4321" \
--key-name SSHkey-205a --public false --disksize 6
{
"server": {
"availability_zone": "eu-de-02", "name": "TEST_VM",
"imageRef": "1cec66fa-036a-4192-835d-75f1568f2821",
"root_volume": { "volumetype": "SATA", "size": "6" },
"flavorRef": "computev1-1", "vpcid": "b4690bfa-6d61-40b4-86b5-e8ab903ef11e",
"security_groups": [ { "id": "fa96c994-b43c-4793-b763-7b624e690511" } ],
"nics": [ { "subnet_id": "c095a6be-0d22-4393-8e46-05660e32da3f" } ],
"key_name": "SSHkey-205a", "adminPass": "Cloud.4321", "count": "1"
}
}
2c9eb2c15487316901549c98469a7e9e
JOBID: https://ecs.eu-de.otc.t-systems.com/v1/a90ddf531f934a9aac8acebf1abdf19a/jobs/2c9eb2c15487316901549c98469a7e9e
{"status":"INIT","entities":{},"job_id":"2c9eb2c15487316901549c98469a7e9e","job_type":"createServer","begin_time":"2016-05-10T21:35:05.879Z","end_time":"","error_code":null,"fail_reason":null}
#{"status":"RUNNING","entities":{},"job_id":"2c9eb2c15487316901549c98469a7e9e","job_type":"createServer","begin_time":"2016-05-10T21:35:05.879Z","end_time":"","error_code":null,"fail_reason":null}
#{"status":"RUNNING","entities":{"sub_jobs_total":1,"sub_jobs":[]},"job_id":"2c9eb2c15487316901549c98469a7e9e","job_type":"createServer","begin_time":"2016-05-10T21:35:05.879Z","end_time":"","error_code":null,"fail_reason":null}
…
#{"status":"RUNNING","entities":{"sub_jobs_total":1,"sub_jobs":[{"status":"RUNNING","entities":{},"job_id":"2c9eb2c15487316901549c9864af7ea1","job_type":"createSingleServer","begin_time":"2016-05-10T21:35:13.583Z","end_time":"","error_code":null,"fail_reason":null}]},"job_id":"2c9eb2c15487316901549c98469a7e9e","job_type":"createServer","begin_time":"2016-05-10T21:35:05.879Z","end_time":"","error_code":null,"fail_reason":null}
#{"status":"SUCCESS","entities":{"sub_jobs_total":1,"sub_jobs":[{"status":"SUCCESS","entities":{"server_id":"03df968c-72d5-40bf-8f7a-0ac2914c864b"},"job_id":"2c9eb2c15487316901549c9864af7ea1","job_type":"createSingleServer","begin_time":"2016-05-10T21:35:13.583Z","end_time":"2016-05-10T21:38:25.672Z","error_code":null,"fail_reason":null}]},"job_id":"2c9eb2c15487316901549c98469a7e9e","job_type":"createServer","begin_time":"2016-05-10T21:35:05.879Z","end_time":"2016-05-10T21:38:34.569Z","error_code":null,"fail_reason":null}
#ECS Creation status: SUCCESS Using OpenStack CLI tools
- Compute (nova v2) - Nova: 2.22.0 – 2.23.3 (Kilo)
- Images (glance v2) - Glance: 0.15.0 – 0.16.0 (0.17.x does not work, 1.1.0 does)
- Block Storage (cinder v2) - Cinder: 1.1.1 – 1.3.1 (all versions tested worked)
- Networking (neutron v2) - Neutron: 2.3.11/2.3.12 (newest versions fail)
- IAM (keystone v3) - Keystone: 1.3.4 (only libs needed, client itself deprecated)
- OpenStack: 1.0.4 – latest
OTC is based on OpenStack Juno plus backports plus extensions ...
Recommended client tools
Preinstalled in openSUSE42.x images (OBS)
docker container (Ubuntu) - docker pull tsiotc/otc-client/
Use pip otherwise
OpenStack CLI example
garloff@ImgFact-JumpHost2:~ [0]$ nova list
No handlers could be found for logger "keystoneclient.auth.identity.generic.base"
+--------------------------------------+--------------------+--------+------------+-------------+---------------------------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+--------------------+--------+------------+-------------+---------------------------------------------------+
| 04213851-9ce9-4fbe-b2eb-42f950822e1c | ecs-8430 | ACTIVE | - | Running | b4690bfa-6d61-40b4-86b5-e8ab903ef11e=192.168.33.5 |
| 3902c40e-8f63-49e7-9a46-3147834c9a7d | test-os421-longssh | ACTIVE | - | Running | b4690bfa-6d61-40b4-86b5-e8ab903ef11e=192.168.33.4 |
+--------------------------------------+--------------------+--------+------------+-------------+---------------------------------------------------+
garloff@ImgFact-JumpHost2:~ [0]$ glance image-list
+--------------------------------------+------------------------------------------------+
| ID | Name |
+--------------------------------------+------------------------------------------------+
| d5fd5821-c8df-4517-9f5c-ab1be9c2f21e | Enterprise_Windows_ENT_2008R2SP1_20160617-0 |
| b66fb2d3-2fc1-426a-a0af-c9dbc7210612 | Enterprise_Windows_STD_2012R2_20160617-0 |
| f6939c3e-d49b-410d-b62c-caa017e4c0c1 | Standard_openSUSE_42.1_JeOS_latest |
| be2fdeeb-5b90-4649-a30c-0ede922ba1ec | Standard_openSUSE_42.1_Docker_latest |
| 59c0757e-dbef-4061-baca-7de06247de57 | Enterprise_SLES12_SP1_latest |
[...]
garloff@ImgFact-JumpHost2:~ [0]$ neutron subnet-list
+--------------------------------------+--------------+-----------------+----------------------------------------------------+
| id | name | cidr | allocation_pools |
+--------------------------------------+--------------+-----------------+----------------------------------------------------+
| 3a3baa38-f977-4824-89a2-cadd3cadce97 | subnet-33$$0 | 192.168.33.0/24 | {"start": "192.168.33.1", "end": "192.168.33.254"} |
| c7950de4-fbb9-4d7d-a5df-68a295a60fa2 | subnet-32 | 192.168.32.0/24 | {"start": "192.168.32.1", "end": "192.168.32.254"} |
+--------------------------------------+--------------+-----------------+----------------------------------------------------+
garloff@ImgFact-JumpHost2:~ [2]$ cinder list
+--------------------------------------+-----------+--------------------------------+------+-------------+----------+--------------------------------------+
| ID | Status | Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+-----------+--------------------------------+------+-------------+----------+--------------------------------------+
| 16a138a9-57a0-444c-9b61-99b0e84cb313 | available | CentOS-67-volume-0000 | 4 | SATA | true | |
| 3bd6bd9e-4399-48e9-be4d-67d390042d51 | in-use | ecs-8430-volume-0000 | 4 | SATA | true | 04213851-9ce9-4fbe-b2eb-42f950822e1c |
| 4084c3b8-bb90-4ba3-b9ee-76236a4a4070 | in-use | test-os421-longssh-volume-0000 | 4 | SATA | true | 3902c40e-8f63-49e7-9a46-3147834c9a7d |
[...]
garloff@ImgFact-JumpHost2:~ [0]$ openstack server list
+--------------------------------------+--------------------+--------+---------------------------------------------------+
| ID | Name | Status | Networks |
+--------------------------------------+--------------------+--------+---------------------------------------------------+
| 04213851-9ce9-4fbe-b2eb-42f950822e1c | ecs-8430 | ACTIVE | b4690bfa-6d61-40b4-86b5-e8ab903ef11e=192.168.33.5 |
| 3902c40e-8f63-49e7-9a46-3147834c9a7d | test-os421-longssh | ACTIVE | b4690bfa-6d61-40b4-86b5-e8ab903ef11e=192.168.33.4 |
+--------------------------------------+--------------------+--------+---------------------------------------------------+
neutron support significantly enhanced recently
OpenStack: Create VM
garloff@ImgFact-JumpHost2:~ [0]$ cinder create --image-id bdf485be-8f6d-416e-aedd-2f4df02897bf \
--availability-zone eu-de-01 --volume-type SATA --name OSTEST_VOLUME 10
+---------------------------------------+--------------------------------------------------------------------------------------------------+
| Property | Value |
+---------------------------------------+--------------------------------------------------------------------------------------------------+
| attachments | [] |
| availability_zone | eu-de-01
[...]
| id | 1b98ad0e-60eb-40aa-bb3c-5dcb3bbbd28d
[...]
| status | creating
[...]
# Wait for availability (polling, sigh!)
garloff@ImgFact-JumpHost2:~ [0]$ cinder show 1b98ad0e-60eb-40aa-bb3c-5dcb3bbbd28d
[...]
| status | available
[...]
garloff@ImgFact-JumpHost2:~ [0]$ nova boot --flavor computev1-1 --boot-volume 1b98ad0e-60eb-40aa-bb3c-5dcb3bbbd28d --nic net-id=c095a6be-0d22-4393-8e46-05660e32da3f --availability-zone eu-de-01 --key-name SSHkey-205a --security_groups fa96c994-b43c-4793-b763-7b624e690511 OSTEST_VM
No handlers could be found for logger "keystoneclient.auth.identity.generic.base"
+--------------------------------------+--------------------------------------------------+
| Property | Value |
+--------------------------------------+--------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | eu-de-01 |
[...]
| id | f38b4383-0007-46de-9791-7ecd11357895 |
[...]
| status | BUILD |
[...]
garloff@ImgFact-JumpHost2:~ [0]$ nova show f38b4383-0007-46de-9791-7ecd11357895
No handlers could be found for logger "keystoneclient.auth.identity.generic.base"
+----------------------------------------------+----------------------------------------------------------+
| Property | Value |
+----------------------------------------------+----------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
[...]
| status | ACTIVE |
[...]
Previously a third step (port create) would have been needed
CLI summary
We cheated:
- Network(VPC), subnet, keypair preexisting
- Can be created using otc or openstack CLI tools
(the latter since shortly, thanks to neutron update) - Complete example will be shown in next blog ...
Real world example:
- Two AZ setup
- JumpHosts, Loadbalancers, Database, BT, Web Tier
Somewhat tedious ...
Loadbalancers need non-native calls
Resources
Open Build Service
RPMs / DEBs (uvp-tools, otc-tools, openstackclient, ...)
OpenTelekomCloud Blog
ImageFactory
Cheat Sheet
| Service | openstack | OTC |
|---|---|---|
| compute (VMs) | nova v2 (server) | ECS |
| block storage | cinder v2 (volume) | EVS |
| object storage | [SWIFT] |
OBS (S3) |
| network |
neutron (net, router, floating IP) | VPC (EIP) |
| images |
glance v2 |
IMS |
| authentication |
(keystone v3) |
IAM / MyWorkPl |
| LoadBalancer |
ELB |
|
| Relat Database |
(trove) | RDS (1.0.1) |
| Containers | (magnum) | CCE (1.0.1) |
API abstraction & orchestration
ansible
Configuration Management Tool
- Suitable for application management
- But is prepared for REST calls
- PoC: ansible-otc on github
# secrets.yml: account data to adjust
USERNAME: "14610140 OTC00000000001000000210"
PASSWORD: "XXXXXXXXXXXXXX"
DOMAIN: "OTC00000000001000000210"
PROJECT_ID: "72007ea3e3324e42ad508c278c33e1f5"
[...]
garloff@ImgFact-JumpHost2:~/Ansible/ansible-otc [0]$ ansible-playbook -i hosts subnet.yml
PLAY [all] *********************************************************************
[...]
TASK [subnet : debug] **********************************************************
ok: [localhost] => {
"msg": {
"subnets": [
{
"availability_zone": "eu-de-02",
"cidr": "192.168.70.0/23",
"dhcp_enable": true,
"gateway_ip": "192.168.70.1",
"id": "8dbf6df6-e904-4824-87df-96594a5a86d7",
"name": "subnet-70",
"primary_dns": "100.125.4.25",
"secondary_dns": "217.150.148.148",
"status": "ACTIVE",
"vpc_id": "a9ab8cde-1e99-40af-830c-d826f09fe475"
},
{
"availability_zone": "eu-de-01",
"cidr": "192.168.68.0/23",
"dhcp_enable": true,
"gateway_ip": "192.168.68.1",
"id": "d83f62df-a36e-4bec-bfc1-84937ee0bd02",
"name": "subnet-68",
"primary_dns": "100.125.4.25",
"secondary_dns": "217.150.148.148",
"status": "ACTIVE",
"vpc_id": "a9ab8cde-1e99-40af-830c-d826f09fe475"
},
{
"availability_zone": "eu-de-02",
"cidr": "192.168.4.0/24",
HEAT
Pros
- Native OpenStack solution - fits the OTC brand
- Supports AWS
- Open source
Cons
- No support for Google, Azure or VM Ware
- No Web GUI
Planned for OTC 1.1 (2016Q4)
heat_template_version: 2015-04-30
description: Simple template to deploy a single compute instance
resources:
my_instance:
type: OS::Nova::Server
properties:
key_name: SSHkey-210
image: Standard_openSUSE_42.1_latest
flavor: m1.smallScalr
Pros
- Simple Paradigm: “If this than that” event based configuration
- Multi platform support including AWS, Google, Azure and OpenStack
- Open source
Cons
- Script callbacks contains logic – low abstraction, implementation can be crude
- No SuSE guests are supported
- No support for application tenant design
OTC support planned 7/2016
Libcloud
Pros
- Multi platform support including AWS, Google, Azure and OpenStack
- Configuration of container and support services (~30 Envs and Services)
- Configuration of Kubernetis Cluster
- Open source (apache foundation)
Cons
- Only python program library as interface
- No support for application tenant design
bosh
Pros
- Unique: Configuration of warden container (Cloudfoundry)
- Multi platform support including AWS, Azure, OpenStack
- Configuration of docker containers
- Open source
Cons
- Currently no support for Google platform, but is planned
- No Web GUI
- Looks “bottom up” – acceptance has to be checked
Support in OTC mostly done
juju
Pros
- Graphical tenant design (CLI is offered too)
- Multi platform support including AWS, Google, Azure and OpenStack
- Management of container* and support services
- Open source, hub for “Bundles” and “Charms”
Cons
- (*) Juju focus on LXC and LXD container, docker container is encapsulated in LXD container
OTC: lack of keystone v2 and nova-network
terraform
Pros
- Multi platform support including AWS, Google, Azure and OpenStack
- Management of container and support services
- Open source
Cons
- No support for application tenant design or templates
- No GUI, unclear if RESTful API
- No Support of VM Ware
OTC: support progressing ...
CliQr
Pros
- Unique: Graphical tenant design (CLI and RESTful API is available too)
- Multi platform support including AWS, Google, Azure, VMware, OpenStack
- Management of container and support services
- Application Templates including market place to share them
- Application Monitoring
Cons
- Not open source, product of Cisco
More ...
Which ones?
You tell us ...
Rundeck (CERN)
GooCloud?
...
Outlook: Containers
... will be the unit of management for many next generation applications
VMs beneath expected to be managed automatically?
Deployment models still emerging
Technologies still emerging
Docker, CoreOS, Kubernetes, Mesos, ...
OTC: Cloud Container Engine (Docker/Kubernetes) in 1.0.1 (7/2016)
Conclusions
API support for Automation:
Needed for Agility and Scaling out
Clear commitment to provide native OpenStack APIs on Open Telekom Cloud is progressing
Demonstrated otc-tools, openstack native and ansible
Vast amount of API abstraction tools available
Broad tool support requires more than DefCore APIs
Questions &
Call to Action
Use Open Telekom Cloud
Use the API and Tools
Feedback welcome - where are we lacking?
Work with us
- or FOR us!
Credits
Hui-Bin Ma: Great support with API enablement
Christian Kortwich: API abstration tools survey
Zsolt Nagy: otc-tools (also python+Java tools)
Frank K / Reik K: ansible-otc
Bernd R & Anthony C: API testing ...
