openstack集群部署—Nova控制节点集群

发表于 | 分类于 | 浏览 次

nova介绍

Nova 是 OpenStack 最核心的服务，负责维护和管理云环境的计算资源。OpenStack 作为 IaaS 的云操作系统，虚拟机生命周期管理也就是通过 Nova 来实现的。

用途与功能 :
1) 实例生命周期管理
2) 管理计算资源
3) 网络和认证管理
4)REST 风格的 API
5) 异步的一致性通信
6)Hypervisor 透明:支持Xen,XenServer/XCP,KVM, UML,VMware vSphere and Hyper-V

部署

创建nova相关数据库

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
# 在任意控制节点创建数据库，后台数据自动同步，以controller01节点为例；
# nova服务含4个数据库，统一授权到nova用户；
# placement主要涉及资源统筹，较常用的api接口是获取备选资源与claim资源等
[root@controller01 ~]# mysql -u root -p
Enter password: 
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 68
Server version: 10.2.29-MariaDB-log MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> show databases;
+--------------------+
| Database           |
+--------------------+
| glance             |
| information_schema |
| keystone           |
| mysql              |
| performance_schema |
+--------------------+
5 rows in set (0.00 sec)

MariaDB [(none)]> CREATE DATABASE nova_api;
Query OK, 1 row affected (0.01 sec)

MariaDB [(none)]> CREATE DATABASE nova;
Query OK, 1 row affected (0.00 sec)

MariaDB [(none)]> CREATE DATABASE nova_cell0;
Query OK, 1 row affected (0.00 sec)

MariaDB [(none)]> CREATE DATABASE nova_placement;
Query OK, 1 row affected (0.01 sec)

MariaDB [(none)]> GRANT ALL PRIVILEGES ON nova_api.* TO 'nova'@'localhost' IDENTIFIED BY '123456';     
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> GRANT ALL PRIVILEGES ON nova_api.* TO 'nova'@'%' IDENTIFIED BY '123456';     
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> GRANT ALL PRIVILEGES ON nova.* TO 'nova'@'localhost' IDENTIFIED BY '123456';     
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> GRANT ALL PRIVILEGES ON nova.* TO 'nova'@'%' IDENTIFIED BY '123456';     
Query OK, 0 rows affected (0.01 sec)

MariaDB [(none)]> GRANT ALL PRIVILEGES ON nova_cell0.* TO 'nova'@'localhost' IDENTIFIED BY '123456';     
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> GRANT ALL PRIVILEGES ON nova_cell0.* TO 'nova'@'%' IDENTIFIED BY '123456';     
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> GRANT ALL PRIVILEGES ON nova_placement.* TO 'nova'@'localhost' IDENTIFIED BY '123456';     
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]>  GRANT ALL PRIVILEGES ON nova_placement.* TO 'nova'@'%' IDENTIFIED BY '123456';     
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> flush privileges;
Query OK, 0 rows affected (0.01 sec)

MariaDB [(none)]> exit; 
Bye

创建计算服务凭据

创建nova用户

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
[root@controller01 ~]# . admin-openrc
[root@controller01 ~]# openstack user create --domain default --password=123456 nova   
+---------------------+----------------------------------+
| Field               | Value                            |
+---------------------+----------------------------------+
| domain_id           | default                          |
| enabled             | True                             |
| id                  | de3f2e2741ec4fe594df50947b15c679 |
| name                | nova                             |
| options             | {}                               |
| password_expires_at | None                             |
+---------------------+----------------------------------+
[root@controller01 ~]# openstack user create --domain default --password=123456 placement        
+---------------------+----------------------------------+
| Field               | Value                            |
+---------------------+----------------------------------+
| domain_id           | default                          |
| enabled             | True                             |
| id                  | b9c6f0054cbb4f84877eb285bef57ced |
| name                | placement                        |
| options             | {}                               |
| password_expires_at | None                             |
+---------------------+----------------------------------+

赋权

1
2
[root@controller01 ~]# openstack role add --project service --user nova admin 
[root@controller01 ~]# openstack role add --project service --user placement admin

创建nova/placement服务实体

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
[root@controller01 ~]# openstack service create --name nova --description "OpenStack Compute" compute
+-------------+----------------------------------+
| Field       | Value                            |
+-------------+----------------------------------+
| description | OpenStack Compute                |
| enabled     | True                             |
| id          | f3b0b1d757c7426bb2863a9128fef978 |
| name        | nova                             |
| type        | compute                          |
+-------------+----------------------------------+
[root@controller01 ~]# openstack service create --name placement --description "Placement API" placement
+-------------+----------------------------------+
| Field       | Value                            |
+-------------+----------------------------------+
| description | Placement API                    |
| enabled     | True                             |
| id          | 94fd6d420f4348aaad3abd682c7b473b |
| name        | placement                        |
| type        | placement                        |
+-------------+----------------------------------+

创建nova/placement-api

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
# 注意--region与初始化admin用户时生成的region一致；
# api地址统一采用vip，如果public/internal/admin分别使用不同的vip，请注意区分；
# nova-api 服务类型为compute，placement-api服务类型为placement；
# nova public api
[root@controller01 ~]# openstack endpoint create --region RegionOne compute public http://controller01:8774/v2.1
+--------------+----------------------------------+
| Field        | Value                            |
+--------------+----------------------------------+
| enabled      | True                             |
| id           | a22329b89cba44a997b50da400c0843d |
| interface    | public                           |
| region       | RegionOne                        |
| region_id    | RegionOne                        |
| service_id   | f3b0b1d757c7426bb2863a9128fef978 |
| service_name | nova                             |
| service_type | compute                          |
| url          | http://controller01:8774/v2.1    |
+--------------+----------------------------------+
[root@controller01 ~]# openstack endpoint create --region RegionOne compute internal http://controller01:8774/v2.1
+--------------+----------------------------------+
| Field        | Value                            |
+--------------+----------------------------------+
| enabled      | True                             |
| id           | 38b56b4c5bf742fbbdbcee017b8d6eb7 |
| interface    | internal                         |
| region       | RegionOne                        |
| region_id    | RegionOne                        |
| service_id   | f3b0b1d757c7426bb2863a9128fef978 |
| service_name | nova                             |
| service_type | compute                          |
| url          | http://controller01:8774/v2.1    |
+--------------+----------------------------------+

[root@controller01 ~]# openstack endpoint create --region RegionOne compute admin http://controller01:8774/v2.1
+--------------+----------------------------------+
| Field        | Value                            |
+--------------+----------------------------------+
| enabled      | True                             |
| id           | 978179c1c3a6453da009429d6e7017d2 |
| interface    | admin                            |
| region       | RegionOne                        |
| region_id    | RegionOne                        |
| service_id   | f3b0b1d757c7426bb2863a9128fef978 |
| service_name | nova                             |
| service_type | compute                          |
| url          | http://controller01:8774/v2.1    |
+--------------+----------------------------------+
1
2
3
[root@controller01 ~]# openstack endpoint create --region RegionOne placement public http://controller01:8778
[root@controller01 ~]# openstack endpoint create --region RegionOne placement internal http://controller01:8778
[root@controller01 ~]# openstack endpoint create --region RegionOne placement admin http://controller01:8778

安装nova

1
2
3
4
# 在全部控制节点安装nova相关服务，以controller01节点为例
[root@controller01 ~]# yum install openstack-nova-api openstack-nova-conductor \
   openstack-nova-console openstack-nova-novncproxy \
   openstack-nova-scheduler openstack-nova-placement-api -y

配置nova.conf

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
# 在全部控制节点操作，以controller01节点为例；
# 注意”my_ip”参数，根据节点修改；
# 注意nova.conf文件的权限：root:nova
[root@controller01 ~]# cp /etc/nova/nova.conf /etc/nova/nova.conf.bak
[root@controller01 ~]# egrep -v "^$|^#" /etc/nova/nova.conf
[DEFAULT]
my_ip=10.21.0.36
use_neutron=true
firewall_driver=nova.virt.firewall.NoopFirewallDriver
enabled_apis=osapi_compute,metadata
osapi_compute_listen=$my_ip
osapi_compute_listen_port=8774
metadata_listen=$my_ip
metadata_listen_port=8775
# 前端采用haproxy时，服务连接rabbitmq会出现连接超时重连的情况，可通过各服务与rabbitmq的日志查看；
# # transport_url=rabbit://openstack:rabbitmq_pass@controller:5673
# # rabbitmq本身具备集群机制，官方文档建议直接连接rabbitmq集群；但采用此方式时服务启动有时会报错，原因不明；如果没有此现象，强烈建议连接rabbitmq直接对接集群而非通过前端haproxy
transport_url=rabbit://openstack:123456@controller01:5672,controller02:5672
[api]
auth_strategy=keystone
[api_database]
connection=mysql+pymysql://nova:123456@controller01/nova_api
[barbican]
[cache]
backend=oslo_cache.memcache_pool
enabled=True
memcache_servers=controller01:11211,controller02:11211
[cells]
[cinder]
[compute]
[conductor]
[console]
[consoleauth]
[cors]
[crypto]
[database]
connection = mysql+pymysql://nova:nova_d&OmJVoX@controller01/nova
[devices]
[ephemeral_storage_encryption]
[filter_scheduler]
[glance]
api_servers = http://controller01:9292
[guestfs]
[healthcheck]
[hyperv]
[ironic]
[key_manager]
[keystone]
[keystone_authtoken]
auth_url = http://controller:5000/v3
#auth_url = http://controller:35357
memcached_servers = controller01:11211,controller02:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = nova
password = nova
[libvirt]
[matchmaker_redis]
[metrics]
[mks]
[neutron]
[notifications]
[osapi_v21]
[oslo_concurrency]
lock_path=/var/lib/nova/tmp
[oslo_messaging_amqp]
[oslo_messaging_kafka]
[oslo_messaging_notifications]
[oslo_messaging_rabbit]
[oslo_messaging_zmq]
[oslo_middleware]
[oslo_policy]
[pci]
[placement]
region_name = RegionOne
project_domain_name = Default
project_name = service
auth_type = password
user_domain_name = Default
auth_url = http://controller:5000/v3
username = placement
password = placement
[quota]
[rdp]
[remote_debug]
[scheduler]
[serial_console]
[service_user]
[spice]
[upgrade_levels]
[vault]
[vendordata_dynamic_auth]
[vmware]
[vnc]
enabled=true
server_listen=$my_ip
server_proxyclient_address=$my_ip
novncproxy_base_url=http://$my_ip:6080/vnc_auto.html
novncproxy_host=$my_ip
novncproxy_port=6080
[workarounds]
[wsgi]
[xenserver]
[xvp]

配置00-nova-placement-api.conf

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# 在全部控制节点操作，以controller01节点为例；
# 注意根据不同节点修改监听地址
[root@controller01 ~]# cp /etc/httpd/conf.d/00-nova-placement-api.conf /etc/httpd/conf.d/00-nova-placement-api.conf.bak
[root@controller01 ~]# sed -i "s/Listen\ 8778/Listen\ 192.168.182.131:8778/g" /etc/httpd/conf.d/00-nova-placement-api.conf
[root@controller01 ~]# sed -i "s/*:8778/192.168.182.131:8778/g" /etc/httpd/conf.d/00-nova-placement-api.conf
[root@controller01 ~]# echo "

#Placement API
<Directory /usr/bin>
   <IfVersion >= 2.4>
      Require all granted
   </IfVersion>
   <IfVersion < 2.4>
      Order allow,deny
      Allow from all
   </IfVersion>
</Directory>
" >> /etc/httpd/conf.d/00-nova-placement-api.conf

# 重启httpd服务，启动placement-api监听端口
[root@controller01 ~]# systemctl restart httpd

同步nova相关数据库

同步nova相关数据库

1
2
3
4
5
6
7
8
9
10
11
12
13
# 任意控制节点操作；
# 同步nova-api数据库
[root@controller01 ~]# su -s /bin/sh -c "nova-manage api_db sync" nova

# 注册cell0数据库
[root@controller01 ~]# su -s /bin/sh -c "nova-manage cell_v2 map_cell0" nova

# 创建cell1 cell
[root@controller01 ~]# su -s /bin/sh -c "nova-manage cell_v2 create_cell --name=cell1 --verbose" nova

# 同步nova数据库；
# 忽略”deprecated”信息
[root@controller01 ~]# su -s /bin/sh -c "nova-manage db sync" nova

补充：

此版本在向数据库同步导入数据表时，报错：/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py:332: NotSupportedWarning: Configuration option(s) [‘use_tpool’] not supported

exception.NotSupportedWarning

解决方案如下：

bug：https://bugs.launchpad.net/nova/+bug/1746530

pacth：https://github.com/openstack/oslo.db/commit/c432d9e93884d6962592f6d19aaec3f8f66ac3a2

验证

1
2
3
4
5
6
7
8
9
Verify nova cell0 and cell1 are registered correctly:

# su -s /bin/sh -c "nova-manage cell_v2 list_cells" nova
+-------+--------------------------------------+
| Name  | UUID                                 |
+-------+--------------------------------------+
| cell1 | 109e1d4b-536a-40d0-83c6-5f121b82b650 |
| cell0 | 00000000-0000-0000-0000-000000000000 |
+-------+--------------------------------------+

启动服务

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
# 在全部控制节点操作，以controller01节点为例；
# 开机启动
[root@controller01 ~]# systemctl enable openstack-nova-api.service \
  openstack-nova-consoleauth.service \
openstack-nova-scheduler.service \
openstack-nova-conductor.service \
openstack-nova-novncproxy.service

# 启动
[root@controller01 ~]# systemctl restart openstack-nova-api.service
[root@controller01 ~]# systemctl restart openstack-nova-consoleauth.service
[root@controller01 ~]# systemctl restart openstack-nova-scheduler.service
[root@controller01 ~]# systemctl restart openstack-nova-conductor.service
[root@controller01 ~]# systemctl restart openstack-nova-novncproxy.service

# 查看状态
[root@controller01 ~]# systemctl status openstack-nova-api.service \
openstack-nova-consoleauth.service \
openstack-nova-scheduler.service \
openstack-nova-conductor.service \
openstack-nova-novncproxy.service

# 查看端口
[root@controller01 ~]# netstat -tunlp | egrep '8774|8775|8778|6080'

验证

1
2
3
4
5
6
7
8
9
[root@controller01 ~]# . admin-openrc

# 列出各服务组件，查看状态；
# 也可使用命令” nova service-list”
[root@controller01 ~]# openstack compute service list
# 展示api端点
[root@controller01 ~]# openstack catalog list
# 检查cell与placement api运行正常
[root@controller01 ~]# nova-status upgrade check
Donate