UPDATE 29/04/2017
Feedback accepted at ask.openstack.org
https://ask.openstack.org/en/question/105905/vm-supposed-to-have-two-nic-cant-bring-up-second-one/
END UPDATE
Posting bellow addresses one of currently pending questions at
ask.openstack,org regarding adding second vnic to currently
running cloud guest which private IP and VNIC associated with neutron
port are located on different private subnet. The one which was
selected booting this guest originally.
For RH's overcloud guests as F24 or CentOS7 adding instruction
"GATEWAYDEV=eth0"
to /etc/sysconfig/network works on RDO Ocata as smoothly as it did
in IceHouse times,preventing original gateway device from changing
after attaching neutron port on new network.Attaching new neutron
port happens during overcloud VM runtime.
Creating ifcfg-eth1 allows to bring up eth1 via dhcp request been
sent to another dnsmasq daemon serving new private network.
*****************************************
Topology been utilized for testing :-
*****************************************
[stack@undercloud ~]$ date
Thu Apr 27 20:34:50 UTC 2017
[stack@undercloud ~]$ nova-manage --version
15.0.4
[stack@undercloud ~]$ openstack server list
+----------------------+----------------------+--------+-----------------------+----------------+
| ID | Name | Status | Networks | Image Name |
+----------------------+----------------------+--------+-----------------------+----------------+
| 134f6705-8156-4284 | overcloud- | ACTIVE |ctlplane=192.168.24.6 | overcloud-full |
| -acce-0610ceec8c39 | controller-1 | | | |
| 53f45785-5a47-4008-9 | overcloud- | ACTIVE |ctlplane=192.168.24.18 | overcloud-full |
| 147-12c32b7fbdbc | controller-2 | | | |
| 5aab1903-7cdf-4d00 | overcloud- | ACTIVE |ctlplane=192.168.24.13 | overcloud-full |
| -afce-060b2f5fac64 | novacompute-0 | | | |
| 9928671e-81ae-466c- | overcloud- | ACTIVE |ctlplane=192.168.24.11 | overcloud-full |
| b65b-d04b9fa0b3fd | controller-0 | | | |
| 97167614-961e-46f3 | overcloud- | ACTIVE | ctlplane=192.168.24.7 | overcloud-full |
| -841c-6f834d42162d | novacompute-1 | | | |
+----------------------+----------------------+--------+-----------------------+----------------+
*****************
Launch VM
*****************
[root@overcloud-controller-0 ~]# openstack image list
+--------------------------------------+-----------------+--------+
| ID | Name | Status |
+--------------------------------------+-----------------+--------+
| b2123182-3350-4073-a405-4999383d2c9b | VF24Cloud-image | active |
+--------------------------------------+-----------------+--------+
[root@overcloud-controller-0 ~]# openstack network list
+-------------------------------+-------------------------------+-------------------------------+
| ID | Name | Subnets |
+-------------------------------+-------------------------------+-------------------------------+
| 02db6681-fec5-4d10-ae38-18021 | internal01 | a7b8e363-0f2b-4eda- |
| d6b0132 | | 8ea7-23d396a95e70 |
| 2f954150-fe15-4e4f- | ext-net | 8265b729-64dc-4c6d-bf8b- |
| 9d26-60ecfd334186 | | 77ef40828fd1 |
| 373f2bc3-c078-42dc- | HA network tenant 0a3c66ba5fc | ac059474-3f58-48c9-b433-d3701 |
| ba97-c91d490f9364 | 74b72bb44d1620b36fb30 | be6d15c |
| fa984290-816e- | internal02 | 0b3115c1-88d3-4eb7-b120-9a099 |
| 4c17-bec1-0163aa68098a | | 4456378 |
+-------------------------------+-------------------------------+-------------------------------+
[root@overcloud-controller-0 ~]# openstack server create --flavor 2 \
--key-name oskey042717 --image b2123182-3350-4073-a405-4999383d2c9b \
--nic net-id=02db6681-fec5-4d10-ae38-18021d6b0132 vf24devs01
+-------------------------------------+--------------------------------------------------------+
| Field | Value |
+-------------------------------------+--------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | |
| OS-EXT-SRV-ATTR:host | None |
| OS-EXT-SRV-ATTR:hypervisor_hostname | None |
| OS-EXT-SRV-ATTR:instance_name | |
| OS-EXT-STS:power_state | NOSTATE |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| OS-SRV-USG:launched_at | None |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | |
| adminPass | krWUrXZPzG3h |
| config_drive | |
| created | 2017-04-27T18:46:30Z |
| flavor | m1.small (2) |
| hostId | |
| id | b9ba03c7-8336-4ff9-b861-60ee292c8491 |
| image | VF24Cloud-image (b2123182-3350-4073-a405-4999383d2c9b) |
| key_name | oskey042717 |
| name | vf24devs01 |
| progress | 0 |
| project_id | 0a3c66ba5fc74b72bb44d1620b36fb30 |
| properties | |
| security_groups | name='default' |
| status | BUILD |
| updated | 2017-04-27T18:46:30Z |
| user_id | 363f2f57000b4d378514aadaab142804 |
| volumes_attached | |
+-------------------------------------+--------------------------------------------------------+
[root@overcloud-controller-0 ~]# nova list
+--------------------------------------+------------+--------+------------+-------------+---------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+------------+--------+------------+-------------+---------------------+
| b9ba03c7-8336-4ff9-b861-60ee292c8491 | vf24devs01 | ACTIVE | - | Running | internal01=50.0.0.7 |
+--------------------------------------+------------+--------+------------+-------------+----------------------
Assign FIP (192.168.24.105) via dashboard
[root@overcloud-controller-0 ~]# nova list
+--------------------------------------+------------+--------+------------+-------------+-----------------------------------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+------------+--------+------------+-------------+-----------------------------------------------------------+
| b9ba03c7-8336-4ff9-b861-60ee292c8491 | vf24devs01 | ACTIVE | - | Running | internal01=50.0.0.7, 192.168.24.105; internal02=70.0.0.12 |
+--------------------------------------+------------+--------+------------+-------------+-----------------------------------------------------------+
[root@overcloud-controller-0 ~]# openstack network list
+-------------------------------+-------------------------------+-------------------------------+
| ID | Name | Subnets |
+-------------------------------+-------------------------------+-------------------------------+
| 02db6681-fec5-4d10-ae38-18021 | internal01 | a7b8e363-0f2b-4eda- |
| d6b0132 | | 8ea7-23d396a95e70 |
| 2f954150-fe15-4e4f- | ext-net | 8265b729-64dc-4c6d-bf8b- |
| 9d26-60ecfd334186 | | 77ef40828fd1 |
| 373f2bc3-c078-42dc- | HA network tenant 0a3c66ba5fc | ac059474-3f58-48c9-b433-d3701 |
| ba97-c91d490f9364 | 74b72bb44d1620b36fb30 | be6d15c |
| fa984290-816e- | internal02 | 0b3115c1-88d3-4eb7-b120-9a099 |
| 4c17-bec1-0163aa68098a | | 4456378 |
+-------------------------------+-------------------------------+-------------------------------+
Here we create new neutron port belongs newly created network :-
[root@overcloud-controller-0 ~]# neutron port-create internal02
Created a new port:
+-----------------------+-----------------------------------------------------------------------+
| Field | Value |
+-----------------------+-----------------------------------------------------------------------+
| admin_state_up | True |
| allowed_address_pairs | |
| binding:host_id | |
| binding:profile | {} |
| binding:vif_details | {} |
| binding:vif_type | unbound |
| binding:vnic_type | normal |
| created_at | 2017-04-27T18:54:11Z |
| description | |
| device_id | |
| device_owner | |
| extra_dhcp_opts | |
| fixed_ips | {"subnet_id": "0b3115c1-88d3-4eb7-b120-9a0994456378", "ip_address": |
| | "70.0.0.12"} |
| id | 35f5c0a8-41a2-47b6-8a16-0ac8f77d0f93 |
| mac_address | fa:16:3e:fd:d1:ea |
| name | |
| network_id | fa984290-816e-4c17-bec1-0163aa68098a |
| port_security_enabled | True |
| project_id | 0a3c66ba5fc74b72bb44d1620b36fb30 |
| qos_policy_id | |
| revision_number | 5 |
| security_groups | d799177b-f826-472b-9540-befd8b0da76f |
| status | DOWN |
| tags | |
| tenant_id | 0a3c66ba5fc74b72bb44d1620b36fb30 |
| updated_at | 2017-04-27T18:54:12Z |
+-----------------------+-----------------------------------------------------------------------+
Here we attach neutron port belongs newly created network to cloud VM running on different
private network:-
[root@overcloud-controller-0 ~]# nova interface-attach --port-id \
35f5c0a8-41a2-47b6-8a16-0ac8f77d0f93 vf24devs01
[boris@fedora24wks ~]$ ssh -i oskey042717.pem fedora@192.168.24.105
Last login: Thu Apr 27 19:05:16 2017 from 192.168.24.1
[fedora@vf24devs01 ~]$ sudo su -
[root@vf24devs01 ~]# cat /etc/sysconfig/network
NETWORKING=yes
NOZEROCONF=yes
DEVTIMEOUT=10
GATEWAYDEV=eth0<==== eth0 is set manually
as gateway device for running VMs
after attaching port belongs to internal02
Next commands
# cd /etc/sysconfig/network-scripts******************************************************************************************* Skipping setting GATEWAYDEV will result to hang ssh connection via FIP and force to manage like bellow:- ********************************************************************************************
# cp ifcfg-eth0 ifcfg-eth1
Edit ifcfg-eth1 to match "eth1" entry
# ifup eth1
[root@overcloud-controller-0 ~]# ip netns
qdhcp-773be634-da70-4213-b653-7a510097f3c1
qdhcp-1bec9cb9-fc19-4686-bb30-d24c909a11b9
qrouter-f0da93c3-3e1f-4eb5-883e-c86692b07ff9
[root@overcloud-controller-0 ~]# ip netns exec \
qdhcp-773be634-da70-4213-b653-7a510097f3c1 \
ssh -i oskey042817.pem fedora@70.0.0.12
The authenticity of host '70.0.0.12 (70.0.0.12)' can't be established.
ECDSA key fingerprint is 52:23:09:65:a3:78:90:85:e5:80:79:89:de:d3:48:a2.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '70.0.0.12' (ECDSA) to the list of known hosts.
Last login: Fri Apr 28 15:06:46 2017 from 192.168.24.1
[fedora@vf24devs01 ~]$ sudo su -Same procedure works fine on CentOS 7.3 Cloud Guest
[root@vf24devs01 ~]# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450
inet 50.0.0.13 netmask 255.255.255.0 broadcast 50.0.0.255
inet6 fe80::f816:3eff:fec0:eb37 prefixlen 64 scopeid 0x20<link>
ether fa:16:3e:c0:eb:37 txqueuelen 1000 (Ethernet)
RX packets 576 bytes 58794 (57.4 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 481 bytes 63460 (61.9 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450
inet 70.0.0.12 netmask 255.255.255.0 broadcast 70.0.0.255
inet6 fe80::f816:3eff:fe06:86b0 prefixlen 64 scopeid 0x20<link>
ether fa:16:3e:06:86:b0 txqueuelen 1000 (Ethernet)
RX packets 81 bytes 10785 (10.5 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 399 bytes 23427 (22.8 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1 (Local Loopback)
RX packets 139 bytes 39092 (38.1 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 139 bytes 39092 (38.1 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@vf24devs01 ~]# ip route
default via 70.0.0.1 dev eth1
50.0.0.0/24 dev eth0 proto kernel scope link src 50.0.0.13
70.0.0.0/24 dev eth1 proto kernel scope link src 70.0.0.12
169.254.169.254 via 50.0.0.1 dev eth0 proto static
******************************************************************************************
Update /etc/sysconfig/network as requested followed by cloud VM restart
That will switch back to default Gateway device eth0 and recover floating IP for VM
******************************************************************************************[root@vf24devs01 ~]# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450
inet 50.0.0.7 netmask 255.255.255.0 broadcast 50.0.0.255
inet6 fe80::f816:3eff:feab:739e prefixlen 64 scopeid 0x20<link>
ether fa:16:3e:ab:73:9e txqueuelen 1000 (Ethernet)
RX packets 384 bytes 43397 (42.3 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 383 bytes 41874 (40.8 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450
inet 70.0.0.12 netmask 255.255.255.0 broadcast 70.0.0.255
inet6 fe80::f816:3eff:fefd:d1ea prefixlen 64 scopeid 0x20<link>
ether fa:16:3e:fd:d1:ea txqueuelen 1000 (Ethernet)
RX packets 6 bytes 760 (760.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 11 bytes 1048 (1.0 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1 (Local Loopback)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@overcloud-controller-0 ~]# openstack port list | grep 50.0.0.7
| 95a4666d-eac8-461c-87b8-3fcaf797c263 | | fa:16:3e:ab:73:9e | ip_address='50.0.0.7', subnet_id='a7b8e363-0f2b-4eda-8ea7-23d396a95e70' | ACTIVE |
[root@overcloud-controller-0 ~]# openstack port list | grep 70.0.0.12
| 35f5c0a8-41a2-47b6-8a16-0ac8f77d0f93 | | fa:16:3e:fd:d1:ea | ip_address='70.0.0.12', subnet_id='0b3115c1-88d3-4eb7-b120-9a0994456378' | ACTIVE |
[root@overcloud-controller-0 ~]# openstack subnet list
+------------------------+------------------------+--------------------------+------------------+
| ID | Name | Network | Subnet |
+------------------------+------------------------+--------------------------+------------------+
| 0b3115c1-88d3-4eb7-b12 | sub-internal02 | fa984290-816e- | 70.0.0.0/24 |
| 0-9a0994456378 | | 4c17-bec1-0163aa68098a | |
| 8265b729-64dc-4c6d- | ext-subnet | 2f954150-fe15-4e4f- | 192.168.24.0/24 |
| bf8b-77ef40828fd1 | | 9d26-60ecfd334186 | |
| a7b8e363-0f2b-4eda- | sub-internal01 | 02db6681-fec5-4d10-ae38- | 50.0.0.0/24 |
| 8ea7-23d396a95e70 | | 18021d6b0132 | |
| ac059474-3f58-48c9-b43 | HA subnet tenant 0a3c6 | 373f2bc3-c078-42dc- | 169.254.192.0/18 |
| 3-d3701be6d15c | 6ba5fc74b72bb44d1620b3 | ba97-c91d490f9364 | |
| | 6fb30 | | |
+------------------------+------------------------+--------------------------+------------------+![]()
Instance restartedLogin into VM via qdhcp namespace
References
1.https://thornelabs.net/2014/09/03/configure-multiple-network-interfaces-on-an-openstack-instance.html