This article is part of a series of articles about issues I encountered during implementation of a vSphere stretched cluster based on vSphere 6.7 U1.
You can find the introduction article here
Issue 3
This issue is irritating, to say at least.
You configured your first ESXi host in a cluster, polished it etc… and created a host profile from this host. And then it should be easy to configure the other hosts. But, when you apply the host profile…. the host disconnects (because of a reboot….)… but never comes back….
What happened ??
What happened is that in the wisdom of the vSphere environment the vCenter instructed the host to delete vmk0… yes the kernel interface that vCenter uses to connect to ESXi (assuming the vSphere host is clean installed and has just one kernel port). Delete it to create vmk0 again…..
But it never gets to that part….
Do you recognize this issue ??
I googled for this issue and found blog posts going back to vSphere 5.1. So it looks like a difference in interpretation of how it should work.
Cause
I expect the cause has something to do with the fact that the kernel port is created on installation of ESXi, and not via the vCenter. Assigning it a MAC address that is unknown by vCenter.
Workaround / Solution
The solution is to re-create the kernel port, reboot the host, create the host profile from this host and used it for the others.
The re-creation is done via the server console (iDrac, iLO etc…) due to the fact that by removing the kernel port you also have no connection to manage it.
In short
Below the commands that are used.
assumptions:
- the original vmk0 has a static ipv4 address
- vmk0 is connected to a distributed vSwitch
esxcfg-vswitch -l | grep vmk0 | awk '{print $1}'
esxcli system coredump network set --enable=false
esxcli network ip interface remove --interface-name=vmk0
esxcli network ip interface add --interface-name=vmk0 --dvs-name=<name of dvSwitch> --dvport-id=<port ID for vmk0 on the dvSwitch>
esxcli network ip interface ipv4 set --interface-name=vmk0 --ipv4=<ipv4 address of kernel port> --netmask < subnetmask of kernel port> --type=static
esxcfg-route -a default <ipv4 gateway address>
esxcli network ip interface tag add --interface-name=vmk0 -t Management
esxcli system coredump network set --enable=true
The steps
0 Put the ESXi host in maintenance mode
We put it in maintenance mode so no VMs are present on the host, and that vCenter knows that it can’t use the host at all.
1 (optional) write down the port ID
Look up on the dvSwitch what the port ID is for this vmk0. You need this info later.
This applies only if you use a dvSwitch and vmk0 is connected to the dvSwitch.
esxcfg-vswitch -l | grep vmk0 | awk '{print $1}'
2 make sure the ESXi shell is running
You can do this in the webclient, with a powerCLI command or via the ESXi shell. The ESXi shell instruction is
3 open a session to the server console
4 Go to the ESXi shell
Normaly the server console will display the DCUI interface of ESXi. Switch over to the shell with ALT+F1. A login prompt should be shown. If not then you forgot step 1.
Logon with the root account.
5 (optional) disable network coredump
If network coredump is enabled and configured to use vmk0 disabled it before removing.
esxcli system coredump network set --enable=false
6 remove vmk0 – the management kernel port
esxcli network ip interface remove --interface-name=vmk0
6 Create the a new kernel port
esxcli network ip interface add --interface-name=vmk0 --dvs-name=<name of dvSwitch> --dvport-id=<saved port ID from step 1>
7 configure the port
Assuming that vmk0 has a static IPv4 address.
esxcli network ip interface ipv4 set --interface-name=vmk0 --ipv4=<IPv4 Address of kernel port> --netmask=<subnetMask of kernel port> --static
8 update gateway
esxcfg-route -a default <ipv4 address of vmk0>
9 tag management service
esxcli network ip interface tag add --interface-name=vmk0 -t Management
10 (optional) Enable network coredump
If disabled in step 5, enabled it again.
esxcli system coredump network set --enable=true
11 check that vCenter is connected to the ESXi host
If all went well then the ESXi host should show connected in vCenter.
12 create a Host Profile from this host
Abstract a new host profile from this host, edit the host customization.
13 edit the host customization for all hosts
For the hosts where this host profile should be applied on, edit the host customization
14 apply the host profile
Apply the host profile to the other hosts and reboot them.
I hope this post was usefull, use the comments below if needed.
vB(art)log 