This issue occurs when a node rejoins a Proxmox cluster using the same IP address or there are no static DNS entries for Proxmox nodes.
Even if the passwordless SSH works between nodes, we may see an error as following through GUI when trying to migrating or replicating:
2018-11-09 08:48:23 # /usr/bin/ssh -e none -o 'BatchMode=yes' -o 'HostKeyAlias=PMX01' This email address is being protected from spambots. You need JavaScript enabled to view it. /bin/true
2018-11-09 08:48:23 @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
2018-11-09 08:48:23 @ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
2018-11-09 08:48:23 @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
2018-11-09 08:48:23 IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
2018-11-09 08:48:23 Someone could be eavesdropping on you right now (man-in-the-middle attack)!
2018-11-09 08:48:23 It is also possible that a host key has just been changed.
2018-11-09 08:48:23 The fingerprint for the RSA key sent by the remote host is
2018-11-09 08:48:23 SHA256:AwjDV7HjOjWaRruzdf4453452223JIkugHk1I7HFcVLfG+lx+wOAg.
2018-11-09 08:48:23 Please contact your system administrator.
2018-11-09 08:48:23 Add correct host key in /root/.ssh/known_hosts to get rid of this message.
2018-11-09 08:48:23 Offending RSA key in /etc/ssh/ssh_known_hosts:11
2018-11-09 08:48:23 remove with:
2018-11-09 08:48:23 ssh-keygen -f "/etc/ssh/ssh_known_hosts" -R pmx01
2018-11-09 08:48:23 RSA host key for pmx01 has changed and you have requested strict checking.
2018-11-09 08:48:23 Host key verification failed.
2018-11-09 08:48:23 ERROR: migration aborted (duration 00:00:00): Can't connect to destination address using public key
TASK ERROR: migration aborted
The reason for this error is Scripts uses the hostname rather than IP address to access other Proxmox nodes. So there need to be SSH keys attached to the hostname. If Proxmox nodes are set up with DNS entries when they are joined to the cluster, the joining process creates the SSH keys and attaches the hostname with the keys.
The Solution
1. First, ensure that passwordless SSH works as expected by logging into one of the Proxmox nodes then accessing the node causing the host key verification issue using the following command:
$ ssh <destination_IP>
2. Add static DNS entries as following in /etc/hosts file or in the DNS servers the Proxmox nodes are pointed to:
X.X.X.X <hostname>
3. Run the following command from the source Proxmox node to copy ssh key for the destination host:
$ ssh-copy-id <destination_hostname>
Note here that do not use the IP address of the destination node. Use hostname only.
4. Test that issue is now fixed by accessing the destination node through SSH using hostname as follows:
$ ssh <destination_hostname>
Tips
Depending on how many nodes are having this issue, you may have to follow this instruction multiple times for the nodes. This solution can also be applied to any Linux distribution having SSH host key verification issue.
