Problems with CPU in incompatible error during VMWare vMotion
One of our users reported problems with vMotion after firmware upgrade of IBM server. Firmware were upgraded using BOMC (utility to create ISO images to upgrade IBM server to the lastest versions available).
After upgrade user was not able to do vMotion. They received following error:
Host CPU is incompatible with the virtual machine’s requirements at CPUID level 0x1 register ‘ecx’.
After couple minutes of googling I found out following official articles:
-
What the AES support is
Based on information there is change when you do firmware upgrade. Sometime (from some version) AES feature is Enabled and sometime is disabled. This setting cannot be changed in BIOS and this is really sad. So let’s look at the way you can change it.
In second mentioned article IBM suppose to download some prepared ISO image called BoMC-2.20-uEFI-AesEnable-to-enabled-vmotion-fix.iso. But I couldn’t find this ISO image. So I had to use other way around. I downloaded IBM Advanced Setting Utility. I downloaded and installes version for Windows 64b. You need your RSA card to be accessible through network. Then you can run following command to check actual value:
asu64.exe showvalues UEFI.AesEnable –host RSA_IP_ADDRESS –user RSA_USER –password RSA_PASSWORD
You can check it on all ESX servers. To decide whether feature should be enabled or disabled you this article. We need to change it to the same setting using following command:
asu64.exe set UEFI.AesEnable Disable –host RSA_IP_ADDRESS –user RSA_USER –password RSA_PASSWORD
When values are the same on all ESX server you are able to do vMotion. I was kinda mad on IBM, because I would preffer to change this value in BIOS instead of some application.
I have to do this. Can you install the windows version on your desktop and then connect to the ESXi host? What I read you had to copy the file to the hard drive of the esxi server and then launch it? If you could give me some more steps on how you actually used the application would help me. Thanks!
@Ryan
Hello Ryan. You need to download IBM Advanced Settings Utility on some Windows/Linux machine. This machine has to have network connectivity to your ESXi server’s RSA card. When you download it, you can run command to change the settings on ESX server.
There is nothing to to copy into ESXi server and nothing to launch on ESXi server.
All the trick is to change some UEFI setting using remote application downloaded on some Windows/Linux machine.
I hope not it’s clear to you 🙂
Hello,
with asu64.exe on flex x240 I’m runing command:
asu64.exe showvalues UEFI.AesEnable –host XXX.XXX.XXX.XXX –user USERNAME –password HOST_PASSWORD
And I got this:
IBM Advanced Settings Utility version 9.61.85B
Licensed Materials – Property of IBM
(C) Copyright IBM Corp. 2007-2014 All Rights Reserved
Connected to IMM at IP address XXX.XXX.XXX.XXX
Could not find setting UEFI.AesEnable
OS in ESXi 5.1 Any idea?
@Grega
Is your server one of those mentioned in http://www-947.ibm.com/support/entry/portal/docdisplay?lndocid=MIGR-5086963 ? Do you have updated BIOS/UEFI?
And it looks this is not affected on ESX version 5.1 (http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1991)
@Ondrej
No, our server is not on that list. We have flex x240. I have update UEFI to the latest.
@Grega
It means this problems “should not” apply to you. Are you having some problems?
@Ondrej
Yes, vMotion is not working. I get this message:
The virtual machine requires hardware features that are unsupported or disabled on the target host:
* Carryless multiply (PCLMULQDQ) (see KB 1034926)
* SSE4.2
* AES instructions (AES-NI) (see KB 1034926)
@Grega
Have you checked http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1034926 ? Look into BIOS if you have AES enabled.
Other way I suppose you should open ticket @VMWare or @IBM. Your server is not listed under affected servers so it can be caused by something else.
Hi Ondrej,
I have an IBM 3650 M2 server, and I have a problem vmotion. I’ve tried to disable AES, but there is an error message “could not find the setting uEFI.AesEnable” “vmotion fix failed”.
if you have a solution?
thank you.
@rony
Hi Rony. What ESX version do you have? What processor do you have in this server?
@Ondrej Žilinec
vmware 5.5
IBM System x3650 M2 7947 – Xeon E5520 2.26 GHz
But this should be a issue only up to ESXi version 5.0.
Do you have virtualization support enabled on your CPUs in BIOS?
Hi guys, someone could correct the error in the server x240? I have the same problem
Hi!
I can not activate AES-NI on my IBM x3400 M3 server. In uEFI settings there is no option for AES-NI Enable/Disable. I updated uEFI firmware but again the same problem occurs.
IBM Advanced Settings Utility version 9.63.86D
Licensed Materials – Property of IBM
(C) Copyright IBM Corp. 2007-2014 All Rights Reserved
Successfully discovered the IMM via SLP.
Discovered IMM at IP address 169.254.95.118
Connected to IMM at IP address 169.254.95.118
Could not find setting UEFI.AesEnable