After upgrading to vCenter Server 5.5 Update 1, logging in to vCenter Server reports the error: Failed to verify the SSL certificate for one or more vCenter Server systems (2074942)

Well this certificate Issue generates a multiple of probllems. After upgrading an old installation to accommodate for vmware Horizon 6 i discovered that view storage acceleration was not available from my vCenter. One thing led to another and it turns out it’s a certificate issue. My installation was living all this time with a 512bit Key which needed upgrading. After this you need to reregister all vmware servivces relevant.
Here goes some cut and paste

Symptoms

After upgrading from vCenter Server 4.x or 5.x to 5.5 Update 1, you may experience these symptoms:

  • After you log in to vCenter Server using the vSphere Web Client or vSphere Client, you see an error similar to:

    Failed to verify the SSL certificate for one or more vCenter Server systems:https://vc55.domain.com:443/sdk

  • The Performance Charts tab fails and reports the error:

    Perf Charts service experienced an internal error

  • The Host Hardware Status Tab for the ESXi fails and reports the error:

    Cannot access the hardware monitoring service

  • The Storage Views Tab fails and reports the error:

    The server ‘vcenter_domain_name’ could not interpret the client’s request. (The remote server returned an error: (503) Server Unavailable.)

  • The Inventory Service ds.log file (located at C:\ProgramData\VMware\Infrastructure\Inventory Service\Logs) contains entries similar to:

    21:28:19,755 pool-19-thread-2 ERROR com.vmware.vim.dataservices.provider.VcProvider] Cannot login: com.vmware.vim.vmomi.client.exception.SslException: com.vmware.vim.vmomi.core.exception.CertificateValidationException: Server certificate chain not verified
    21:28:19,755 pool-19-thread-2 INFO com.vmware.vim.query.server.provider.impl.ProviderManagerServiceImpl] Cannot connect to provider: com.vmware.vim.query.server.store.exception.UnauthorizedException: not connected
    21:28:20,507 pool-19-thread-2 ERROR com.vmware.vim.dataservices.provider.VcProvider] Cannot login: com.vmware.vim.vmomi.client.exception.SslException: com.vmware.vim.vmomi.core.exception.CertificateValidationException: Server certificate chain not verified
    21:28:20,507 pool-19-thread-2 INFO com.vmware.vim.query.server.provider.impl.ProviderManagerServiceImpl] Cannot connect to provider: com.vmware.vim.query.server.store.exception.UnauthorizedException: not connected
    21:28:21,269 pool-19-thread-2 ERROR com.vmware.vim.dataservices.provider.VcProvider] Cannot login: com.vmware.vim.vmomi.client.exception.SslException: com.vmware.vim.vmomi.core.exception.CertificateValidationException: Server certificate chain not verified
    21:28:21,269 pool-19-thread-2 INFO com.vmware.vim.query.server.provider.impl.ProviderManagerServiceImpl] Cannot connect to provider: com.vmware.vim.query.server.store.exception.UnauthorizedException: not connected

 

  • The vSphere Web Client log vsphere_client_virgo.log file (located at C:\ProgramData\VMware\vSphere Web Client\serviceability\logs) contains entries similar to:

    21:26:51.262] [INFO ] http-bio-9443-exec-7 70000055 100001 200001 com.vmware.vise.vim.commons.vcservice.impl.VcServiceImpl Initializing vmomi for vc – https://vc55.domain.com:443/sdk at VMODL version com.vmware.vim.binding.vim.version.internal.version9
    21:26:51.286] [ERROR] http-bio-9443-exec-7 70000055 100001 200001 com.vmware.vsphere.client.security.VimAuthenticationHandler Connection failure to vc https://vc55.domain.com:443/sdk com.vmware.vim.vmomi.core.exception.CertificateValidationException: Server certificate chain not verified

  • The vCenter Management Web Services vws.log file (located at C:\ProgramData\VMware\VMware VirtualCenter\Logs) contains entries similar to:

    21:25:57,927 localhost-startStop-1 INFO com.vmware.vim.vimclient.VimClientFactory] VMODL context has been initialized for CMS
    21:25:58,191 localhost-startStop-1 ERROR com.vmware.vim.vimclient.VimClientFactory] Failed VC client creation with exception
    com.vmware.vim.vmomi.client.exception.SslException: com.vmware.vim.vmomi.core.exception.CertificateValidationException: Server certificate chain not verified
    . . .
    Caused by: com.vmware.vim.vmomi.core.exception.CertificateValidationException: Server certificate chain not verified
    . . .
    Caused by: javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
    . . .
    21:25:58,194 localhost-startStop-1 ERROR com.vmware.vim.cimmonitor.qs.provider.impl.QsHelperImpl] Vim configuration exception occured while registering provder
    com.vmware.vim.vimclient.exception.VimConfigException: Failed VC client creation with exception

  • The vCenter Server vpxd.log file (located at C:\ProgramData\VMware\VMware VirtualCenter\Logs\) contains entries similar to:

    T21:30:41.084Z [04712 warning ‘ProxySvc’] SSL Handshake failed for stream <io_obj p:0x00000000095fdd88, h:2540, <TCP ‘192.168.2.55:443’>, <TCP ‘192.168.2.55:57823’>>, error: class Vmacore::Ssl::SSLException(SSL Exception: error:14094416:SSL routines:SSL3_READ_BYTES:sslv3 alert certificate unknown)
    T21:30:41.836Z [04712 warning ‘ProxySvc’] SSL Handshake failed for stream <io_obj p:0x0000000009609338, h:2624, <TCP ‘192.168.2.55:443’>, <TCP ‘192.168.2.55:57824’>>, error: class Vmacore::Ssl::SSLException(SSL Exception: error:14094416:SSL routines:SSL3_READ_BYTES:sslv3 alert certificate unknown)
    T21:30:42.587Z [02620 warning ‘ProxySvc’] SSL Handshake failed for stream <io_obj p:0x0000000009608ef8, h:2540, <TCP ‘192.168.2.55:443’>, <TCP ‘192.168.2.55:57825’>>, error: class Vmacore::Ssl::SSLException(SSL Exception: error:14094416:SSL routines:SSL3_READ_BYTES:sslv3 alert certificate unknown)

Purpose

This article describes a specific issue. If you experience all of the above symptoms, consult the sections below. If you are experiencing some but not all of these symptoms, your issue is not related to this article.

Cause

If you experience all of the symptoms listed, this issue can occur because the vCenter Server SSL certificate has a low bit strength of less than 1024 bits. vCenter Server 5.5 Update 1 updates the Java Runtime Environment (JRE) to version 7.0.450.18 which no longer supports a key length of less than 1024 bits.

Note: vCenter Server 5.x does not support SSL certificates with a key length of less than 1024 bits.

To verify the key length of the vCenter Server certificate:

  1. Open the rui.crt file located at C:\ProgramData\VMware\VMware VirtualCenter\SSL.
  2. Click the Details Tab and scroll to the Public Key field.
  3. Verify if the Value is less than 1024 Bits.

Resolution

To resolve this issue, regenerate the vCenter Server certificate using a stronger public key strength.

Caution: These caveats apply to replacing a vCenter Server certificate:

  • Replacing the vCenter Server certificate may result in ESXi Hosts becoming disconnected from vCenter Server. A Manual reconnection of the ESXi Hosts may be required.
  • Plug-in components such as Update Manager, Site Recovery Manager, vCloud Director, Horizon View, etc, may need to be re-registered with vCenter Server.

To replace the vCenter Server SSL certificate, perform these steps:

  1. Using a text editor, copy openssl_config.cfg text to a file . Edit the portions highlighted in red to match your environment.

    [ req ]
    default_bits = 2048
    default_keyfile = rui.key
    distinguished_name = req_distinguished_name
    encrypt_key = no
    prompt = no
    string_mask = nombstr
    req_extensions = v3_req

    [ v3_req ]
    basicConstraints = CA:FALSE
    keyUsage = digitalSignature, keyEncipherment, dataEncipherment
    extendedKeyUsage = serverAuth, clientAuth
    subjectAltName = DNS: vc55-1, IP:10.0.0.10DNS:vc51-1.vmware.com, DNS: 10.0.0.10

    [ req_distinguished_name ]
    countryName = US
    stateOrProvinceName = NY
    localityName = New York
    0.organizationName = VMWare
    organizationalUnitName = vCenterUniqueServer
    commonName = vc55-1.vmware.com

  2. Save the openssl_config.cfg file to C:\Program Files\VMware\Infrastructure\Inventory Service\bin.
  3. Open a Windows command prompt as Administrator and change the directory to:

    C:\Program Files\VMware\Infrastructure\Inventory Service\bin

  4. Regenerate a self-signed certificate and key file using this command:

    openssl req -x509 -nodes -days 3650 -newkey rsa:2048 -keyout rui.key -out rui.crt -config openssl_config.cfg -extensions v3_req

  5. Create the vCenter Server PFX file using this command:

    openssl pkcs12 -export -in rui.crt -inkey rui.key -name rui -passout pass:testpassword -out rui.pfx

    Note
    : Do not replace rui or testpassword with any other values.

  6. When the rui.crt, rui.key and rui.pfx files regenerate, replace the vCenter Server SSL certificate with the steps inConfiguring CA signed certificates for vCenter Server 5.5 (2061973).

 

Configuring CA signed certificates for vCenter Server 5.5 (2061973)

Purpose

Note: This article is specifically for vSphere 5.5. If you are using vSphere 5.1, see Configuring CA signed certificates for vCenter Server 5.1 (2035005). If you are using vSphere 5.0, see Implementing CA signed SSL Certificates with vSphere 5.0 (2015383).
This article guides you through the configuration of Certificate Authority (CA) certificates for a vCenter Server 5.1 and vCenter Server 5.5. VMware has released a tool to automate much of the described process below.  See Deploying and using the SSL Certificate Automation Tool 5.5 (2057340) before following the steps in this article.
In the case that you are unable to use the tool this article helps you eliminate common causes for problems during certificate implementation, including configuration steps and details, and helps avoid common misconfigurations in the implementation of custom certificates in your environment.

Resolution

Note: This article is part of a resolution path. See Implementing CA signed SSL certificates with vSphere 5.x (2034833) before following the steps in this article.

Creating CA assigned certificates for vCenter Server is a complex task. In many organizations it is required to maintain proper security for regulatory requirements. There are several different work flows required for successful implementation:

  • Creating the certificate request
  • Getting the certificate
  • Installation and configuration of the certificate in vCenter Server
These steps must be followed to ensure successful implementation of a custom certificate for vCenter Server. Before attempting these steps ensure that:

Installation and configuration of the certificate in vCenter Server

After the certificate has been created, follow these steps to complete the installation and configuration of the certificate in vCenter Server:
  1. Log in to vCenter Server as an administrator.
  2. If you have not already imported it, double-click on the c:\certs\Root64.cer file and import the certificate into the Trusted Root Certificate Authorities > Local Computer Windows certificate store. This ensures that the certificate server is trusted.
  3. Backup the certificates for the VMware vCenter Server:

    C:\ProgramData\VMware\VMware VirtualCenter\SSL

  4. Copy the new certificate files into the above folder. If you are following this resolution path, the proper certificate is in c:\certs\vCenter.
  5. Open rui.crt in a text editor and validate that the first line of the file begins with —–BEGIN CERTIFICATE—–.  If there is any text prior to this, remove it.  The code that validates the certificate may fail in Step 5 if there is additional text.
  6. Go to https://localhost/mob/?moid=vpxd-securitymanager&vmodl=1 on the vCenter Server and load the certificates for the configuration by using the Managed Object Browser.
  7. Click continue if you are prompted with a certificate warning.
  8. Enter a vCenter Server administrator username and password when prompted.
  9. Click reloadSslCertificate.
  10. Click Invoke Method. If successful, the window shows this message: Method Invocation Result: void.
  11. Close both windows.
  12. Open a command prompt on vCenter Server and change to the isregtool directory. By default, this is C:\Program Files\VMware\Infrastructure\VirtualCenter Server\isregtool.
  13. Run this command to register the vCenter Server to the inventory service:

    register-is.bat vCenter_Server_URL Inventory_Service_URL SSO_Lookup_Service_URL

    Where these URLs are the typical URL (modify if ports are different):

    • vCenter_Server_URL is https://server.domain.com/sdk
    • Inventory_Service_URL is https://server.domain.com:10443/
    • SSO_Lookup_Service_URL is https://server.domain.com:7444/lookupservice/sdk
      If the command is successful, you see a message similar to:

      Note: If the return code is not 0 0, an error has likely occurred in the command. Review the text to see the error. The most common error is a mistyped URL in one of the three services.

  14. Change to the vCenter Server directory. By default, this is C:\Program Files\VMware\Infrastructure\VirtualCenter Server\.
  15. Run this command:

    vpxd -p

  16. Type the password for the vCenter Server database user to encrypt the password with the new certificate.
  17. Restart the VMware VirtualCenter Server service from the service control manager (services.msc)
  18. Restart the VMware vSphere Profile Driven Storage Service.
  19. After the initial restart of the services, wait for 5 minutes. If the VMware vSphere Profile Driven Storage service stops during this time, restart it.
  20. Navigate to https://vcenterserver.domain.com/ and validate the certificate.
The configuration of the custom certificates is now complete for vCenter Server. Next, continue to install the custom certificates for the vSphere Web Client. For more information, see Configuring CA signed SSL certificates for the vSphere Web Client and Log Browser in vCenter Server 5.5 (2061975).

Re-Register to vCenter Server using Update Manager Utility

Re-Register to vCenter Server using Update Manager Utility

It is necessary to re-register vCenter Server from Update Manager, for example when you change IP Address, Hostname or Certificate of vCenter Server. When you get an error message like below, then you know that you will need to re-register vCenter Server from Update Manager.

There was an error connection to VMware vSphere Manager
sysimage.fault.SSLCertificateError

image

When Update Manager is installed, it installs VMwareUpdateManagerUtility under  C:\Program Files (x86)\VMware\Infrastructure\Update Manager folder.

 

Step 1
Run VMwareUpdateManagerUtility, then select Re-register to vCenter Server from the left pane, then re-enter vCenter Sever IP Address, Username and password, then click Apply.

image

Step 2
When you get a message below. Then restart Update Manager service.

image

Step 3
Try accessing vCenter Server using vSphere Client, you will not get an error message anymore and Update Manager is available.

image

Comments are closed.