66.vsd mount returned critical failure

An ESX 4.0 host becomes unresponsive when it is put into maintenance mode. The ESX host does not boot and drops into Troubleshooting (busy box) mode.

This issue is resolved in vSphere 4.0 Update 3.

 

This issue occurs if an ESX host cannot identify the esxconsole.vmdk file in which the service console resides. 

 

To troubleshoot this issue: 

1. Go to the console of the ESX host. After the error message, ESX drops into Troubleshooting (busy box) mode.
2. Find the .vmdk for the service console by running the command:
   
   grep "/boot/cosvmdk" /etc/vmware/esx.conf
   
   The output is similar to:
   
   /boot/cosvmdk = "<uudi>/<dir>/esxconsole.vmdk"
   
   For example:
   
   /boot/cosvmdk = "/vmfs/volumes/4a14d968-88bf7161-700f-00145ef48f76/esxconsole-4a14d906-2f96-7956-7284-00145ef48f74/esxconsole.vmdk"
    
3. Make note of the <uuid> and the <dir> values in the output.
4. Verify that the files exist by running the command:
   
   ls -al /vmfs/volumes/<uuid>/<dir>/*.vmdk
   
   Where <uuid> and <path> is from the output of step 2. 
   
   The output is similar to:
   
   total 7906560
   drwxr-xr-x 1 root root        840 May 21 00:45 .
   drwxr-xr-t 1 root root       2660 Oct 21 09:10 ..
   -rw------- 1 root root 8095006720 Oct 26 15:37 esxconsole-flat.vmdk
   -rw------- 1 root root        475 May 21 00:32 esxconsole.vmdk
   drwxr-xr-x 1 root root        980 May 21 00:45 logs
   
   
   
5. Ensure that both the esxconsole-flat.vmdk and the esxconsole.vmdk files exist. 
   
   If the esxconsole.vmdk file does not exist, see Recreating a missing virtual disk (VMDK) header/descriptor file (1002511) and ensure that the following settings are in place:
   • ddb.adapterType = "buslogic"
   • ddb.consoleOsDisk = "True"
      
6. Ensure that both the esxconsole-flat.vmdk exists.
   
   If the esxconsole-flat file does not exist, you must re-install the ESX host to recreate the service consoleNote:
   
7. If you recieve following error on step 4r: ls: /vmfs/volumes/4a14d968-88bf7161-700f-00145ef48f76/esxconsole-4a14d906-2f96-7956-7284-045ef48f74/: No such file or directory.  Then follow bellow steps.
8. Run this command to enable resignaturing on the VMware ESX machine:
   
   esxcfg-advcfg -s 1 /LVM/EnableResignature
   
   You must get an output similar to:
   
   Value of EnableResignature is 1.
   
   Note: If the root is mounted as read only, run the command  mount -o remount / to remount the volumes so that they are in a writable state.
    
9. Run this command to unload the VMFS drivers:
    
   vmkload_mod -u vmfs3
    
10. Run this command to load the VMFS drivers:
    
    vmkload_mod vmfs3
     
11. Run this command to detect new VMFS volumes and resignature the volume:
    
    vmkfstools -V
    
12. Run this command to identify the full path of the esxconsole.vmdk file:
    
    find /vmfs/volumes/ -name esxconsole.vmdk
    
    The output appears similar to:
    
    /vmfs/volumes/4a14d968-88bf7161-700f-00145ef48f76/esxconsole-4a14d906-2f96-7956-7284-00145ef48f74/esxconsole.vmdk
     
    Note: Make a note of this full path.
    
    
13. Restart the VMware ESX machine. You see a menu provided by the grub boot loader. 
14. Press e to edit the grub entries manually.
15. Scroll down to the line that starts with kernel /vmlinuz (it is indented under the VMware ESX 4.0 heading).
16. Go to the end of the line and include the following entry after a space:
    
    /boot/cosvmdk=<path>/esxconsole.vmdk
    
    Where <path> is the full path identified in step 6.
     
17. Press Enter to accept the changes.
18. Press b to boot using the modified settings. The ESX host successfully boots. 
    
    Note: The changes made to the boot options are not saved. They only apply to the current boot process. The changes need to be made to the boot configuration files as described in the following steps.
     
19. Log into the console as root.
20. Edit the /etc/vmware/esx.conf file with a text editor and modify the following lines:
    
    /adv/Misc/CosCorefile = "/vmfs/volumes/<path>/core-dumps/cos-core"
    /boot/cosvmdk = "/vmfs/volumes/<path>/esxconsole.vmdk" 
    
    Where <path> is the full path identified in step 6.
     
21. Run this command to update the boot configuration files:
    
    esxcfg-boot -b