Wednesday, July 22, 2009
I became DAD again
Friday, July 17, 2009
vCenter 2.5u4 Performance Overview - Chart could not be loaded .Error :The URI is empty
When I was trying to install plugin using plugin manager I was getting “Invalid URI: The URI is empty ”
VMTM forum suggested
http://communities.vmware.com/thread/196315
“ dlane's post above put me on the right track to a fix in our environment, so thanks for that! Ultimately I had a service ticket go all the way to VMware Engineering to verify what was wrong and nail down the process to fix it, but this was the key. Our vCenter server had two NICs, one of which is on a private non-routed subnet. The URL to the statsReportExtension.xml file was pointing to the unroutable IP instead of the one it should be, so clients couldn't reach it. Manually pointing a web browser to the URL with the correct IP can confirm this.
So, fixing this is a two step process. First of all, find the offending XML file and correct the IPs in it (should be like 3 instances, one in each stanza).This will point clients to the right data once they get the correct XML file. You should be able to find it under Program Files\VMware\Infrastructure\VirtualCenter Server\tomcat\webapps\statsreport\extension or something similar. Confirm the correct values by manually pointing to the URL with correct IP from a client system.
Second, you need to correct the values in the vCenter DB that tell the clients how to reach the XML file in the first place. You need to stop the vCenter service and Web Infrstructure Access service first. I also advise a backup of the vCenter database. Use a SQL query editor to connect to the DB and look at the entries in the VPX_EXT_SERVER table. Find the entry with EXT_ID=com.vmware.vim.stats.report, and correct the IP address in the URL column - don't change anything else, just the IP in that one entry. Save the changes and restart the vCenter and Web services. This should now correctly point clients to the XML file. Confirm that Perf Overview still works OK on the server itself (open VI3 client to "localhost"), and if that's working, see if it fixes the plugin on the client side as well by opening a new VI3 client session to your server. You should see that the Performance Overview tab appears when you have a Host, VM, or Resource Pool selected. You should NOT need to go to Manage Plugins and download anything, it should just be there when you start the client and select the inventory item... Trying to download the "report" plugin is NOT going to work and will result in the "Invalid URI" error.
If you're still having problems, it's probably a different issue... try some of the other fixes/articles already linked here, or open a Support case with VMware. Good luck! ”
One More VMTM thread discussed different solution
http://communities.vmware.com/thread/196237
But I got it resolved this by restarting
Installing Performance Overview Plugin for VC2.5 U4
Trying to better understand your virtual machine's utilization?
Well look no further. Virtual Center 2.5 update 4 (released on Feb 23, 2009) has a great new plug-in that summaries all your virtual machines metrics,
from one convenient screen! If you have used the performance charts in Virtual Center, then you know how difficult they can be to work with. Analyzing a virtual machines performance can be a tedious task, especially when trying to understand capacity and doing trend analysis. Until now, the performance charts were the only tools available to monitor system performance. You would have to navigate through several different charts to understand a virtual machine CPU, memory, I/O and network utilization – quite a lot of work to say the least. New in Virtual Center update 4, is a less well known performance "overview" plug-in that makes virtual machine analysis a breeze. Unfortunately the plug-in doesn't come built-in to Virtual Center. Not to worry though, in the following pages, we will walk you through the steps necessary to install the new performance charts and have you analyzing your virtual machines performance in minutes!
Note: This plug-in is only offered in Virtual Center update 4 (build 147633 or greater). If you are not running VC update 4, you must first upgrade your Virtual Center instance. Please see VMware's release notes for details on upgrading.
We'll begin our installation with Java. If you already have the Java SE development kit installed, you can begin with the section below entitled Configuring Java Environment Variables
Download Java
Download and install the Sun Java SE development kit on your Virtual Center Server. Unless you are an advanced user, select the defaults through the completion of the installer. Take note of the install location as we will need this path later when configuring Java's environment variables.
Configuring Java Environment Variables
We must now set the appropriate environment variables for Java to run correctly on our Virtual Center server.
-
Right-click ‘My Computer’ and click ‘Properties’
2. From the ‘Advanced’ tab, click ‘Environment Variables’
-
Select the 'Path' system variable and click 'Edit.' Append the location Java was installed from the Download Java section above. If you used the defaults during the installation, this should be ';C:\Program Files\Java\jdk1.6.0_11\bin\'
Note: Ensure to include the ‘;’ – this acts as the delimiter between field elements In the System variable list, click ‘New’
-
In the 'Variable name', enter JAVA_HOME and in 'Variable value', enter 'C:\Program Files\Java\jdk1.6.0_11' or the path where Java was installed (from the Download Java section above)
Click through the remaining screens selecting the 'OK' option until the changes have been saved.
-
Click through the remaining screens selecting the 'OK' option until the changes have been saved.
Install Virtual Center Performance Charts
Now that Java is configured, we are ready to install the performance chart plug-in.
-
Open the command prompt on your Virtual Center server:
Start >> Run >> CMD
-
Run install.bat from your VMware Virtual Center installation media:
D:\vpx\perfCharts>install.bat
**************** VMWare Stats Installer ******************************
Usage: install vcUserName vcPassword
vcUserName is the user name you use to log in to the virtual center
vcPassword is the password you use to log into the virtual center
*****************************************************************
D:\vpx\perfCharts>install.bat administrator password
-
Once the installer has completed, run the Virtual Infrastructure client from your desktop
-
Click 'Manage Plugins...' from the Plugins menu
-
On the 'Available' tab, click the 'Download and install...' button on the viClientsScripts plug-in
-
Once the installation has completed, restart the Virtual Infrastructure client
-
From the Plugins Manager, click the 'Installed' tab and click the 'Enabled' check box next to the viClientScripts plugin
-
You can now access the plug-in from the 'Performance Overview' tab in the Virtual Infrastructure client
-
If you install client from VC2.5 U 4 DVD then there is no need of following above steps
Once plugin is installed then you can view under VC for host as well as VM
-
Tuesday, July 14, 2009
Web Access of VC4.0
I discovered new feature of ESX 4.0 that you can manage VC4.0 from web. No it’s not that this feature was not available with ESX3.5 . It was there but there is something which I am excited about
1. Brand new screen which gives you feel of VC.
2. Addition of Remote Console which was used for earlier version of ESX2.X
So when you try to popup VM it will ask you to install plugin compatible to browser (IE/Firefox)
3. Using this plugin you can manage VM more effectively then earlier version of ESX. If you recall this screen looks familiar to VMware Player. So did they included that ?
4. This is how the web managed VC looks like
If you closely watch, you can even see “Datacenter” where as this is not available with 2.5U5 . VC 2.5 U5 shows you only folder and VM level hierarchy. Bottom part of web VC also gave you same feel of VC client.
That is all I have discovered so far.
Monday, July 13, 2009
Hot Memory Add feature of VM on ESX4
I was trying to figure out Hot Memory add feature in ESX4.0. To enable this feature VM should be shut down. Very important to note that this feature is supported for “windows 2003 EE” not for “Standard edition”.
Once this feature is enabled then we can increase the memory when VM is powered on.
You can only add the memory but you cannot decrease the memory .
Sunday, July 12, 2009
Not able to add ESX host into Virtual Center
I was trying to resolve following error message ,this look straight forward?
No it is not as simple as it looks. So I started troubleshooting this
- I restarted vpx/hostd using following command “service mgmt-vmware restart”. This had no effect at all. Tried couple of time without any luck.
- We then disconnect the host and tried to reconnect but it was not responding at all.
- I was able to ping host by netbios name and IP address as well. Host was up and running.
- I was not able to login to the host bypassing VC. This seems to be bigger problem then it has been anticipated.
- Surprising part was when I restart mgmt-vmware service all seems to be OK but even though I decided to reinstall the VPX agent
service mgmt-vmware stop
service vmware-vpxa stop
Get the currently installed vpx version:
rpm -qa grep vpxa
This should return something like "VMware-vpxa-2.5.0-84767"
Now remove the vpxa agent:
rpm -e VMware-vpxa-2.5.0-84767
This also can be done using following way :
- On the vCenter Server, look for the following files in C:\Program Files\VMware\Infrastructure\VirtualCenter Server\upgrade
vpx-upgrade-esx-7-linux-104215
vpx-upgrade-esx-7-linux-104215.sig
- copy the files to the ESX host and run the following commands:
sh vpx-upgrade-esx-7-linux-104215
service mgmt-vmware restart
- Now I thought let me reconnect the host back because that is the time it does push the VC agent back into ESX host. But it did not respond back to reconnect command .
- Though this host was in cluster but I will not be able to put host into maintenance mode since it was in disconnected from VC.Now it looks like I really have to reboot the host but I did have live production VM.
- I checked the vm status and all were up and running. We decided to check logs under vmfs partition and guess what we were not able to cd into it. This is the problem because vmfs partition does response to “hostd ” and if there is something wrong with vmfs then we can anticipate such kind of problem.
- We tried rescanning HBA and it was getting into hung state. At this time we decided to check the health of filer and as soon as I logged in I found following comment
Can this be the problem ? No this was not the problem. Storage admin added some more space and this message went off. But I was not still able to get my HBA response to rescan.
- I then called my onsite engineer to have physical look at a HBA and guess what he told me. He told light at HBA is not blinking. Is my HBA dead (We have 4 port but still using one port ,reason not to be reveled ) ? I asked him to put in next card on the port and light started blinking. I still can not do anything by rescanning. I have to reboot the host. I killed all the VM’s and then rebooted the host.
This should have been straight forward issue but guess where it turn out to be. Nice troubleshooting experience.
What to do if “ifcfg-vswif0” file is deleted?
I usually change host name and IP address by following my earlier blog. What happened this time was not sure but I lost "ifcfg-vswif0" file and host were out of network. I still can see ""Service Console " and vSwitch0 had all the port group. Now this is very confusing situation , if I create new service console ,will I lost all the portgroup . The answer is NO. So I went ahead and created new Service Console port using following command
[root@server root]# esxcfg-vswif –a vswif0 –i 192.168.1.10 –n 255.255.255.0 –p "Service Console"
['Vnic' warning] Generated New Mac address, 00:50:xx:xx:xx:xx for vswif0
To my surprise I found that all the port group were intact and I got back my "ifcfg-vswif0" file with new MAC address.
Wednesday, July 8, 2009
Experience Installing VM tools on Citrix P2V server
One of the Admin did P2V for Citrix Server and was not able to install VM tools onto it. While I wrote couple of blogs for Citrix Virtualization
1. Best Practices for deploying Citrix on VMware ESX
2. Troubleshooting P2V Citrix Box on Vmware
But my dear college fail to refer these and complain that he is not able to power on the VM after P2V.
1. I tried and status of VM remain same . Red and Green button both were ON
2. So I decided that I will create a new VM and attach the HDD and see if I am able to power on VM. This is make the thing clear if P2V was an issue OS . When I attach this P2V HDD to the newly created VM , I got a strange message that file is locked.
How come file is locked . I have shutdown and unregister the VM . I logged into service console to check the status of VM.
‘Ps –ef ‘ did not help. The one which help was
‘ps axuw | grep vmx’
‘ps axuw | grep vmware-vmx’
‘ps auxfww | grep <vm name>’
To my surprise this shows that VM is still running. So it was clear why it was not allowing me to attach HDD to new VM.
Even ‘esxtop ’ shows that vm was running. Killed all this PID and I was successfully able to attach the HDD to newly created VM and was able to power on. This clear my doubts that it was not the VM neither the OS.
This also raises doubts how this file did got locked after P2V. May be he was following wrong method? Since this happed second time I have decided to investigate entire P2V process which our folks were following.
3. HDD was reverted back to original P2V machine and powered back ON. VM tool was not installed so I decided to install VM tool. I got following error message .
Why not since this is running Citrix and it is suppose to run in Application mode. Message which comes is really confusing and need not to be followed.
You can change terminal service mode from the command line itself
To determine the current mode on the Terminal Server server, run the change user /query command at a command prompt.
Click Start, and then click Run.
In the Open box, type cmd, and then click OK.
At the command prompt, type change user /install, and then press ENTER. The following message appears:
User session is ready to install applications.
Type exit, and then press ENTER.
Add or remove the programs that you want.
To switch a Terminal Services server into Execute mode, follow these steps:
Click Start, and then click Run.
In the Open box, type cmd, and then click OK.
At the command prompt, type change user /iexecute, and then press ENTER. The following message appears:
User session is ready to install applications.
Type exit, and then press ENTER.
When you install programs in Install mode, Terminal Services tracks all registry entries, and the HKEY_CURRENT_USER information is primarily written to the following registry key:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Terminal Server\Install
When you finish the program installation, by clicking Finish or by typing change user /execute, the system returns to Execute mode. The registry information that was written to the HKEY_CURRENT_USER registry hive during installation is written to the HKEY_CURRENT_USER registry hive for each user when they log on to the Terminal Server
SOURCE : KB252330
After this I was able to install VM tools
Friday, July 3, 2009
Upgrading VC2.5 to vSphare2.5 U4
Our environment was running virtual center 2.5 base edition. SQL was running one a shared database and virtual center server on one of the virtual machine. When we planned to upgrade to vShpare2.5 U4 we decided to move everything to physical server with database running on the same server. We shut down the VC service and disabled it so that no updates are happing on the database. After that we planned to perform database from SQL2000 to SQL2005. Once backup completed then we install SQL2005 on windows2003 server.
We then restored the old database on new SQL2005 server. Created new ODBC for SQL server.
With this we are ready to install vShare2.5 U4. It is quite simple and straight forward. It will detect the version of database and prompt for upgrade. We also have to insure that we have shut down VC license server and copy the license file to the new VC server.
After the upgrade is over we have to setup license server pointing to license file. Update the DNS and alias pointing to new vSphare 2.5 U4 server. We are ready to fire our VC client and point to the same "alias" . We may prompt to upgrade the client or it may work with same client.
Here are some surprise:
You will find all the host in disconnect state. DO NOT PANIC. This is normal because all the agent which runs on host is still pointing to old VC server. We have to perform reconnect with all the esx host. Yeah it is pain in the a#$%& and there is no work around. But good part was no permission change, no change in folder hierarchy . This gave me little bit of relief. I had pain with my ESX 2.X host connecting back to new VC. But everything went very smoothly .
It was cool experience to share.
Now new feature of vSphare 2.5 U4 which I did notice was :
You can perform hot cloning , yes hot cloning . You can clone running virtual machine without any packets drop. When it is at 3% then it think a lot J but after that it is very fast and user does not realize that machine is getting cloned . What I want to figure out is till what time that changes are captured? This will help me understand more about hot cloning.
How present NetAPP ISCSI LUN to The Windows Host
I was give assignment to present ISCSI LUN to windows box. I have never done this before. We have to present storage from NetAPP filer FAS3050.This windows box was suppose to run SQL database. Earlier to present storage from filer ,MS ISCSI initiator along with SNAP mirroring were used. It was costing more than HBA’s .So we connected Qlogic HBA “QLE4062C ” and tested it from BIOS (Ctrl + S) to the filer.
1. Once windows is installed then make sure that we have driver for Qlogic card is installed. Because driver don’t get installed by default. Once driver is installed then you can find Qlogic card under device manager
2. After Qlogic card is installed then we have can also test the connectivity to the filer from card using device
3. But this is not the end of the story . My real pain started how to enter target IP .After searching Qlogic website I found that “SANSurger HBA Manger “ is my friend
4. Download and Install SANSurfer iSCSI HBA Manager in client mode
5. Make sure you choose “iSCSI GUI and Agent” so that you can have agent and we can also do some additional troubleshooting in case there is problem
6. It will ask for destination to install
7. Select for all the Users Profile
8. Once it is installed lunch it using “local host”
9. Once it is connected it will show all the physical card as well as port
You have to expand the card to see all the ports. Before we configure the lun , we have to ensure that SAN admin has created Lun for WINDOWS. We also have to share IQN with SAN admin
10. You can find the link status and IQN number from this, This needs to be shared with Storage person
11. Select wizard from the top and select “General Configuration Wizard”. Select HBA port and then next
12. It will give driver version and MAC address
13. Provide IP address for the HBA
14. Don’t choose anything if you are not using IPV6
15. Don’t select iSNS if you are not uins Storage Network Server
16. This part is very crucial . We have to add the target IP address of the filer. Click on the green + tab. It will ask for target IP address for
17. Select next after adding. It will give summary
18. Once we hit “Next” then it will give warning message
19. Now it is very important steps. It will ask for password and the password is “config”. This information you can find in admin guide. This is not root password not anything but default password for SAN surfer manager
. Once it finished you can find space under “Disk Management”