If you ever tried to search through task and events in the vSphere client, you know this can take quite some time, dependent on your environment size and how much is going on in the vCenter. For that reason and because you “only” get the last 1000 events, it can be beneficial to be able to do an event search.

When I have a challenge like this I jump to use the trusted tools of PowerCli. For this task the good folks at VMware has make a cmdlet called Get-ViEvent. If you simply run it you will get a long list of events.

 

As I stated in the beginning dependent on size this might be useful, just remember that doing these query means that vCenter has to query the DB an return the answer, so in small environments or where the number of events you need to get through are small, you’ll get your answer fast.

I recall a case where it wasn’t so, I had to search through 100.000+ event, and it would take hours before the query finally finished. This was clearly an extreme situation, as what we later found out to be a “wrong” setting on the storage array, that cause the storage array to drop the connection. It also cause a lot of events to be logged. I mean think about what happens when you get en all path down event, then think this would happen at a data center wide level and to add to this, it would happen every 30 minutes or so, and then 5 minutes later I would all connect again, only to repeat it self. Damn that was a lot of events. At the time there wasn’t a real syslog solution implemented – I mean prior to Log Insight, VMware’s offering wasn’t really of much value. We did try to direct our VMware logs at the network teams Splunk installation, but it just choked at the overwhelming amount of lot a vSphere environment creates.

 

Event search script

The script below was created to search for events related to the above case, it quite a simple construction in the second line I included the “MaxSamples” parameter in order to get Get-VIEvent to search more events than it does by default. The “foreach” part is simply to create a report, which in the last line is exported as a csv file.

$Details = @()
$data = Get-VIEvent -MaxSamples "100000" | where {$_.EventTypeId -eq "esx.problem.vmfs.heartbeat.timedout" -or $_.EventTypeId -eq "esx.problem.vmfs.heartbeat.recovered" -or $_.EventTypeId -eq "esx.problem.storage.redundancy.lost"}
foreach($msg in $data){
$Report = "" | Select-Object -Property Host,Date,DataStore,DataCenter,Cluster,FFM
$Report.Host = $msg.Host.name
$Report.DataStore = ($msg.Arguments | where {$_.key -eq "2"}).value
$Report.DataCenter = $msg.Datacenter.Name
$Report.Cluster = $msg.ComputeResource.Name
$Report.Date = $msg.CreatedTime
$Report.FFM = $msg.FullFormattedMessage
$Details += $Report
}
$Details | Export-Csv -Delimiter ";" Heartbeat_full.csv -NoTypeInformation

 

Happy searching