The need
So here’s the deal. We’re performing some regular maintenance on our Hyper-V 2016 S2D – i.e. patching. This includes rebooting nodes. While node is not online, Cluster performs storage repair jobs to keep our 3 way healthy. It’s not good to reboot another node while repair job is in progress. To check the state of CSVs I can either use GUI:
or PowerShell:
With this I will see which drive is in degradaded state or repairing.
I can use another cmdlet to get the status of the job:
This on the other hand shows me how’s the repair going, how long tasks are running or how much data is already processed. What I don’t get from here is which job relates to which drive. This can be useful. Imagine you’ve got one repair job that is stuck or taking a long time. I’d like to know which CSV (Virtual Drive) is affected.
The Search
Both objects returned by either Get-StorageJob or Get-VirtualDisk have an object called ObjectID, which looks like this:
Seems like the thing I’m looking. Now I just need to parse the string to get the last guid-like string between { and } and match it with Get-VirtualDisk’s output same position. Let’s use some regex. As I’m new in this area I’ve used this site to get my regex right. Just paste your string and try different matching till you get it right. Seems like this will do the trick:
([A-Za-z0-9]{8}\-?){1}([A-Za-z0-9]{4}\-?){3}([A-Za-z0-9]{12})
Got it – Let’s try it:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
$StorageJobIDs = Get-StorageJob | Select-Object ObjectID | Select-String –Pattern '([A-Za-z0-9]{8}\-?){1}([A-Za-z0-9]{4}\-?){3}([A-Za-z0-9]{12})' –AllMatches | | |
Select-Object –ExpandProperty Matches | | |
Select-Object –ExpandProperty Value –Last 1 | |
foreach ($objectId in $StorageJobIDs) { | |
Get-VirtualDisk | Where-Object {$_.ObjectId -match $objectID} | |
} |
And nothing. No output. Verifying both objects, and it seems they differ with one char. StorageJob seems to have +1 on 18th position comparing to VirtualDisk.
Ok, let’s adjust my regex to match new condition:
([A-Za-z0-9]{8}\-?){1}([A-Za-z0-9]{4}){1}
The resolution
Now I know I can corelate repair job to specific CSV. Let’s get some additional date from both commands. I’d like to know which drive is being repaired, the status, percent complete and amount of data.
It’s now just a matter of creating a custom object in a foreach loop:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
$StorageJobs = Get-StorageJob | |
foreach ($stJob in $StorageJobs) { | |
foreach ($jobObjectID in ($stJob | Select-Object –ExpandProperty ObjectId)) { | |
$objectID = $jobObjectID | Select-String –Pattern '([A-Za-z0-9]{8}\-?){1}([A-Za-z0-9]{4}){1}' –AllMatches | | |
Select-Object –ExpandProperty Matches | | |
Select-Object –ExpandProperty Value –Last 1 | |
$vdisk = Get-VirtualDisk | Where-Object {$_.ObjectId -match $objectID} | |
[PSCustomObject]@{ | |
VirtualDiskName = $vdisk.FriendlyName | |
HealthStatus = $vdisk.HealthStatus | |
JobState = $stJob.JobState | |
PercentComplete= $stJob.PercentComplete | |
Name = $stJob.Name | |
BytesProcessed = $stJob.BytesProcessed | |
BytesTotal = $stJob.BytesTotal | |
} | |
} | |
} |
Running it locally on a cluster node though is not a way I like it. Let’s use Invoke-Command and target the Cluster Owner node for information. Also, let’s add Credential parameter – so I can query cluster from my own workstation without admin privileges. I’ll end up with a function like this:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
function Get-StorageJobReport { | |
<# | |
.SYNOPSIS | |
Get current storage jobs on cluster. Returns additional information from Get-VirtualDisk. | |
.DESCRIPTION | |
If there are storageJobs running on the cluster will match each job to virtual disk and return custom object. | |
.PARAMETER Cluster | |
Hyper-V cluster name. | |
.PARAMETER Credential | |
Credentials to connect to Cluster. | |
.EXAMPLE | |
Get-StorageJobReport -Cluster 'SomeCluster' | |
Will use current user credentials to connect to SomeCluster, check for StorageJobs, match with VirtualDisks and return custom objects. | |
.EXAMPLE | |
Get-StorageJobReport -Cluster 'SomeCluster' -Credential (Get-Credential) | |
Will use provided credentials to connect to SomeCluster and validate if provided VHDPath exists. | |
#> | |
[CmdletBinding()] | |
[OutputType([PSObject])] | |
Param( | |
[Parameter(Mandatory,HelpMessage='Provide Hyper-V Cluster Name', | |
ValueFromPipeline,ValueFromPipelineByPropertyName)] | |
[string] | |
$Cluster, | |
[Parameter(Mandatory=$false,HelpMessage='Provide Credentials to access cluster', | |
ValueFromPipeline,ValueFromPipelineByPropertyName)] | |
[System.Management.Automation.Credential()][System.Management.Automation.PSCredential] | |
$Credential = [System.Management.Automation.PSCredential]::Empty | |
) | |
process{ | |
#region invoke-Command connection properties | |
$invokeProps = @{ | |
Cluster = $Cluster | |
} | |
if($PSBoundParameters.ContainsKey('Credential')) { | |
Write-Verbose "Credential {$($Credential.UserName)} will be used to connect to Cluster {$Cluster}" | |
$invokeProps.Credential = $Credential | |
} | |
else { | |
Write-Verbose "Processing Cluster {$Cluster} with default credentials of user {$($env:USERNAME)}" | |
} | |
#endregion | |
Invoke-Command –ComputerName $Cluster –Credential $Credential –ScriptBlock { | |
$StorageJobs = Get-StorageJob | |
if ($StorageJobs) { | |
foreach ($stJob in $StorageJobs) { | |
foreach ($jobObjectID in ($stJob | Select-Object –ExpandProperty ObjectId)) { | |
$objectID = $jobObjectID | Select-String –Pattern '([A-Za-z0-9]{8}\-?){1}([A-Za-z0-9]{4}){1}' –AllMatches | | |
Select-Object –ExpandProperty Matches | | |
Select-Object –ExpandProperty Value –Last 1 | |
$vdisk = Get-VirtualDisk | Where-Object {$_.ObjectId -match $objectID} | |
[PSCustomObject]@{ | |
VirtualDiskName = $vdisk.FriendlyName | |
HealthStatus = $vdisk.HealthStatus | |
JobState = $stJob.JobState | |
PercentComplete= $stJob.PercentComplete | |
Name = $stJob.Name | |
BytesProcessed = $stJob.BytesProcessed | |
BytesTotal = $stJob.BytesTotal | |
} | |
} | |
} | |
} | |
else { | |
Write-Verbose "No Storage Jobs Running on Cluster {$USING:Cluster}" | |
return $null | |
} | |
} | |
} | |
} |