Michael Ryom (.dk)

Import-DSvRepo - One Script to Import Everything

2026-04-25T08:51:41+02:00

Import-DSvRepo - One Script to Import Everything

This is the third piece of the air-gapped VMware patching puzzle. DSvClient downloads the repository from Broadcom, DSvRepoImport is the PowerShell module that knows how to talk to each vCenter API, and Import-DSvRepo is the wrapper script that ties them together into a single command.

If you’ve read my posts on DSvClient 0.8.0 and DSvRepoImport, you know the individual pieces. This post covers how the wrapper orchestrates them.

Why a Wrapper Script?

The DSvRepoImport module exports eight functions, each handling a different data type. Calling them one by one with the right parameters and credentials gets tedious fast — especially when you’re doing it regularly across multiple vCenter instances.

Import-DSvRepo.ps1 reduces it to:

.\Import-DSvRepo.ps1 -VCenterServer vcsa.domain.local `
    -RepoPath D:\VMware-repo `
    -VersionFilter '8.0*' `
    -BuildFilter '2*'

Two credential prompts (one for the vCenter REST API, one for VCSA OS root), then it runs through everything automatically.

What It Does, In Order

The script calls the module functions in dependency order:

Import-VsanHclDatabase — uploads the vSAN Hardware Compatibility List JSON from hcl/all.json in the repo
Import-VsanReleaseCatalog — uploads the vSAN Release Catalog from vcg/vsan_release_catalog.json
Import-VcgDatabase — uploads the VCG compatibility database from vvs_data.gz
Import-VlcmOfflineBundle — scans the repo for all four depot types (main/, addon-main/, iovp-main/, vmtools-main/) and imports each as vLCM offline depot bundles. Handles version/build filtering, size-based chunking for the 2 GB API limit, and deduplication of already-imported depots.

VCSA appliance patching (Install-VcsaPatch) is intentionally not called by the wrapper. Patching is destructive — it restarts all vCenter services and can take 30+ minutes — so it should be a conscious decision, not something that runs as part of a bulk import.

Parameters

Parameter	Description
`-VCenterServer`	FQDN or IP of the vCenter instance
`-RepoPath`	Path to the DSvClient-downloaded repository root
`-VCSAUser`	Username for VCSA OS access (default: `root`)
`-VersionFilter`	Wildcard filter for ESXi/vCenter versions to import (e.g. `8.0*`)
`-BuildFilter`	Wildcard filter for build numbers
`-CleanupBeforeImport`	Remove all existing vLCM offline depots before importing. Off by default — use this when you want a clean slate.

Version and Build Filtering

The -VersionFilter and -BuildFilter parameters are passed through to Import-VlcmOfflineBundle and control which ESXi builds get packaged into offline depot bundles. If your repo contains every ESXi 7.x and 8.x build from the last two years but you only care about 8.0 Update 3, filtering saves significant time and disk space on the VCSA:

# Only import ESXi 8.0 builds starting with "24" or "25" (U3 era)
.\Import-DSvRepo.ps1 -VCenterServer vcsa.domain.local `
    -RepoPath D:\VMware-repo `
    -VersionFilter '8.0*' `
    -BuildFilter '2[45]*'

Without filters, everything in the repo gets imported — which is fine if that’s what you want, just slower.

The Typical Air-Gapped Workflow

Here’s how the three tools fit together end-to-end:

On your internet-connected machine:

# 1. Download everything from Broadcom
./DSvClient --config sources.toml --output /mnt/usb/VMware-repo

Transfer the repo to the air-gapped network (USB drive, data diode, sneakernet, whatever your security policy requires).

On your management workstation inside the air gap:

# 2. Import depot data into vCenter
.\Import-DSvRepo.ps1 -VCenterServer vcsa.domain.local `
    -RepoPath D:\VMware-repo

# 3. If there's a VCSA appliance patch to apply
Install-VcsaPatch -VCSAHost vcsa.domain.local -Credential $cred `
    -SourcePath 'D:\VMware-repo\valm\8.0.3.00800' -Install

Step 2 takes 10-30 minutes depending on how many depot bundles need uploading (the SFTP transfer of multi-GB bundles to the appliance is the bottleneck). Step 3 takes another 20-40 minutes for the actual patch installation.

Prerequisites

The script requires the DSvRepoImport module and its dependencies:

Install-Module VMware.PowerCLI -Scope CurrentUser
Install-Module WinSCP -Scope CurrentUser
Install-Module Posh-SSH -Scope CurrentUser

Place DSvRepoImport.psm1 and DSvRepoImport.psd1 somewhere on your $env:PSModulePath, or Import-Module them explicitly before running the wrapper.

What’s Next

For the next iteration I’m considering adding a -WhatIf mode that scans the repo and reports what would be imported without actually doing it — useful for reviewing the scope of an import before committing. Also on the list is support for generating a diff report between what’s already in vCenter’s vLCM depot and what’s in the local repo, so you can see exactly what’s new before importing.

Both the module and the wrapper script are on my OneDev instance. DSvClient is on GitHub. Questions or issues — reach out. 🛠️

DSvRepoImport - Importing VMware Repos into Air-Gapped vCenter

2026-04-25T08:48:22+02:00

DSvRepoImport - Importing VMware Repos into Air-Gapped vCenter

Available now

If you run vCenter in an air-gapped environment — no internet access, no Broadcom CDN connectivity — getting vSAN HCL databases, VCG compatibility data, vLCM offline depot bundles, and VCSA appliance patches onto the appliance is a multi-step manual process that’s easy to get wrong.

I wrote DSvRepoImport to automate that entire pipeline. It’s a PowerShell module that takes a DSvClient-downloaded repository and imports everything into vCenter over SSH and REST API, with no internet connectivity required on the vCenter side.

What It Imports

The module handles six distinct data types, each with its own import mechanism:

Function	What It Imports	API Used
`Import-VsanHclDatabase`	vSAN Hardware Compatibility List	REST API upload
`Import-VsanReleaseCatalog`	vSAN Release Catalog	REST API upload
`Import-VcgDatabase`	VCG Compatibility Database	REST API upload
`Import-VlcmOfflineBundle`	vLCM offline depot bundles (ESXi base, OEM addons, IO drivers, VMware Tools)	REST API + SFTP + on-appliance HTTP server
`New-VcsaPatchIso`	Builds a VCSA patch ISO from unpacked `valm/` folder	Local IMAPI2FS COM objects
`Install-VcsaPatch`	Stages and installs VCSA appliance patches via loopback-mounted ISOs	SSH + SFTP

The vLCM Bundle Challenge

The vLCM offline depot import was the trickiest part. vCenter’s offline depot API only accepts http(s):// URLs — no file:/// paths, no local directories. In an air-gapped environment, there’s nowhere to host an HTTP server that vCenter can reach.

The solution: SFTP the bundle to the VCSA, spin up a temporary Python HTTP server on the appliance itself, point the vLCM API at http://localhost:8888/, wait for the import task to complete, then tear down the server.

There’s also a 2 GB hard limit on individual bundles — vCenter’s download manager uses a signed 32-bit byte counter that overflows at 2 GB. The module enforces a 1800 MB ceiling and bin-packs VIBs into chunks for large flat-catalog depot types like vmtools-main/.

VCSA Appliance Patching Without VM Console Access

The Install-VcsaPatch function is designed for environments where you have SSH access to the VCSA but not hypervisor-level access to attach a virtual CD-ROM. The VCSA’s software-packages command only supports --iso and --url for patch media — it expects the patch contents mounted at /mnt/iso-contents.

The module replicates that mount point with a loopback mount:

Build a flat ISO from the valm// folder using Windows IMAPI2FS (no external tools needed)
SFTP the ~8 GB ISO to /storage/core/ on the VCSA
losetup + /dev/cdromN symlink so the VCSA’s mountISO() function finds it
Run software-packages stage --iso --acceptEulas then software-packages install --staged
Clean up loop device, symlink, and ISO file

Two modes:

# Validate a patch without installing (stage only)
Install-VcsaPatch -VCSAHost vcsa.domain.local -Credential $cred `
    -SourcePath D:\VMware-repo\valm\8.0.3.00800 -Stage

# Stage + install in one shot
Install-VcsaPatch -VCSAHost vcsa.domain.local -Credential $cred `
    -SourcePath D:\VMware-repo\valm\8.0.3.00800 -Install

A Key Discovery: VCSA Patch ISOs Are Flat

One thing I learned the hard way: a VCSA patch ISO must be completely flat — every file at the mount root, no package-pool/ subdirectory. Even though manifest-latest.xml references RPMs as package-pool/foo.rpm, the VCSA’s stager code (functions_target.py::parseRpmXmlManifestToJson) calls os.path.basename() to strip that prefix before looking up files on the ISO.

The prefix only matters in online-URL mode (software-packages stage --url), where it becomes part of the HTTP request path. For ISO staging it’s discarded. I spent quite a few iterations figuring this out by reverse-engineering the stager Python code from inside target-patch-scripts.zip.

Pre-Flight and Post-Install Checks

After hitting several avoidable issues during patching (stale loop devices from prior failed runs, stuck INSTALL_FAILED state in the appliance’s CLI wrapper, duplicate authz roles breaking a post-install hook), I added automated checks:

# Pre-flight readiness check
Test-VcsaPatchReadiness -VCSAHost vcsa.domain.local -Credential $cred |
    Format-Table Name, Status, Detail -AutoSize

# Post-install verification
Test-VcsaPatchInstalled -VCSAHost vcsa.domain.local -Credential $cred `
    -ExpectedVersion '8.0.3.00800' -ExpectedBuild '25197330' |
    Format-Table Name, Status, Detail -AutoSize

Test-VcsaPatchReadiness checks SSH connectivity, core services, disk space on all storage partitions, vCHA state, hostname/PNID match, stuck failed-install state, and orphaned loop devices. Test-VcsaPatchInstalled verifies the installed version/build, services, residual state, and HTTPS endpoint.

Both return structured Name / Status / Detail / Remediation records so you can script around them.

The Cleanup Landmine

One pattern that bit me repeatedly: when a -Stage or -Install run fails mid-flight, the loopback device and /dev/cdromN symlink are deliberately left in place so you can investigate. But the next run creates a new loop device at a different /dev/cdromN slot, and the VCSA’s mountISO() iterates /dev/cdrom* alphabetically — picking up the stale one first.

The module now auto-cleans residual loop devices and cdrom symlinks at the start of every run, and verifies each cleanup command’s exit status individually at the end. No more silent cleanup failures that poison the next attempt.

Dependencies

Install-Module VCF.PowerCLI -Scope CurrentUser #Real Requirements: VMware.VimAutomation.Core, VMware.VimAutomation.Cis.Core, VMware.VimAutomation.Storage
Install-Module WinSCP -Scope CurrentUser
Install-Module Posh-SSH -Scope CurrentUser

The module requires PowerShell 5.1+, VMware/VCF PowerCLI (for vSAN/VCG/vLCM REST API access), WinSCP (for SFTP with the custom SftpServer raw setting that VCSA needs), and Posh-SSH (for SSH command execution).

Getting Started

The companion wrapper script Import-DSvRepo.ps1 calls all the import functions in sequence:

.\Import-DSvRepo.ps1 -VCenterServer vcsa.domain.local `
    -RepoPath D:\VMware-repo `
    -VersionFilter '8.0*' `
    -BuildFilter '2*'

Both repos are available on my OneDev instance. DSvClient (the downloader that produces the repo) is on GitHub. 🔧

DSvClient 0.8.0 - VCSA Patch Layout Fix and On-Disk CDN Mirror

2026-04-13T15:10:48+02:00

A few critical bugs have been fixed in the DSvClient to make it work when patching vCenter, some blobs where missing and layout didnt align with vCenters expectations when used for an online repo. This make it ready to use with the up coming patch module for vCenter, very useful in airgapped environments.

Here's what changed and why.

Why This Release Exists

Previous DSvClient versions downloaded every VCSA patch file — RPMs, container image blobs, `.manifest` files, script zips, and root metadata — into a single flat directory at `valm//`. That worked for local inspection, but it didn't match how the Broadcom CDN actually serves those files.

On the CDN, the layout looks like this:

valm/8.0.3.00800/
manifest-latest.xml
manifest-latest.xml.sha256
manifest-latest.xml.sign
rpm-manifest.json
rpm-manifest.json.sha256
rpm-manifest.json.sign
package-pool/
*.rpm
*.blob
*.manifest
patch-metadata-scripts.zip
target-patch-scripts.zip

Only the six metadata files live at the root. Everything else — the hundreds of RPMs, the container image blobs, the `.manifest` container descriptors, and the two `*-patch-scripts.zip` script bundles — lives under `package-pool/`.

What Changed in 0.8.0

The `get_vcsa_file_path` function in `downloader.rs` now preserves the `package-pool/` prefix from each package's `` or `relativepath` entry, instead of stripping it to the basename:

fn get_vcsa_file_path(&self, version: &str, file: &str) -> PathBuf {
    let mut path = self.base_path.join("valm").join(version);
    for segment in file.split('/')
        .filter(|s| !s.is_empty() && *s != ".." && *s != ".")
    {
        path = path.join(segment);
    }
    path
}

The `..` and `.` segments are stripped defensively so a hostile manifest can't write outside the version directory. The `download_file` function already creates missing parent directories via `create_dir_all`, so the `package-pool/` subdirectory is created on demand.

Why This Matters

With the on-disk layout now matching the CDN, a freshly-downloaded `valm//` tree can be:

1. **Served directly over HTTP** as a local mirror of the Broadcom CDN for `software-packages stage --url http://host/valm//`
2. **Turned into a VCSA patch ISO** via the DSvRepoImport PowerShell module
3. **Inspected locally** the same way as before — only the directory structure changed

Migrating an Existing Flat Repo

If you already downloaded a repo with DSvClient 0.6 or earlier and don't want to re-download ~8 GB, a quick shell one-liner reshuffles the files:

cd /path/to/VMware-repo/valm/8.0.3.00800
mkdir -p package-pool
for f in *; do
  case "$f" in
    package-pool|manifest-latest.xml|manifest-latest.xml.sha256|\
    manifest-latest.xml.sign|rpm-manifest.json|\
    rpm-manifest.json.sha256|rpm-manifest.json.sign) ;;
    *) mv -- "$f" package-pool/ ;;
  esac
done

PowerShell equivalent:

$src = 'C:\path\to\VMware-repo\valm\8.0.3.00800'
$keep = @('manifest-latest.xml','manifest-latest.xml.sha256',
          'manifest-latest.xml.sign','rpm-manifest.json',
          'rpm-manifest.json.sha256','rpm-manifest.json.sign')
New-Item -ItemType Directory -Path (Join-Path $src 'package-pool') -Force | Out-Null
Get-ChildItem -Path $src -File |
  Where-Object { $keep -notcontains $_.Name } |
  Move-Item -Destination (Join-Path $src 'package-pool')

After migration, running DSvClient 0.8.0 again will verify all checksums and skip already-valid files — no re-download.

Also in This Release

- **Version 0.7.0** (bundled in this release) added `parse_vcsa_rpm_manifest_json` to download `.blob` and `.manifest` files referenced by `rpm-manifest.json` that weren't previously being fetched. Without these 24+ container image blobs and 2 container manifests, VCSA patching fails silently during the staging phase.

DSvClient is open source and available on GitHub.
Grab the latest release binary or build from source with `cargo build --release`. 🚀

Capturing VM Switch‑Port Traffic with a simple script

2025-09-16T04:17:58+02:00

When I’m troubleshooting a virtual machine’s network behavior, the first thing I reach for is a packet capture. In theory it’s simple: locate the switch‑port the VM is attached to, start a capture on that port, and save the results for later analysis. In practice I kept forgetting the exact commands—especially the quirky combination of pktcap-uw and tcpdump-uw that ESXi require.

To solve that, I wrote a tiny shell script that does everything in one go. Below I walk through the script, explain each step, so you can use it in your own environment.

The script can be found here over on Codeberg.org

Why This Script Exists

Forgetfulness is real. Every time I needed a capture I’d open a terminal, remember the command net‑stats -l or using esxcli and whatever parameters is needed, extract the port number, remember the right flags for pktcap-uw, pipe it into tcpdump-uw, and finally name the output file and forget to give it a new one when doing multiple takes.
Consistency matters. Having a reproducible command chain eliminates human error—no more “oops, I used the wrong flag or command”
Convenience. With a single argument () the script does all the heavy lifting, timestamps the output, ready for you to analyze the .pcapoutput.

How to Use It

chmod +x capture_vm_switchport_traffic.sh
./capture_vm_switchport_traffic.sh my-vm

Replace my-vm with any unique substring of the VM’s name. The script will:

Find the correct switch‑port.
Start capturing traffic.
Save a timestamped .pcap under /tmp.

When you’re done, simply press Ctrl‑C. The capture ends, and you’ll see the location of the file printed to the console.

The Script, Line by Line

#!/bin/sh # --------------------------------------------------------------- #

capture_vm_switchport_traffic.sh

# # Usage: ./capture_vm_switchport_traffic.sh

# # What it does:

# 1️⃣ Finds the switch‑port ID (PortNum) for the given VM

# using `net‑stats -l`. The VM name appears in column 6;

# we also capture the exact name reported by the system.

# 2️⃣ Starts a packet capture on that switch‑port.

# 3️⃣ Saves the capture file under /tmp with a timestamp.

# ---------------------------------------------------------------

The header explains purpose, usage, and the three core steps.

set -euo pipefail # safer scripting

Enables strict error handling: abort on any failure, treat unset variables as errors, and propagate failures through pipelines.

1️⃣ Argument Validation

if [ "$#" -ne 1 ]; then

echo "Error: Exactly one argument (the VM/client name) is required."

echo "Usage: $0 "

exit 1 fiINPUT_VM=$1 # what the user typed (may be partial)

We insist on a single argument – the (partial) VM name – and store it for later pattern matching.

2️⃣ Locate the Switch‑Port

NET_OUT=$(net-stats -l)

MATCH=$(printf "%s\n" "$NET_OUT" | awk -v pat="$INPUT_VM" 'tolower($6) ~ tolower(pat) {print $1, $6}' | head -n1)

PORT_NUM=$(printf "%s" "$MATCH" | awk '{print $1}') REAL_VM=$(printf "%s" "$MATCH" | awk '{print $2}' | cut -d '.' -f1)

net‑stats -l lists all live interfaces.
We pipe the output through awk, performing a case‑insensitive match against column 6 (the VM name).
The first match gives us two pieces of data: the PortNum (column 1) and the exact VM identifier.
cut -d '.' -f1 strips any domain suffix that might appear.

If nothing matches, we bail out with a clear error:

if [ -z "${PORT_NUM:-}" ] || [ -z "${REAL_VM:-}" ]; then
    echo "Error: No matching entry found for client \"$INPUT_VM\"."    exit 2fi

3️⃣ Timestamped Filename

TIMESTAMP=$(date +%Y%m%d_%H%M%S)PCAP_FILE="/tmp/${REAL_VM}_${TIMESTAMP}.pcap"

A unique filename prevents accidental overwrites and makes it easy to locate captures later.

4️⃣ Run the Capture

echo "Starting capture on switch‑port $PORT_NUM → $PCAP_FILE"
echo "Press Ctrl‑C to stop the capture."
( pktcap-uw --switchport "$PORT_NUM" --dir 2 -o - | tcpdump-uw -r - -w "$PCAP_FILE")

pktcap-uw grabs raw packets from the specified switch‑port (--dir 2 means both inbound and outbound).
The -o - flag streams the output to stdout, which we pipe directly into tcpdump-uw.
tcpdump-uw reads from stdin (-r -) and writes a proper .pcap file (-w).

Because the whole pipeline runs inside a subshell, pressing Ctrl‑C cleanly terminates both processes.

5️⃣ Wrap‑Up Message

echo "Capture finished. PCAP saved to: $PCAP_FILE"
echo "You can inspect it with:"
echo "    tcpdump-uw -r $PCAP_FILE"
echo "    wireshark $PCAP_FILE   # after copying to a workstation with Wireshark"

A friendly reminder of the next steps: analyze with tcpdump on the host or pull the file to a workstation and open it in Wireshark.

Takeaways

One source of truth. By encapsulating the lookup and capture logic, you eliminate the mental overhead of remembering several unrelated commands.
Safety first. set -euo pipefail ensures the script stops on any unexpected condition, protecting you from generating empty or misleading captures.
Reusability. Ready to troubleshoot VMs instantly.

Next time you’re staring at a VM and need to see what’s really happening on the wire, just run the script and let it do the heavy lifting. Happy sniffing! 🐶

Automating VMware/Broadcom URL Whitelisting Checks with a Simple Bash Script

2025-09-08T03:38:14+02:00

I wrote another piece of bash script that I think others might fine useful, hence the blog post :) It is some what inspired by DSvClient which I also recently did a new release of check it out here on MichaelRyom.dk or here on GitHub!

Why I wrote this script

I ran into issues working with vCenter and network security. It was a very practical problem: firewall and proxy rules were silently breaking essential VMware/Broadcom services.

Broadcom publishes a public URL list that must be whitelisted for things like vSphere Update Manager, vSAN Cloud Health, and the VVS API. The list lives here:

🔗 https://knowledge.broadcom.com/external/article/327186/public-url-list-for-sddc-manager.html

Back in the day my go‑to method was a blunt‑force curl -k -vvv and then manually scanning the verbose output for errors or a successful 200 OK or finding proxy denies. That works, but it’s noisy, time‑consuming, and impossible to run reliably across dozens of vCenters.

I wanted a single, repeatable command that would:

Hit every URL on the official whitelist (that I and most need).
Show a concise status (HTTP code + short note - I broke the note part a some point, so mainly HTTP code).
Handle the one oddball Broadcom VVS endpoint that requires an OAuth bearer token.

That’s exactly what the script does. It condenses the whole checklist into a tidy table that anyone can read at a glance.

By the way if you wanna jump straight to the script it is available here on codeberg.org

What the script actually does

Below is a high‑level walk‑through of the script’s flow. I’ve kept the code deliberately short and commented, so you can see the logic without getting lost in Bash minutiae.

#!/usr/bin/env bashset -euo pipefail          
# Fail fast on errors
# -------------------------------------------------------------------------
# 1️⃣  Arguments – you must supply the  that replaces the#     placeholder in one of the URLs.
# -------------------------------------------------------------------------
if [[ $# -ne 1 ]]; then
    echo "Usage: $0 "
    exit 1
fi

DOWNLOAD_TOKEN="$1"

1. Embedded URL list (TOML)

The URLs are stored in a small TOML block right inside the script. Each line follows the same url = "…" syntax, which makes it easy to add or remove entries later.

TOML=$(cat <<'EOS'url = "https://dl.broadcom.com//PROD/COMP/ESX_HOST/addon-main/vmw-depot-index.xml"url = "https://partnerweb.vmware.com/service/vsan/all.json"url = "https://vcsa.vmware.com/ph/api/v1/results?deploymentId=2d02e861-7e93-4954-9a73-b08692a330d1&collectorId=VsanCloudHealth.6_5&objectId=0c3e9009-ba5d-4e5f6-bae8-f25ec506d219&type=vsan-updates-json"url = "https://vvs.broadcom.com/v1/compatible/vcg/bundles/all?format=gz"EOS)

2. Pull out just the URL lines

mapfile -t URL_LINES < <(grep -E '^url[[:space:]]*=' <<<"$TOML")

Now URL_LINES holds each definition, ready for processing.

3. Detect whether we need an OAuth token

Only the vvs.broadcom.com endpoint requires a bearer token. The script scans the list; if it finds such a URL it performs a client‑credentials grant against Broadcom’s auth service.

if $needs_oauth; then    token_resp=$(curl -s -X POST "$AUTH_URL" \        -H 'Content-Type: application/json' \        -d "$(jq -nc --arg cid "$CLIENT_ID" --arg cs "$CLIENT_SECRET" \                '{grant_type:"client_credentials",client_id:$cid,client_secret:$cs}')")    ACCESS_TOKEN=$(jq -r '.access_token // empty' <<<"$token_resp")fi

4. Probe each URL

The heart of the script is the probe() helper. It tries a lightweight HEAD request first (fast, no body). If the response isn’t a 2xx, it falls back to a silent GET. The function returns a string like 200|OK.

probe() {    local url=$1 extra=${2:-}    local http_code note    http_code=$(curl --silent --show-error --head --max-time $MAX_TIME \                     $extra "$url" -w "%{http_code}" -o /dev/null) || true    …}

For the OAuth‑protected VVS endpoint we send the token in a custom header (X-Vmw-Esp-Client) and parse the first response line.

5. Print a compact table

Finally the script prints a nicely aligned table:

printf "\n%-95s %-6s %s\n" "Result" "Code" "Note"printf "%-95s %-6s %s\n" "$(printf '=%.0s' {1..95})" "------" "----"…printf "%-95s %-6s %s\n" "$src_url" "$code" "$note"

The output looks like this (example run with a dummy token):

Result Code Note=============================================================================================== ------ ----https://dl.broadcom.com//PROD/COMP/ESX_HOST/addon-main/vmw-depot-index.xml 403 403 https://partnerweb.vmware.com/service/vsan/all.json 200 200 https://vcsa.vmware.com/ph/api/v1/results?deploymentId=2d02e861-7e93-4954-9a73-b08692a330d1&collectorId=VsanCloudHealth.6_5&objectId=0c3e9009-ba5d-4e5f6-bae8-f25ec506d219&type=vsan-updates-json 200 405 https://vvs.broadcom.com/v1/compatible/vcg/bundles/all?format=gz 200 200

Result – The full URL that was tested (the placeholder is replaced by the token you passed, but not viable in output).
Code – The HTTP status code returned by the server. Anything in the 200‑range means the endpoint is reachable; 3xx indicates a redirect, 4xx/5xx signals a problem.
Note – SHOULD HAVE BEEN: The short description taken from the first line of the HTTP response (e.g., “OK”, “Moved Permanently”, “Unauthorized”). It gives a quick hint about why a non‑200 code appeared.

How to use the script

Make it executable

chmod +x vmware_broadcom_url_check.sh

Run it, passing the download token you got from Broadcom

./vmware_broadcom_url_check.sh <your-download-token>

Replace with the real token (or a test value if you just want to see the table format).

Read the table – any row that isn’t 200 OK is a candidate for firewall or proxy adjustment.

Keeping the URL list up‑to‑date

The official list lives at:

🔗 https://knowledge.broadcom.com/external/article/327186/public-url-list-for-sddc-manager.html

Broadcom occasionally adds or deprecates entries. Whenever you spot a change, simply update the TOML block in the script and re‑run. Because the script is self‑contained, there’s no external configuration file to manage.

Final thoughts

What started as a handful of manual curl commands turned into a repeatable, auditable health‑check that I now run whenever I touch firewall rules, spin up a new vCenter instance.

If you’re managing vSphere, vSAN, or any Broadcom‑backed SDDC component, I hope you find this script as useful as I do. Feel free to fork it, tweak the output format, or integrate it into your own monitoring stack. A big shout‑out to my other project DSvClient which started this for me.

Happy testing! 🚀

DSvClient 0.6 - Broadcom ready and bug fixes

2025-09-03T22:13:14+02:00

Hi,

Since the initial release, I've been addressing quite a few bugs that came with the new Broadcom token, as well as other minor issues and slowdowns.

I've also open-sourced the software. It's still not perfect, but hey, whose code that, right? 😉

Going forward, I'll be supporting Windows and Linux only. If you need it for other operating systems, you'll have to build it yourself.

The latest version, 0.6, can be found here: https://github.com/MichaelRyom/DSvClient/tree/Main/releases/latest

I've implemented a new system for GitHub automation to help with releases when needed.

Here are some of the key improvements and fixes:

Instead of maintaining a list of files that were not available, this release creates a file with possible exclusions if a file is not present. You can enable this in the config.toml file. This way, you won't see as many 403/Access denied errors, and these files won't be reported on.
I've made output and exit codes more useful and require less manual monitoring.
I've added a new URL for the VVS compatibility data API, which is a redirect to where Broadcom has saved the latest VCG Compatibility bundle. It used to be on AWS S3, but has recently been moved to Google Storage.
The command-line interface now shows both version and help information, which is very nice and much-needed.

For more details, check out the GitHub repository: https://github.com/MichaelRyom/DSvClient

DSvClient - New patch downloading tool for vCenter

2025-04-18T08:07:32+02:00

Streamlining VMware Update Management with DSvClient: A vExpert's tool

In the ever-evolving landscape of VMware infrastructure management, keeping components updated while ensuring consistency across environments has become increasingly complex. As a VMware administrator, you're likely familiar with the challenges of managing patches, updates, and ensuring file integrity across multiple environments. Today, I'd like to introduce you to a powerful utility that I have created.

I started this project for fun while experimenting with Rust during last year's Christmas holiday. It has since evolved into an application that I believe has multiple applications in enterprise environments.

What Is DSvClient?

DSvClient is a lightweight, cross-platform command-line utility designed specifically for VMware administrators who need to efficiently manage update repositories, verify file integrity, and ensure consistent deployment of VMware components. Built with performance in mind using Rust, DSvClient offers remarkable speed even when processing large repositories or verifying extensive file content.

Available for Windows, Linux, macOS, and even FreeBSD, DSvClient can be downloaded from Codeberg, making it accessible regardless of your management platform.

Click OS icon to download DSvClient for your OS

Download config.toml

Download sources.toml

Why DSvClient Matters

If you've ever managed a disconnected vSphere environment or needed to maintain consistent patch levels across multiple sites, you'll immediately understand DSvClient's value. Here are the key reasons why this tool deserves a place in your VMware toolkit:

Offline Repository Management Perfect for air-gapped environments where direct internet access is prohibited.
File Integrity Verification Automatically verifies checksums of downloaded components, ensuring you're deploying exactly what VMware released.
Consistent Update Strategy Simplifies maintaining identical patch levels across development, testing, and production environments.
Bandwidth Optimization Download once, deploy many times, dramatically reducing bandwidth usage in multi-site deployments.
Automation-Friendly Easily integrated into CI/CD pipelines and automation workflows.

Key Use Cases for DSvClient

1. Creating and Maintaining Offline Update Repositories

Many enterprise environments, particularly in financial, healthcare, and government sectors, operate in restricted network environments. DSvClient excels in these scenarios:

DSvClient C:\VMware\UpdateRepo

With this simple command, DSvClient will download the latest patches and updates from the configured sources, creating a perfectly structured, local repository that can be used by Update Manager or other deployment tools.

2. Validating Update Repository Integrity

Have you ever deployed an update only to discover file corruption midway through? DSvClient's verification capabilities prevent this nightmare scenario:

DSvClient C:\VMware\UpdateRepo --verify

This command meticulously checks each file against its expected checksum, producing a detailed report of any inconsistencies. It has saved me countless hours of troubleshooting corrupted installations.

3. Maintaining Consistent vSphere Environments

For organizations with development, testing, and production environments, consistency is critical. I use DSvClient to ensure identical patch baselines across all environments:

# Download once DSvClient /mnt/shared/VMware/Updates# Verify before deployment to each environmentDSvClient /mnt/shared/VMware/Updates --verify

4. Complete vCenter Upgrade Management

One of my favorite features is DSvClient's ability to handle VCSA upgrade paths:

# Configure DSvClient to download specific VCSA versionsvim sources.toml# Download all necessary VCSA componentsDSvClient /storage/vcsa-updates

This allows you to maintain a complete library of VCSA upgrade artifacts, ready for deployment when needed.

5. Centralized vSAN HCL Management

Managing the vSAN Hardware Compatibility List offline has traditionally been challenging. DSvClient simplifies this:

# The sources.toml already includes vSAN HCL sourcesDSvClient /path/to/vsan-repository

Now your vSAN health checks can reference a local, always-available HCL list and catalog.

Getting Started with DSvClient

Starting with DSvClient is straightforward, making it accessible even for administrators new to command-line tools:

1. Download the appropriate binary for your platform from Codeberg.

2. Configure your sources by customizing the sources.toml file. The default configuration includes:

ESXi base updates and patches
Vendor and partner provided add-ons
VMware certified async drivers
vSAN HCL list and catalog
VCSA upgrade paths

3. Run your first repository synchronization:

DSvClient /path/to/repository

4. Verify your repository:

DSvClient /path/to/repository --verify

5. Customize behavior through the config.toml file to adjust performance parameters based on your environment:

Concurrent download limits
Verification thread count
Logging verbosity
Exclude patterns for unwanted components

Advanced Configuration Tips

After using DSvClient extensively in various environments, I've found these configuration adjustments particularly helpful:

1. Tune concurrent downloads based on your internet connection:

[download]max_concurrent_downloads = 10

2. Optimize verification on powerful systems:

[verification]max_concurrent_verifications = 1000

3. Exclude unnecessary components to save space and time:

These are default, as they are not available on the VMware download site.

[exclude]patterns = [      "addon-main/addon/NEC/vib20/qedentv/QLC_bootbank_qedentv_3.40.9.0-1OEM.700.1.0.15525992.vib",      "addon-main/addon/NEC/vib20/qedf/QLC_bootbank_qedf_2.2.4.0-1OEM.700.1.0.15525992.vib",      "addon-main/addon/NEC/vib20/qedi/QLC_bootbank_qedi_2.19.5.0-1OEM.700.1.0.15525992.vib",      "addon-main/addon/NEC/vib20/qedrntv/QLC_bootbank_qedrntv_3.40.9.1-1OEM.700.1.0.15525992.vib",      "addon-main/addon/NEC/vib20/smx-provider/HPE_bootbank_smx-provider_700.03.16.00.12-14828939.vib",      "addon-main/addon/NEC/vib20/i40en/INT_bootbank_i40en_1.10.9.0-1OEM.700.1.0.15525992.vib",      "addon-main/addon/NEC/vib20/lpfc/EMU_bootbank_lpfc_12.6.228.4-1OEM.700.1.0.15525992.vib",      "addon-main/addon/NEC/vib20/qfle3f/QLC_bootbank_qfle3f_2.1.6.0-1OEM.700.1.0.15525992.vib",      "addon-main/addon/NEC/vib20/qcnic/QLC_bootbank_qcnic_2.0.0.0-1OEM.700.1.0.15525992.vib",      "addon-main/addon/NEC/vib20/qfle3/QLC_bootbank_qfle3_1.4.6.0-1OEM.700.1.0.15525992.vib",      "addon-main/addon/NEC/vib20/qfle3i/QLC_bootbank_qfle3i_2.1.2.0-1OEM.700.1.0.15525992.vib",      "addon-main/addon/NEC/vib20/amscli/HPE_bootbank_amscli_11.5.0.22-1OEM.700.0.0.15525992.vib",      "addon-main/addon/NEC/vib20/amsd/HPE_bootbank_amsd_700.11.5.0.28-1OEM.700.1.0.15525992.vib",      "addon-main/addon/NEC/vib20/ilo/HPE_bootbank_ilo_700.10.1.0.16-1OEM.700.0.0.14828939.vib",      "addon-main/addon/NEC/vib20/ixgben/INT_bootbank_ixgben_1.8.9.0-1OEM.700.1.0.15525992.vib",      "addon-main/addon/NEC/vib20/igbn/INT_bootbank_igbn_1.4.11.0-1OEM.700.1.0.15525992.vib",      "addon-main/addon/NEC/vib20/qlnativefc/Marvell_bootbank_qlnativefc_4.1.9.0-1OEM.700.1.0.15525992.vib",      "addon-main/addon/HTI/vib20/qedentv/QLC_bootbank_qedentv_3.40.9.0-1OEM.700.1.0.15525992.vib",      "addon-main/addon/HTI/vib20/qedi/QLC_bootbank_qedi_2.19.5.0-1OEM.700.1.0.15525992.vib",      "addon-main/addon/HTI/vib20/qedrntv/QLC_bootbank_qedrntv_3.40.9.1-1OEM.700.1.0.15525992.vib",      "addon-main/addon/HTI/vib20/qedf/QLC_bootbank_qedf_2.2.4.0-1OEM.700.1.0.15525992.vib",      "addon-main/addon/HTI/vib20/sut/HPE_bootbank_sut_700.2.5.5-1OEM.700.1.0.15525992.vib",      "addon-main/addon/HTI/vib20/smx-provider/HPE_bootbank_smx-provider_700.03.16.00.12-14828939.vib",      "addon-main/addon/HTI/vib20/hpessacli/HPE_bootbank_hpessacli_4.18.1.0-7.0.0.15525992.hpe.vib",      "addon-main/addon/HTI/vib20/lpfc/EMU_bootbank_lpfc_12.6.228.4-1OEM.700.1.0.15525992.vib",      "addon-main/addon/HTI/vib20/ilo/HPE_bootbank_ilo_700.10.1.0.16-1OEM.700.0.0.14828939.vib",      "addon-main/addon/HTI/vib20/amsd/HPE_bootbank_amsd_700.11.5.0.28-1OEM.700.1.0.15525992.vib",      "addon-main/addon/HTI/vib20/amscli/HPE_bootbank_amscli_11.5.0.22-1OEM.700.0.0.15525992.vib",      "addon-main/addon/HTI/vib20/qfle3i/QLC_bootbank_qfle3i_2.1.2.0-1OEM.700.1.0.15525992.vib",      "addon-main/addon/HTI/vib20/qfle3f/QLC_bootbank_qfle3f_2.1.6.0-1OEM.700.1.0.15525992.vib",      "addon-main/addon/HTI/vib20/qcnic/QLC_bootbank_qcnic_2.0.0.0-1OEM.700.1.0.15525992.vib",      "addon-main/addon/HTI/vib20/qfle3/QLC_bootbank_qfle3_1.4.6.0-1OEM.700.1.0.15525992.vib",      "addon-main/addon/HTI/vib20/qlnativefc/Marvell_bootbank_qlnativefc_4.1.9.0-1OEM.700.1.0.15525992.vib",      "addon-main/addon/HTI/vib20/ixgben/INT_bootbank_ixgben_1.8.9.0-1OEM.700.1.0.15525992.vib",      "addon-main/addon/HTI/vib20/bootcfg/HPE_bootbank_bootcfg_700.10.5.0.23-7.0.0.15525992.vib",      "addon-main/addon/HTI/vib20/hponcfg/HPE_bootbank_hponcfg_700.10.5.0.25-7.0.0.15525992.vib",      "addon-main/addon/HTI/vib20/testevent/HPE_bootbank_testevent_700.10.5.0.24-7.0.0.15525992.vib",      "addon-main/addon/HTI/vib20/conrep/HPE_bootbank_conrep_700.10.5.0.34-7.0.0.15525992.vib",      "addon-main/addon/HTI/vib20/cru/HPE_bootbank_cru_700.10.16-1OEM.700.0.0.14828939.vib",      "addon-main/addon/HTI/vib20/igbn/INT_bootbank_igbn_1.4.11.0-1OEM.700.1.0.15525992.vib",      "addon-main/addon/HTI/vib20/fc-enablement/HPE_bootbank_fc-enablement_700.3.5.0.40-1OEM.700.0.0.15525992.vib",]

4. Enhance logging for troubleshooting:

[logging]term_level = "Debug"

Integrating with Automation

DSvClient truly shines when integrated into automation workflows. You can incorporated it into your monthly patch management process:

Scheduled download of updates using a simple cron job or Task Scheduler task
Automatic verification after download completion
Integration with Update Manager through internal webserver
Notifications of download completion and verification status

The consistent exit codes and structured output make DSvClient particularly suitable for automation scenarios, enabling fully unattended update management.

DSvClient Application Flow Diagram

To understand how DSvClient works, here's a visual representation of the application's flow from configuration to repository creation:

┌────────────────────────────────────┐│ Configuration Files                ││ ┌──────────────┐  ┌──────────────┐ ││ │ config.toml  │  │ sources.toml │ ││ │  - Download  │  │  - VMware    │ ││ │    settings  │  │    updates   │ ││ │  - Verify    │  │  - VCSA      │ ││ │    settings  │  │    versions  │ ││ │  - Logging   │  │  - vSAN HCL  │ ││ │  - Exclusions│  │  - Vendor    │ ││ └──────────────┘  │    add-ons   │ ││                   └──────────────┘ │└───────────────────┬────────────────┘                    │                    ▼┌────────────────────────────────────┐│ DSvClient Core Process             ││ ┌──────────────────────────────┐   ││ │ 1. Parse configuration       │   ││ │ 2. Initialize components     │   ││ │ 3. Set up logging            │   ││ │ 4. Process command arguments │   ││ └──────────────┬───────────────┘   ││                │                   ││                ▼                   ││ ┌──────────────────────────────┐   ││ │         Downloader           │   ││ │ ┌────────────────────────┐   │   ││ │ │ ESXi Updates Pipeline  │   │   ││ │ └────────────────────────┘   │   ││ │ ┌────────────────────────┐   │   ││ │ │ VCSA Updates Pipeline  │   │   ││ │ └────────────────────────┘   │   ││ │ ┌────────────────────────┐   │   ││ │ │ vSAN HCL Pipeline      │   │   ││ │ └────────────────────────┘   │   ││ │ ┌────────────────────────┐   │   ││ │ │ Vendor Add-on Pipeline │   │   ││ │ └────────────────────────┘   │   ││ └──────────────┬───────────────┘   ││                │                   ││                ▼                   ││ ┌──────────────────────────────┐   ││ │          Verifier            │   ││ │ - SHA-1/SHA-256 verification │   ││ │ - Parallel processing        │   ││ │ - Error tracking             │   ││ └──────────────────────────────┘   │└────────────────┬───────────────────┘                 │                 ▼┌────────────────────────────────────┐│ Local Repository Structure          ││ /download_path/                    ││ ├── depot/                         ││ │   ├── index.xml                  ││ │   ├── vmw-depot-index-*.xml      ││ │   └── com.vmware.vsan.*          ││ │                                  ││ ├── addon-main/                    ││ │   ├── index.xml                  ││ │   └── addon/                     ││ │       ├── HTI/ (HPE)             ││ │       ├── DELL/                  ││ │       └── Other vendors...       ││ │                                  ││ ├── hcl.json                       ││ │                                  ││ └── vcsa/                          ││     └── version folders (e.g. 8.0.2)││        ├── manifest-latest.xml     ││        └── vmware-vcsa-* files     │└──────────────────┬─────────────────┘                   │                   ▼┌────────────────────────────────────┐│ Optional Web Server Deployment     ││ ┌──────────────┐  ┌──────────────┐ ││ │ Apache/NGINX │  │  Docker      │ ││ └──────┬───────┘  └────┬─────────┘ ││        │               │           ││        └───────┬───────┘           ││                │                   ││                ▼                   ││ ┌──────────────────────────────┐   ││ │ Internal Web Server Endpoint │   ││ │ http://updates.internal.com/ │   ││ └──────────────────────────────┘   │└────────────────┬───────────────────┘                 │                 ▼┌────────────────────────────────────┐│ Consumption Methods                ││ ┌──────────────┐  ┌──────────────┐ ││ │ vCenter      │  │ Hierarchical │ ││ │ Update       │  │ Multi-Site   │ ││ │ Manager      │  │ Distribution │ ││ └──────────────┘  └──────────────┘ │└────────────────────────────────────┘

This diagram illustrates the complete flow of DSvClient from configuration to consumption:

Configuration Files: DSvClient begins by reading two key configuration files:
- config.toml - Controls performance parameters, logging, and exclusions
- sources.toml - Defines what content to download and from where
Core Processing: The application initializes components, parses arguments, and prepares the environment
Download Pipelines: Multiple specialized pipelines handle different content types:
- ESXi updates (patches, base images)
- VCSA updates (appliance upgrades)
- vSAN HCL (hardware compatibility lists)
- Vendor-specific add-ons (drivers, utilities)
Verification: All downloaded content undergoes cryptographic verification:
- SHA-1 and SHA-256 checksum validation
- Parallel processing for performance
- Error tracking for failed verifications
Repository Structure: Content is organized into a VMware-compatible folder hierarchy
Optional Web Deployment: The repository can be exposed via web servers for network distribution
Consumption: The repository serves as a source for Update Manager or multi-site distribution

The separation of concerns in DSvClient's architecture makes it efficient and maintainable. Each pipeline handles specific content types while sharing common infrastructure for downloading, verification, and error handling. This modular approach also allows for easy extension to support new content types or sources as VMware's ecosystem evolves.

DSvClient Repository Structure: A Visual Breakdown

When you run DSvClient with a download path, it creates an organized repository structure that mirrors VMware's official update repositories. Here's a visual breakdown of the typical folder structure you'll see based on the default sources.toml configuration:

/download_path││├── addon-main                     # Hardware vendor add-ons│   ├── index.xml                  # Add-on index file│   └── addon│       ├── HTI                    # HP components│       │   └── vib20│       │       ├── ilo│       │       ├── amsd│       │       └── ... (other HP VIBs)│       ││       ├── DELL                   # Dell components│       │   └── vib20│       │       ├── iDRAC│       │       ├── BIOS│       │       └── ... (other Dell VIBs)│       ││       └── NEC                    # NEC components│           └── vib20│               └── ... (NEC VIBs)│└── ...│└── ...│└── ...│└── vcsa                           # vCenter Server Appliance updates    └── 8.0.2                      # Version-specific folders        ├── manifest-latest.xml    # Update manifest        ├── vmware-vcsa-8.0.2.0*   # VCSA ISO components        └── ... (other VCSA files)

This hierarchy is automatically created by DSvClient based on the sources defined in your sources.toml file. The default sources.toml includes entries for:

ESXi Base Repository - Contains core ESXi updates and patches
Hardware Vendor Add-ons - Drivers and utilities from partners like HPE, Dell and Cisco
vSAN HCL Download - The vSAN Hardware Compatibility List
VCSA Update Repositories - vCenter Server Appliance updates for various versions
Async Driver Updates - Out-of-band driver releases from VMware

When DSvClient runs, it traverses the URLs in sources.toml, intelligently downloading and organizing files to create this consistent structure. VIB files (VMware Installation Bundles) are stored in their appropriate vendor folders, while metadata files like XML indexes are placed at the proper hierarchy levels.

The organization automatically matches what vCenter Update Manager expects, allowing you to point your internal web server directly at this folder structure without any additional configuration. This careful mirroring of VMware's repository structure is what makes DSvClient so effective for maintaining internal update repositories.

The exclude section in config.toml lets you omit specific paths (like those "NEC" VIBs shown in the example configuration) to optimize storage space and download time, while still maintaining the overall repository structure.

DSvClient Decision Flow Chart

Here is a simplified visual representation of the applications decision flow

┌─────────────────────────┐│ DSvClient Start         │└──────────────┬──────────┘               │               ▼┌─────────────────────────┐│ Parse Configuration     ││ - Read config.toml      ││ - Read sources.toml     │└──────────────┬──────────┘               │               ▼┌─────────────────────────┐│ Process Command Args    │└──────────────┬──────────┘               │           ┌───┴────┐           ▼        ▼┌────────────┐    ┌────────────┐│ Path Only  │    │ Path with  ││ (Download) │    │ --verify   │└─────┬──────┘    └──────┬─────┘      │                  │      ▼                  │┌────────────────────────┐│ Download Mode          │◄─────────────────┐├────────────────────────┤                  ││ For each source in     │                  ││ sources.toml:          │                  │└──────────┬─────────────┘                  │           │                                │           ▼                                │┌────────────────────────┐                  ││ Source Type Decision   │                  │└──────────┬─────────────┘                  │           │                                │     ┌─────┼────────┬─────────┬────────┐    │     ▼     ▼        ▼         ▼        ▼    │┌─────────┐ ┌───────┐ ┌───────┐ ┌──────────┐││ ESXi    │ │ VCSA  │ │ vSAN  │ │ Add-ons  │││ Updates │ │ Files │ │ HCL   │ │ & Drivers││└────┬────┘ └───┬───┘ └───┬───┘ └────┬─────┘│     │          │         │          │      │     └──────────┴────┬────┴──────────┘      │                     │                       │                     ▼                       │┌────────────────────────┐                   ││ For each file:         │                   │└──────────┬─────────────┘                   │           │                                 │           ▼                                 │┌────────────────────────┐                   ││ File Exists?           │                   │└──────────┬─────────────┘                   │           │                                 │      ┌────┴─────┐                           │      ▼          ▼                           │┌──────────┐  ┌──────────┐                   ││   No     │  │   Yes    │                   │└────┬─────┘  └────┬─────┘                   │     │             │                         │     │             ▼                         │     │      ┌────────────────────┐           │     │      │ Verify Checksum    │           │     │      └────────┬───────────┘           │     │               │                       │     │        ┌──────┴─────┐                 │     │        ▼            ▼                 │     │  ┌──────────┐  ┌──────────┐           │     │  │ Valid    │  │ Invalid  │           │     │  └──────────┘  └────┬─────┘           │     │        │            │                 │     └────────┐            │                 │              │            │                 │              │            ▼                 │              │    ┌─────────────────────┐   │              │    │ Delete Invalid File │   │              │    └──────────┬──────────┘   │              │               │              │              └───────────────┘              │                              │              │                              ▼              │                     ┌─────────────────────┐ │                     │ Download File       │ │                     └──────────┬──────────┘ │                                │            │                                ▼            │                     ┌─────────────────────┐ │                     │ Add to Processed    │ │                     │ Files List          │ │                     └──────────┬──────────┘ │                                │            │                                └────────────┘                                │                                ▼           ┌─────────────────────────┐           │ --verify Flag Present?  │           └──────────┬──────────────┘                      │               ┌──────┴─────┐               ▼            ▼        ┌──────────┐  ┌──────────┐        │   No     │  │   Yes    │        └────┬─────┘  └────┬─────┘             │             │             │             ▼             │     ┌────────────────────┐             │     │ Verification Mode  │             │     ├────────────────────┤             │     │ For each file in   │             │     │ repository:        │             │     └────────┬───────────┘             │              │             │              ▼             │     ┌────────────────────┐             │     │ Verify Checksum    │             │     └────────┬───────────┘             │              │             │          ┌───┴────┐             │          ▼        ▼             │    ┌──────────┐ ┌──────────┐             │    │  Valid   │ │ Invalid  │             │    └─────┬────┘ └────┬─────┘             │          │           │             │          │           ▼             │          │    ┌─────────────────────┐             │          │    │ Report Mismatch     │             │          │    └─────────┬───────────┘             │          │              │             │          └──────────────┘             │                         │             └─────────────────────────┘                                      │                                      ▼                          ┌──────────────────────┐                          │ Exit with status     │                          │ code based on errors │                          └──────────────────────┘

Decision Flow Explanation

DSvClient follows a logical decision tree to determine what actions to take. Here's how the application makes decisions:

1. Initial Configuration

The application starts by parsing two key configuration files:

config.toml - Contains operational parameters like concurrent download limits, verification settings, and exclusion patterns
sources.toml - Defines what content to download and from where (ESXi updates, VCSA files, vSAN HCL, vendor add-ons)

2. Command Argument Processing

DSvClient then processes the command-line arguments to determine the primary mode of operation:

Path only (e.g., DSvClient /path/to/repo) - Default download mode
Path with --verify flag (e.g., DSvClient /path/to/repo --verify) - Verification mode

3. Download Mode Logic

When in download mode, DSvClient:

Processes each source defined in sources.toml
For each source, determines the appropriate handler based on the content type:
- ESXi updates processor
- VCSA files processor
- vSAN HCL processor
- Add-ons and drivers processor
For each file within each source:
- Checks if the file already exists locally
- If it exists, verifies its checksum
- If the checksum is valid, skips the file
- If the file doesn't exist or has an invalid checksum, downloads the file
- Adds the file to the processed files list to avoid redundant operations

4. Verification Mode Logic

When the --verify flag is provided (either directly or after download):

The application scans all files in the repository
For each file, it verifies the checksum against expected values
Valid files are noted as passing verification
Invalid files are reported as mismatches
A summary report is generated showing verification results

This decision flow ensures that DSvClient operates efficiently, avoiding unnecessary downloads while maintaining repository integrity. The semaphore-based concurrency controls (visible in the code) ensure that the application can process multiple files simultaneously without overwhelming system resources.

Serving Internal Update Repositories with DSvClient

A powerful extension to DSvClient's capabilities is setting up an internal web server to distribute your meticulously maintained repositories across your organization. This approach offers significant advantages for enterprises with multiple VMware environments or distributed teams.

Why Expose DSvClient Repositories via Web Server?

By serving your DSvClient repositories through a web server, you can:

Create a centralized update source for all your environments
Reduce external bandwidth consumption by downloading updates once for the entire organization
Enforce update standardization across teams and departments
Implement controlled update rollouts through repository staging
Provide secure access to updates in restricted network segments

Implementation Options

Setting up a web server to host your DSvClient repository is straightforward. Here are three effective approaches:

1. Simple HTTP Server for Internal Networks

For basic needs, a lightweight HTTP server works perfectly:

# On Linux with Apache sudo mkdir -p /var/www/html/vmware-updates sudo ln -s /path/to/dsvclient-repository /var/www/html/vmware-updates sudo systemctl restart apache2 # On Windows with IIS New-WebVirtualDirectory -Site "Default Web Site" -Name "vmware-updates" -PhysicalPath "C:\VMware\UpdateRepo"

2. Authenticated Access with NGINX

For environments requiring user authentication:

server { listen 80; server_name updates.internal.example.com; location / { root /path/to/dsvclient-repository; autoindex on; auth_basic "VMware Update Repository"; auth_basic_user_file /etc/nginx/.htpasswd; } }

3. Docker-Based Solution for Quick Deployment

For portable, consistent deployments:

docker run -d \ --name vmware-update-repo \ --restart=always \ -p 8080:80 \ -v /path/to/dsvclient-repository:/usr/share/nginx/html:ro \ -e NGINX_ENTRYPOINT_QUIET_LOGS=1 \ nginx:alpine

Implementation: Multi-Site Architecture

Implementing a hierarchical update distribution system using DSvClient:

1. Headquarters: Maintains the master repository with DSvClient and serves it via HTTPS

# Weekly update job 0 2 * * 0 /usr/local/bin/dsvclient /opt/vmware/master-repo >> /var/log/dsvclient.log 2>&1

2. Regional Offices: Synchronize from headquarters using a combination of rsync and DSvClient verification

# Pull updates from HQ rsync -avz --delete https://hq-updates.example.com/vmware-updates/ /opt/vmware/regional-repo/ # Verify integrity 
/usr/local/bin/dsvclient /opt/vmware/regional-repo --verify

3. Local Sites: Configure Update Manager to use their regional repository URL

This architecture reduced their external bandwidth usage by over 95% while ensuring consistent patch levels across all sites.

Integrating with vCenter Update Manager

The web-served repositories integrate seamlessly with vCenter:

In the vSphere Client, navigate to Update Manager > Settings
Select Download Settings
Choose Use a shared repository
Enter your internal URL: http://updates.internal.example.com/vmware-updates/

Now your Update Manager will source all patches and updates from your carefully maintained internal repository.

Security Considerations

When serving update repositories internally:

Consider HTTPS for sensitive networks, using internal CA certificates
Implement access controls based on network segmentation or authentication
Validate checksums at each distribution tier with DSvClient's --verify option
Maintain logs of repository access for compliance purposes
Regularly scan repository content with security tools

This approach provides a perfect balance of security and accessibility for your VMware updates.

By leveraging DSvClient with an internal web server, you create a robust, efficient distribution system for VMware updates that minimizes external dependencies while maximizing control over your update process. The result is a more resilient, bandwidth-efficient infrastructure that maintains consistent standards across your entire VMware estate.

Note on Broadcom download token

I have not have time to test DSvClient with the new download system Broadcom has put in place.

As it is compatible with Update Manager without a special release, I'm expecting that the data flow remains the same.

So only a url change may be needed.

Summary

DSvClient represents a significant advancement in VMware update and repository management. By providing a fast, reliable way to download, verify, and maintain VMware component repositories, it addresses key challenges facing VMware administrators in modern environments:

It simplifies offline repository management for security-conscious organizations
It ensures file integrity through comprehensive verification
It enables consistent environments through centralized update management
It optimizes bandwidth usage in distributed deployments
It integrates seamlessly with automation workflows

As environments grow increasingly complex and update cadences accelerate, tools like DSvClient become essential components of a well-managed VMware infrastructure. Whether you're managing a small cluster or an enterprise-spanning deployment, DSvClient offers tangible benefits in efficiency, reliability, and consistency.

I encourage you to download DSvClient from Codeberg and experience these benefits firsthand. Your update management workflow will never be the same.

Have you tried DSvClient in your environment? I'd love to hear about your experiences in the comments below!

VMUGDK Presentation - PowerShell: From Zero To Hero

2022-12-15T16:34:55+01:00

Hi all

Did a presentation on Powershell yesterday and only got around halfway through, before I ran out of time.

I have publish the entire thing on github if anyone want a peak at what I talk or didn't. No powerpoint just powershell code.

The presentation is an attempt to start slow and end up where modules and functions gets built.

https://github.com/MichaelRyom/VMUGDK-12-2022

Enjoy

Part 3: SuperMicro SuperServer E300-8D 2 years later

2019-08-05T21:37:25+02:00

How has it been

Before I jump into all the new stuff. Let me look at how it has been the last two years with the SuperMicro server in the lab. It has been excellent, no issues at all I have flashed BIOS, with no hiccups. IPMI is a life saver, not because I have had issues, but it is just to nice to be able do something stupid network wise and still be able to log in to the hosts and troubleshoot the issue. Still my only complaint about the SuperMicro, is that I need to buy license to use all the IPMI features like flashing the BIOS the easy way, instead of needing to create an USB stick and plug it in. It the same for all server vendors like HPE and DELL. Never the less it is just another cash cow. One you always want to have.

The upgrade

There are a few things that I have changed. Lets start with a list of items that I have bought.

6 x 32 GB Samsung DDR4 PC2666 REG (M393A4K40BB2-CTD)
2 x WD Blue 3D NAND SATA-SSD 2TB 6GB/s M.2 2280 (WDS200T2B0B)
Intel NUC NUC8i3BEK2 - WHAT!?!
Intel SSD Solid-State Drive 660p Series 512GB M.2 PCI Express 3.0 x4 (NVMe)
(SSDPEKNW512G8X1)
2 x 3PORT M.2 NGFF SSD CARD ADAPTER PCI EXPRESS 3.0 M.2 NGFF CARD
(PEXM2SAT32N1)
2 x RSC-RR1U-E8 RISERCARD 1HE | PASSIVE PCI-E X8 (RSC-RR1U-E8)

Also bought Noctua fans and a Micro SD card which turned out to be mistakes.

More memory

The price for memory has been high since I bought the 32 GB memory there is in each of my E300-8D's. Therefore I have never found the cash to buy the 128 GB each E300-8D can handle, until now! Since I already had two sticks of memory from another vendor, I have group those together with the new Samsung memory and there has been no issues. This was one of my biggest concerns, but I took a leap of fate and I worked no issues. So the lab has 256 GB of memory and it is still no enough, someday, just someday I'll hope that will be the case until then this will have to do :)

vSAN

I needed to be able to run a vSAN cluster, without to much trouble, when upgrade time comes around and without having to rebuild the home lab. That made me do two thing.

1. Buy a Intel NUC, as the SuperMicro server is just too expensive. This will be my "management cluster", single node. With Intel 660p M.2 and 32 GB memory. All it does is host the vSAN witness node and vCenter. Giving me 256 GB for all the fun stuff. A NSX manager will most likely also find its way over there, but for now just two VMs.

2. Storage upgrade! I ran hybrid mode, with an SATA cable to an external HDD. Not optimal and slow. When I first bough the E300-8D's I also bough some ASUS PCIe M.2 adaptors, but they are just one or two millimeters to tall, to fit inside the case. So I dropped them. Had a chat with a Canadian company, but it was WAY to much trouble buying from them, asking me to fax stuff to them!!!

Finally I found the Startech PEXM2SAT32N1, which is a PCIe card the holds one PCIe M.2 NVMe and two SATA M.2 cards. Just perfect for my needs, only one problem, it was also to tall. Luckly SuperMicro has a riser for PCIe slot. Making this the perfect solution. I could get more fast storage in the E300-8D and I could also expand the storage if you should need to do so at a later time.

I already had Samsung 960 Pro as cache tier, so I looked a something like it, but I found the options to be too expensive. So I settled for the Western Digital 2 TB Blue Line M.2 card. This should give me some decent amount of storage when combined with vSAN.

Now for some pictures

Intel NUC

Not much to say here. I had bought a micro SD card I thought I could install ESXi on, but It would not detect it. So I used an USB stick instead. I did not install on the Intel 660p card as I would like as much space available for VMs, after all it is only 512 GB of Storage, so I will be gone fast.

Startech PEXM2SAT32N1

This is a very cheap card, I paid less than 50 USD per card. Very cheap if you ask me, the riser, which I will come to in a moment, was less than 30 USD.

1 NVMe M.2 card which uses the PCIe port on the motherboard and two SATA M.2 cards with has to be connected to the motherboard via one SATA cable each. Power is served from the PCIe port.

Not much else to say all the bits are there including a small screwdriver, which is used to change the back plate of the card. The screwdriver does not work with the screws used to hold the M.2 card in place, which is a bit weird. Why ship a screwdriver with the package if it only solves one problem.

RSC-RR1U-E8 RISERCARD 1HE | PASSIVE PCI-E X8 (RSC-RR1U-E8)

I started out lazy. Did not want to put I all together to test if the StarTech card worked. So here you can see how the riser looks and fit if only half assembled.

Note the use need to break parts of the riser to make it fit. I had to break even more of to make it fit inside the Riser card bracket, but on that in a moment.

Riser card bracket

This come with the E300-8D as default. First I mounted the riser card in the bracket and then I mounted the StarTech card with the SATA M.2 card. I have tested it with two SATA and one NVMe on card and one on the motherboard and ESXi detected them all :)

SATA cabling

The SATA cabling is a bit of a hassle to fit. A SATA cable with both ends being L shaped, I think would be better. I any case this works as well.

The only issue

The SATADOM, which I use for ESXi, touches the Riser card bracket. Is this an issue, I do not know, but just to be on the safe side. I have moved the satadom to the other dedicated slot just next to it. So be warned if you want to have multiple satadom, you might need a sata cable or verify if this is an issue.

Lastly this is what 128 GB of memory looks like an small form factor!

That is all for this time thanks for reading this far!

ps. the Noctua fans did not work out for multiple reasons.

I bought the wrong once - This is my mistake.

1. They are too small compared to the once ready in
2. I'm not a fan of the rubber fittings that came with fans. They broke to easily. I used strips instead.
3. They are way too low in RPMs - The E300-8D get hot with them in. Around 80-90 degrees Celsius with "normal" load. I have not tested with 100% load.

The first picture shows the new fan. Second mounted. Third The old once. Fourth old vs new.

vRops 7.5 Active Directory authentication fails

2019-04-29T09:00:32+02:00

vRops 7.5 has been out little over two weeks now and from the looks of it, its an epic release with so many improvements and a great deal of new and exciting features.

I have upgraded a few environments with no problems at all and then last week yet another one got upgraded. As it does most times the upgraded went fine and the pre-upgrade bundle had no changes which would affect the customers environment.

I then went to the login page, to try to login to vRops. Selected the Active Directory source, typed my credentials and hit the login button. "Incorrect user name/password". I tried a few times more, thinking I might have fat fingered the password or username, but no dice. I logged in as local admin in vRops and looked at the vRops version, it had upgraded to 7.5. Check authentication sources under administration->access, and compared it to the customers second environment, it is all an exact copy of the environment I had issues with, only difference was the other environment had not been upgraded yet, so it is still running vRops 7.0.

Log Insight to the rescue

Lucky, I thought, they have Log Insight so that might help me troubleshoot me issue. I set the source to the vRops node and search for my username.

It worked, but the messages were a bit cryptic, messages like:

com.vmware.vcops.auth.server.util.AuthUtils.searchUserByNameAndSourceId - Search for user by name: Michael and sourceId: 3262cfdb-e8f4-4f7d-af92-0d291ca8fd88 returned 0 entries. Must return only one entry. Something wrong

and

com.vmware.vcops.platform.gemfire.GemfireFunction.invokeHandlerMethod - Failed to invoke the method 'UserAuthentication.authenticateUser' : InvalidCredentialsException: No user exists with userName: Michael and sourceId: 3262cfdb-e8f4-4f7d-af92-0d291ca8fd88

I looked at authentication sources and access control lots of times, before I noticed something, which I thought might be a clue. In access control the Active Directory users username is in User Principle Name (UPN) style format. Meaning username@domain.com was shown as the username, but since I had always logged in with just the username, SAM Account Name style, I had only tested that way of logging in. Could this be why the logs stated it could not find my username?

I quickly logout of vRops and login with the UPN style naming convention. I works! So something has been changed. I reach out on twitter and slack, nobody has noticed any login issues after upgrading to vRops 7.5.

The problem

I scratch my head. As I already have upgrade my homelab vRops and had not noticed anything in that either. I logon to my homelab to verify if I can replicate the issue. Login just works, I scratch me head somewhat harder. Then I notice the UPN suffix is different from the domain name to which the customer is using.

Back into the homelab, logon to the domain controller, open domain and trusts, right click on the root and properties and add the UPN of vmware.com, what better UPN to test with.

Then I go into users and computers, find my user, right click properties, account and change the UPN suffix to @vmware.com.

The old login still works. So I go to access control and see the UPN is still the old one of MichaelRyomDK.root. Under authentication sources I click on the update icon, to synchronize the authentication source.

This time under access control, the UPN suffix has changed to @vmware.com.

So I once again logout and try to login again, SAM style meaning only using my username. Hurry, it failed to login! I successfully replicated the error.

So while trying to troubleshoot this issue one of the things that struck me as a bit odd was that under authentication sources, if I edit my source and looked under details the common name is set to userPrincipleName. This by the way is default when you add a new authentication source. This could explain why I was force to use UPN style logins, but it does not explain why if I am part of the original domain name or UPN suffix, I can do without and login with just my username.

I change the common name to samAccountName, hit ok and synced the authentication source again. Now the users under access control, showed up with no UPN suffix. I logout of vRops and back in with just my username. It worked!

Conclusion

So it seems that there has been a change in the way vRops 7.5 authenticates it users, so if you are using multiple or non default UPN suffix's, what you might see after upgrading to vRops 7.5 is issues with authenticating users.

The quick fix is to under authentication source change the common name to samAccountName.

I have been in contact with Sunny Dua which is looking into the problem with vRops team.

With that said this was the only issue I had when upgrading to vRops 7.5 and for now there is a quick workaround to get this working again. So now you are truly ready to upgrade to vRops 7.5.

Thanks and take care.