A self-shutting-down Linux VM for a weekly batch job on Azure (.NET 8 + CZDS zone files)

I needed to process DNS zone files for about 1,000 top-level domains, load ~280 million domain names into SQL, and do it on a schedule without paying for a server that sits idle 95% of the time. This is how I set it up, what the stack looks like, and the mistakes I made along the way.

The problem

ICANN's Centralized Zone Data Service (CZDS) gives approved users a daily snapshot of every gTLD zone file: .biz, .info, .xyz, all the way to .com. Each file lists every registered domain in that TLD. I wanted to turn those into one table: label -> number of TLDs it is registered in, so that for a keyword like cloudapp we can show "registered in 37 TLDs".

Requirements:

Run once a week for now, nightly later.
Big memory for the largest zones, so a real VM, not a serverless function.
Cost close to zero when not running.
No one logging in to press Start.

The stack

Layer	Choice	Why
Worker	.NET 8 console app, published self-contained for `linux-x64`	Streams the gzip, parses ~4M lines in seconds, `SqlBulkCopy` to Azure SQL. No runtime install on the server.
Server	Azure `Standard_D8ds_v5` (8 vCPU, 32 GB), Ubuntu 24.04 LTS	Enough RAM to hold a whole zone's labels in a `HashSet`. Linux because it boots fast and a systemd unit is all the scheduler I need.
Storage	256 GB Standard SSD data disk at `/data`	Yesterday's label file is needed to compute today's diff. Must survive shutdown.
First boot	cloud-init	Formats and mounts the data disk, installs `jq` and `curl`. Pasted as "custom data" when creating the VM.
Scheduler on the VM	systemd `oneshot` service enabled at boot	The VM booting is the trigger. No cron, no Hangfire.
Wake-up	Azure Logic App (Consumption) with a Recurrence trigger	One HTTP action calling ARM `POST .../virtualMachines/x/start` with managed identity. Costs a fraction of a cent per month.
Shutdown	The VM deallocates itself via the ARM API using its own managed identity	Only deallocated VMs stop billing.
Safety net	DevTest auto-shutdown at 6 PM Pacific	If the job hangs, the VM still goes off.
Database	Azure SQL: staging table + set-based `MERGE`	1.3M rows staged in 27 s, full load in 3.5 min.
Archive	Azure Blob (cool tier, 30-day lifecycle)	Raw `.gz` files, in case we need to reprocess.

How a run works

Saturday 08:00 PT   Logic App --POST /start--> VM boots
                                                |
                    systemd: zoneworker-nightly.service (runs on every boot)
                                                |
                    run-nightly.sh
                      1. read VM tags from the Instance Metadata Service
                         AutoRun=false?  -> exit, leave VM on (maintenance boot)
                      2. ZoneWorker run-all: auth once, download 4 zones in parallel,
                         parse -> diff vs yesterday -> apply adds/removes to SQL
                      3. POST webhook if any zone failed
                      4. POST .../deallocate on itself via managed identity

The two VM tags turned out to be the nicest part of the design. AutoRun=false lets me start the VM from the Portal to deploy a new build or read logs without triggering a 3-hour run. AutoDeallocate=false lets a run finish and keep the machine on so I can inspect it. Both are read from inside the VM through the metadata endpoint:

curl -s -H Metadata:true \
  "http://169.254.169.254/metadata/instance/compute/tagsList?api-version=2021-02-01" \
  | jq -r '.[] | select(.name=="AutoRun") | .value'

Self-deallocation needs no Azure CLI on the box, just a token from the same endpoint:

TOK=$(curl -s -H Metadata:true "http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=https%3A%2F%2Fmanagement.azure.com%2F" | jq -r .access_token)
RID=$(curl -s -H Metadata:true "http://169.254.169.254/metadata/instance/compute/resourceId?api-version=2021-02-01&format=text")
curl -X POST -H "Authorization: Bearer $TOK" -H "Content-Length: 0" \
  "https://management.azure.com${RID}/deallocate?api-version=2024-07-01"

For this to work the VM's system-assigned identity needs the Virtual Machine Contributor role scoped to itself. More on that below.

Numbers from the first real run (`.biz`)

Step	Result
Download	42.6 MB gzip in 1.5 s
Parse	3.39M lines, 3.2M NS records, 1,286,102 unique labels in 4.6 s
Stage to SQL (`SqlBulkCopy`)	1.29M rows in 27 s
Full load + aggregate `MERGE`	3 min 26 s
Whole run, boot to power-off	about 4 minutes

The second run on the same day found zero changes and skipped SQL entirely. The diff is a merge of two sorted files, so it costs nothing.

Things that bit me

1. Pasting the SSH public key without its prefix. The Portal text box happily accepted AAAAC3Nza... and I only noticed later that the line must start with ssh-ed25519. Copy the whole .pub line with Get-Content ~/.ssh/id_ed25519.pub | clip.

2. "Classic Virtual Machine Contributor" is not "Virtual Machine Contributor". They sit next to each other in the role picker. The Classic one is for the old Service Manager model and gives zero permissions on Microsoft.Compute. My deallocate calls failed five times with a 403 before I spotted the word Classic in the IAM list. Same fix for the Logic App's identity.

3. Wrong resource group in the ARM URL returns Forbidden, not Not Found. I had created the VM in an existing resource group but typed the planned one into the Logic App's URI. ARM answers 403 when the caller has no rights at that scope, which is misleading when the real issue is a typo. Copy the resource ID from the VM's Properties page instead of typing it.

4. Multi-line pastes into an SSH window get garbled. Twenty lines of install commands ended up interleaved (df -h /dsudo mkdir ...) and half of them never ran. Bash's bracketed-paste also waits for an Enter that is easy to miss, and Ctrl+C silently discards the whole block. Fix: ship an install.sh with the build and run one command, or join commands with && on one line ending in && echo INSTALL-OK.

5. Stop is not Deallocate. shutdown -h now inside the VM leaves it in "Stopped" state, and you keep paying for the cores. Only the ARM deallocate action (or the Portal's Stop button, which calls it) releases the hardware.

6. The 300 GB temp disk is wiped on every deallocate. Tempting to use for the download cache, useless for anything that must survive the night. The persistent data disk at /data is the only place for yesterday's label files.

7. LUN 0 matters for cloud-init. The data disk appears at /dev/disk/azure/scsi1/lun0 only if you attached it at LUN 0. My cloud-init disk_setup hard-codes that path.

8. Secrets in an env file, read literally. The CZDS password contains !, $ and ^. Sourcing the file with . env would expand $74 to nothing. The script reads it line by line with export "$line", so no shell expansion happens. chmod 640 root:zoneworker.

9. Set the safety-net shutdown after the job can possibly finish. My first instinct was 10 AM for an 8 AM start. The first run over 1,000 zones is a full load and could take hours; a too-early auto-shutdown would kill it halfway and leave SQL half-updated. It is a safety net, not the schedule.

10. .com is a different animal. 160 million labels in a HashSet<string> is roughly 12 to 16 GB of managed heap, plus sort buffers. It may fit in 32 GB, it may not. It is excluded from the weekly run until the parser gets an external-sort path. Everything else, about 1,064 zones and 120 million labels, fits comfortably.

Cost

Weekly schedule, pay-as-you-go in West US:

Item	Per month
D8ds_v5 compute, about 6 h/week	about $12
64 GB OS disk + 256 GB data disk (Standard SSD)	about $22
Static public IP	about $4
Logic App Consumption, ~16 executions	under $0.01
Total	about $38

Moving to nightly runs raises compute to roughly $50 a month. Disks and the IP bill whether the VM runs or not.

What I would do the same next time

Let the boot be the trigger. A oneshot systemd unit plus "start the VM on a schedule" is simpler than any in-VM scheduler and survives reboots for free.
Tags as a control plane. Two tags replaced a config service, a maintenance flag file, and a lot of SSH-ing.
Managed identity everywhere. No secrets for the deallocate call, no secrets for the Logic App, and RBAC scoped to a single VM.
Build a vertical slice on the smallest useful zone first. Everything that worked for .biz (auth, streaming parse, bulk load, diff) is reused unchanged at full scale. Only parallelism and memory handling change.

What is next

Add a memory-bounded parser for .com, skip zones whose Last-Modified header has not changed, and a weekly job that renews expiring CZDS grants. Then move from Saturdays to nightly.

Search This Blog

Tan LE