A self-shutting-down Linux VM for a weekly batch job on Azure (.NET 8 + CZDS zone files)
I needed to process DNS zone files for about 1,000 top-level domains, load ~280 million domain names into SQL, and do it on a schedule without paying for a server that sits idle 95% of the time. This is how I set it up, what the stack looks like, and the mistakes I made along the way. The problem ICANN's Centralized Zone Data Service (CZDS) gives approved users a daily snapshot of every gTLD zone file: .biz , .info , .xyz , all the way to .com . Each file lists every registered domain in that TLD. I wanted to turn those into one table: label -> number of TLDs it is registered in , so that for a keyword like cloudapp we can show "registered in 37 TLDs". Requirements: Run once a week for now, nightly later. Big memory for the largest zones, so a real VM, not a serverless function. Cost close to zero when not running. No one logging in to press Start. The stack Layer Choice Why Worker .NET 8 console app, published self-contained for linux-x64 Str...