Setting Up Local Routines with AI Agents: What Worked and What Didn't

Over the past few months I have moved a handful of recurring back-office jobs from "a person does this every morning" to "an AI agent does this every morning, and a person checks the summary". Not chatbots, not a one-off prompt: routines that run on a schedule, touch real databases, and have to be right. This post is about the setup work behind that — the boring parts that decide whether an agent routine is trustworthy or just impressive in a demo.

The running example is a small tool I built for a domain-name business: every day it pulls a few thousand domain roots that still lack keyword metadata, has an AI agent split each root into words (worldcup2026update → world cup 2026 update), validates the result, and writes the keywords back to SQL. Simple on paper. Most of the lessons below came from making it boring in practice.

Start with the shape of the job, not the agent

My first instinct was to hand the agent the whole task: "connect to the database, fetch the pending rows, split them, write them back." It worked once. It was also unreviewable, non-repeatable, and one bad run away from overwriting good data.

What actually worked was splitting the routine into stages with a clear boundary around the part that needs intelligence:

• Export — deterministic Python. Query the database, apply the same pre-filters the downstream consumer uses, write CSV files into a dated run folder.
• AI pass — the only stage where the agent thinks. It reads the input files and writes result files with the same names. Nothing else.
• Validate — deterministic Python again. Every split must concatenate back to the original root letter-for-letter; anything else is rejected with an issue code and queued for a retry file.
• Load — deterministic, guarded, and the only stage that writes to the database. It updates only rows that still have no keywords, so re-running it is harmless.

The agent never touches the database. It cannot skip validation because validation is a separate command. And because each run lives in its own folder with a metadata file, I can open any day's run and see exactly what went in, what came out, and what was rejected.

Write instructions the way you would write them for a new hire

The prompt the agent follows is a Markdown file that the export step generates per run, with the absolute run path filled in. A few things made a measurable difference in result quality:

• One hard rule, stated first. "The words must concatenate back to the root exactly. Never drop, add, or change a letter." Everything else is guidance; this one is enforced by the validator, and telling the agent that upfront cut rejected rows dramatically.
• Examples of both directions. Not just what a good split looks like, but what a tempting wrong split looks like: sojoband is sojo band, not so job and. Agents generalize from counterexamples much better than from abstract rules.
• An explicit stopping condition. "Run the validator, read its summary, retry the rejected rows once, stop when a retry no longer reduces the count." Without this the agent either quits too early or loops forever polishing.
• A list of files it must never edit. Inputs, the dictionary-split results, the export manifest. Agents are helpful by default, and helpful includes "fixing" your input data if you let them.

Don't send the agent work a dictionary can do

Roughly a fifth of the domain roots were trivially splittable with off-the-shelf word-segmentation libraries. Those now get answered locally before the agent ever sees them, with a conservative confidence threshold: at most three words, every word must be a frequent English unigram, no stopwords, no www-style repeats. Anything the dictionary is unsure about goes to the agent.

This is less about cost than about attention. The agent's error rate on the hard names went down when the easy names stopped padding out its input files, and the daily run got faster for free.

Chunk the input; the agent will thank you

A single 2,000-row CSV is a bad unit of work for an agent. It is too big to hold in context comfortably, a mistake on row 1,400 forces a redo of everything, and there is no natural checkpoint. I chunk to 100 rows per file (part-001.csv, part-002.csv, …) and the agent writes one result file per input file. Validation is per file, retries are per row, and a partially completed run is still a useful run.

Make the routine survive where it runs

The local version runs on a Windows box with a virtualenv and an .env file. I then wanted the slow AI pass to run in a cloud coding session so the local machine was not tied up. That surfaced a whole class of setup problems that have nothing to do with AI:

• Provisioning has to be a script, not a memory. The cloud environment needs an ODBC driver, a handful of Python packages and an editable install of the project. That lives in a cloud-setup.sh that always exits 0 (a failing setup script blocks the session from starting) and can also be re-run inside a session to repair a stale snapshot.
• Network is the real boundary. The cloud session could reach PyPI and Microsoft's package repo, run the test suite, and complete the AI pass — but it could not open a TCP connection to Azure SQL on port 1433. Everything leaves through an HTTPS proxy. I measured it, documented the exact failure mode, and stopped fighting it.
• Hybrid beats heroic. The database stages stay on the local machine; only the AI pass moves to the cloud. The hand-off is a git branch: export locally, commit the run's input files, let the cloud session write the results and push, pull and validate and load locally. It is not elegant, but every step is inspectable and nothing runs where it cannot be trusted.
• Secrets scope down, not up. Environment variables in a shared cloud environment are visible to every command the agent runs. The database login for this tool can SELECT the source query and run one specific UPDATE, nothing more. The agent is a capable colleague; it still does not get the admin password.

Small things that saved me repeatedly

• A latest pointer. A file containing the path of the most recent run means every downstream command can default to "the latest export" instead of needing a path argument.
• Dry runs on anything that writes. load --dry-run stages the rows and prints how many would change, and the agent is instructed to run it before the real thing.
• Idempotency by design. The load statement filters on "still has no keywords", so a routine that fires twice does nothing the second time. This removed an entire category of "what if the scheduler double-fires" worries.
• Tests for the deterministic parts. Thirty-odd pytest cases cover the exporter, the validator and the loader. The AI pass is not unit-tested; it is validated at runtime instead, which is the right place for a non-deterministic component.
• A README that records measurements, not intentions. "The filter keeps about 72% of names, the dictionary answers about 20% of those, the query takes about 7 seconds." Numbers with dates on them are what you need when a run looks off six weeks later.

What I would tell someone starting today

• The agent is a stage, not the system. Put deterministic code on both sides of it. Let code decide what the agent sees and whether the agent's output is accepted.
• Optimize for reviewability first. Dated run folders, per-row status, issue codes. Speed and cost come later and are much easier to fix.
• Write the instructions as a document the agent reads at runtime, versioned with the code, with the run-specific bits templated in. Prompts in chat history are not a routine.
• Assume the environment will change under you. Setup scripts, pinned dependency ranges, explicit network assumptions written down with the date you checked them.
• Grant the least access that lets the job finish. Then let the agent be as autonomous as you like inside that box.

None of this is specific to domain names. The same shape — deterministic export, chunked agent pass, strict validation, guarded write-back — is what I now reach for whenever a recurring job has a fuzzy step in the middle and a database at the end.

---

More case studies on the Featured Projects page, or reach me at letanphp@gmail.com.