Classifying Event-Ticket Emails with AI: From Prompt Engineering to Fine-Tuning GPT-4o-mini

 Post 2 in my series on an automation hub for a ticket-resale business. This one covers the AI pipeline that reads and classifies event-ticket emails — from prompt engineering and multimodal handling (emails with images) to building my own dataset and fine-tuning GPT-4o-mini.

The problem

The purchasing team's inbox receives hundreds of emails a day: Ticketmaster presale announcements, venues releasing new ticket blocks, artists sending promo codes, cancelled orders that need refunds... Missing a "tickets released" email can mean losing a chance to buy at face value; missing a cancellation email means losing refund money. But nobody can read hundreds of emails a day by hand.

The requirement: classify emails into categories (PRESALES, TICKETS_RELEASED, TICKETS_ON_SALE, ALMOST_SOLD_OUT, PROMOTION_DISCOUNT, Cancelled...) and extract structured data: performer, venue, event date, purchase link, promo code — and for cancellation emails, the order number, vendor, quantity, and amount.

The pipeline

IMAP (MailKit) reads new mail
   → extract text/HTML + inline images (base64 data URLs, max 4)
   → send batch to the LLM (fine-tuned OpenAI / Gemini) with a ~120-line prompt
   → parse the JSON output (with its own defensive layer — see post 3)
   → group by label, send an HTML digest email to the team
   → Cancelled items go straight to the refund sheet for finance

A few technical details worth calling out:

• Incremental reads via IMAP UID: instead of rescanning the whole mailbox, I persist the last processed UID per (account, folder) pair in MySQL, and each run only searches UniqueIdRange(lastUid+1, MaxValue). A cold start takes the 10 newest messages. Nothing is ever processed twice, and there's never a full scan.
• Multimodal: many marketing emails put all the information inside a banner image. I pull the inline images, base64-encode them into data: URLs, and send them as image content parts — the model "sees" the banner instead of reading an empty alt attribute.
• Deep links back to the original mail: every row in the digest email links straight to the source message via mail.google.com/...#search/rfc822msgid:<Message-ID> — one click from the digest to the exact email.
• Two providers, one contract: AnalyzeEmailsWithAI() (OpenAI) and AnalyzeEmailsWithGemini() (Gemini Flash) return the same JSON schema, so I can swap providers to compare cost and quality.

Prompt engineering = bug fixing in natural language

My classification prompt is ~120 lines long and has its own "commit history" — every time the model misclassified something, I added a new rule and kept the old one as a dated comment, like a changelog. Some patterns that emerged:

1. Gating rules instead of vague descriptions

At first I just described "TICKETS_RELEASED means an email announcing new tickets." The model slapped that label on anything containing the word "new." The fix: every label got a gating condition — it can only be assigned when there is "explicit newness evidence," backed by concrete lists of allow-cues and disqualifiers.

2. A time heuristic

An "on sale now" email sent three months ago was still being reported as an upcoming sale. The fix: a rule comparing the sale-start date against the email's sent date, treating it as "fresh" only within a ±7-day window.

3. Banning keyword matching

At one point a band sent "Just released: new T-shirts & ornaments" and the model cheerfully filed it under TICKETS_RELEASED. The fix: a dedicated NON-TICKET CONTENT section with YES/NO few-shot examples, plus one instruction that turned out to be surprisingly effective:

"Do NOT act like a simple search-term filter; never pick a label just because you saw one keyword."

4. Normalizing data inside the prompt

Refund emails usually carry negative quantities and amounts. The prompt requires absolute values — and the C# side still backstops it with Math.Abs. The principle: never trust a prompt to be followed 100% of the time; always enforce it again in code.

5. De-duplication with an actual formula

The same show can appear in five different emails. The prompt defines a literal canonical_key = lower(subject)|lower(performer)|lower(venue)|event_date with merge rules (keep the most specific link, the most specific date, any promo code), plus a priority chain when one email matches multiple labels: Cancelled > ALMOST_SOLD_OUT > TICKETS_RELEASED > PRESALES > PROMOTION_DISCOUNT > TICKETS_ON_SALE.

Fine-tuning: building a dataset out of... PDFs

No matter how good the prompt gets, a generic model has a ceiling. I decided to fine-tune GPT-4o-mini on the team's own email data. The catch: the pre-labelled data lived as... folders of PDFs exported from Gmail, organized by category (PRESALES/, PROMOTION_DISCOUNT/...).

The dataset pipeline:

1. Extract text from each PDF with UglyToad.PdfPig; also render page 1 to PNG (Magick.NET, 150 DPI) and upload it to Azure Blob for a vision variant.
2. Clean up: Gmail PDF exports are full of cruft ("1 message", Reply-to: headers, timestamps, "View in browser", unsubscribe lines...). I wrote a set of regex heuristics to isolate the real subject/body from all that.
3. Emit JSONL in OpenAI's chat format (system/user/assistant), shuffle with a fixed seed (42, of course), and split train/valid 80/20.
4. Drive the raw REST API: multipart file upload with purpose=fine-tune, create a fine-tuning job on the gpt-4o-mini base model, then poll every 8 seconds until succeeded, dumping all job events to the log.

The most memorable bug: "invalid file format"

OpenAI kept rejecting my JSONL file with a vague format error. The cause: the file had been written as UTF-16 with a BOM (a classic Windows/PowerShell artifact). The fix was an EnsureJsonlUtf8 function: sniff for UTF-16 LE/BE BOMs, re-encode to UTF-8 without a BOM, and validate that every NDJSON line is a valid JSON object before uploading. Lesson learned: when uploading files to a third-party API from Windows, check the encoding before you suspect anything else.

Results

The team no longer reads hundreds of emails a day by hand. Each notification type arrives as a single digest email, cancellation emails flow automatically into the finance refund sheet, and the fine-tuned model actually speaks the "language" of ticketing emails instead of guessing like a generic model.

Next post: LLM output is never as clean as you think — how I built a defensive layer for parsing "dirty" JSON in C#.

Tech stack for this post: MailKit/MimeKit (IMAP/SMTP), OpenAI Fine-tuning API, Google Gemini API, UglyToad.PdfPig, Magick.NET, Azure Blob Storage.