Building a Local AI Email Triage System

Start with the problem

Before any AI existed, the workflow looked like this:

That works. But it also means making the same small decision, over and over. The goal wasn't to hand the inbox over — it was to remove the decisions that don't actually need a mind.

Step 1 — running an AI model locally

The first piece of the system was llama.cpp. It lets us run compatible language models directly on our own computer, instead of sending text to an external API:

./build/bin/llama-server \
  -m models/gemma3/gemma-3-4b-it-Q4_K_M.gguf \
  --port 8080

Now our computer has a local endpoint:

http://127.0.0.1:8080

Our JavaScript application can send prompts to that address just like it would talk to any other API. The important difference:

The email never has to leave the machine to get classified.

What does llama-server actually do?

The model and the server are two different pieces:

The model itself doesn't know anything about Gmail. It just receives text and generates text — our application is what gives it a useful job.

Test the server first

Before connecting anything else, we can check whether the local server is alive. While the model is loading, the health check fails — once it's ready, it returns ok:

This is an important development habit:

Test one layer before adding another layer.

If the AI server isn't working, debugging Gmail at the same time just makes the problem harder to find.

Step 2 — reading Gmail

Next, the application connects to Gmail using Google's API and OAuth, and searches specifically for:

is:unread in:inbox category:primary

That distinction matters — I didn't want the model wasting time on hundreds of promotional messages from the Promotions tab.

For every email, the app extracts:

{
  id,
  from,
  subject,
  date,
  body
}

Why clean the email first?

Real emails are messy. They can contain:

The model doesn't need most of that. So before it ever sees an email, the application strips the noise out:

return body
  .replace(/https?:\/\/\S+/g, " ")
  .replace(/<[^>]+>/g, " ")
  .replace(/\s+/g, " ")
  .trim()
  .slice(0, 1500);

We're reducing noise before asking the model to reason. That matters even more when you're running AI on limited hardware.

Step 3 — give the model a specific job

At first, the model returned something simple:

{
  "priority": "high",
  "action_required": true,
  "summary": "Recruiter wants interview availability.",
  "reason": "A response is required."
}

That's already more useful than asking "what do you think about this email?" — we define a schema and force the model into a predictable structure. Later, the classifier became more detailed:

{
  "category": "interview_request",
  "priority": "high",
  "action_required": true,
  "confidence": "high",
  "summary": "Company requests interview availability.",
  "reason": "The sender is asking the recipient to schedule an interview."
}

Now JavaScript can make decisions from the result.

/ DEBUGGING /

Then the model started making mistakes

This is where the project got a lot more interesting. An automated LinkedIn job digest, a Tennis Warehouse marketing email, and an Audible recommendation were all classified as related to a job interview or a personal request. There wasn't an interview in any of them.

See what the model said, and what was actually true:

Technically, the model was following pieces of the prompt. It just wasn't understanding the distinction that actually mattered.

A valid AI response is not necessarily a correct AI response.

The model is not the entire system

My first instinct could have been: get a bigger model. Instead, I changed the architecture. Some email decisions don't require AI at all — if a message comes from a known automated job-alert address, JavaScript can already decide, with no model in the loop:

Why spend thirty seconds asking a language model something we already know?

Rules first, AI second

const JOB_ALERT_PATTERNS = [
  /\bjobs similar to\b/i,
  /\bnew jobs?\b/i,
  /\bjob alert\b/i,
  /\brecommended jobs?\b/i,
];

Now obvious job-board emails never have to reach the model at all.

Rules vs. AI reasoning

I separated two different kinds of knowledge. One file holds things the application already knows — deterministic, same input, same result every time:

const SENDER_RULES = {
  "jobs-noreply@linkedin.com": "job_alert",
  "email@alert.ally.com": "financial_alert",
};

The other holds things the model needs to understand — the ambiguous cases, where language and context actually matter:

A job posting is NOT an interview request.

A marketing call-to-action is NOT a personal request.

A personalized greeting does not mean the email
was personally written for the recipient.

I think of this file similarly to a SKILL.md or instruction file used by AI agents — it teaches the model how to reason about the cases that are genuinely uncertain.

Walk an email through it

Pick a real email below and step through exactly what happens to it — some emails never reach the model at all.

This is much closer to an actual AI system than simply calling a model on every email that arrives.

Step 4 — let code control the action

One more design decision mattered: the model classifies. JavaScript decides what actually happens.

if (
  category === "billing_statement" &&
  confidence === "high"
) {
  return "star";
}

I might personally want every billing statement starred. That's my workflow preference — the model shouldn't have to rediscover it every time it reads an email.

Step 5 — add a safe dry-run mode

Before letting the program modify Gmail, I added a flag. Dry-run mode runs the entire workflow — fetch, rules, AI, classify, decide — but instead of changing anything, it just says what it would have done:

node classify-email.js --dry-run

Flip the switch below and run it both ways on the same three emails.

Only node classify-email.js, with no flag, runs the live version. Before giving AI-assisted software permission to change something, give yourself a way to observe its decisions first.

Comparing models

Once the workflow worked, I could swap models without rewriting the application — the JavaScript always talks to 127.0.0.1:8080, so I can stop one model server and start another. That let me compare:

One model might answer faster but make worse decisions. Another might reason better but take longer. The "best" model isn't necessarily the largest one — it's the model that works best inside the workflow you're building.

What I learned

The biggest lesson from this project wasn't how to install a language model. It was learning where AI actually belongs in an application.

Each piece has a specific responsibility.

/ NEXT LEVEL /

What could we build next?

The next stage could turn this into a background email agent, instead of one you run by hand. Important messages could trigger a notification right away:

While less urgent starred emails accumulate into a digest instead:

At that point, the system isn't just classifying email. It's managing a workflow.

/ CHALLENGE /

Your turn

Build your own small AI classifier. It doesn't have to use Gmail — start with three pieces:

You could classify:

Before writing any code, answer these three questions:

• What decisions are completely predictable? Those probably belong in code.
• Which decisions require understanding language or context? Those are good candidates for AI.
• What happens if the AI gets something wrong? That should decide how much control you give it.

The bigger idea

A local model by itself isn't an automation. The useful part comes from everything built around it:

The goal isn't to use AI everywhere. The goal is to find the places where AI adds judgment that ordinary application logic can't easily provide. That's what turned an old MacBook and a local language model into something that could actually do useful work.