Inbox Agent — AI Classification, Extraction & Evaluation Specification

Status: Implementation Baseline
Audience: Cursor, Charter, engineering, QA
Purpose: Define the AI decision layer used to understand mail while preserving deterministic, auditable behavior.

1. Objectives

The Inbox Agent uses AI to convert unstructured email content into structured, reviewable signals that support inbox triage, search, contact intelligence, follow-up, and future automation. AI output is advisory unless an explicitly authorized workflow permits an action.

Core principles:

• Preserve the source message and provider state as the system of record for mail content.
• Separate model inference from deterministic policy and side effects.
• Require typed, versioned structured output.
• Record provenance, model/prompt versions, confidence, and evaluation results.
• Prefer abstention or human review over unsupported certainty.
• Never infer that an AI result itself authorizes destructive or externally visible actions.
• Make reprocessing possible without corrupting historical inference records.

2. Processing Boundary

AI processing occurs against the canonical MailMessage, with mailbox-specific context supplied from MessageInstance when relevant. A single logical message may have multiple instances; content classification must not be duplicated merely because the same message exists in more than one mailbox/folder/account context.

Processing stages:

1. Canonical message eligibility check.
2. Content normalization and quoted-history/signature handling.
3. Lightweight deterministic extraction.
4. AI classification.
5. AI entity/action extraction.
6. Confidence and validation gates.
7. Persistence of immutable inference result.
8. Policy evaluation.
9. Optional human review or downstream workflow.

3. Classification Taxonomy

The implementation must support a versioned taxonomy rather than hard-coded UI labels. Initial semantic dimensions should include:

• Message intent: informational, request, response, notification, transactional, marketing, social, system-generated, unknown.
• Attention requirement: no action, read/review, reply, perform task, approve/decide, follow up, urgent review.
• Priority: low, normal, high, urgent.
• Relationship context: person-to-person, business/service, automated sender, mailing list, unknown.
• Lifecycle state recommendation: inbox, waiting, follow-up, reference/archive candidate, review required.
• Risk/sensitivity indicators: financial, legal, credential/security, personal/private, attachment-sensitive, suspicious/uncertain.

Taxonomy values are extensible. Unknown/unclassified is always a valid value.

4. Structured Extraction

AI may extract:

• people and organizations;
• sender role and inferred relationship when evidence exists;
• dates, times, deadlines, and temporal expressions;
• commitments and requests;
• action items and responsible party;
• referenced accounts, projects, topics, products, or services;
• monetary amounts and transaction references;
• contact information appearing in message content/signatures;
• attachment semantics;
• reply/follow-up expectations;
• concise message/thread summary;
• user-facing rationale for classification.

Each extracted item must carry:

• normalized value;
• source/evidence reference when feasible;
• confidence;
• extraction type;
• inference run identifier.

Do not silently overwrite canonical contact data with inferred values. Contact consolidation is a separate governed process.

5. Thread Context

Models may use prior canonical messages in the conversation/thread when needed. Thread context must:

• be bounded to control cost and prompt size;
• identify which messages were supplied;
• avoid treating quoted duplicates as independent evidence;
• distinguish the current message from historical context;
• preserve chronology;
• allow evaluation with and without thread context.

6. Inference Contract

Every model call used for persisted decisions must produce schema-validated output. Persist at minimum:

• inference ID;
• MailMessage ID;
• optional MessageInstance ID when instance context materially affected the result;
• task type;
• model/provider identifier;
• prompt/template version;
• taxonomy/schema version;
• processing timestamp;
• input content fingerprint;
• structured output;
• per-field confidence where applicable;
• overall confidence;
• validation status;
• token/cost/latency telemetry when available;
• error/abstention reason.

Invalid structured output must not be coerced into valid business data without an explicit repair/validation step.

7. Confidence and Human Review

Confidence thresholds are configuration, not embedded constants.

Recommended behavior:

• High confidence: persist and expose normally.
• Medium confidence: persist, expose with uncertainty, and route to review where material.
• Low confidence: abstain from consequential recommendations or mark as review required.
• Contradictory evidence: prefer review required.

Any future automatic action must have an independent policy threshold stricter than the threshold for simply displaying an AI suggestion.

8. Deterministic Policy Layer

The model describes the message; application policy decides what may happen.

Examples of deterministic checks:

• whether an account permits write-back;
• whether the action is destructive;
• whether the sender is trusted;
• whether confidence meets the configured threshold;
• whether the user has approved the automation category;
• whether provider state has changed since inference;
• whether the proposed action is still applicable.

No model response may directly invoke provider mutation.

9. Prompt and Model Versioning

Prompts are source-controlled implementation assets. Changes that can alter persisted semantics require a new prompt/version identifier. Model changes must be independently versioned.

Reprocessing must create a new inference record rather than erase the previous result. The application may designate one result as current while retaining history for audit and evaluation.

10. Evaluation Framework

Evaluation must cover classification, extraction, summaries, abstention, and policy-supporting signals.

Maintain a curated evaluation corpus containing representative:

• personal correspondence;
• work/business messages;
• automated notifications;
• newsletters/marketing;
• receipts and transactional mail;
• long threads;
• forwarded messages;
• messages with signatures and quoted history;
• ambiguous requests;
• deadlines;
• attachments;
• adversarial/prompt-injection-like content.

Metrics should include:

• precision/recall/F1 for categorical labels;
• exact/normalized match for structured fields;
• deadline/date accuracy;
• action-item precision and recall;
• calibration by confidence band;
• abstention quality;
• false-positive rate for urgent/action-required labels;
• summary factuality;
• regression against the approved baseline;
• latency and cost distributions.

High-impact false positives must be weighted more heavily than cosmetic classification errors.

11. Golden Set and Regression Gates

A human-reviewed golden set is required before classification behavior is considered production-ready. Each material prompt/model/taxonomy change must run against the golden set.

Release gates:

• no unexplained regression in critical extraction fields;
• no material increase in high-impact false positives;
• schema validity at the required target;
• latency/cost within service-quality boundaries;
• reviewed examples for newly introduced taxonomy values.

Evaluation results must be reproducible from model/prompt/schema version metadata.

12. Security and Prompt Injection

Email content is untrusted data. Instructions contained in email bodies, attachments, signatures, or quoted messages must never be treated as system/developer instructions.

The AI layer must not:

• disclose secrets or credentials;
• follow embedded instructions to contact third parties;
• execute links or attachments merely because content requests it;
• bypass application policy;
• modify mail or contacts directly.

13. Failure Handling

Failures are classified as retryable or terminal.

Retryable examples: transient provider/model errors, timeouts, rate limits.
Terminal examples: unsupported content, repeated schema failure, permanently unavailable content.

A failed inference must not block synchronization of the underlying mail. Processing state and retry metadata are persisted separately from the canonical message.

14. Acceptance Criteria

The AI subsystem is implementation-ready when:

• all persisted AI tasks have typed contracts;
• inference history is versioned and auditable;
• MailMessage-level processing avoids duplicate inference across instances;
• confidence/abstention behavior is defined;
• model output cannot directly cause provider mutations;
• a golden evaluation corpus and regression runner exist;
• prompt injection is explicitly treated as untrusted content;
• failures can be retried without duplicate side effects.