DoorDash processes millions of chat messages every day across a real-time marketplace where dashers, customers, and merchants communicate constantly. The stakes are high: harassment, fraud, and safety incidents can erode trust and damage the platform. Yet running every message through a premium LLM would be prohibitively expensive at that scale.

Bruna Pereira from DoorDash recently presented how the company solved this by building SafeChat, a content-agnostic AI moderation platform that combines lightweight models with strategic LLM use. The result? Reduced safety incidents, lower operational costs, and a system that non-engineers can tune without deploying new code.

For founders building trust and safety, moderation, or compliance products, DoorDash's architecture offers a practical blueprint for scaling AI systems that actually work in production.

The Cost Problem With LLM-Only Moderation

The obvious approach to content moderation in 2024 is to throw everything at a large language model. ChatGPT, Claude, or Gemini can handle nuanced context, multiple languages, and subtle violations with a single API call.

But at DoorDash's scale—millions of messages daily—this becomes economically unsustainable. Even at a few cents per call, the cost compounds quickly. Worse, latency matters in a real-time chat system. Users won't tolerate multi-second delays for simple messages.

DoorDash's solution was to stop treating moderation as a single-model problem.

Hybrid Architecture: Fast Filters Plus Smart Routing

SafeChat uses a tiered approach that mirrors how human moderators would triage content:

Tier 1: Lightweight internal models handle the obvious cases. Simple profanity, known scam patterns, and clear violations can be caught by fast, cheap classifiers trained on DoorDash's own data. These models run in milliseconds and cost a fraction of LLM calls.

Tier 2: LLMs for nuance. When a message isn't clearly safe or unsafe—context-dependent slang, sarcasm, cultural references—it gets routed to an LLM for multi-axis scoring. The LLM evaluates harassment, fraud risk, policy violations, and context in one pass, returning structured scores rather than binary yes/no decisions.

Tier 3: No-code policy workflows. Non-engineers—trust and safety teams, policy specialists—can define rules and thresholds in a no-code interface. If a message scores high on fraud but low on harassment, route it to review. If it's flagged in a specific city or merchant category, apply stricter rules. These workflows are version-controlled and backtested against historical data before deployment.

This architecture means DoorDash only pays for expensive LLM inference on the small percentage of messages that genuinely need it.

Backtesting and Continuous Improvement

One of the smartest design choices in SafeChat is the backtesting layer. Before deploying a new policy or threshold, DoorDash runs it against weeks of historical data to see how many incidents it would have caught and how many false positives it would have created.

This is critical for trust and safety products because the cost of false positives—blocking legitimate users or messages—can be as damaging as missing real violations. Backtesting gives teams confidence that policy changes improve precision and recall, not just one at the expense of the other.

It also means the system improves over time. As new patterns emerge—novel scam tactics, seasonal fraud spikes—teams can iterate quickly without waiting for engineering sprints.

Key Takeaways for Founders Building Moderation or Safety Products

If you're pitching a product in this space, DoorDash's approach shows that "AI-powered" isn't enough. You need a thoughtful architecture that balances cost, accuracy, and the realities of operating at scale.

Most founders spend months debating architecture and tooling when they should be validating product-market fit with real users. Get your MVP built in 3 days and start gathering the metrics and feedback that matter.

Sources: https://www.infoq.com/presentations/doordash-llm-ai-moderation-platform/