How It Works

ANT is the intelligent routing engine that sits between your application and LLM providers. Every request goes through a multi-stage pipeline to find the optimal model for your task.

┌──────────┐   ┌─────────────┐   ┌──────────────┐   ┌──────────┐   ┌──────────┐
│ Your App │──▶│ Classify LLM│──▶│ Optimize LLM │──▶│ Route AI │──▶│   LLM    │
└──────────┘   └─────────────┘   └──────────────┘   └──────────┘   └──────────┘
                Task type &        Compress           Score &        2,500+ models
                complexity         (optional)         select         30+ providers

Stage 1: Classify LLM

ANT's classification LLM analyzes your prompt in real-time to determine the task type, complexity level, and required capabilities. This runs as parallel micro-inferences in under 15ms, producing structured route tags that drive all downstream decisions.

Task Types

coding - Code generation, debugging, refactoring, review
reasoning - Math, logic, multi-step problem solving
analysis - Data analysis, research, structured extraction
creative - Writing, brainstorming, storytelling
chat - General conversation, Q&A, support
translation - Language translation, localization
vision - Image analysis, OCR, visual understanding
summarization - Document summarization, key point extraction

Complexity Levels

simpleQuick facts, lookups, greetings, basic formatting — routed to fast/free models
mediumMulti-step reasoning, explanations, summaries, code with context — balanced models
complexDeep analysis, novel architecture, creative synthesis, long-context — premium models

Stage 2: Optimize LLM (Optional)

When enabled, ANT's optimization LLM compresses your prompt using a trained language model that identifies and removes redundant tokens while preserving semantic meaning. This typically saves 20-40% on tokens with no quality loss.

Light

Remove filler words, standardize formatting

Normal

Compress verbose instructions, merge contexts

Aggressive

Rewrite for minimal tokens while preserving intent

Enable this in your dashboard under Settings > Prompt Optimization.

Stage 3: Route AI

ANT's routing AI combines the classification tags with your preference weights to score every available model in real-time. It factors in each model's capabilities, current latency, cost, provider health, and context window requirements to select the optimal LLM for this specific request.

Scoring Formula

score = speed_pref × speed
      + quality_pref × quality
      + cost_pref × cost
      + task_bonus × capability_match
      + context_bonus × fits_window

speed            = 1 - normalized_latency
quality          = task_capability_rating (per task type)
cost             = 1 - normalized_cost_per_token
capability_match = model's strength on detected task type
fits_window      = prompt fits within model's context window

You can adjust your speed/quality/cost preferences in the dashboard, or use virtual models like ant:fast or ant:best for preset weights. You can also create custom nests that restrict routing to a specific set of models — see the Virtual Models docs.

Stage 4: Execution

ANT forwards your request to the selected LLM. The response is streamed back to your application in real-time, with headers that tell you exactly what happened:

Header	Description
`X-Antbase-Model`	Actual model used
`X-Antbase-Task-Type`	Detected task category
`X-Antbase-Complexity`	simple, medium, or complex
`X-Antbase-Tokens-Saved`	Tokens saved by prompt optimization

Automatic Failover

If the selected LLM returns an error or times out, ANT automatically fails over to the next-best scored model from a different provider. Circuit breakers track provider health in real-time and temporarily remove unhealthy providers from the scoring pool.

Retry logic: Up to 3 attempts, each targeting a different provider. ANT re-scores remaining healthy models between retries, so the fallback is always the best available option — not just the next one in a static list.

If all providers for the requested capability are down, the error is returned with the ALL_PROVIDERS_DOWN code and a list of attempted providers for debugging.