The Optimization Mindset

Most teams use AI inefficiently:

Current state (typical team):
  All requests → Claude Sonnet → $3,000/month

Optimized state (same team, same quality):
  Simple tasks (60%) → Flash/Haiku     → $120/month
  Standard tasks (30%) → Sonnet/GPT-4o → $900/month
  Complex tasks (10%) → Opus/GPT-5     → $450/month
                        Total:           $1,470/month
                        Savings:         51%

The optimization process:

1. Instrument every request (model, tokens, task type, quality)
2. Analyze: which tasks use expensive models unnecessarily?
3. Test: can a cheaper model handle this task at acceptable quality?
4. Route: direct each task type to its optimal model
5. Monitor: ensure quality doesn't degrade

Building a Task-Model Matrix

Map your organization's AI tasks to models:

Task	Current Model	Optimal Model	Cost Reduction
Ticket classification	Sonnet	Gemini Flash	95%
Code autocomplete	GPT-4o	Codestral/local	99%
Code review	Sonnet	Sonnet (keep)	0%
Architecture design	Sonnet	Opus	-200% (worth it)
Test generation	GPT-4o	Haiku	85%
Doc generation	GPT-4o	Sonnet	50%
Data extraction	Sonnet	GPT-4o mini	93%
Email drafting	Sonnet	Haiku	85%
Bug investigation	Sonnet	Opus	-200% (worth it)

Key insight: Some tasks should upgrade to MORE expensive models. Better diagnosis on the first try saves hours of developer time — worth the extra $0.20 per request.

Systematic Quality Testing

Before downgrading a task to a cheaper model, test thoroughly:

Step 1: Collect representative samples

Gather 50-100 real requests for the task type
Include edge cases and difficult examples
Record the current model's responses as baseline

Step 2: Run candidates

For each candidate model:
  Run all 100 samples
  Record responses
  Measure: latency, tokens, cost

Step 3: Evaluate quality

Option A — Human evaluation:
  Rate each response: Acceptable / Degraded / Unacceptable
  If >95% Acceptable → model qualifies

Option B — LLM evaluation:
  Use a frontier model to compare responses
  "Is Response B as good as Response A for this task?"
  If >90% "yes" → model qualifies

Option C — Automated metrics:
  For structured output: accuracy, completeness, format compliance
  For code: tests pass, no new linting errors

The awesome-opencode Ecosystem

A growing collection of tools and integrations for AI development:

Core Tools:

• OpenCode — open-source AI coding agent
• Model Prism — multi-provider gateway and router
• Prompt Flux — dynamic prompt composition

MCP Servers:

• File system, Git, GitHub, GitLab
• Database (Postgres, MongoDB, SQLite)
• Browser automation
• Monitoring (Prometheus, Grafana)
• Communication (Slack, Telegram, Email)

Skills & Commands:

• /review — standardized code review
• /test — test generation
• /docs — documentation generation
• /security — security audit
• /deploy-check — pre-deployment validation

Community Resources:

• Shared AGENTS.md templates for common tech stacks
• Skill libraries for different domains
• Model comparison benchmarks
• Cost optimization guides

Building Your Optimization Pipeline

A systematic approach to continuous optimization:

┌─────────────────────────────────────────┐
│  1. INSTRUMENT                          │
│  Tag every request: task_type, model,   │
│  tokens, latency, cost, quality_score   │
├─────────────────────────────────────────┤
│  2. ANALYZE (weekly)                    │
│  Which task types use expensive models? │
│  Where is quality over-provisioned?     │
│  Where is quality under-provisioned?    │
├─────────────────────────────────────────┤
│  3. EXPERIMENT                          │
│  A/B test cheaper models per task type  │
│  Measure quality impact                 │
│  Calculate savings potential            │
├─────────────────────────────────────────┤
│  4. DEPLOY                              │
│  Update routing rules in Model Prism    │
│  Set alerts for quality regression      │
│  Monitor for 2 weeks                    │
├─────────────────────────────────────────┤
│  5. REPEAT                              │
│  New models release monthly             │
│  Re-evaluate every quarter              │
│  The optimal mapping is always changing │
└─────────────────────────────────────────┘

Advanced Optimization Techniques

For teams ready to go further:

Prompt caching:

Cache long system prompts server-side
Only send the unique user message each time
Savings: 30-50% on input tokens for repeated patterns

Semantic caching:

"What's the capital of France?" → cache hit
"Tell me France's capital city" → semantic match → cache hit
"Capital of France?"            → semantic match → cache hit

Batch API discounts:

Many providers offer 50% discount for batch processing:
- Collect non-urgent requests throughout the day
- Submit as a batch at midnight
- Results available by morning
- Perfect for: report generation, data analysis, bulk classification

Fine-tuning (the nuclear option):

If you send >50,000 similar requests per month:
- Fine-tune a small model on your specific task
- Often matches GPT-4 quality at GPT-4o-mini cost
- Requires ML expertise and labeled training data
- Consider only after exhausting routing optimizations

The Complete AI Stack

Putting it all together — a mature AI infrastructure:

┌─ Developer Experience ─────────────────┐
│  IDE (Cursor/VS Code) + CLI (OpenCode) │
│  Slash commands (/review, /test, /docs)│
│  Remote control (Telegram/Slack)       │
├─ Gateway Layer ────────────────────────┤
│  Model Prism                           │
│  Auto-routing, cost tracking, quotas   │
│  Model aliasing, tier boost            │
├─ Provider Layer ───────────────────────┤
│  Cloud: OpenAI, Anthropic, Google      │
│  Managed: AWS Bedrock, Azure           │
│  Self-hosted: Ollama, vLLM             │
├─ Observability ────────────────────────┤
│  Prometheus metrics, Grafana dashboards│
│  Cost analytics, quality monitoring    │
│  Audit logs, usage reports             │
└────────────────────────────────────────┘

This isn't built in a day. Start with one layer (gateway), add others as you grow. The goal is a system that gets better and cheaper over time — automatically.

---quiz question: What is the typical cost savings from systematic task-model optimization? options:

• { text: "About 5-10%", correct: false }
• { text: "40-60% while maintaining the same quality", correct: true }
• { text: "100% — optimization makes AI free", correct: false } feedback: Systematic optimization typically saves 40-60% by routing simple tasks (which are the majority) to cheaper models while reserving expensive models for genuinely complex work. Quality remains the same because each task gets a model that's fully capable of handling it.

---quiz question: Why should some tasks be UPGRADED to more expensive models? options:

• { text: "Because expensive models are always better", correct: false }
• { text: "Better diagnosis on the first try saves hours of developer time, making the extra $0.20 per request worthwhile", correct: true }
• { text: "To use up the AI budget", correct: false } feedback: For complex tasks like bug investigation and architecture design, a frontier model that gets it right on the first try saves hours of developer time — making the small cost increase highly profitable when measured in total cost (AI + human time).

---quiz question: How often should task-model mappings be re-evaluated? options:

• { text: "Once, when first configured", correct: false }
• { text: "Every quarter at minimum, because new models release monthly and the optimal mapping changes", correct: true }
• { text: "Only when costs increase", correct: false } feedback: The AI model landscape changes rapidly — new models, new pricing, new capabilities every month. Quarterly re-evaluation ensures you're always using the best model for each task, capturing savings from newer, cheaper models as they become available.