Orion is a Python-based terminal assistant built to act as an intelligent broker for AI queries.
While Atlas abstracts the provider (“I want to use different models without opening four apps”), Orion abstracts the decision: “I don’t care which model answers this. Figure it out.”
Instead of manually choosing an LLM, I simply type a question. Orion analyzes the intent and determines whether the prompt needs a fast local model, a heavyweight reasoning model, a specialized coder, or live web search—then executes it automatically.
Intelligent Routing
I built this to explore the systems engineering behind LLMs—focusing on routing, latency, and cost.
- Dynamic Routing: Uses fast heuristics to classify intents before falling back to a small local classifier model.
- Escalation: If a small model returns a weak response, the orchestrator automatically escalates the prompt to a larger reasoning model.
- Cost-Aware: Evaluates estimated API costs to select the cheapest capable provider.
- Local-First: The pipeline runs locally, dynamically loading and unloading models to aggressively manage GPU memory.
Evolution into Atlas
Orion served as a critical testing ground, proving the value of intelligent routing and local-first orchestration.
However, the terminal REPL proved too limiting for daily use. I ultimately abandoned the CLI and ported Orion’s best architectural ideas—like its model-agnostic routing and lightweight web search—into a native macOS application. That project became Atlas.