Back to Research

LAB NOTE // 003

Lab NoteJul 19, 20266 min

Hydra: composing specialized models instead of scaling one

Our systems aren't one model doing everything. A high-capacity core coordinates models dedicated to specific roles — orchestration, audio, vision — over an internal nervous system we call Tesseract. What we built, and what is still design.

Abstract

Hydra is the lab's architectural thesis: instead of scaling a single model until it does everything, compose a system out of specialized models coordinated by a high-capacity core. This note documents the pattern at the two levels where we apply it — specialized models running side by side inside one system, and specialized cognitive modules within that system's runtime — plus the internal fabric that connects them, which we call Tesseract. Parts of this are running code with four years of iteration behind them; parts are still design. We say which is which.

1. The question

The default answer to "make the AI better" is "use a bigger model." That answer is expensive, it centralizes, and on owned hardware it hits a wall fast: our lab server has four consumer GPUs and roughly 37GB of VRAM total. You cannot brute-force your way out of that.

So we asked the other question: what if capability comes from composition rather than scale? One core model with the breadth to coordinate, plus smaller models that are genuinely good at one thing each — and an internal fabric that lets them behave as a single system rather than a pile of services.

2. The pattern, at two levels

Between models. Our current system runs several models simultaneously, each in its own container with its own GPU allocation, each with a single job: a multimodal model as the main reasoning core; a dedicated speech-recognition model for hearing; a dedicated vision model for understanding what the camera sees; a dedicated text-to-speech model for voice. And — the piece we think matters most — a separate model whose only role is orchestration: a small routing model that decides where work goes, rather than making the large model spend its capacity on traffic direction.

That last one is the thesis in miniature. Orchestration is a real job. Giving it its own specialist, instead of a slice of the core's attention, is exactly what the composition argument predicts you should do.

Inside the system. The same principle repeats one level down, among cognitive modules rather than models: perception, inner monologue, autonomy (belief–desire–intention planning), goal decomposition, gated self-improvement, and a deliberative council that consults several models before consequential decisions. These are specialized components with narrow contracts, not one monolithic loop.

Two levels of abstraction, one idea: specialists coordinated by a core, rather than a generalist doing everything.

3. Tesseract: the internal nervous system

For the composition to behave like one system, the parts need a shared body. Tesseract is our name for that fabric, and it is deliberately internal — not a network between machines, not a mesh between installations. In its original formulation it maps to biology: a message bus acting as synapses for fast signalling between components, a vector store acting as long-term memory, embeddings as the associative layer that lets one part recall what another learned.

We state this precisely because the name invites a bigger claim than the truth. Tesseract is the nervous system of one organism. What we have built across machines is more mundane and more real: our other systems connect to the same memory store and message fabric through a purpose-built remote-agent protocol — one brain, several specialized bodies. That is the distributed part, and it exists because it was useful, not because it was visionary.

4. Evidence, and its limits

The multi-model composition is not a diagram: it is four generations of running configuration, in code, with containers and GPU assignments we can point at. The cognitive-module architecture is documented in the system's own theory document, written before the current generation existed, and implemented in a package whose structure still matches that theory today.

The honest boundary: a coordination pattern we relied on early — one role proposing actions, another auditing and approving them — ran as two roles inside a single model, played through prompting, rather than as two separate models. That was a deliberate test step: validate that a role behaves correctly before committing hardware to it. It worked, and it is also exactly the kind of detail that gets quietly upgraded into "we run separate specialist models for governance" if nobody insists on precision. We insist. The specialization that is real today is real: core, audio, vision, orchestration. The specialization by security posture — dedicated models for internal defense and external assessment — is designed, named internally, and not yet built as separate models.

5. Why this matters beyond our lab

If capability can come from composition, then sovereign AI stops being a luxury argument. You don't need a datacenter to run a system with genuine breadth — you need a coherent architecture, a fabric that lets specialists share memory, and the discipline to give each job to something actually good at it. That is a thesis a small lab can test, and we have been testing it on our own metal for two years.

6. Limitations and what's next

The system this note describes is dormant — its host was decommissioned in April 2026, the stack is preserved, redeployment is staged. Composition is measured operationally, not benchmarked: we have no formal comparison of "composed system versus single large model" on identical tasks, and that comparison is the obvious next experiment. Parts of the roster are design rather than implementation, marked as such above. And the deliberate act of naming the pattern — Hydra — is younger than the pattern itself, which is a normal hazard of building faster than you document.

Compose the specialists. Give orchestration its own specialist too. Connect them with a nervous system, not an API contract. That is the whole thesis, and the parts of it that are built have been running long enough for us to say they work.


Research home: blackicelabs.ca/research

Written by Levon Azevedo

Get the next one by email.

Subscribe
Hydra: composing specialized models instead of scaling one | Black Ice Labs