Distributed inference · self-hosted
Your next GPU cluster is already on your desks.
Nodea pools the idle memory in the laptops, towers and GPU workstations your organization already owns into one private inference fleet. Install the node app, point the OpenAI SDK at your own endpoint. No new hardware, no prompts leaving the network.
- device
- free memory
- model
- serving
How it works
From idle machine to inference node in four steps.
One bidirectional gRPC stream carries registration, model assignment and every token the fleet generates.
Install
The Nodea node app runs on macOS, Windows or Linux — on machines the organization already manages.
Register
It opens a JWT-authenticated gRPC stream and announces a device identity: name, type, free GPU memory.
Assign
The orchestrator fits a model to that memory footprint and pushes a load-model command down the same stream.
Serve
Requests route to a node with the right model loaded; generated tokens stream back to the caller in real time.
Adoption
Point your stack at it. Nothing else changes.
Same SDK, same schema, same streaming semantics — inference just happens on your own floor.
from openai import OpenAI client = OpenAI( api_key="<your-api-key>", base_url="https://your-org.getnodea.com/openai/v1", ) stream = client.chat.completions.create( model="gemma-2-2b", messages=[{"role": "user", "content": "Summarize Q3 churn."}], stream=True, ) # tokens arrive from a workstation two desks away
Visibility
You can see the fleet you are running.
| Device | Model | Status |
|---|---|---|
| studio-04 · GPU workstation | DeepSeek-R1-8B | serving |
| mbp-14 · design | Gemma-2-2B | serving |
| tower-09 · finance | Gemma-2-2B | idle |
| mbp-16 · platform | Llama-3.2-1B | serving |
| tower-02 · support | — | offline |
Connection events, per-device token counts and org-wide usage, streamed live over SSE.
Every organization already owns an idle AI datacenter.
Nodea turns it into a private, OpenAI-compatible inference cloud you control — billed to hardware you have already paid for.