Deep Dive into a ReAct Runtime in Go: Building an MCP-Compatible AI Agent

J.A.R.V.I.S. is an AI Agent platform I built from scratch. The ReAct (Reason + Act) runtime is written entirely in Go 1.25+.

The sections below are the parts where I tried the wrong thing, changed direction, and only then landed on the current design.

Multi-agent routing: one agent per kind of work

Instead of one agent that “knows everything” handling every kind of request, J.A.R.V.I.S. routes each request to a specialized agent (coding, research, scheduling, casual conversation). The routing is based on a lightweight intent classification step that runs before the main LLM call.

Why this design instead of a single mega-prompt:

• Lower token cost: a specialized agent needs only a short system prompt focused on its own domain, rather than cramming every instruction and tool definition of the whole system into one context window.
• Higher accuracy: a “coding” agent tuned specifically for reading code and calling search/file tools drifts less in its answers than a general agent that has to balance too many jobs.
• Easier to extend: adding a new agent (say a “finance” agent for budget questions) only means registering a new route, with no edits to the other agents’ prompts.

The classification step uses a small, low-latency model (under 100ms) to label the intent. Only then does the router forward the request to the right agent, along with a trimmed-down context.

A request flows through the runtime like this:

flowchart TD
    U["User request"] --> IC["Intent classification<br/>small model, under 100ms"]
    IC --> RT{"Router picks an agent"}
    RT --> A1["Coding agent"]
    RT --> A2["Research agent"]
    RT --> A3["Scheduling agent"]
    RT --> A4["Casual conversation agent"]
    A1 --> CTX["Trimmed context"]
    A2 --> CTX
    A3 --> CTX
    A4 --> CTX
    CTX --> LLM["That agent's main LLM"]

Running tools in parallel with errgroup

When a task needs several independent tools (web search, a database query, reading a file), the runtime uses errgroup to run them in parallel instead of one after another. This is where Go clearly beats the other languages I’ve tried for runtimes like this:

func ExecuteToolsParallel(ctx context.Context, calls []ToolCall) ([]ToolResult, error) {
	g, ctx := errgroup.WithContext(ctx)
	results := make([]ToolResult, len(calls))

	for i, call := range calls {
		i, call := i, call // capture loop variables correctly
		g.Go(func() error {
			res, err := executeTool(ctx, call)
			if err != nil {
				return fmt.Errorf("tool %s failed: %w", call.Name, err)
			}
			results[i] = res
			return nil
		})
	}

	if err := g.Wait(); err != nil {
		return nil, err
	}
	return results, nil
}

What matters about errgroup.WithContext: if one tool call fails, the context is cancelled immediately, and the other tool calls still running get the cancel signal instead of carrying on for nothing. This brings the total latency of a “reasoning step” with 5 tool calls from ~4 seconds (sequential) to under 1 second.

The parallel path and the failure path look like this:

flowchart TD
    S["Reasoning step<br/>5 independent tool calls"] --> EG["errgroup: run in parallel<br/>one goroutine per tool"]
    EG --> OK["All tools finish"]
    EG --> FAIL["One tool fails"]
    OK --> RES["Collect results: under 1 second<br/>instead of ~4 seconds sequentially"]
    FAIL --> STOP["context is cancelled immediately<br/>the remaining tools get the cancel signal and stop running for nothing"]

A separate timeout for each kind of tool

Not every tool needs the same timeout. A tool calling a search engine can wait 3 seconds, but a tool querying the internal database should wait at most 500ms before it counts as a failure. J.A.R.V.I.S.’s runtime attaches a timeout per tool type using context.WithTimeout nested inside each goroutine, rather than one shared timeout for the whole batch.

3-tier memory: Vector, Graph and Redis

A useful AI Agent has to remember more than what was said 5 minutes ago. But “remember everything” by stuffing the entire history into the context window is both token-hungry and ineffective once a conversation runs for weeks. J.A.R.V.I.S. splits memory into 3 tiers, each serving a different kind of query:

• Vector memory (MongoDB Atlas Vector Search): semantic recall for long-term conversation. When a user returns to a topic discussed weeks ago, the system searches by embedding similarity instead of exact keyword search.
• Graph memory: stores relationships between entities (people, projects, tools) to answer multi-hop questions. For example, “what’s the project Trinh is working on with the Backend team?” needs to join several relationship edges, not just find one relevant piece of text.
• Redis: short-term working memory for the current context window, with a TTL tied to the session. This is the fastest tier, serving the current ReAct loop directly with no round trip to the vector DB.

To handle a question, the runtime queries all 3 tiers in parallel (using errgroup again, like the tool execution above), then merges the results in priority order: Redis (immediate context) → Graph (relationships) → Vector (long-term semantics).

The three tiers run in parallel and only differ at merge time:

flowchart TD
    Q["New question"] --> RV["Query all 3 tiers in parallel<br/>also using errgroup"]
    RV --> RM["Redis<br/>working memory, TTL per session"]
    RV --> GM["Graph memory<br/>relationships between entities"]
    RV --> VM["Vector memory<br/>MongoDB Atlas Vector Search"]
    RM --> MRG["Merge in priority order"]
    GM --> MRG
    VM --> MRG
    MRG --> OUT["Context for the current ReAct loop"]

MCP client/server and SSE streaming

J.A.R.V.I.S. implements both an MCP (Model Context Protocol) client and a server, so external tools can be plugged in without touching the core runtime:

• MCP server: exposes J.A.R.V.I.S.’s internal tools (calendar queries, reading project files) to the outside through the standard, so other MCP clients (an MCP-capable IDE, for instance) can call them directly.
• MCP client: lets J.A.R.V.I.S. connect to third-party MCP servers (say, a server managing a company’s internal documents) without writing a custom integration for each system.

The result of every processing step is streamed back to the client over Server-Sent Events (SSE) through a Fastify gateway, keeping the time to the first visible token under 300ms. That matters: the “thinking” experience feels natural, instead of the user staring at a blank screen until the whole answer lands.

This modular design (multi-agent routing, parallel tool execution, tiered memory, and MCP as a standard interface) means adding a new agent or tool takes a few hours instead of changes to the core runtime. The rule I settled on from the very first draft: every extension must be additive. Never invasive.