Build Log: An MCP Server for Internal Docs Search
· 10 min read

Build Log: An MCP Server for Internal Docs Search

A beginner-friendly guide to a Python MCP server that searches internal markdown with a local Chroma index, plus an optional local Ollama embeddings path.

What this guide covers

MCP (Model Context Protocol) is a standard that lets AI clients discover and call tools exposed by a server. This guide walks through a small Python MCP server that searches a folder of internal markdown documents using a local Chroma vector store, with optional fully local embeddings through Ollama. It assumes no prior MCP experience and describes a setup intended to run on your machine. MCP’s architecture guide and Python SDK documentation explain the client-server model and server tools.

This guide is assembled from official documentation and community reports — we did not run this build ourselves.

Build difficulty: 5/10 — the pieces are approachable, but indexing and server startup need to fit together, and stdio logging has an easy-to-miss constraint.

The idea and stack

An MCP server exposes tools an AI client can discover and call; here, one tool searches indexed markdown and returns snippets with their source paths. Python, Chroma, and optional Ollama make up the proposed stack. These choices favor a readable local starting point, not a measured claim about search quality or speed.

The official Python SDK is a natural fit for a small example: install it with uv add "mcp[cli]" or pip install "mcp[cli]", define a tool with @mcp.tool(), and use the SDK’s mcp dev inspector workflow. A TypeScript SDK is available too; Python is chosen here for readability, not because it is required. The Python SDK documents the decorator and development command, while the TypeScript SDK documents the alternative.

Chroma provides a local in-memory client, which this guide uses for simplicity. Its create_collection and get_or_create_collection methods create or retrieve a collection; collection.add embeds documents automatically when embeddings are not supplied. Chroma’s getting-started guide documents those operations.

Ollama is optional. Its embedding models let you keep embedding generation local; the Ollama blog shows embeddings being added to and queried from Chroma. The blog lists mxbai-embed-large, nomic-embed-text, and all-minilm; models may have newer versions, so treat that list as a starting point rather than a current-version guarantee. Ollama’s embedding-model guide has the examples and model details.

Step 1: Install the pieces

For the simplest version, install the Python MCP SDK and Chroma, then skip Ollama. Chroma supplies embeddings automatically when you add documents without providing embeddings. If you want local embeddings instead, install Ollama separately and pull a model. These are setup choices, not recommendations based on a benchmark.

uv add "mcp[cli]"
pip install chromadb

The SDK also documents pip install "mcp[cli]" as an alternative to the uv command. For Ollama, the documented example is:

ollama pull mxbai-embed-large

The Ollama blog also lists nomic-embed-text and all-minilm as model options. The model names and details above come from Ollama’s guide; check that source for its embedding example before adapting the optional path.

Step 2: Sketch the server tool

Start with a tool that accepts a search query and returns matching text. In the Python SDK, the @mcp.tool() decorator exposes a type-hinted function as a tool. Its docstring describes the tool, and its type hints contribute to the generated input schema. MCP clients discover tools through tools/list and invoke them through tools/call.

from mcp.server import MCPServer

mcp = MCPServer("docs-search")


@mcp.tool()
def search_docs(query: str, top_k: int = 5) -> list[str]:
    """Search the internal docs folder. Returns top matches as 'source path — snippet'."""
    ...

The ellipsis marks where the search logic will go; this is a skeleton, not a complete running server. The top_k default is an example parameter, not a claim that this is the best number of results. The SDK documents how a decorated function’s type hints become its schema, which clients receive with the tool definition. Read the Python SDK guide for its tool example.

Step 3: Index the markdown documents

Before searching, split the markdown into chunks, then add each chunk to a collection with a unique string ID and metadata containing its source path. This example splits at markdown headings. Chunking policy can vary; the available sources do not establish a best chunk size or method.

from pathlib import Path
import re

import chromadb

client = chromadb.Client()
collection = client.get_or_create_collection(name="internal-docs")


def markdown_chunks(text: str) -> list[str]:
    parts = re.split(r"(?m)(?=^#{1,6}\s)", text)
    return [part.strip() for part in parts if part.strip()]


def index_docs() -> None:
    for path in sorted(Path("docs").rglob("*.md")):
        chunks = markdown_chunks(path.read_text(encoding="utf-8"))
        ids = [f"{path}:{index}" for index, _ in enumerate(chunks)]
        metadatas = [{"source": str(path)} for _ in chunks]

        if chunks:
            collection.add(
                ids=ids,
                documents=chunks,
                metadatas=metadatas,
            )

The IDs combine a path and chunk position so each chunk has its own string ID. Chroma requires unique string IDs, and collection.add accepts IDs, documents, and metadata. When you omit embeddings, Chroma embeds the documents automatically. Chroma’s guide covers adding documents and collection setup.

One important detail: chromadb.Client() is in-memory. Keep indexing and searching in the same server process for this example, and run the indexing routine before serving searches. Running this indexing code in a separate process would not make its in-memory collection available to another process. Persistent storage is outside this guide because the fetched Chroma documentation does not detail its setup.

Step 4: Connect the tool to the collection

The tool can query Chroma with the user’s text and return each result alongside the source path saved during indexing. Pass the function’s top_k value as n_results so the query count is explicit. Chroma’s documented default is ten results when n_results is not provided; that default is not a quality recommendation.

@mcp.tool()
def search_docs(query: str, top_k: int = 5) -> list[str]:
    """Search the internal docs folder. Returns top matches as 'source path — snippet'."""
    matches = collection.query(
        query_texts=[query],
        n_results=top_k,
    )

    documents = matches["documents"][0]
    metadatas = matches["metadatas"][0]

    return [
        f"{metadata['source']} — {document}"
        for metadata, document in zip(metadatas, documents)
    ]

This version uses Chroma’s automatic embedding path: query_texts accepts the search text, and Chroma handles embedding. With Ollama, the alternative is to create query embeddings using ollama.embed(model=..., input=...), then pass them to collection.query(query_embeddings=..., n_results=...). The Ollama example also supplies embeddings when adding documents. Use the same embedding approach for indexed documents and queries; refer to Ollama’s example for its Chroma workflow.

Step 5: Test and connect

Use the Python SDK’s development command to open the MCP Inspector and explore the server’s tools. The Inspector is also available through npx @modelcontextprotocol/inspector, with command-line and terminal UI modes. Once the server is ready, Claude Code can connect to a local stdio server using the command below.

uv run mcp dev server.py
claude mcp add --transport stdio docs-search -- uv run mcp run server.py

The first command opens the Inspector for development; the MCP Inspector project documents its web, CLI, and TUI options. The second follows the stdio form in Claude Code’s MCP documentation. If you change docs-search or the server command, adjust those arguments to match your setup.

For multiple clients, use Streamable HTTP rather than stdio. The Python SDK documents running the server with uv run mcp run server.py --transport streamable-http, with http://localhost:8000/mcp as the example endpoint. Claude Code documents claude mcp add --transport http <name> <url> for HTTP connections. Its documentation marks SSE as deprecated. Compare MCP tools and approaches if you are weighing other options.

What to pick

Choose based on how you want the server to connect, where embeddings should run, and whether resetting the index is acceptable. The documented starting point is stdio with Chroma’s automatic embeddings and an in-memory collection. Streamable HTTP and Ollama offer alternatives, but neither changes the need to index the documents you want searched.

DecisionOptions
Transportstdio — the simplest local setup; typically serves one client and must never write to stdout. Streamable HTTP — typically serves many clients; the SDK’s example uses uv run mcp run server.py --transport streamable-http and http://localhost:8000/mcp.
EmbeddingsChroma default — no embedding setup required; Chroma embeds automatically. Ollama local — embeddings can be generated locally. The Ollama blog lists mxbai-embed-large (334M), nomic-embed-text (137M), and all-minilm (23M).
StorageIn-memory — chromadb.Client() resets between runs, so re-index when starting the server. Persistent — not detailed in the fetched Chroma documentation, so it is out of scope here.

The transport distinctions come from MCP’s architecture documentation: local stdio servers typically serve one MCP client, while remote Streamable HTTP servers typically serve many. HTTP servers may log to stdout, but stdio servers must not. Browse MCP server tools for related options.

Tradeoffs to keep in view

This guide describes a configuration, not a measured result. We did not benchmark retrieval quality, latency, index size, or cost, and the sources do not establish a best chunking policy or result count. The example uses an in-memory collection because the fetched Chroma page does not explain persistent setup. For stdio, protect stdout from logs.

In particular, a print statement on stdout can corrupt the JSON-RPC messages exchanged over stdio. Send logs to stderr or a file instead; MCP’s documentation also recommends stderr logging for stdio or OpenTelemetry. The MCP server guide explains the logging rule and distinguishes it from HTTP servers, which may log to stdout.

What can break / limits

The documented constraints point to a few things to plan for: stdout logging can break stdio communication, an index can become stale after its source documents change, and this guide does not cover access control for internal documents. These are limitations of the documented setup, not issues observed in a hands-on build.

  • Stdout logging: On stdio, writing logs to stdout can corrupt JSON-RPC. Use stderr or a file.
  • Stale index: Editing source documents does not update an existing snapshot automatically in this example. Re-run indexing after edits. The fetched Chroma documentation mentions upsert as a way to avoid re-adding the same documents, but it does not cover a full update strategy.
  • Access control: The sources do not explain how to restrict which users or clients can search which documents. Do not treat this guide as a complete access-control design.
  • No performance claims: There are no benchmarks here, and no claims about search speed, accuracy, or cost.

One adjacent operational detail in Claude Code’s documentation is that, on macOS and Linux, disconnecting a stdio server sends SIGINT, then SIGTERM, then SIGKILL if the process is still running. These details do not provide access control for the indexed docs; see Claude Code’s MCP documentation and the Inspector project.

FAQ

The short answers: Ollama is optional, stdio logs belong on stderr rather than stdout, and stdio is typically for one client. For multiple clients, the documented alternative is Streamable HTTP. These answers describe the tools and transports covered here; they do not imply a benchmark or a complete security configuration.

Do I need Ollama?

No. Chroma embeds documents automatically when you add them without supplied embeddings. Ollama is an optional route for generating embeddings locally; the Ollama blog demonstrates using its embeddings with Chroma. Choose it when that local embedding workflow suits your setup, not because this guide establishes a quality advantage.

Where do my logs go?

For stdio, do not write logs to stdout: that can corrupt JSON-RPC communication. Use stderr or a file. The MCP documentation says HTTP servers may log to stdout, but logging to stderr remains a clear option when you want to keep diagnostic output separate from responses.

Can more than one client connect?

A local stdio server typically serves a single MCP client. For multiple clients, use Streamable HTTP; Claude Code documents claude mcp add --transport http <name> <url> for connecting to an HTTP server. Its docs also say SSE is deprecated. See the transport documentation for the connection details.

📊 See how MCP tools compare → ToolBrain comparisons

  • ToolBrain — tool reviews, LLM comparisons, and AI workflow guides
  • Hermes Tutorials — Hermes Agent setup, configuration, and advanced workflows
  • CodeIntel Log — code quality, debugging, and software engineering benchmarks
  • NiteAgent — AI agent development, frameworks, and production patterns

Cross-links automatically generated from None.

Back to all posts