Evidence observability for RAG systems

See what your RAG system actually saw.

SourceMapR traces every answer back to the exact document evidence that produced it — and shows how that evidence was created (parse → chunk → embed → retrieve → answer).

Get started View workflow

Frameworks:

LlamaIndex

LangChain

LLM API:

OpenAI

Answer → Evidence PDF overlays Chunk inspection LLM tracking

SourceMapR Dashboard

Question

What is the attention mechanism in transformers?

Evidence used

3 chunks

attention_paper.pdf Page 5

chunk #12 · score: 0.89

attention_paper.pdf Page 3

chunk #7 · score: 0.84

llama2_paper.pdf Page 8

chunk #31 · score: 0.76

LLM Call

gpt-4o-mini

• Prompt tokens: 1,247
• Response tokens: 312
• Latency: 1.2s

Your app stays the same — SourceMapR shows the evidence lineage.

Supported Frameworks

Drop-in instrumentation for the most popular RAG frameworks

LlamaIndex

Full pipeline instrumentation

Supported

SimpleDirectoryReader (document loading)

SentenceSplitter & NodeParser (chunking)

VectorStoreIndex (indexing)

Query callbacks (retrieval + LLM)

HuggingFace embeddings

LangChain

Callback-based tracing

Supported

PyPDFLoader, DirectoryLoader, TextLoader

RecursiveCharacterTextSplitter

VectorStore retrievers

LLM & ChatModel callbacks

Token usage tracking

LLM Providers

Track token usage, latency, and costs

OpenAI

Supported

Anthropic

Coming soon

Google

Coming soon

More frameworks coming soon: Haystack · Semantic Kernel · DSPy

How It Works

SourceMapR traces every step of your RAG pipeline

Load Documents

Parse Extract text

Chunk Split text

Embed Vectorize

Retrieve Find similar

Generate LLM answer

Full Evidence Lineage

For every answer, SourceMapR shows you: which chunks were retrieved, their similarity scores, where they came from in the original document (with PDF highlighting), what prompt was sent to the LLM, and how many tokens were used. Debug your RAG pipeline without guessing.

Features

Answer → Evidence mapping

Trace every response to the exact chunks that were retrieved. See similarity scores and rankings.

PDF viewer with highlights

Click any chunk to see it highlighted in the original PDF. Split view shows chunk and source side by side.

Full LLM call capture

See the exact prompt sent to the model, the response, token counts, and latency for every query.

Experiment tracking

Organize runs into experiments. Compare "chunk-size-256" vs "chunk-size-512" side by side.

Two lines of code

Add SourceMapR to your existing pipeline. Your code stays the same — we just watch.

# pip install sourcemapr llama-index

from sourcemapr import init_tracing, stop_tracing
init_tracing(endpoint="http://localhost:5000")

# Your existing LlamaIndex code — unchanged
from llama_index.core import SimpleDirectoryReader, VectorStoreIndex

documents = SimpleDirectoryReader("./papers").load_data()
index = VectorStoreIndex.from_documents(documents)

response = index.as_query_engine().query("What is attention?")
print(response)

stop_tracing()

# pip install sourcemapr langchain langchain-community

from sourcemapr import init_tracing, stop_tracing
init_tracing(endpoint="http://localhost:5000")

# Your existing LangChain code — unchanged
from langchain_community.document_loaders import PyPDFLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_community.vectorstores import FAISS

loader = PyPDFLoader("./papers/attention.pdf")
documents = loader.load()

splitter = RecursiveCharacterTextSplitter(chunk_size=512)
chunks = splitter.split_documents(documents)

vectorstore = FAISS.from_documents(chunks, embeddings)
results = vectorstore.similarity_search("What is attention?")

stop_tracing()

Then open http://localhost:5000 to see the full evidence lineage.

Open source first

Built for developers who are tired of print-debugging RAG pipelines. Run locally. Own your data. See your evidence.

MIT License Local-first SQLite storage Zero config

Get started in 60 seconds

$ git clone https://github.com/kamathhrishi/sourcemapr.git

$ cd sourcemapr && pip install -e .

$ sourcemapr server

Server running at http://localhost:5000

View on GitHub Read docs