Repository status
Code complete · 35/35 tests passing · Published on GitHub. Everything below is the real repository, verbatim — not a summary, not pseudocode. The sample knowledge base is a fictional dental clinic's front-desk manual, so nothing here is client data.
File tree
rag-ai-chatbot-knowledge-base/
├── README.md
├── LICENSE
├── requirements.txt # pytest only — the pipeline is dependency-free
├── src/rag/
│ ├── __init__.py
│ ├── chunker.py # overlapping word windows, citation metadata
│ ├── retriever.py # TF-IDF-style scoring, tiny stemmer, no deps
│ ├── composer.py # extractive answers, enforced refusal
│ ├── whatsapp.py # WhatsApp intake for the chat receptionist
│ ├── kb.py # loads the sample markdown manual
│ └── kb/ # 4 sample docs: hours, booking, cancellation, insurance
└── tests/
├── conftest.py
├── test_chunker.py
├── test_retriever.py
├── test_composer.py # incl. out-of-KB refusal, low scores, empty KB
└── test_whatsapp.py
The parts that matter
1 · The no-guess rule, enforced in code
This is the heart of the build — composer.py. If no passage clears the relevance bar, it returns the refusal. There is no path from here to an invented answer.
REFUSAL = (
"I couldn't find that in the front-desk manual, so I can't answer it from "
"our documents. Please ask the front-desk team directly and they'll help you."
)
def answer(question, retrieved, min_score=0.095, max_bullets=2):
"""Build a cited answer from scored passages, or refuse."""
usable = [r for r in retrieved if r["score"] >= min_score]
if not usable:
return {"answer": REFUSAL, "refused": True, "citations": []}
...
picked = (strong or whole)[:max_bullets]
if not picked:
return {"answer": REFUSAL, "refused": True, "citations": []}
Extractive on purpose: answers are assembled from verbatim sentences, each cited [doc:chunk]. A generative rewriter could paraphrase a fact into a falsehood; quoting cannot.
2 · The chunker: nothing falls through the cracks
chunker.py splits documents into overlapping windows, so a fact sitting on a boundary is never cut in half. Every chunk carries its document id and index for citations.
def chunk_text(doc_id, title, text, chunk_size=60, overlap=15):
words = text.split()
if not words:
return []
# Overlap can never swallow the window or loop forever on short texts.
overlap = max(0, min(overlap, chunk_size - 1, len(words) - 1))
step = chunk_size - overlap
chunks = []
i, idx = 0, 0
while i < len(words):
chunk_words = words[i : i + chunk_size]
chunks.append({"doc": doc_id, "title": title,
"chunk": idx, "text": " ".join(chunk_words)})
if i + chunk_size >= len(words):
break
i += step
idx += 1
return chunks
3 · The retriever: dependency-free on purpose
retriever.py tokenizes, drops stopwords, applies a tiny stemmer, and scores chunks by term-frequency × inverse-document-frequency. Swap this module for embeddings in production and the composer never notices.
def _stem(token):
"""Tiny stemmer: strips -ing/-ed and plural -s so 'booked' matches 'book'."""
t = token
if len(t) > 4:
for suffix in ("ing", "ed"):
if t.endswith(suffix):
t = t[: -len(suffix)]
break
if len(t) > 3 and t.endswith("s") and not t.endswith("ss"):
t = t[:-1]
return t
Get the full repo
Run it yourself: pip install -r requirements.txt && pytest — 35 tests, all passing.
Go deeper: the interactive demo — ask the manual and watch retrieval work — and the article on the repeated-question tax.
VROOM ANALYTICS