Build a local, dependency-free sentence-window retrieval prototype, understand why precise retrieval needs surrounding context, and evaluate the evidence before connecting the pattern to a production RAG stack.
What this tutorial covers
Retrieval-augmented generation, usually shortened to RAG, gives an answer system external text to consult at query time. A common implementation choice is to split every document into fixed-size chunks and retrieve the chunks most related to a question. That approach is useful, but it creates a persistent design trade-off. Small chunks can make retrieval precise while removing definitions, conditions, and exceptions. Large chunks can restore context while adding irrelevant material to the prompt.
Sentence-window retrieval addresses that trade-off by separating the unit used for retrieval from the unit used for interpretation. The system indexes individual sentences. When a sentence is selected, the system expands it into a local window containing nearby sentences from the same document. The retrieval signal remains precise, while the reader receives a fuller passage.
This tutorial intentionally uses only the Python standard library. The verified context does not establish current APIs, package versions, model availability, or persistence behavior for a particular RAG framework or model provider. A local prototype is therefore the most accurate way to demonstrate the technique without presenting unverified architecture as fact. Once the behavior is understood and tested, map the same concepts to the components that your organization has independently verified.
Why local context matters in RAG
A sentence often contains the words that best match a user question but not the complete meaning. Consider a policy passage with a rule, an exception, and a deadline. A query may match the sentence containing the deadline, while the preceding sentence says the policy applies only to a particular role. Returning the deadline alone can create a misleading answer.
The verified research context identifies a related problem in conventional RAG: retrieving too much information can create token-limit pressure and the “lost in the middle” problem, where relevant details become less useful among excessive context. The same research proposes retrieving chunks at multiple abstraction levels, including multi-sentence, paragraph, section, and document levels. In its Glycoscience-paper evaluation, that approach improved AI-evaluated question-answer correctness by 25.739% compared with a traditional single-level approach. This is a research result for that evaluation, not a promise that every corpus or sentence-window configuration will improve by the same amount.
A sentence window is one practical multi-sentence context pattern. It is especially appropriate when facts and their qualifications are usually located near each other. It is less suitable when the evidence required to answer a question is dispersed across distant sections or multiple documents. In those cases, a system may need broader retrieval, additional abstraction levels, or a document structure designed for the task.
Prerequisites
- Python 3.10 or later.
- A terminal capable of running Python commands.
- A small set of trusted UTF-8 plain-text or Markdown documents.
- Familiarity with basic command-line navigation and Python files.
This prototype does not call a model API. It retrieves evidence and prints the selected context windows. That boundary is deliberate: it lets you inspect whether retrieval has selected adequate evidence before introducing answer generation.
Step 1: Create a small corpus with rules and exceptions
Create a project directory and two short Markdown documents. The sample corpus is fictional. Its purpose is to make it easy to see why a sentence match alone may not carry enough context.
mkdir sentence-window-rag
cd sentence-window-rag
mkdir data
cat > data/travel_policy.md <<'EOF'
# Travel Policy
Employees must use the approved travel portal when inventory is available. Economy class is required for flights shorter than six hours. Premium economy may be booked for flights of six hours or longer.
Business class requires written approval from a vice president before booking. A manager approval is not sufficient. The approval email must be attached to the expense report.
EOF
cat > data/security_policy.md <<'EOF'
# Security Policy
Privileged production access requires multi-factor authentication and an approved access request. Shared user accounts are prohibited. Temporary production access expires automatically after eight hours unless an incident commander extends it during an active incident.
Employees must report suspected security incidents immediately through the incident portal. If the portal is unavailable, employees must contact the on-call security engineer.
EOF
Keep documents that you index within the authorization boundary of the intended users. This local example has no authentication, filtering, or remote service. Do not treat it as a ready-made system for confidential documents.
Step 2: Build a sentence index and local context windows
Create sentence_window_rag.py. The script reads Markdown and text files, separates text into simple sentence-like units, calculates a transparent lexical relevance score, and expands every result into neighboring sentences from the same source file. It is a learning implementation, not a linguistic sentence parser or a semantic-vector retrieval engine.
from __future__ import annotationsimport argparse
import math
import re
from collections import Counter
from dataclasses import dataclass
from pathlib import PathTOKEN_PATTERN = re.compile(r"[a-z0-9]+")
SENTENCE_PATTERN = re.compile(r"(?<=[.!?])\s+")@dataclass(frozen=True)
class SentenceRecord:
source_file: str
position: int
text: strdef tokenize(text: str) -> list[str]:
return TOKEN_PATTERN.findall(text.lower())def split_sentences(text: str) -> list[str]:
cleaned = re.sub(r"^#+\s+.*$", "", text, flags=re.MULTILINE)
cleaned = re.sub(r"\s+", " ", cleaned).strip()
if not cleaned:
return []
return [part.strip() for part in SENTENCE_PATTERN.split(cleaned) if part.strip()]def load_records(data_dir: Path) -> list[SentenceRecord]:
records: list[SentenceRecord] = []
for path in sorted(data_dir.rglob("*")):
...Continue Reading
Log in for free to read the rest of this article and access exclusive AI tools.
Log in / Register