Preserve Context at AI Document Chunk Boundaries

Breaking a document into pieces is a design decision about what evidence stays together. A chunk can contain the sentence a reader needs while omitting the heading, exception or version that makes the sentence meaningful. Before comparing retrieval tools or asking an AI assistant to answer from documents, decide which relationships each retrieved piece must preserve. This guide uses original fictional notes and manual expectations. It does not run a model, embedding system, search index or client retrieval pipeline.

The practical aim is a chunk contract that an implementer and a process owner can discuss. It should say how a piece identifies its source, what context remains attached, how overlaps are represented and what evidence a question requires. A universal chunk size cannot answer those questions. A successful text split is not automatically a successful retrieval, and a retrieved passage is not automatically a supported answer. Keep those stages separate when designing a useful test.

Begin with the questions the documents should answer

List several concrete questions before choosing a splitting strategy. Ask which source statements would be needed to answer each one and which qualifications would change the answer. A question about whether a desk accepts sealed kits may need one sentence and its heading. A question about open kits may need a nearby exception. A question about the current rule also needs the document version. These are different evidence requirements even when they concern the same short note.

The original teaching source is called Workshop Note V3. Its heading is Cedar Return Desk. Sentence S1 says that the Cedar desk accepts sealed kits on Tuesday. Sentence S2 says that open kits need review before return. Sentence S3 says that Maple desk rules are separate. These are invented statements about invented operations. They are not company policy, a real desk schedule or instructions for an actual return. Their limited purpose is to expose context relationships.

Write manual evidence expectations for the questions. For a question about sealed kits at Cedar, the heading and S1 identify the intended scope. For a question about open kits, S2 is indispensable in this exercise. S3 prevents the Cedar statement from being generalized to Maple. A test that rewards any mention of Tuesday would not distinguish those questions. Record which statements matter and why instead of treating all matching words as equivalent evidence.

Case A: a heading can carry the missing subject

The first fictional candidate contains a sentence equivalent to accepts sealed kits on Tuesday but omits the heading and subject. The authored expectation is that the desk identity remains unresolved in that candidate. The restored candidate pairs the Cedar Return Desk heading with S1. This is a static comparison of written examples, not a measured failure of a named chunking library or an observed answer from an AI system.

A heading can establish which product, location, team or document section a statement concerns. Removing it may leave grammatically readable text whose scope is unclear. Adding every heading in the whole document is not the only possible solution. A source path, verified section title or another explicit context field may carry the needed relationship. The important check is whether the supplied representation identifies the subject required by the actual question.

Do not fill the missing subject from whichever nearby business name looks most plausible. If a candidate lacks the identity evidence the question requires, mark that limitation or obtain a better candidate. A generated context sentence is another assertion to verify, not a substitute for the source relationship. If it identifies the wrong desk, it can make the fragment sound clearer while making the evidence less faithful.

Case B: keep an exception with the question it changes

The second example supplies only S1, the statement about sealed kits, for a question about open kits. The manual expectation requires S2 and leaves the open-kit answer unsupported by S1 alone. This does not mean every chunk must contain every exception in an entire knowledge base. It means the evidence supplied for this particular question must include the qualification that controls it. The fixture identifies that relationship explicitly.

A useful boundary review looks at the text immediately before and after a proposed split. Ask whether a heading, definition, exception or continuation crosses it. If it does, choose whether to preserve a larger unit, attach a verified context field or allow a later retrieval step to obtain the related unit. Those are design alternatives to test. This article does not declare one alternative universally superior or promise that a particular overlap length solves every boundary problem.

Counterexamples make the review more concrete. A test asking only about sealed kits could pass even when the open-kit qualification has been lost. Add a question whose answer changes when the exception is absent. Then a person can inspect whether the selected evidence contains the required sentence. That is an evidence-selection check. It remains separate from measuring how a real model interprets the sentence once it has been supplied.

Case C: overlapping text is not independent corroboration

The third fictional arrangement has one chunk containing S1 and S2 and another containing S2 and S3. Together they contain four sentence occurrences but only three unique sentence identifiers. S2 appears twice because the chunks overlap. It is not two independent observations or two separate sources confirming the same policy. A result that reports two matching passages needs to preserve that relationship before someone interprets the count as strength of evidence.

Keep both the chunk identifier and the underlying source reference. Overlap can be useful for preserving context, but it changes how retrieved material should be counted and displayed. If the same passage appears in several results, a summary may need to identify it as repeated source text. Do not silently delete repeated material when order or boundary context matters, but do not promote it into independent support either. The intended use determines the appropriate presentation.

The fixture checks four occurrences, three unique identifiers and two occurrences of S2. Those are authored representation counts, not retrieval precision or model accuracy. They do not establish whether an actual index duplicates records or whether a real search result should be deduplicated. A production design needs its own source keys and tests. The example simply makes one common source relationship visible before a count is given a larger meaning.

Case D: a chunk label needs a document and version

The fourth fictional case contains a chunk labeled C1 from Workshop Note V2 and another chunk labeled C1 from Workshop Note V3. Equal chunk labels do not make them the same evidence. The full identity in this exercise includes document, version and chunk label. A comparison that joins only on C1 would collapse two distinct references. This is an original identity example, not an observation of a real document store.

Version handling should be explicit in the retrieval task. A question may ask for the current source, a historical comparison or a summary of a supplied snapshot. Those are different scopes. Do not assume the latest-looking date is authoritative without a source-selection rule. If the project has an approved version policy, preserve its evidence and owner. This guide invents no approval policy and does not decide which real document is current.

When a source is updated, consider how chunk identifiers are rebuilt and how older references remain interpretable. A changed split can move the same sentence into another chunk, while a stable label can point to changed content. Keep enough source context to locate the evidence used for an answer. The right technical implementation depends on the system, but the review question is stable: can another person tell which actual source and version the result used?

Distinguish source text from added context

Anthropic's primary contextual-retrieval article describes adding explanatory context to chunks before retrieval and discusses the problem of fragments losing their surrounding context. This guide uses that limited design motivation, without importing its reported performance results into these examples. The fictional cases here do not reproduce that method, benchmark a model or establish that contextual enrichment improves a particular business system.

If an implementation adds a context prefix, label it separately from verbatim source text. A reviewer should be able to see which words came from the document and which were generated or assembled by another step. The prefix may be helpful, but it can also introduce an incorrect subject, version or qualification. Verify important added assertions against the source. Do not quote a generated prefix as though it were an original document statement.

Keep provenance attached when moving between formats. A chunk displayed in a search result, copied into a prompt and cited in a response should retain an intelligible path back to the source. That does not require exposing private identifiers in a public answer; the actual project must choose an appropriate representation. It does require a way for authorized reviewers to inspect the relationship. Fluent citation text alone does not demonstrate that the relationship exists.

Case E: a budget is meaningful only under a declared measure

The fifth fictional case assigns size units of 1,200, 800 and 900 to three candidate pieces. The supplied limit is 2,500 units. Their sum is 2,900, which exceeds the limit by 400. These are invented budget units, not measured tokens from a tokenizer or a statement about any model's context window. The arithmetic checks the authored allocation only. It does not predict cost, latency or whether a real API request would be accepted.

Name the measure in an actual project. Characters, words, bytes and model-specific tokens are different quantities. A fixed number of characters does not establish an exact token count. If the limit applies to a real model request, account for all relevant request components using the actual system's rules and tooling. This guide supplies no tokenizer execution and no universal conversion factor. A numerical size field should not acquire a different unit in a summary.

If selected evidence exceeds a real budget, do not drop the controlling exception simply because it is inconvenient. Revisit the question, candidate selection or representation and then test the result. A shorter context can be insufficient, while a longer context can contain unrelated material. The practical decision concerns which necessary evidence is present under the chosen constraints. A size target by itself does not identify the best boundary or establish answer quality.

Case F: a reference can remain unresolved across a split

The final fictional candidate contains the sentence It must be reviewed first, with no supplied antecedent for it. The expected result is an unresolved referent. The sentence might be grammatical, but the reader cannot tell from this candidate what must be reviewed. Adding the first noun found in another unrelated result would invent a relationship. This example does not claim that a particular model would make that mistake; it defines a boundary the test should preserve.

For an actual document, inspect the surrounding source to see whether the referent is established. If the relevant context is available and appropriate to use, preserve it with the candidate or retrieve it through a named step. If it is not supplied, narrow the answer or identify the missing relationship. Do not ask an assistant to certify a fragment's meaning from an incomplete representation. The text can support only what its available context establishes.

Pronouns are only one form of dependency. Phrases such as the previous condition, the second option or the exception below can also point outside a fragment. A useful review sample includes examples of these references when they occur in the actual material. The six cases here are not exhaustive. They illustrate how to design tests around source relationships instead of assuming that every paragraph can be interpreted independently.

Build a small question-to-evidence test set

For each selected question, record the source version, necessary statements, acceptable scope and a reason an incomplete candidate would fail. Keep the manual expectation independent from the system output being evaluated. If a real retrieval system later returns a different passage, inspect whether it supplies the same necessary evidence rather than checking only exact text equality. That semantic review requires judgment and should not be mislabeled as a purely mechanical count.

Separate several outcomes in the notes. No candidate returned is different from a candidate returned without the exception. A complete candidate followed by an unsupported generated answer is a later-stage issue. A wrong source version is another issue. These distinctions make debugging useful: they indicate which stage needs investigation without inventing a cause. The local fixture checks on this page concern authored data consistency, not the behavior of any of those actual stages.

If the project uses a scoring rubric, state its denominator and unit of evaluation. A fraction of questions with necessary evidence is different from a fraction of matching chunks, and neither is automatically answer accuracy. Several overlapping chunks may satisfy one question, while one chunk may support several questions. Preserve that mapping before aggregating a result. This guide's counts are intentionally small and literal so their scope can be inspected directly.

Review a proposed change against the same questions

When changing a split rule, compare the candidates for the same question set and source version. A new strategy may preserve the exception in one case while creating ambiguity in another. Record those concrete differences rather than declaring that larger or smaller chunks are inherently better. A process owner can then see which question becomes easier to answer and which relationship remains unresolved. No before-and-after system performance is reported here.

Changing the source at the same time as the splitting strategy makes interpretation harder. If the document wording, version and chunk boundaries all change, a different result cannot be attributed to the split alone. Keep the comparison scope explicit and avoid causal claims from a mixed change. For a real evaluation, preserve actual inputs, configuration and outputs so later reviewers can inspect what changed. The fictional V2/V3 identity example is not such an evaluation.

A useful review also considers the display that people will read. If a search interface truncates the heading or hides the version, a backend representation that preserves them may not be enough for the human task. Check the actual handoff surface when it is part of the project. This article does not inspect a client interface or supply an interactive utility. It provides static cases that can inform a later, separately documented check.

Keep the final answer within the supplied evidence

An AI instruction can ask for answers only from the supplied source passages, with unresolved subjects or missing qualifications identified. It can request source references under the declared convention and distinguish source wording from added explanation. Those are original instruction ideas, not observed model outputs. A real response would still need review against the actual source and manual expectations. Prompt wording alone does not establish compliance.

Do not treat chunk design as an authorization or security boundary. Preserving headings and versions does not demonstrate retrieval permissions, protection from prompt injection or permission to disclose a document. Those concerns need their own design and verification in a real system. The narrow subject here is evidence context: whether a selected fragment carries the relationships required for a particular reading question. Its success does not settle other workflow requirements.

A completed chunk contract therefore names the source identity, necessary context, boundary-sensitive questions, overlap representation and size measure. It leaves actual performance to actual tests. The downloadable examples check the authored relationships described here locally and after download. They make no claim about a real model, customer dataset, embedding system, index, professional review or operating approval. Their value is to make a later evaluation ask a clear question.

Assign someone to maintain the question-to-evidence expectations when the source changes. A fixture can remain technically readable while its expected relationship is no longer appropriate for a revised document. The review owner should explain whether a changed result reflects a new source statement, a different boundary or a different task. Keeping those reasons separate makes later checks interpretable without pretending that a permanent test file can freeze a changing knowledge base.

Frequently asked questions

Is there one correct chunk size for every document?

No universal size is established here. The useful boundary depends on the questions, source structure and actual system constraints. A heading, exception or reference may need to remain connected even when a numerical target suggests a split. Test the necessary relationships for the intended task rather than treating a size value as a quality guarantee.

Do two overlapping results count as two independent sources?

Not merely because two chunks are displayed. In Case C, four sentence occurrences represent three unique sentence identifiers because S2 appears twice. Preserve the underlying source references before interpreting a result count as corroboration. This fixture does not decide a universal deduplication policy for actual retrieval systems.

Can an added context sentence be quoted as original source text?

Only actual source wording should be represented as source wording. A generated or assembled prefix is a separate assertion that needs verification. Keep it labeled so reviewers can distinguish it from the document. This guide describes the design distinction but generates no model context prefixes and reports no accuracy result for them.

Are the budget numbers actual model token counts?

No. The 1,200, 800 and 900 values and the 2,500 limit are invented units. Their sum exceeds the authored limit by 400. An actual model budget requires the real measure and tooling, including relevant request components. No tokenizer, pricing calculation, context-limit test or latency measurement was performed for this guide.

Does a correct chunk contract establish document permissions?

No. Source context, access control and operating authorization are different questions. This page does not test permissions, security or prompt-injection defenses. A real retrieval project needs appropriate separate checks for those concerns. The fictional cases only show how headings, exceptions, identities and references affect evidence for a reading task.

Were retrieval tools or AI models tested here?

No. The source sentences, candidate arrangements and expectations are original fictional teaching material. Local and downloaded checks establish their representation and arithmetic consistency. They are not observations of a model, embedding service, search index, client workflow or professional review, and they do not approve a production deployment.