Chunking is not a neutral step

Before it is indexed, a document is split into passages. This step long passed for a simple preliminary, settled by a fixed length and an overlap picked by eye. Yet it decides what the system will be able to retrieve: too short a passage loses the context that made its content interpretable, too long a one dilutes the relevant information among off-topic text, and a badly placed cut separates a statement from the condition that qualified it. None of these losses can be recovered downstream, since the passage is the unit retrieval operates on.

The team at Wrocław University of Science and Technology published in 2026 the first systematic evaluation of chunking methods, compared across several data types and scenarios. Its value lies as much in its method as in its results: the authors report each method's computational cost against its gain, where papers proposing a new chunking strategy generally compare it to fixed-length chunking alone, on their own dataset, without pricing it. The overall conclusion is a measured one: several sophisticated methods do not hold up outside the setting they were presented in.

We take a rule of conduct from it rather than a method to implement. DeepView chunks along the structure recovered during layout analysis rather than by fixed length, which this evaluation supports for structured documents. Above all it guards us against a common failing: adopting the latest published strategy because it posts better numbers on an academic dataset. Chunking is chosen against the corpus, and the question reopens with every deployment whose documentation does not resemble the last.