RAG — Retrieval-Augmented Generation — is often explained as "add a vector database in front of your LLM". In simple demos and general knowledge applications, that description is close enough to be useful. But in industrial domains, where documents are dense, interdependent, and version-controlled across decades, the gap between a demo and a production system is significant.
Technical information in these environments is not cleanly contained in single documents. It is spread across P&IDs, specifications, datasheets, inspection reports, vendor submissions, change logs, and engineering models — often with the same concept described differently in each. Some documents reference others. Some supersede others. Some are outdated but still in circulation, either because no one updated the index or because they remain contractually relevant for a specific project phase.
When you feed this kind of knowledge into a basic RAG pipeline and ask an engineer a question, what you get back depends almost entirely on how thoughtfully you designed the information architecture — not on how sophisticated the model is.
- *Why document structure matters more than embedding quality**
A common mistake in early RAG implementations is treating documents as monolithic blobs of text. A 200-page engineering specification gets chunked at fixed intervals, embedded, and indexed. When a user asks a question, the system retrieves the three most semantically similar chunks and passes them to the model.
This works reasonably well for general information retrieval. It fails for technical questions where the answer depends on the relationship between specific sections. If the question is "what is the calibration interval for instrument tag FT-1021?", the answer might be in Section 7.3 of the datasheet, which references a schedule in Appendix B, which was updated by a revision notice issued 18 months later.
A flat embedding index does not know that these things are related. It just sees chunks. The retrieval step will return whichever chunks happen to be most similar to the question string, and the model will synthesise something plausible from whatever lands in the context window.
This is why document structure — how documents are parsed, sectioned, and linked — is the real differentiator in industrial RAG. Parsing a technical PDF is not a text extraction problem. It is a knowledge modelling problem.
- *Metadata and identifiers as the connective tissue**
The second critical design decision is metadata. In industrial environments, good metadata is the difference between a knowledge base and a knowledge pile.
Every chunk, section, or concept node in a RAG index should carry structured metadata: the source document, the document type, the revision number, the date, the project phase, the discipline, and any explicit relationships to other documents or identifiers. Tag numbers, equipment IDs, and document codes are particularly important — they are the shared language that connects information across different systems and sources.
When a retrieval step can filter on metadata before doing semantic search, the quality of results improves dramatically. "Find information about FT-1021" becomes a structured query against a well-organised index, not a semantic search that might surface a different instrument with a similar name from a different project.
This is not a new idea in enterprise information management. What AI changes is that metadata-enriched retrieval can now be combined with generative synthesis — so the system not only finds the right information but assembles it into a useful, coherent response. The metadata does the retrieval; the model does the reasoning.
- *Handling conflicts and versioning**
One of the hardest problems in industrial RAG is deciding what to do when documents conflict. In a large EPC project, it is not unusual for a specification and a datasheet to carry different values for the same attribute. One was updated; the other was not. Or they were written by different teams with slightly different assumptions.
A general-purpose RAG system will retrieve both and pass them to the model, which will either pick one, average them, or produce a response that acknowledges the conflict without resolving it. For an engineer making a procurement decision or writing an inspection procedure, none of those outcomes are acceptable.
Good industrial RAG systems need to encode a hierarchy of authority. When a conflict exists, which document wins? Usually this is not ambiguous — there is a contractual or project-level answer. The system needs to know that answer and apply it consistently during retrieval.
This means the information architecture has to capture not just what documents say, but what authority they carry and what they supersede. Building this into the retrieval layer, rather than leaving it to the model to figure out at inference time, produces dramatically more reliable outputs.
- *Layers of confidence, not one answer**
The design pattern I find most useful for industrial RAG is thinking in layers of confidence rather than aiming for one definitive answer.
The first layer is what we know for certain — information that has a single authoritative source, is current, and has no conflicts. The system can return this with high confidence.
The second layer is what the system suggests — information that is likely correct based on multiple consistent sources, but where a human cross-check is warranted.
The third layer is what requires human confirmation — information where conflicts exist, sources are outdated, or the query touches something outside the system's reliable coverage.
Presenting answers this way does not make the system less useful. It makes it more trustworthy. Engineers are accustomed to knowing the provenance and confidence level of the information they use. A RAG system that communicates these things clearly fits their mental model. One that presents everything with equal confidence forces them to distrust everything equally — which defeats the purpose.
RAG is not just a technical pattern for connecting documents to models. In industrial domains, it is a way of encoding how an organisation understands, updates, and stands behind its own knowledge. Getting that right requires as much attention to information architecture as to model selection — and that is a product design challenge as much as an engineering one.