The RET-RAG pilot indexed the last few issues of Shell’s TechXplorer Digest, and the article reports that the semantic search reliably brought up relevant context for most test questions. The archive the tool is meant to serve consists of more than 50 volumes of the magazine’s internal and external editions, documenting Shell’s scientific research and technological developments. Text is split into chunks of around 256 tokens, and each query retrieves the top five most relevant chunks.
The magazine had a basic keyword search that can retrieve relevant article PDFs but cannot extract specific information from them. An internship project, in collaboration with Energy Transition Campus Amsterdam partner Huracán Technical Advisory, adapted Huracán’s commercially available semantic search tool, which combines retrieval-augmented generation with requirement extraction tools in a technique called RET-RAG, to TechXplorer Digest content. The pilot was developed jointly by Shell and Huracán as a proof of concept to validate the utility of such an AI tool for Shell’s internal use, with success defined as accurate retrieval of information without losing key points.
The system follows a four-stage pipeline: document ingestion, embedding and storage, query handling and an LLM interface. A backend RET parser extracts text and metadata such as headings, sections, page numbers and figure captions from uploaded PDFs or Word files; for the pilot, article name and year of publication were added to the metadata. Each chunk is converted into a 384-dimensional vector by the GTE-large sentence transformer model and stored in the open-source ChromaDB vector database, and the original PDFs are registered in the knowledge-base collection of the self-hosted AI platform Open WebUI. A user’s question is embedded with the same model and compared with the stored chunks by cosine similarity, with k = 5 by default.
The retrieved chunks are combined with the question into an augmented prompt for a local LLM. The team tested Gemma 3B and 27B, served via the Ollama runtime, and Qwen-7B, all hosted on Huracán’s servers with no external API calls. Containerised Python back-end services run with Docker, so the system is self-contained and Shell’s data remain private. A post-processing step inserts inline citations that link to the TechXplorer Digest issue and page, and Shell engineers use the tool through a chat interface in Open WebUI.
The article concludes that the pilot validated the RAG architecture for this use case and laid the groundwork for future scaling and integration.