Retrieval-Augmented Generation (RAG) systems address critical limitations of large language models (LLMs) such as hallucinations, static knowledge, and context constraints-by dynamically integrating organizational data to enhance accuracy and relevance in knowledge management (KM). This RAG LLM development integrates the generative strengths of large language models with the precision of real-time information retrieval, enabling responses grounded in both structured and unstructured organizational data. The development process involves data collection, preprocessing, embedding, and indexing, followed by retrieval and prompt augmentation, ensuring that the model can access and utilize up-to-date, domain-specific knowledge at response time. By leveraging authoritative internal sources and advanced vector search, the RAG system overcomes the static knowledge and hallucination limitations of traditional LLMs, delivering more accurate, contextually relevant, and transparent answers. This approach not only enhances the reliability of generative AI in business scenarios but also offers a scalable, low-code framework adaptable to diverse enterprise needs. Our research employs low code approach for RAG system development and open source LLM.
Research results demonstrate how a low-code RAG architecture, leveraging open-source LLMs and real-time data retrieval, transforms KM through three key contributions: (i) enhanced accuracy and operational efficiency, (ii) cost-effective scalability and customization, (iii) managerial and strategic impact. The RAG system reduced hallucination rates compared to standalone LLMs, achieving higher accuracy in various tasks.
Future work aims to integrate multimodal data and hybrid architectures, further advancing collaborative human-AI knowledge ecosystems. Addressing these challenges involves robust system architecture, scalable data pipelines, advanced retrieval and ranking techniques, and strong governance over data quality and security.