LongRAG A Dual Perspective Retrieval Augmented Generation Paradigm for Long Context Question Answering

The paper introduces LongRAG, a novel approach designed to enhance the performance of Retrieval Augmented Generation (RAG) systems in Long Context Question Answering (LCQA). LCQA involves reasoning over lengthy documents to provide accurate answers, a task where existing Large Language Models (LLMs) often struggle due to the "lost in the middle" issue. LongRAG aims to address this by improving the understanding of both global information and factual details within long contexts.

Introduction

The paper introduces LongRAG, a novel approach designed to enhance the performance of Retrieval Augmented Generation (RAG) systems in Long Context Question Answering (LCQA). LCQA involves reasoning over lengthy documents to provide accurate answers, a task where existing Large Language Models (LLMs) often struggle due to the "lost in the middle" issue. LongRAG aims to address this by improving the understanding of both global information and factual details within long contexts.

Dual Perspective Approach

LongRAG employs a dual perspective strategy to enhance the understanding of complex long context knowledge. This includes both global information and specific factual details, addressing the limitations of existing RAG systems.

Plug and Play Components

The system is designed with four plug and play components a hybrid retriever, an LLM augmented information extractor, a CoT guided filter, and an LLM augmented generator. These components work together to refine global information and contextual structure among chunks and improve evidence density.

Superior Performance

Extensive experiments on three multi hop datasets demonstrate that LongRAG significantly outperforms long context LLMs (by 6.94%), advanced RAG systems (by 6.16%), and Vanilla RAG (by 17.25%).

Automated Fine Tuning Pipeline

The paper introduces a novel automated instruction data pipeline for constructing high quality datasets for fine tuning. This pipeline enhances the system's "instruction following" capabilities and facilitates its transferability to other domains.

Conclusion

LongRAG represents a significant advancement in the field of LCQA by addressing the limitations of current RAG systems. Its dual perspective approach and plug and play components enable it to effectively mine global information and identify factual details, leading to superior performance. The automated fine tuning pipeline further enhances its robustness and transferability, making it a versatile tool for various domains.

Source(s):

• LongRAG A Dual Perspective Retrieval Augmented Generation Paradigm for Long Context Question Answering