Easy Guide: Installing and Running RagFlow Locally with DeepSeek - An End-to-End Tutorial

Introduction

RagFlow is an innovative, open-source RAG (Retrieval-Augmented Generation) engine developed by Infiniflow, available on GitHub. It leverages deep document understanding to extract knowledge from unstructured data and combines it with large language models (LLMs), specifically DeepSeek, to deliver grounded, citation-backed answers. RagFlow is designed to process complex documents, reduce hallucinations, and provide traceable responses, making it a powerful tool for AI-powered workflows.

In this post, we will guide you through the process of installing and running RagFlow locally using DeepSeek as the LLM model. We will utilize DeepSeek for chat interactions and Mistral for generating embeddings. This step-by-step tutorial will help you set up the engine, configure it, and begin using it effectively in your own environment.

Running Environment

Our evaluation of RagFlow was conducted on a Ubuntu virtual machine running via WSL (Windows Subsystem for Linux) on a Windows 11 PC. This setup provided a robust and flexible environment for testing RagFlow's capabilities, allowing us to simulate real-world deployment scenarios while leveraging the convenience of a Windows-based development ecosystem.

Installation Instructions

To get started with RagFlow, follow these steps:

1. Clone the Repository: Begin by cloning the RagFlow repository from GitHub using the following command:

   git clone https://github.com/infiniflow/ragflow.git
   

2. Set Up the Environment: Navigate to the cloned repository and install the required dependencies using the following command:

   pip install -r requirements.txt
   

For additional instructions, refer to the official RagFlow GitHub repository.

Running RagFlow in Containers

To simplify the deployment process, RagFlow can be run using Docker containers. The repository includes a docker-compose.yml file in the docker directory, which allows you to spin up all the necessary services with a single command. To start RagFlow in detached mode (running in the background), use the following command:

docker compose -f docker/docker-compose.yml up -d

This command will initialize the required containers and ensure that RagFlow is up and running, ready for evaluation or integration into your workflows. For detailed instructions, refer to the official RagFlow GitHub repository.

By default, RagFlow uses port 80 for HTTP serving. However, this port can be reconfigured to meet your specific deployment requirements by modifying the docker-compose.yml file or adjusting the relevant environment variables.

To confirm that RagFlow has started correctly, you can check the logs of the ragflow-server container using the following command:

docker logs -f ragflow-server

If RagFlow is running successfully, you should see output similar to the following:

* Running on all addresses (0.0.0.0)
* Running on http://127.0.0.1:9380
* Running on http://x.x.x.x:9380
INFO:werkzeug:Press CTRL+C to quit

This output indicates that RagFlow is actively serving requests on the specified addresses and ports.

Accessing RagFlow from Browser

To access RagFlow from your browser, enter the IP address of your server and log into RagFlow. In my case, since I use the default port 80 and run it in a WSL Ubuntu VM on my PC, I simply navigate to http://localhost.

If you are a first-time user, you will need to sign up to create an account. The sign-up process is straightforward and only requires an email address, a nickname and a password. Once registered, you can log in and start exploring RagFlow's features.

Setting Up Chat and Embedding Models

In our evaluation, we will use deepseek-chat as the chat model and mistral-embed as the embedding model. To set up these models, follow the steps below:

1. Add LLM Providers:

   • Navigate to the Model providers section under user settings in the RagFlow interface.
   • Add the necessary API keys for DeepSeek and Mistral.
   • Save the configuration.
   

2. Set the Models:

   • Press System Model Settings.
   • Select deepseek-chat as the chat model and mistral-embed as the embedding model.
   • Save the settings to apply the changes.
   

Once the models are configured, RagFlow will be ready to leverage DeepSeek for chat interactions and Mistral for generating embeddings.

Creating a Knowledge Base and Adding Documents

In this evaluation, we created a new Knowledge Base called AI-papers and added 20 PDF papers retrieved from arXiv by searching for the keyword DeepSeek. Here’s how we did it:

1. Creating the Knowledge Base:

   • Navigate to the Knowledge Base section in the RagFlow interface.
   • Create a new Knowledge Base named AI-papers.
   • Save the configuration.
   

2. Keeping Default Mode for Document Chunking and Embedding model:

   • We retained the default chunking method for processing the documents. Also for embedding model we kept the default model configured before mistral-embed.
   

3. Adding PDF Documents to the Knowledge Base:

   • Upload the 20 PDF papers to the AI-papers Knowledge Base.
   • Ensure all documents are properly selected for processing.
   

4. Parsing All Documents as a Bulk Operation:

   • Initiate the parsing process for all uploaded documents in bulk.
   • The total time for parsing all 20 documents was approximately 35 minutes.
   

Once the parsing is complete, the Knowledge Base will be ready for querying, enabling RagFlow to retrieve and generate responses based on the content of the uploaded papers.

Creating a New Chat Assistant

To create a new Chat Assistant called AI-papers Assistant that utilizes the AI-papers Knowledge Base, follow these steps:

1. Navigate to Chat in the RagFlow interface

2. Create a New Assistant:

   • Select the option to create a new Assistant.
   • Name it AI-papers Assistant.
   • Choose the AI-papers Knowledge Base from the available options.
   

3. Configure Prompt Engine and Model Settings:

   • Keep the default values for the Prompt Engine and Model Settings tabs.
   • Under the model configuration, ensure that the deepseek-chat model is selected as configured previously.
   

4. Save the Configuration:

   • Save the settings to finalize the creation of the AI-papers Assistant.

After completing these steps, the AI-papers Assistant will be ready to assist users by utilizing the AI-papers Knowledge Base, delivering relevant responses based on the uploaded documents.

Example Query to AI-papers Assistant

One of the queries you can ask the AI-papers Assistant is: "What are the key innovations in the DeepSeek Reasoner model?" To start a chat using the AI-papers Assistant, follow these steps:

1. Navigate to the Chat Interface:

   • Go to the chat interface section in the RagFlow application.

2. Select the AI-papers Assistant:

   • Choose AI-papers Assistant from the list of available assistants.

3. Ask the Question:

   • Input the question in the chat interface.
   

4. Receive Responses:

   • The AI-papers Assistant will respond with information regarding the key innovations in the DeepSeek Reasoner model, including references to the source information used for the response.
   

Conclusion

In conclusion, RagFlow presents several compelling features that make it an attractive choice for users looking for an effective RAG solution:

1. Open Source Solution: As an open-source project, RagFlow allows users to access, modify, and contribute to its development freely.

2. Easy Installation: With Docker containers, setting up and running RagFlow locally is straightforward and efficient, simplifying the deployment process.

3. Adaptability: The platform is highly adaptable, supporting a wide selection of LLM providers for both chat and embedding models, catering to various user needs.

4. Intuitive User Interface: RagFlow's user interface is designed for ease of use, providing a seamless experience for managing knowledge bases and engaging in chat.

5. Inclusion of References: The inclusion of references in chat replies enhances the credibility of the information provided, making it easier for users to verify sources.

Overall, RagFlow stands out as a powerful and user-friendly tool for harnessing the capabilities of retrieval-augmented generation technology.

Enjoyed this announcement? Found it insightful? Feel free to leave a comment below to share your thoughts or ask questions. A GitHub account is required to join the discussion.