# AnythingLLM and ChromaDB Integration **Introduction** AnythingLLM is an AI-powered application that enables users to interact with documents using Large Language Models (LLMs). It supports Retrieval-Augmented Generation (RAG), document-based chat systems, vector database integration, and AI agents. This deployment follows the previously configured ChromaDB setup running on port 8000. In this implementation, AnythingLLM was deployed using Docker on a NeevCloud Ubuntu server and integrated with the existing ChromaDB instance as the vector database backend. The integration enables document embedding storage, semantic search, contextual retrieval, and AI-powered document interaction workflows. #### Phase 1 Deployment and configuration of ChromaDB as the vector database backend is complete you can follow this [ChromaDB](https://app.gitbook.com/o/5yGrSBNeMSj6H9BXuf0s/s/EC5NwtFshv6EATOemuUn/~/edit/~/changes/KIzh3eF0Y2x2oCZk7hJG/neevcloud-guide/neevcloud-knowledgebase/setting-up-chromadb-on-neevcloud) #### Phase 2 Deployment of AnythingLLM and integration with the existing ChromaDB instance to enable Retrieval-Augmented Generation (RAG), semantic document search, and AI-powered document interaction. This document focuses on configuring the AI application layer and integrating it with the previously deployed ChromaDB server. *** **System Requirements** * Ubuntu 22.04 Server * Minimum 8 GB RAM * Docker * Docker Compose * Public IP Address * ChromaDB running on Port 8000 *** **Server Information** | Component | Details | | ----------------- | ------------ | | Cloud Provider | NeevCloud | | Operating System | Ubuntu 22.04 | | AnythingLLM Port | 3001 | | ChromaDB Port | 8000 | | Deployment Method | Docker | *** **Step 1: Connect to Server**\ After establishing SSH connectivity with the server, the environment was prepared for software installation and deployment activities. ```bash ssh root@SERVER_IP ``` *** **Step 2: Install Docker** Update packages:\ Updating system packages ensures that all dependencies and security patches are up to date before deployment. ```bash sudo apt update ``` Install Docker:\ Docker was configured successfully to provide a containerized deployment environment for AnythingLLM. ```bash sudo apt install docker.io -y ``` Start Docker service: ```bash sudo systemctl start docker sudo systemctl enable docker ``` Verify installation: ```bash docker --version ``` *** **Step 3: Install Docker Compose**\ Docker Compose was installed to simplify multi-container deployment and configuration management. Download Docker Compose: ```bash sudo curl -L "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose ``` Make executable: ```bash sudo chmod +x /usr/local/bin/docker-compose ``` Verify installation: ```bash docker-compose --version ```
*** **Step 4: Create AnythingLLM Directory**\ The application directory structure was prepared for persistent storage and configuration management. ```bash mkdir -p ~/anythingllm cd ~/anythingllm mkdir storage ``` *** **Step 5: Create Docker Compose File**\ The deployment configuration file defines the container settings, port mappings, and storage volumes required for AnythingLLM. Create configuration file: ```bash nano docker-compose.yml ``` Add the following configuration: ```yaml services: anythingllm: image: mintplexlabs/anythingllm:latest container_name: anythingllm ports: - "3001:3001" environment: - STORAGE_DIR=/app/server/storage - SERVER_PORT=3001 volumes: - ./storage:/app/server/storage restart: unless-stopped ``` Save the file. *** **Step 6: Fix Folder Permissions** ```bash chmod -R 777 ~/anythingllm ``` This step prevents SQLite database permission errors inside the container. *** **Step 7: Start AnythingLLM Container**\ The AnythingLLM container was deployed successfully and initialized using the configured Docker environment. ```bash docker-compose up -d ```
it's may take some time (\~20 min) Check running containers: ```bash docker ps ``` Expected output: ``` anythingllm Up (healthy) ```
*** **Step 8: View Logs** ```bash docker logs anythingllm ``` Expected output: ``` Document processor app listening on port 8888 ``` This confirms the service started successfully. *** **Step 9: Configure Firewall** Allow port 3001: ```bash sudo ufw allow 3001/tcp ``` Reload firewall: ```bash sudo ufw reload ``` *** **Step 10: Access AnythingLLM Dashboar**d\ Successful dashboard accessibility confirmed that the application deployment was completed correctly. Open browser: ``` http://SERVER_IP:3001 ``` The AnythingLLM dashboard should appear successfully.
*** **Step 11: Configure ChromaDB Integration** After successfully accessing the AnythingLLM dashboard, the next phase involved integrating the previously deployed ChromaDB instance as the vector database backend. Navigate to: ``` Settings → Vector Database ``` Select: * Provider: Chroma Enter endpoint: ``` http://:8000 ``` Leave the following fields blank: * API Header * API Key Click: * Save The configuration should be saved successfully, establishing communication between AnythingLLM and the ChromaDB vector database service.
*** **Step 12: Upload Documents** After successful integration of ChromaDB, the next step involved uploading documents into AnythingLLM for vector processing and semantic retrieval operations. From the dashboard: * Click “Upload a Document” * Upload PDF, DOCX, or TXT files AnythingLLM automatically: * Splits documents into chunks * Generates embeddings * Stores vectors inside ChromaDB This workflow enables semantic search and Retrieval-Augmented Generation (RAG) capabilities for uploaded documents. *** **Step 13: Install Ollama (Optional Local LLM)**\ To enable local AI response generation without relying on external APIs, Ollama was installed as the local Large Language Model (LLM) provider. Install Ollama using: ```bash curl -fsSL https://ollama.com/install.sh | sh ``` Ollama provides support for running open-source LLMs locally and integrates directly with AnythingLLM for contextual AI-based responses.
Start service: ```bash sudo systemctl start ollama sudo systemctl enable ollama ``` Pull a model: ```bash ollama pull llama2 ``` Verify installation: ```bash ollama list ``` *** **Step 14: Configure Ollama in AnythingLLM** After installing Ollama successfully, the next phase involved configuring AnythingLLM to communicate with the local LLM service. Navigate to: ``` Settings → LLM ``` Enter the following configuration: | Field | Value | | -------- | ----------------------------------------------------------------------- | | Provider | Ollama | | URL | [http://host.docker.internal:11434](http://host.docker.internal:11434/) | | Model | llama2 | Save the configuration. This setup enables AnythingLLM to send contextual queries to the locally deployed Ollama LLM service for AI-generated responses. {% hint style="info" %} The URL allows the Docker container to communicate with the Ollama service running on the host machine. {% endhint %} *** **Step 15: Test the System** After completing the deployment and integration process, the final step involved validating the end-to-end functionality of the AI document assistant system. Upload a document and enter the following query: ``` Explain the uploaded document. ``` During this process, the following workflow is executed: 1. AnythingLLM retrieves relevant vector embeddings from ChromaDB. 2. Relevant contextual information is extracted based on semantic similarity. 3. The retrieved context is sent to the configured LLM through Ollama. 4. The LLM generates an AI-powered contextual response for the user.
*** **Use Cases of AnythingLLM** AnythingLLM can be used for: * AI document assistants * Enterprise knowledge bases * RAG applications * Internal AI chatbots * Semantic document search * AI-powered support systems *** **Conclusion** AnythingLLM was successfully deployed on a NeevCloud Ubuntu server using Docker and integrated with ChromaDB as the vector database backend. The deployment supports: * Document uploads * Semantic search * Retrieval-Augmented Generation workflows * AI-powered contextual querying * Optional local LLM integration using Ollama This deployment was implemented on a NeevCloud CPU Cloud environment using open-source AI infrastructure components. --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.neevcloud.com/neevcloud-guide/neevcloud-knowledgebase/anythingllm-and-chromadb-integration.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.