> ## Documentation Index
> Fetch the complete documentation index at: https://docs.hiperai.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Knowledge Bases & RAG

# Knowledge Bases & RAG System

SecureAI uses **Retrieval-Augmented Generation (RAG)** to connect AI models with your private data and documents, ensuring secure, accurate, and context-aware responses.

***

## What is RAG?

**Retrieval-Augmented Generation (RAG)** is an AI technique that combines language models with external knowledge sources. Instead of relying solely on the model's pre-trained data:

1. **Query Processing**: The system analyzes the user prompt to identify relevant knowledge requirements.
2. **Knowledge Retrieval**: Semantic search retrieves relevant document chunks from your index.
3. **Context Augmentation**: Retrieved context is injected into the model prompt.
4. **Response Generation**: The AI generates a factual, grounded response citing your documents.

***

## Index Types in SecureAI

Indexes are structured knowledge bases categorized by accessibility:

| Index Type           | Scope               | Access & Privacy                                                                | Best For                                                       |
| -------------------- | ------------------- | ------------------------------------------------------------------------------- | -------------------------------------------------------------- |
| **Personal Indexes** | Individual user     | End-to-end encrypted; zero-knowledge storage (even admins cannot read contents) | Personal notes, private research, individual project documents |
| **Group Indexes**    | Teams / Departments | Role-based access controlled by group membership                                | Department wikis, team project documentation, shared SOPs      |
| **Global Indexes**   | Organization-wide   | Read access for all company users                                               | Company policies, HR guidelines, company-wide manuals          |

### 1. Personal Indexes

* **Zero-Knowledge Privacy**: Your files and conversations are encrypted with tenant/user keys.
* **Creation**:
  * **Users**: Open the chat interface, click the index selector next to the model picker, switch to the *Personal* tab, and click **+**.
  * **Admins**: Go to **Admin ? Index Management ? Create Index** and assign it to a specific user.

### 2. Group Indexes

* **Collaboration**: Multiple team members can query and contribute documents to the same knowledge base.
* **Creation**:
  * Go to **Admin ? Group Management ? Create Group**, assign members, and link a shared Group Index.

### 3. Global Indexes

* **Company-wide Standards**: Centralized repositories for enterprise compliance, legal terms, and operational guidelines.
* **Creation**: Go to **Admin ? Index Management ? Create Index** and set scope to **Global**.

***

## How Indexing Works

When you upload files (PDFs, Word docs, CSV, JSON, Markdown, or plain text):

`Raw Document  --?  Text Extraction  --?  Semantic Chunking  --?  Vector Embeddings  --?  Vector Storage (Index)`

1. **Document Processing**: Text and metadata (timestamps, file names) are extracted and cleaned.
2. **Chunking**: Content is split into optimized semantic chunks with overlap to maintain contextual continuity.
3. **Embedding Generation**: Chunks are converted into high-dimensional vector embeddings.
4. **Vector Storage**: Embeddings are stored in high-performance vector databases for instant similarity retrieval.

***

## Best Practices

* **Naming**: Use clear, descriptive names for indexes and uploaded files (e.g., HR-Policy-2026 instead of doc1).
* **File Preparation**: Ensure uploaded documents are clear text; clean up low-quality scans before uploading.
* **File Limits**: Standard uploads support files up to 10MB per batch.
* **Scope Appropriately**: Keep sensitive personal notes in Personal Indexes; move team resources into Group Indexes.