Llama 2

LLama

LLamaIndex Installation and Setup

LLamaIndex Installation and Setup

LlamaIndex, previously known as GPT Index, is a data framework designed for LLM applications, facilitating the intake, organization, and retrieval of specialized or private data. Here’s what LlamaIndex offers:

  • Data Connectors: These allow for the seamless import of your data, irrespective of its original source or format, be it APIs, PDFs, SQL, among others.
  • Data Indexes: They organize your data into formats that are both efficient and easily digestible for LLMs.
  • Engines: These grant natural language-based access to your data. To illustrate:
  • Query Engines: Robust retrieval systems that enhance knowledge outputs.
  • Chat Engines: Interactive platforms enabling dynamic, two-way conversations with your data.
  • Data Agents: These are LLM-driven knowledge operatives enhanced with a range of tools, from basic helper functions to comprehensive API integrations.
  • Application Integrations: These seamlessly weave LlamaIndex into your broader ecosystem, be it LangChain, Flask, Docker, ChatGPT, or any other platform.

Installation from Pip

You can simply do:

pip install llama-index

IMPORTANT: LlamaIndex might download and save files locally from different packages like NLTK, HuggingFace, etc. To specify the storage location for these files, set the “LLAMA_INDEX_CACHE_DIR” environment variable.

Installing LlamaIndex from Source

  1. First, clone the repository using Git:
git clone https://github.com/jerryjliu/llama_index.git
  1. For an editable installation (which allows you to modify source files) of only the package:
pip install -e .
  1. To install both optional dependencies and those needed for development (like unit testing):
pip install -r requirements.txt

OpenAI Environment Setup

By default, we use the OpenAI gpt-3.5-turbo model for text generation and text-embedding-ada-002 for retrieval and embeddings. In order to use this, you must have an OPENAI_API_KEY setup. You can register an API key by logging into OpenAI’s page and creating a new API token.

Setting Up LlamaIndex in a Local Environment

If you prefer not to use OpenAI, the system will default to LlamaCPP and llama2-chat-13B for text generation, and BAAI/bge-small-en for retrieval and embeddings. All these models operate locally.

Steps to Set Up LlamaCPP:

  1. Refer to the installation guide for LlamaCPP. (Note: The link placeholder is provided; ensure you replace it with the actual link to the guide.)
  2. Install the llama-cpp-python package. It’s recommended to get a version that’s compatible with your GPU. Setting up this package will consume approximately 11.5GB of memory split between the CPU and GPU.

For Local Embeddings Setup:

  1. Simply execute the following command:
pip install sentence-transformers

The local embedding model will use around 500MB of memory.

Read related articles:

Tags: