Tag: Models
-
WordLLama
WordLlama is a utility for NLP and word embedding that repurposes components from large language models (LLMs) to generate efficient and compact word representations, similar to GloVe, Word2Vec, or FastText. It starts by extracting the token embedding codebook from a state-of-the-art LLM (e.g., LLaMA3 70B) and trains a small, context-free model within a general-purpose embedding…
-
Llama 3.1 405B
In April 2024, Meta launched Llama 3, the latest generation of advanced, open-source large language models. The initial release featured Llama 3 8B and Llama 3 70B, both setting new performance benchmarks for LLMs in their respective sizes. However, within three months, several other models surpassed these benchmarks, highlighting the rapid advancements in artificial intelligence.…
-
MetaGPT
MetaGPT (Hong et al.) is an awesome multi-agent framework (ICLR 2024) that models agents as a software company following a structured SOP – PMs, architects, engineers, and more communicate with each other to solve the task at hand. LLamaIndex team excited to feature RAG-enhanced MetaGPT, powered by @llama_index modules – this allows agents to tap…