Llama 2

LLama

OpenAI Lifeboat by Replicate

OpenAI Lifeboat by Replicate

Replicate introducing the OpenAI Lifeboat. They made a proxy so you can switch your language model provider to Llama 70B with 3 lines of code. It’s free for the next week.

Quick Start

To get started, create a Replicate account and copy your API token. Set your token as an environment variable. Install the OpenAI client if you haven’t already.Next:

  1. Change your api_key from your OpenAI key to your REPLICATE_API_TOKEN environment variable wherever you initialize the OpenAI client.
  2. Point your base_url to https://openai-proxy.replicate.com/v1
  3. Set your model to meta/llama-2-70b-chat

Your code should now look like this:

import OpenAI from "openai";

const openai = new OpenAI({
  apiKey: process.env["REPLICATE_API_TOKEN"],
  baseURL: "https://openai-proxy.replicate.com/v1",
});

const completions = await openai.chat.completions.create({
  model: "meta/llama-2-70b-chat",
  messages: [
    {
      role: "user",
      content: "Write a haiku about camelids",
    },
  ],
  maxTokens: 64,
  stream: true,
});
for await (const part of completions) {
  process.stdout.write(part.choices[0]?.delta || "");
}

That’s it! You’re now using an open source model and it’s free for the next week. Enjoy.

Llama 70B outperforms GPT-3.5-turbo. It’s not as smart as GPT-4, but you can fine-tune it to do things that aren’t possible with larger models.

Read other related articles:

Tags:

,