Together AI: A Complete Guide to Inference and Fine-Tuning Open Source Models with One API

Hey everyone! This time I want to talk about a platform that I have been using a lot lately for my AI projects, and it is called Together AI. For those of you who so far only know OpenAI or Anthropic as language model API providers, Together AI offers something different: access to popular open source models like Llama, Qwen, DeepSeek, Mixtral, and dozens of other models, through a single API that is compatible with the OpenAI format. So if you are already used to coding with the OpenAI library, there is almost no learning curve at all.

What makes me stick with Together AI is not just about inference, meaning running a model to generate text. The platform also provides fine-tuning that is super easy, embeddings for semantic search, image generation models, plus function calling and structured output. So it is genuinely a one-stop shop for almost all your AI needs. In this tutorial I will explain everything from scratch: how to get an API key, install the library, all the way to Python code examples you can run right away on your laptop or server.

I wrote this tutorial assuming you already understand the basics of Python and know a little bit about LLMs. But do not worry, I will explain things slowly. Let us get started!

Introduction

Before we jump into code, I want to give you an overview of why Together AI is interesting and when you should use this platform.

What is Together AI

Together AI is a cloud platform that focuses on running open source AI models at scale. Imagine you want to use Llama 3.3 70B or DeepSeek R1 for your application. If you run it yourself, you need expensive GPUs, a complicated setup, and electricity costs that are not cheap. Together AI provides all of that as a service, so you just call the API and pay per token you use. No need to think about infrastructure at all.

There are a few reasons why I really like this platform:

First, the model collection is very broad. From chat models like Llama, Qwen, Mistral, and DeepSeek, to models specialized for coding, reasoning, embeddings, and even image generation. You are free to pick whichever model fits your use case best without being locked into a single vendor.

Second, the pricing is transparent and competitive. Because we use open source models, the cost is much cheaper than proprietary models. For startups or indie developers like us, this helps a lot in saving budget.

Third, and this is important, the API is OpenAI-compatible. So if you have old code that uses the openai library, you just swap the base URL and API key, and it runs immediately. Migration becomes super easy.

When should you use Together AI

In my opinion Together AI is a great fit when you want more control over the model you use, when you want to fine-tune your own model with your data, or when you want to save cost while still getting good quality. If you need data privacy and do not want your data used for training, Together AI also has a clear policy on this. For me personally, this platform is my go-to choice when I am building fast prototypes or production applications that need open source models.

Installation

Okay now we get into the technical part. First we need to get an API key and install the library.

Get an API Key

The first step, open the together.ai website and register an account. The process is quick, just use email or sign in with Google. After you enter the dashboard, look for the Settings or API Keys menu. There you can generate a new API key. Keep this key safe, do not let it get committed to a public repository or leaked to other people. Usually you also get free credits when you first sign up, so you can experiment right away without spending money.

The safest way to store an API key is using an environment variable. So do not hardcode it in your code. In the terminal, you can set it like this:

export TOGETHERAPIKEY="your-api-key-here"

If you are on Windows using PowerShell:

$env:TOGETHERAPIKEY="your-api-key-here"

Or even neater, create a .env file in your project folder and store the key there. I will show you how to load it later.

Install the Python Library

Now we install the official Together AI library. Open your terminal and run:

pip install together

If you want to load the API key from a .env file, also install python-dotenv:

pip install together python-dotenv