MergeKit Tutorial: Merging LLM Models to Create Custom AI Models

Open-source large language models (LLMs) continue to evolve rapidly. With numerous fine-tuned models available on Hugging Face, an interesting technique has emerged that allows us to combine multiple models into a single new model without any retraining. This technique is called model merging, and MergeKit is the most popular toolkit for doing it.

MergeKit, developed by Arcee AI, enables us to merge two or more transformer models using various methods such as SLERP, TIES, DARE, and linear merge. The result is a new model that combines capabilities from the source models without requiring GPU resources for training.

In this tutorial, we will learn how to use MergeKit from installation to creating a merged model ready for deployment.

Why Model Merging?

Before diving into the techniques, it is important to understand why model merging has become popular:

Installing MergeKit

Prerequisites

• Python 3.10 or newer
• Minimum 16 GB RAM (depends on model size)
• Sufficient disk space to store models (each 7B model requires approximately 14 GB in fp16)

Install from PyPI

pip install mergekit

Install from Source (Recommended)

To get the latest features:

git clone https://github.com/arcee-ai/mergekit.git
cd mergekit
pip install -e .

If you want to use the evolution feature (evolutionary merge):

pip install -e ".[evolve]"

Verify Installation

mergekit-yaml --help

If this command displays help options, the installation was successful.

Core Concepts of Model Merging

Supported Architectures

MergeKit supports transformer-based models, including:

• LLaMA and derivatives (Mistral, Qwen, etc.)
• GPT-NeoX
• Phi
• Gemma
• StableLM
• And other compatible architectures

Available Merge Methods

MergeKit provides several merge methods:

| Method | Description | When to Use |

|--------|-------------|-------------|

| Linear | Simple weighted average of model weights | Baseline, fast, easy to understand |

| SLERP | Spherical Linear Interpolation | Merging 2 models with smooth transition |

| TIES | TrIm, Elect, and merGe | Reducing inter-task interference |

| DARE | Drop And REscale | Similar to TIES with a different approach |

| Passthrough | Concatenating layers sequentially | Creating larger "frankenmerge" models |

| Model Stock | Geometric merge in weight space | More robust merging |

Creating Your First Merge Configuration

MergeKit uses YAML files to define merge configurations. Let us start with simple examples.

Linear Merge (Weighted Average)

Create a file linear-merge.yml:

mergemethod: linear
dtype: float16
models:
model: NousResearch/Hermes-2-Pro-Mistral-7B
    parameters:

      weight: 0.6
model: teknium/OpenHermes-2.5-Mistral-7B
    parameters:

      weight: 0.4

mergekit-yaml linear-merge.yml ./output-linear-merge --cuda --lazy-unpickle

mergemethod: slerp
dtype: float16
basemodel: NousResearch/Hermes-2-Pro-Mistral-7B

parameters:
  t: 0.5
models:
model: NousResearch/Hermes-2-Pro-Mistral-7B
model: teknium/OpenHermes-2.5-Mistral-7B

mergekit-yaml slerp-merge.yml ./output-slerp-merge --cuda

mergemethod: ties
dtype: float16
basemodel: mistralai/Mistral-7B-v0.1

parameters:
  density: 0.5
  normalize: true
models:
model: NousResearch/Hermes-2-Pro-Mistral-7B
    parameters:

      weight: 1.0
      density: 0.5
model: teknium/OpenHermes-2.5-Mistral-7B
    parameters:

      weight: 0.5
      density: 0.5

• density: Proportion of parameters to keep (0.0-1.0). Lower values mean more parameters are trimmed.
• normalize: Normalize weights after merging.
• basemodel: Reference model for computing delta weights.

mergekit-yaml ties-merge.yml ./output-ties-merge --cuda

DARE Merge

DARE (Drop And REscale) is similar to TIES but uses random dropping. Create a file dare-merge.yml:

mergemethod: dareties
dtype: float16
basemodel: mistralai/Mistral-7B-v0.1

parameters:
  normalize: true
models:
model: NousResearch/Hermes-2-Pro-Mistral-7B
    parameters:

      weight: 1.0
      density: 0.6
model: teknium/OpenHermes-2.5-Mistral-7B
    parameters:

      weight: 0.5
      density: 0.6

mergekit-yaml dare-merge.yml ./output-dare-merge --cuda

mergemethod: passthrough
dtype: float16
slices:
sources:
model: NousResearch/Hermes-2-Pro-Mistral-7B
        layerrange: [0, 16]

sources:
model: teknium/OpenHermes-2.5-Mistral-7B
        layerrange: [8, 24]

sources:
model: NousResearch/Hermes-2-Pro-Mistral-7B
        layerrange: [16, 32]

mergekit-yaml franken-merge.yml ./output-franken-merge --cuda

mergemethod: slerp
dtype: float16
basemodel: NousResearch/Hermes-2-Pro-Mistral-7B

parameters:
  t:
filter: selfattn
      value: [0.0, 0.3, 0.5, 0.7, 1.0]

filter: mlp
      value: [1.0, 0.7, 0.5, 0.3, 0.0]

value: 0.5
models:

model: NousResearch/Hermes-2-Pro-Mistral-7B
model: teknium/OpenHermes-2.5-Mistral-7B

This configuration uses different gradients for attention layers and MLP layers, providing granular control over the merge process.

Merge with Custom Tokenizer

If the models being merged have different tokenizers:

mergemethod: linear
dtype: float16
tokenizersource: union
models:
model: modelA

    parameters:

      weight: 0.5
model: modelB
    parameters:

      weight: 0.5

mergemethod: slerp
dtype: float16
basemodel: /path/to/local/model-a

parameters:
  t: 0.5
models:
model: /path/to/local/model-a
model: /path/to/local/model-b

from transformers import AutoModelForCausalLM, AutoTokenizer

modelpath = "./output-slerp-merge"
tokenizer = AutoTokenizer.frompretrained(modelpath)
model = AutoModelForCausalLM.frompretrained(

    modelpath,
    torchdtype="auto",

    devicemap="auto"
)

prompt = "Explain machine learning in 3 sentences:"
inputs = tokenizer(prompt, returntensors="pt").to(model.device)

outputs = model.generate(inputs, maxnewtokens=200, temperature=0.7)

print(tokenizer.decode(outputs[0], skipspecialtokens=True))

pip install lm-eval

lmeval --model hf \
  --modelargs pretrained=./output-slerp-merge \

  --tasks hellaswag,arceasy,mmlu \
  --batchsize 8

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

def generateresponse(modelpath, prompt, maxtokens=200):
    tokenizer = AutoTokenizer.frompretrained(modelpath)
    model = AutoModelForCausalLM.frompretrained(

        modelpath, torchdtype=torch.float16, devicemap="auto"
    )
    inputs = tokenizer(prompt, returntensors="pt").to(model.device)

    outputs = model.generate(inputs, maxnewtokens=maxtokens, temperature=0.7)
    return tokenizer.decode(outputs[0], skipspecialtokens=True)

testprompts = [

    "Write a Python function to calculate fibonacci numbers:",
    "Explain quantum computing to a 5-year-old:",
    "What are the best practices for database indexing?",
]

models = {
    "Model A": "NousResearch/Hermes-2-Pro-Mistral-7B",
    "Merged": "./output-slerp-merge",
}

for prompt in testprompts:
    print(f"\nPrompt: {prompt}")
    print("=" * 60)
    for name, path in models.items():
        response = generateresponse(path, prompt)

        print(f"\n[{name}]:")
        print(response)

from huggingfacehub import HfApi

api = HfApi()

api.createrepo(

    repoid="username/my-merged-model",
    repotype="model",

    private=False
)

api.uploadfolder(
    folderpath="./output-slerp-merge",

    repoid="username/my-merged-model",
    repotype="model",

)

huggingface-cli upload username/my-merged-model ./output-slerp-merge

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp

python converthftogguf.py ../output-slerp-merge --outfile merged-model.gguf --outtype f16

For quantization (reducing file size):

./llama-quantize merged-model.gguf merged-model-Q4KM.gguf Q4KM

Popular quantization levels:

• Q4KM: Balance between size and quality (recommended)
• Q5KM: Higher quality, larger file
• Q80: Quality nearly equivalent to fp16, large file

RuntimeError: CUDA out of memory

RuntimeError: Incompatible architectures

If models have different vocabularies, use tokenizersource: union or select a tokenizer from one of the models.

Conclusion

MergeKit is an incredibly useful toolkit for creating custom LLM models without any training. By understanding the various merge methods and available parameters, you can experiment and find the optimal combination for your specific needs.

Key takeaways:

• Start with simple methods (linear or SLERP) before trying more complex ones
• Always evaluate merged models thoroughly
• Document every experiment for reproducibility
• The best merged models typically combine models that excel in different areas

With model merging techniques and MergeKit, you can create AI models tailored to your needs without expensive training costs. Happy experimenting!