Complete Transformers.js Tutorial: Running AI Models Directly in the Browser and Node.js
Transformers.js is an open-source library that allows you to run Hugging Face models directly in the browser or Node.js without requiring a GPU server. The library uses ONNX Runtime as its backend, enabling models to run efficiently on CPU or WebGPU. In this tutorial, we will learn how to use Transformers.js from installation to implementing advanced features like text generation, image classification, and speech recognition.
Why Transformers.js?
Before we begin, here are several reasons why Transformers.js deserves consideration in your projects:
Installation
Installation for Node.js
npm install @huggingface/transformers
Installation for Deno
import { pipeline } from "https://cdn.jsdelivr.net/npm/@huggingface/transformers@3";
Browser Usage via CDN
Usage with Bundlers (Webpack/Vite)
npm install @huggingface/transformers
import { pipeline } from "@huggingface/transformers";
Basic Usage: Pipeline API
The Pipeline API is the easiest way to use Transformers.js. Pipelines simplify the process of loading models and tokenizers into a single easy-to-use function.
Text Classification (Sentiment Analysis)
import { pipeline } from "@huggingface/transformers";
const classifier = await pipeline("sentiment-analysis");
const result = await classifier("This product is amazing and high quality!");
console.log(result);
// Output: [{ label: 'POSITIVE', score: 0.9998 }]
const results = await classifier([
"I am very happy with this service",
"The delivery was extremely slow and disappointing"
]);
console.log(results);
// Output: [
// { label: 'POSITIVE', score: 0.9995 },
// { label: 'NEGATIVE', score: 0.9987 }
// ]
Text Generation
import { pipeline } from "@huggingface/transformers";
const generator = await pipeline("text-generation", "Xenova/gpt2");
const output = await generator("Machine learning is", {
maxnewtokens: 50,
temperature: 0.7,
topk: 50,
});
console.log(output[0].generatedtext);
Question Answering
import { pipeline } from "@huggingface/transformers";
const qa = await pipeline("question-answering");
const answer = await qa({
question: "Who coined the term machine learning?",
context: "Arthur Samuel was an American scientist who in 1959 coined the term machine learning. He defined it as the field of study that gives computers the ability to learn without being explicitly programmed."
});
console.log(answer);
// Output: { answer: 'Arthur Samuel', score: 0.95, start: 0, end: 14 }
Translation
import { pipeline } from "@huggingface/transformers";
const translator = await pipeline(
"translation",
"Xenova/nllb-200-distilled-600M"
);
const result = await translator("Artificial intelligence is transforming the world.", {
srclang: "engLatn",
tgtlang: "indLatn",
});
console.log(result[0].translationtext);
// Output: "Kecerdasan buatan mengubah dunia."
Summarization
import { pipeline } from "@huggingface/transformers";;const summarizer = await pipeline("summarization", "Xenova/distilbart-cnn-6-6");
const article =
Machine learning has become an extremely important technology across various industries.
From healthcare to finance, AI models are used to analyze data, make predictions, and
automate business processes. The growth in machine learning adoption is driven by the
availability of abundant data, increased computational power, and advances in deep
learning algorithms. Major companies like Google, Meta, and Microsoft continue to
invest in AI research to develop more sophisticated models.
const summary = await summarizer(article, {
maxlength: 50,
minlength: 20,
});
console.log(summary[0].summarytext);
Computer Vision Features
Transformers.js doesn't only support NLP but also various computer vision tasks.
Image Classification
import { pipeline } from "@huggingface/transformers";
const classifier = await pipeline(
"image-classification",
"Xenova/vit-base-patch16-224"
);
const result = await classifier("https://example.com/cat.jpg");
console.log(result);
// Output: [{ label: 'Egyptian cat', score: 0.89 }, ...]
Object Detection
import { pipeline } from "@huggingface/transformers";
const detector = await pipeline(
"object-detection",
"Xenova/detr-resnet-50"
);
const result = await detector("https://example.com/street.jpg");
console.log(result);
// Output: [
// { label: 'car', score: 0.95, box: { xmin: 10, ymin: 20, xmax: 200, ymax: 150 } },
// { label: 'person', score: 0.92, box: { xmin: 250, ymin: 30, xmax: 350, ymax: 200 } }
// ]
Image Segmentation
import { pipeline } from "@huggingface/transformers";
const segmenter = await pipeline(
"image-segmentation",
"Xenova/detr-resnet-50-panoptic"
);
const result = await segmenter("https://example.com/room.jpg");
console.log(result);
// Output: Array of segments with labels and masks
Audio Features
Speech Recognition (Automatic Speech Recognition)
import { pipeline } from "@huggingface/transformers";
const transcriber = await pipeline(
"automatic-speech-recognition",
"Xenova/whisper-small"
);
const result = await transcriber("audiosample.wav");
console.log(result.text);
// Output: "Hello, welcome to the Transformers.js tutorial"
Audio Classification
import { pipeline } from "@huggingface/transformers";
const audioClassifier = await pipeline(
"audio-classification",
"Xenova/wav2vec2-large-xlsr-53-gender-recognition-librispeech"
);
const result = await audioClassifier("audiosample.wav");
console.log(result);
// Output: [{ label: 'male', score: 0.95 }, { label: 'female', score: 0.05 }]
Embeddings and Semantic Search
One of the most popular uses of Transformers.js is generating embeddings for semantic search.
Feature Extraction (Embeddings)
import { pipeline } from "@huggingface/transformers";
const extractor = await pipeline(
"feature-extraction",
"Xenova/all-MiniLM-L6-v2"
);
const embedding = await extractor("Machine learning tutorial for beginners", {
pooling: "mean",
normalize: true,
});
console.log(embedding.dims); // [1, 384]
Implementing Semantic Search
import { pipeline, cossim } from "@huggingface/transformers";
const extractor = await pipeline(
"feature-extraction",
"Xenova/all-MiniLM-L6-v2"
);
const documents = [
"Python is a popular programming language for data science",
"JavaScript is used for frontend and backend web development",
"Machine learning requires large datasets for training",
"Docker simplifies application deployment in containers",
"Neural networks are inspired by how the human brain works"
];
const query = "How do you train an AI model?";
async function semanticSearch(query, documents) {
const queryEmbedding = await extractor(query, {
pooling: "mean",
normalize: true,
});
const docEmbeddings = await Promise.all(
documents.map(doc =>
extractor(doc, { pooling: "mean", normalize: true })
)
);
const scores = docEmbeddings.map((docEmb, i) => ({
document: documents[i],
score: cos
sim(queryEmbedding.data, docEmb.data),
}));
return scores.sort((a, b) => b.score - a.score);
}
const results = await semanticSearch(query, documents);
console.log("Semantic search results:");
results.forEach((r, i) => {
console.log(${i + 1}. [${r.score.toFixed(4)}] ${r.document});
});
Advanced Usage
Model Configuration and Quantization
Transformers.js supports quantized models for smaller size and faster inference.
import { pipeline, env } from "@huggingface/transformers";
env.allowLocalModels = false;
const classifier = await pipeline("sentiment-analysis", "Xenova/distilbert-base-uncased-finetuned-sst-2-english", {
dtype: "q8", // use 8-bit quantized model
device: "webgpu", // use WebGPU if available
});
Cache Configuration
import { env } from "@huggingface/transformers";
env.cacheDir = "./modelscache";
env.useBrowserCache = true;
Using WebGPU for Acceleration
WebGPU provides significant acceleration for model inference. Here's how to enable it:
import { pipeline } from "@huggingface/transformers";
const classifier = await pipeline("sentiment-analysis", "model-name", {
device: "webgpu",
});
const result = await classifier("Text to analyze");
console.log(result);
To check if WebGPU is available:
if (navigator.gpu) {
console.log("WebGPU available, using GPU acceleration");
} else {
console.log("WebGPU not available, falling back to CPU");
}
Streaming Text Generation
import { pipeline, TextStreamer } from "@huggingface/transformers";
const generator = await pipeline(
"text-generation",
"Xenova/Phi-3-mini-4k-instruct",
{ dtype: "q4" }
);
const streamer = new TextStreamer(generator.tokenizer, {
skipprompt: true,
callbackfunction: (text) => {
process.stdout.write(text);
},
});
await generator("Explain what machine learning is in 3 sentences:", {
maxnewtokens: 128,
streamer,
});
Custom Model Loading
import {
AutoTokenizer,
AutoModelForSequenceClassification,
} from "@huggingface/transformers";
const tokenizer = await AutoTokenizer.frompretrained("Xenova/bert-base-uncased");
const model = await AutoModelForSequenceClassification.frompretrained(
"Xenova/bert-base-uncased",
{ dtype: "q8" }
);
const inputs = tokenizer("Transformers.js is very easy to use!");
const { logits } = await model(inputs);
console.log("Logits:", logits.data);
Integration with Web Frameworks
React + Transformers.js
import { useState, useEffect, useRef } from "react";
import { pipeline } from "@huggingface/transformers";
function SentimentAnalyzer() {
const [result, setResult] = useState(null);
const [loading, setLoading] = useState(true);
const classifierRef = useRef(null);
useEffect(() => {
async function loadModel() {
classifierRef.current = await pipeline("sentiment-analysis");
setLoading(false);
}
loadModel();
}, []);
async function analyze(text) {
if (!classifierRef.current) return;
const output = await classifierRef.current(text);
setResult(output[0]);
}
if (loading) return
Loading model...
;
return (
)}
);
}
Next.js + Web Worker
For better performance, run models in a Web Worker to avoid blocking the UI thread:
// worker.js
import { pipeline } from "@huggingface/transformers";
let classifier = null;
self.onmessage = async (event) => {
if (!classifier) {
self.postMessage({ status: "loading" });
classifier = await pipeline("sentiment-analysis");
self.postMessage({ status: "ready" });
}
if (event.data.type === "classify") {
const result = await classifier(event.data.text);
self.postMessage({ status: "result", data: result });
}
};
// component.jsx
import { useEffect, useRef, useState } from "react";
export default function ClassifierComponent() {
const workerRef = useRef(null);
const [status, setStatus] = useState("initializing");
useEffect(() => {
workerRef.current = new Worker(
new URL("./worker.js", import.meta.url),
{ type: "module" }
);
workerRef.current.onmessage = (event) => {
if (event.data.status === "ready") setStatus("ready");
if (event.data.status === "result") {
console.log("Result:", event.data.data);
}
};
return () => workerRef.current?.terminate();
}, []);
function classify(text) {
workerRef.current?.postMessage({ type: "classify", text });
}
return (
Status: {status}
Analyze
);
}
Practical Project: Local RAG Chatbot
Here's an example of building a simple chatbot that runs entirely in the browser using RAG (Retrieval-Augmented Generation):
import { pipeline, cossim } from "@huggingface/transformers";;class LocalRAGChatbot {
constructor() {
this.embedder = null;
this.generator = null;
this.knowledgeBase = [];
this.embeddings = [];
}
async initialize() {
console.log("Loading embedding model...");
this.embedder = await pipeline(
"feature-extraction",
"Xenova/all-MiniLM-L6-v2"
);
console.log("Loading text generation model...");
this.generator = await pipeline(
"text2text-generation",
"Xenova/flan-t5-small"
);
console.log("Chatbot ready!");
}
async addDocument(text) {
const embedding = await this.embedder(text, {
pooling: "mean",
normalize: true,
});
this.knowledgeBase.push(text);
this.embeddings.push(embedding);
}
async findRelevantDocs(query, topK = 3) {
const queryEmbedding = await this.embedder(query, {
pooling: "mean",
normalize: true,
});
const scores = this.embeddings.map((emb, i) => ({
index: i,
score: cossim(queryEmbedding.data, emb.data),
}));
return scores
.sort((a, b) => b.score - a.score)
.slice(0, topK)
.map((s) => this.knowledgeBase[s.index]);
}
async chat(query) {
const relevantDocs = await this.findRelevantDocs(query);
const context = relevantDocs.join("\n");
const prompt =
Based on the following context, answer the question.Context: ${context}
Question: ${query}
Answer:
const response = await this.generator(prompt, {
maxnewtokens: 100,
});
return response[0].generatedtext;
}
}
// Usage
const chatbot = new LocalRAGChatbot();
await chatbot.initialize();
await chatbot.addDocument("Transformers.js supports over 1000 models from Hugging Face.");
await chatbot.addDocument("WebGPU provides GPU acceleration for inference in the browser.");
await chatbot.addDocument("4-bit quantized models are 75% smaller than the original models.");
const answer = await chatbot.chat("How many models are supported?");
console.log(answer);
Best Practices
1. Optimize Model Size
Always use quantized models for production browser applications:
const pipe = await pipeline("task", "model-name", {
dtype: "q4",
});
Model size comparison:
- fp32: Full size (baseline)
- fp16: 50% of fp32
- q8: 25% of fp32
- q4: 12.5% of fp32
2. Lazy Loading and Caching
let classifierInstance = null;
async function getClassifier() {
if (!classifierInstance) {
classifierInstance = await pipeline("sentiment-analysis");
}
return classifierInstance;
}
3. Use Web Workers for Heavy Tasks
Always run model inference in a Web Worker to maintain UI responsiveness. Never run models on the main thread for production applications.
4. Progress Callback for Better UX
const classifier = await pipeline("sentiment-analysis", "model-name", {
progress_callback: (progress) => {
if (progress.status === "downloading") {
console.log(Downloading: ${(progress.progress).toFixed(1)}%);
}
if (progress.status === "ready") {
console.log("Model loaded!");
}
},
});
5. Error Handling
async function safeInference(text) {
try {
const classifier = await pipeline("sentiment-analysis");
return await classifier(text);
} catch (error) {
if (error.message.includes("WebGPU")) {
console.warn("WebGPU not available, falling back to CPU");
const classifier = await pipeline("sentiment-analysis", null, {
device: "cpu",
});
return await classifier(text);
}
throw error;
}
}
6. Limit Input Length
function truncateInput(text, maxLength = 512) {
const words = text.split(" ");
if (words.length > maxLength) {
return words.slice(0, maxLength).join(" ");
}
return text;
}
Comparison with Cloud API Solutions
| Aspect | Transformers.js (Local) | Cloud API |
|--------|------------------------|-----------|
| Privacy | Data stays on device | Data sent to server |
| Cost | Free | Pay per request |
| Latency | Low (no network) | Network dependent |
| Model Size | Limited (~2GB max) | Unlimited |
| Accuracy | Smaller models | Large, accurate models |
| Offline | Yes | No |
Troubleshooting
Model Download Failures
import { env } from "@huggingface/transformers";
env.useBrowserCache = false;
env.useCustomCache = false;
Out of Memory in Browser
Use smaller or quantized models:
const pipe = await pipeline("task", "smaller-model", {
dtype: "q4",
});
WebGPU Not Available
Ensure the browser supports WebGPU (Chrome 113+, Edge 113+). For browsers that don't support it, Transformers.js automatically falls back to WASM/CPU.
Conclusion
Transformers.js opens up new possibilities for running AI models directly in the browser and Node.js without dependency on GPU servers. With support for over 1000 models from the Hugging Face Hub, quantization for efficiency, and WebGPU acceleration, this library is an excellent choice for building AI applications that prioritize privacy and performance.
Key takeaways to remember:
- Use the Pipeline API for ease of use
- Always use quantized models for browser applications
- Run inference in Web Workers to maintain UI responsiveness
- Leverage WebGPU for acceleration when available
- Consider the trade-offs between model size and accuracy
With Transformers.js, you can build powerful AI applications entirely on the client side, from sentiment analysis and semantic search to RAG-powered chatbots, all running in the user's browser.