Complete Transformers.js Tutorial: Running AI Models Directly in the Browser and Node.js

# Tutorial Lengkap Transformers.js: Menjalankan Model AI Langsung di Browser dan Node.js Transformers.js adalah library open-source yang memungkinkan Anda menjalankan model-model Hugging Face langsun...

By Ruby Abdullah · · tutorial
Transformers.jsHugging FaceWebGPUJavaScriptBrowser AI

Complete Transformers.js Tutorial: Running AI Models Directly in the Browser and Node.js

Transformers.js is an open-source library that allows you to run Hugging Face models directly in the browser or Node.js without requiring a GPU server. The library uses ONNX Runtime as its backend, enabling models to run efficiently on CPU or WebGPU. In this tutorial, we will learn how to use Transformers.js from installation to implementing advanced features like text generation, image classification, and speech recognition.

Why Transformers.js?

Before we begin, here are several reasons why Transformers.js deserves consideration in your projects:

  • Data Privacy: All inference is performed locally, user data never needs to be sent to external servers
  • No GPU Server Costs: No need to rent cloud GPUs to run AI models
  • Low Latency: No network round-trip since models run on the user's device
  • Offline Capable: Once the model is downloaded, the application can run without an internet connection
  • Wide Compatibility: Supports over 1000 models from the Hugging Face Hub in ONNX format
  • Installation

    Installation for Node.js

    npm install @huggingface/transformers
    

    Installation for Deno

    import { pipeline } from "https://cdn.jsdelivr.net/npm/@huggingface/transformers@3";
    

    Browser Usage via CDN

    Usage with Bundlers (Webpack/Vite)

    npm install @huggingface/transformers
    

    import { pipeline } from "@huggingface/transformers";
    

    Basic Usage: Pipeline API

    The Pipeline API is the easiest way to use Transformers.js. Pipelines simplify the process of loading models and tokenizers into a single easy-to-use function.

    Text Classification (Sentiment Analysis)

    import { pipeline } from "@huggingface/transformers";
    
    

    const classifier = await pipeline("sentiment-analysis");

    const result = await classifier("This product is amazing and high quality!");

    console.log(result);

    // Output: [{ label: 'POSITIVE', score: 0.9998 }]

    const results = await classifier([

    "I am very happy with this service",

    "The delivery was extremely slow and disappointing"

    ]);

    console.log(results);

    // Output: [

    // { label: 'POSITIVE', score: 0.9995 },

    // { label: 'NEGATIVE', score: 0.9987 }

    // ]

    Text Generation

    import { pipeline } from "@huggingface/transformers";
    
    

    const generator = await pipeline("text-generation", "Xenova/gpt2");

    const output = await generator("Machine learning is", {

    maxnewtokens: 50,

    temperature: 0.7,

    topk: 50,

    });

    console.log(output[0].generatedtext);

    Question Answering

    import { pipeline } from "@huggingface/transformers";
    
    

    const qa = await pipeline("question-answering");

    const answer = await qa({

    question: "Who coined the term machine learning?",

    context: "Arthur Samuel was an American scientist who in 1959 coined the term machine learning. He defined it as the field of study that gives computers the ability to learn without being explicitly programmed."

    });

    console.log(answer);

    // Output: { answer: 'Arthur Samuel', score: 0.95, start: 0, end: 14 }

    Translation

    import { pipeline } from "@huggingface/transformers";
    
    

    const translator = await pipeline(

    "translation",

    "Xenova/nllb-200-distilled-600M"

    );

    const result = await translator("Artificial intelligence is transforming the world.", {

    srclang: "engLatn",

    tgtlang: "indLatn",

    });

    console.log(result[0].translationtext);

    // Output: "Kecerdasan buatan mengubah dunia."

    Summarization

    import { pipeline } from "@huggingface/transformers";
    
    

    const summarizer = await pipeline("summarization", "Xenova/distilbart-cnn-6-6");

    const article =

    Machine learning has become an extremely important technology across various industries.

    From healthcare to finance, AI models are used to analyze data, make predictions, and

    automate business processes. The growth in machine learning adoption is driven by the

    availability of abundant data, increased computational power, and advances in deep

    learning algorithms. Major companies like Google, Meta, and Microsoft continue to

    invest in AI research to develop more sophisticated models.

    ;

    const summary = await summarizer(article, {

    maxlength: 50,

    minlength: 20,

    });

    console.log(summary[0].summarytext);

    Computer Vision Features

    Transformers.js doesn't only support NLP but also various computer vision tasks.

    Image Classification

    import { pipeline } from "@huggingface/transformers";
    
    

    const classifier = await pipeline(

    "image-classification",

    "Xenova/vit-base-patch16-224"

    );

    const result = await classifier("https://example.com/cat.jpg");

    console.log(result);

    // Output: [{ label: 'Egyptian cat', score: 0.89 }, ...]

    Object Detection

    import { pipeline } from "@huggingface/transformers";
    
    

    const detector = await pipeline(

    "object-detection",

    "Xenova/detr-resnet-50"

    );

    const result = await detector("https://example.com/street.jpg");

    console.log(result);

    // Output: [

    // { label: 'car', score: 0.95, box: { xmin: 10, ymin: 20, xmax: 200, ymax: 150 } },

    // { label: 'person', score: 0.92, box: { xmin: 250, ymin: 30, xmax: 350, ymax: 200 } }

    // ]

    Image Segmentation

    import { pipeline } from "@huggingface/transformers";
    
    

    const segmenter = await pipeline(

    "image-segmentation",

    "Xenova/detr-resnet-50-panoptic"

    );

    const result = await segmenter("https://example.com/room.jpg");

    console.log(result);

    // Output: Array of segments with labels and masks

    Audio Features

    Speech Recognition (Automatic Speech Recognition)

    import { pipeline } from "@huggingface/transformers";
    
    

    const transcriber = await pipeline(

    "automatic-speech-recognition",

    "Xenova/whisper-small"

    );

    const result = await transcriber("audiosample.wav");

    console.log(result.text);

    // Output: "Hello, welcome to the Transformers.js tutorial"

    Audio Classification

    import { pipeline } from "@huggingface/transformers";
    
    

    const audioClassifier = await pipeline(

    "audio-classification",

    "Xenova/wav2vec2-large-xlsr-53-gender-recognition-librispeech"

    );

    const result = await audioClassifier("audiosample.wav");

    console.log(result);

    // Output: [{ label: 'male', score: 0.95 }, { label: 'female', score: 0.05 }]

    One of the most popular uses of Transformers.js is generating embeddings for semantic search.

    Feature Extraction (Embeddings)

    import { pipeline } from "@huggingface/transformers";
    
    

    const extractor = await pipeline(

    "feature-extraction",

    "Xenova/all-MiniLM-L6-v2"

    );

    const embedding = await extractor("Machine learning tutorial for beginners", {

    pooling: "mean",

    normalize: true,

    });

    console.log(embedding.dims); // [1, 384]

    import { pipeline, cossim } from "@huggingface/transformers";
    
    

    const extractor = await pipeline(

    "feature-extraction",

    "Xenova/all-MiniLM-L6-v2"

    );

    const documents = [

    "Python is a popular programming language for data science",

    "JavaScript is used for frontend and backend web development",

    "Machine learning requires large datasets for training",

    "Docker simplifies application deployment in containers",

    "Neural networks are inspired by how the human brain works"

    ];

    const query = "How do you train an AI model?";

    async function semanticSearch(query, documents) {

    const queryEmbedding = await extractor(query, {

    pooling: "mean",

    normalize: true,

    });

    const docEmbeddings = await Promise.all(

    documents.map(doc =>

    extractor(doc, { pooling: "mean", normalize: true })

    )

    );

    const scores = docEmbeddings.map((docEmb, i) => ({

    document: documents[i],

    score: cossim(queryEmbedding.data, docEmb.data),

    }));

    return scores.sort((a, b) => b.score - a.score);

    }

    const results = await semanticSearch(query, documents);

    console.log("Semantic search results:");

    results.forEach((r, i) => {

    console.log(${i + 1}. [${r.score.toFixed(4)}] ${r.document});

    });

    Advanced Usage

    Model Configuration and Quantization

    Transformers.js supports quantized models for smaller size and faster inference.

    import { pipeline, env } from "@huggingface/transformers";
    
    

    env.allowLocalModels = false;

    const classifier = await pipeline("sentiment-analysis", "Xenova/distilbert-base-uncased-finetuned-sst-2-english", {

    dtype: "q8", // use 8-bit quantized model

    device: "webgpu", // use WebGPU if available

    });

    Cache Configuration

    import { env } from "@huggingface/transformers";
    
    

    env.cacheDir = "./modelscache";

    env.useBrowserCache = true;

    Using WebGPU for Acceleration

    WebGPU provides significant acceleration for model inference. Here's how to enable it:

    import { pipeline } from "@huggingface/transformers";
    
    

    const classifier = await pipeline("sentiment-analysis", "model-name", {

    device: "webgpu",

    });

    const result = await classifier("Text to analyze");

    console.log(result);

    To check if WebGPU is available:

    if (navigator.gpu) {
    

    console.log("WebGPU available, using GPU acceleration");

    } else {

    console.log("WebGPU not available, falling back to CPU");

    }

    Streaming Text Generation

    import { pipeline, TextStreamer } from "@huggingface/transformers";
    
    

    const generator = await pipeline(

    "text-generation",

    "Xenova/Phi-3-mini-4k-instruct",

    { dtype: "q4" }

    );

    const streamer = new TextStreamer(generator.tokenizer, {

    skipprompt: true,

    callbackfunction: (text) => {

    process.stdout.write(text);

    },

    });

    await generator("Explain what machine learning is in 3 sentences:", {

    maxnewtokens: 128,

    streamer,

    });

    Custom Model Loading

    import {
    

    AutoTokenizer,

    AutoModelForSequenceClassification,

    } from "@huggingface/transformers";

    const tokenizer = await AutoTokenizer.frompretrained("Xenova/bert-base-uncased");

    const model = await AutoModelForSequenceClassification.frompretrained(

    "Xenova/bert-base-uncased",

    { dtype: "q8" }

    );

    const inputs = tokenizer("Transformers.js is very easy to use!");

    const { logits } = await model(inputs);

    console.log("Logits:", logits.data);

    Integration with Web Frameworks

    React + Transformers.js

    import { useState, useEffect, useRef } from "react";
    

    import { pipeline } from "@huggingface/transformers";

    function SentimentAnalyzer() {

    const [result, setResult] = useState(null);

    const [loading, setLoading] = useState(true);

    const classifierRef = useRef(null);

    useEffect(() => {

    async function loadModel() {

    classifierRef.current = await pipeline("sentiment-analysis");

    setLoading(false);

    }

    loadModel();

    }, []);

    async function analyze(text) {

    if (!classifierRef.current) return;

    const output = await classifierRef.current(text);

    setResult(output[0]);

    }

    if (loading) return

    Loading model...

    ;

    return (

    onChange={(e) => analyze(e.target.value)}

    placeholder="Type text to analyze..."

    />

    {result && (

    Sentiment: {result.label} (Confidence: {(result.score * 100).toFixed(1)}%)

    )}

    );

    }

    Next.js + Web Worker

    For better performance, run models in a Web Worker to avoid blocking the UI thread:

    // worker.js
    

    import { pipeline } from "@huggingface/transformers";

    let classifier = null;

    self.onmessage = async (event) => {

    if (!classifier) {

    self.postMessage({ status: "loading" });

    classifier = await pipeline("sentiment-analysis");

    self.postMessage({ status: "ready" });

    }

    if (event.data.type === "classify") {

    const result = await classifier(event.data.text);

    self.postMessage({ status: "result", data: result });

    }

    };

    // component.jsx
    

    import { useEffect, useRef, useState } from "react";

    export default function ClassifierComponent() {

    const workerRef = useRef(null);

    const [status, setStatus] = useState("initializing");

    useEffect(() => {

    workerRef.current = new Worker(

    new URL("./worker.js", import.meta.url),

    { type: "module" }

    );

    workerRef.current.onmessage = (event) => {

    if (event.data.status === "ready") setStatus("ready");

    if (event.data.status === "result") {

    console.log("Result:", event.data.data);

    }

    };

    return () => workerRef.current?.terminate();

    }, []);

    function classify(text) {

    workerRef.current?.postMessage({ type: "classify", text });

    }

    return (

    Status: {status}

    );

    }

    Practical Project: Local RAG Chatbot

    Here's an example of building a simple chatbot that runs entirely in the browser using RAG (Retrieval-Augmented Generation):

    import { pipeline, cossim } from "@huggingface/transformers";
    
    

    class LocalRAGChatbot {

    constructor() {

    this.embedder = null;

    this.generator = null;

    this.knowledgeBase = [];

    this.embeddings = [];

    }

    async initialize() {

    console.log("Loading embedding model...");

    this.embedder = await pipeline(

    "feature-extraction",

    "Xenova/all-MiniLM-L6-v2"

    );

    console.log("Loading text generation model...");

    this.generator = await pipeline(

    "text2text-generation",

    "Xenova/flan-t5-small"

    );

    console.log("Chatbot ready!");

    }

    async addDocument(text) {

    const embedding = await this.embedder(text, {

    pooling: "mean",

    normalize: true,

    });

    this.knowledgeBase.push(text);

    this.embeddings.push(embedding);

    }

    async findRelevantDocs(query, topK = 3) {

    const queryEmbedding = await this.embedder(query, {

    pooling: "mean",

    normalize: true,

    });

    const scores = this.embeddings.map((emb, i) => ({

    index: i,

    score: cossim(queryEmbedding.data, emb.data),

    }));

    return scores

    .sort((a, b) => b.score - a.score)

    .slice(0, topK)

    .map((s) => this.knowledgeBase[s.index]);

    }

    async chat(query) {

    const relevantDocs = await this.findRelevantDocs(query);

    const context = relevantDocs.join("\n");

    const prompt = Based on the following context, answer the question.

    Context: ${context}

    Question: ${query}

    Answer:;

    const response = await this.generator(prompt, {

    maxnewtokens: 100,

    });

    return response[0].generatedtext;

    }

    }

    // Usage

    const chatbot = new LocalRAGChatbot();

    await chatbot.initialize();

    await chatbot.addDocument("Transformers.js supports over 1000 models from Hugging Face.");

    await chatbot.addDocument("WebGPU provides GPU acceleration for inference in the browser.");

    await chatbot.addDocument("4-bit quantized models are 75% smaller than the original models.");

    const answer = await chatbot.chat("How many models are supported?");

    console.log(answer);

    Best Practices

    1. Optimize Model Size

    Always use quantized models for production browser applications:

    const pipe = await pipeline("task", "model-name", {
    

    dtype: "q4",

    });

    Model size comparison:

    • fp32: Full size (baseline)
    • fp16: 50% of fp32
    • q8: 25% of fp32
    • q4: 12.5% of fp32

    2. Lazy Loading and Caching

    let classifierInstance = null;
    
    

    async function getClassifier() {

    if (!classifierInstance) {

    classifierInstance = await pipeline("sentiment-analysis");

    }

    return classifierInstance;

    }

    3. Use Web Workers for Heavy Tasks

    Always run model inference in a Web Worker to maintain UI responsiveness. Never run models on the main thread for production applications.

    4. Progress Callback for Better UX

    const classifier = await pipeline("sentiment-analysis", "model-name", {
    

    progress_callback: (progress) => {

    if (progress.status === "downloading") {

    console.log(Downloading: ${(progress.progress).toFixed(1)}%);

    }

    if (progress.status === "ready") {

    console.log("Model loaded!");

    }

    },

    });

    5. Error Handling

    async function safeInference(text) {
    

    try {

    const classifier = await pipeline("sentiment-analysis");

    return await classifier(text);

    } catch (error) {

    if (error.message.includes("WebGPU")) {

    console.warn("WebGPU not available, falling back to CPU");

    const classifier = await pipeline("sentiment-analysis", null, {

    device: "cpu",

    });

    return await classifier(text);

    }

    throw error;

    }

    }

    6. Limit Input Length

    function truncateInput(text, maxLength = 512) {
    

    const words = text.split(" ");

    if (words.length > maxLength) {

    return words.slice(0, maxLength).join(" ");

    }

    return text;

    }

    Comparison with Cloud API Solutions

    | Aspect | Transformers.js (Local) | Cloud API |

    |--------|------------------------|-----------|

    | Privacy | Data stays on device | Data sent to server |

    | Cost | Free | Pay per request |

    | Latency | Low (no network) | Network dependent |

    | Model Size | Limited (~2GB max) | Unlimited |

    | Accuracy | Smaller models | Large, accurate models |

    | Offline | Yes | No |

    Troubleshooting

    Model Download Failures

    import { env } from "@huggingface/transformers";
    

    env.useBrowserCache = false;

    env.useCustomCache = false;

    Out of Memory in Browser

    Use smaller or quantized models:

    const pipe = await pipeline("task", "smaller-model", {
    

    dtype: "q4",

    });

    WebGPU Not Available

    Ensure the browser supports WebGPU (Chrome 113+, Edge 113+). For browsers that don't support it, Transformers.js automatically falls back to WASM/CPU.

    Conclusion

    Transformers.js opens up new possibilities for running AI models directly in the browser and Node.js without dependency on GPU servers. With support for over 1000 models from the Hugging Face Hub, quantization for efficiency, and WebGPU acceleration, this library is an excellent choice for building AI applications that prioritize privacy and performance.

    Key takeaways to remember:

    • Use the Pipeline API for ease of use
    • Always use quantized models for browser applications
    • Run inference in Web Workers to maintain UI responsiveness
    • Leverage WebGPU for acceleration when available
    • Consider the trade-offs between model size and accuracy

    With Transformers.js, you can build powerful AI applications entirely on the client side, from sentiment analysis and semantic search to RAG-powered chatbots, all running in the user's browser.

    Related Articles

    Complete Google Gemma Tutorial: Open-Source Models from Google DeepMind

    Tutorial Lengkap Google Gemma: Model Open-Source dari Google DeepMind Google Gemma adalah keluarga model bahasa open-sou...

    Complete Guide to Hugging Face AutoTrain: Fine-Tune AI Models Without the Hassle

    Tutorial Lengkap Hugging Face AutoTrain: Fine-Tuning Model AI Tanpa Ribet Hugging Face AutoTrain adalah platform dan lib...

    MergeKit Tutorial: Merging LLM Models to Create Custom AI Models

    Tutorial MergeKit: Menggabungkan Model LLM untuk Membuat Model AI Kustom Model bahasa besar (LLM) open-source terus berk...

    Text Generation Inference (TGI) Tutorial: Production LLM Serving

    Menyajikan LLM di Produksi dengan Text Generation Inference (TGI) Text Generation Inference (TGI) adalah toolkit buatan ...