Tutorial 18: TensorFlow Lite - Deploy ML on Mobile

Table of Contents

Introduction

Deploying machine learning models on mobile and edge devices opens up possibilities that cloud-only inference cannot match: offline capability, reduced latency, improved privacy, and lower operational costs. TensorFlow Lite (TFLite) is Google's framework for running ML models on mobile phones, embedded devices, and edge hardware.

This tutorial covers the complete workflow from converting trained models to the TFLite format, applying quantization to reduce model size and improve speed, deploying on Android and iOS, running on Edge TPUs, and benchmarking performance. Whether you are building a real-time image classifier, an on-device NLP model, or a sensor data pipeline, these techniques apply directly.

Prerequisites

• Python 3.9+ with TensorFlow 2.15+
• Android Studio (for Android deployment)
• Xcode 15+ (for iOS deployment)
• Basic understanding of neural network architectures
• A trained model (we will create one for demonstration)

# Install required packages
pip install tensorflow tflite-support onnx onnx-tf torch

import tensorflow as tf
import numpy as np
print(f"TensorFlow version: {tf.version}")

Understanding TensorFlow Lite

TFLite uses a different model format (.tflite) from standard TensorFlow (.pb, SavedModel). The TFLite format is a FlatBuffer-based schema optimized for:

• Small binary size - FlatBuffers are compact and zero-copy
• Fast initialization - no parsing overhead, memory-mapped
• Reduced memory footprint - designed for constrained devices
• Hardware acceleration - supports GPU delegates, NNAPI, Edge TPU

The conversion pipeline looks like this:

TensorFlow Model / PyTorch Model
        |
        v
    TFLite Converter (with optional optimizations)
        |
        v
    .tflite file
        |
        v
    TFLite Interpreter (on device)

Model Conversion from TensorFlow

Converting a Keras Model

import tensorflow as tf
from tensorflow import keras

Create and train a sample image classification model
def buildmodel():

    model = keras.Sequential([
        keras.layers.Conv2D(32, (3, 3), activation='relu',
                           inputshape=(224, 224, 3)),
        keras.layers.MaxPooling2D((2, 2)),
        keras.layers.Conv2D(64, (3, 3), activation='relu'),
        keras.layers.MaxPooling2D((2, 2)),
        keras.layers.Conv2D(128, (3, 3), activation='relu'),
        keras.layers.GlobalAveragePooling2D(),
        keras.layers.Dense(256, activation='relu'),
        keras.layers.Dropout(0.5),
        keras.layers.Dense(10, activation='softmax')
    ])
    model.compile(
        optimizer='adam',
        loss='sparsecategoricalcrossentropy',
        metrics=['accuracy']
    )
    return model

model = buildmodel()


Method 1: Convert from Keras model directly
converter = tf.lite.TFLiteConverter.fromkerasmodel(model)

tflitemodel = converter.convert()

Save the model
with open('model.tflite', 'wb') as f:
    f.write(tflitemodel)


print(f"Model size: {len(tflitemodel) / 1024:.1f} KB")

Converting from SavedModel

# Save as SavedModel first
model.save('savedmodeldir')

Convert from SavedModel
converter = tf.lite.TFLiteConverter.fromsavedmodel('savedmodeldir')
tflitemodel = converter.convert()


with open('modelfromsaved.tflite', 'wb') as f:

    f.write(tflitemodel)

Setting Input/Output Details

# Convert with explicit input shape and signature
converter = tf.lite.TFLiteConverter.fromkerasmodel(model)

Set input shape for dynamic batch size models
converter.inputshape = {model.input.name: [1, 224, 224, 3]}


tflitemodel = converter.convert()

Verify input/output details
interpreter = tf.lite.Interpreter(modelcontent=tflitemodel)
interpreter.allocatetensors()


inputdetails = interpreter.getinputdetails()
outputdetails = interpreter.getoutputdetails()


print("Input details:")
for inp in inputdetails:
    print(f"  Name: {inp['name']}, Shape: {inp['shape']}, "
          f"Type: {inp['dtype']}")

print("Output details:")
for out in outputdetails:

    print(f"  Name: {out['name']}, Shape: {out['shape']}, "
          f"Type: {out['dtype']}")

import torch
import torch.nn as nn

Define a PyTorch model
class PyTorchClassifier(nn.Module):
    def init(self, numclasses=10):
        super().init()
        self.features = nn.Sequential(
            nn.Conv2d(3, 32, 3, padding=1),
            nn.ReLU(),
            nn.MaxPool2d(2),
            nn.Conv2d(32, 64, 3, padding=1),
            nn.ReLU(),
            nn.AdaptiveAvgPool2d(1),
        )
        self.classifier = nn.Sequential(
            nn.Flatten(),
            nn.Linear(64, 128),
            nn.ReLU(),
            nn.Linear(128, numclasses),

        )

    def forward(self, x):
        x = self.features(x)
        x = self.classifier(x)
        return x

Step 1: Export PyTorch to ONNX
pytorchmodel = PyTorchClassifier()
pytorchmodel.eval()


dummyinput = torch.randn(1, 3, 224, 224)
torch.onnx.export(
    pytorchmodel,

    dummyinput,
    "model.onnx",
    inputnames=['input'],

    outputnames=['output'],
    dynamicaxes={'input': {0: 'batchsize'},
                  'output': {0: 'batchsize'}},

    opsetversion=13
)

Step 2: Convert ONNX to TensorFlow SavedModel
import onnx
from onnxtf.backend import prepare


onnxmodel = onnx.load("model.onnx")
tfrep = prepare(onnxmodel)
tfrep.exportgraph("pytorchsavedmodel")

Step 3: Convert SavedModel to TFLite
converter = tf.lite.TFLiteConverter.fromsavedmodel("pytorchsavedmodel")
tflitemodel = converter.convert()


with open("pytorchmodel.tflite", "wb") as f:
    f.write(tflitemodel)


print(f"PyTorch -> TFLite model size: {len(tflitemodel) / 1024:.1f} KB")

Post-Training Quantization

Quantization reduces model size and increases inference speed by converting 32-bit floating-point weights to lower-precision representations.

Dynamic Range Quantization

# Dynamic range quantization (simplest, good default)
converter = tf.lite.TFLiteConverter.fromkerasmodel(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]

tflitequantmodel = converter.convert()

with open('modeldynamicquant.tflite', 'wb') as f:
    f.write(tflitequantmodel)

originalsize = len(tflitemodel)
quantsize = len(tflitequantmodel)

print(f"Original: {originalsize / 1024:.1f} KB")
print(f"Quantized: {quantsize / 1024:.1f} KB")

print(f"Reduction: {(1 - quantsize / originalsize)  100:.1f}%")

# Full integer quantization requires a representative dataset
def representativedataset():
    """Generate representative data for calibration."""
    for  in range(100):

        data = np.random.rand(1, 224, 224, 3).astype(np.float32)
        yield [data]

converter = tf.lite.TFLiteConverter.fromkerasmodel(model)

converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representativedataset = representativedataset


Force full integer quantization
converter.targetspec.supportedops = [

    tf.lite.OpsSet.TFLITEBUILTINSINT8

]
converter.inferenceinputtype = tf.uint8

converter.inferenceoutputtype = tf.uint8


tfliteint8model = converter.convert()


with open('modelint8.tflite', 'wb') as f:
    f.write(tfliteint8model)

print(f"INT8 model size: {len(tfliteint8model) / 1024:.1f} KB")

# Float16 quantization - good balance of size and accuracy
converter = tf.lite.TFLiteConverter.fromkerasmodel(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.targetspec.supportedtypes = [tf.float16]

tflitefp16model = converter.convert()

with open('modelfp16.tflite', 'wb') as f:

    f.write(tflitefp16model)


print(f"FP16 model size: {len(tflitefp16model) / 1024:.1f} KB")

import tensorflowmodeloptimization as tfmot

Start with a pre-trained model
basemodel = buildmodel()

basemodel.fit(traindata, trainlabels, epochs=10)  # Pre-train first

Apply quantization-aware training
quantizemodel = tfmot.quantization.keras.quantizemodel

qatmodel = quantizemodel(basemodel)


qatmodel.compile(
    optimizer=keras.optimizers.Adam(learningrate=1e-4),

    loss='sparsecategoricalcrossentropy',

    metrics=['accuracy']
)

print("QAT Model Summary:")
qatmodel.summary()

Fine-tune with QAT (use lower learning rate, fewer epochs)
qatmodel.fit(traindata, trainlabels, epochs=3, validationsplit=0.1)

Convert QAT model to TFLite
converter = tf.lite.TFLiteConverter.fromkerasmodel(qatmodel)

converter.optimizations = [tf.lite.Optimize.DEFAULT]

qattflitemodel = converter.convert()


with open('modelqat.tflite', 'wb') as f:
    f.write(qattflitemodel)

print(f"QAT model size: {len(qattflitemodel) / 1024:.1f} KB")

# Quantize only specific layers
def applyquantizationtodense(layer):

    if isinstance(layer, keras.layers.Dense):
        return tfmot.quantization.keras.quantizeannotatelayer(layer)

    return layer

annotatedmodel = keras.models.clonemodel(

    basemodel,
    clonefunction=applyquantizationtodense
)

selectiveqatmodel = tfmot.quantization.keras.quantizeapply(

    annotatedmodel
)

selectiveqatmodel.compile(
    optimizer='adam',
    loss='sparsecategoricalcrossentropy',
    metrics=['accuracy']
)

import tensorflowmodeloptimization as tfmot

Apply pruning to reduce model complexity
prunelowmagnitude = tfmot.sparsity.keras.prunelowmagnitude

pruningparams = {

    'pruningschedule': tfmot.sparsity.keras.PolynomialDecay(
        initialsparsity=0.30,

        finalsparsity=0.80,
        beginstep=0,

        endstep=1000
    )
}

prunedmodel = prunelowmagnitude(basemodel, pruningparams)


prunedmodel.compile(
    optimizer='adam',
    loss='sparsecategoricalcrossentropy',
    metrics=['accuracy']
)

Train with pruning callbacks
callbacks = [tfmot.sparsity.keras.UpdatePruningStep()]
prunedmodel.fit(traindata, trainlabels, epochs=5,

callbacks=callbacks)

Strip pruning wrappers and convert
strippedmodel = tfmot.sparsity.keras.strippruning(prunedmodel)
converter = tf.lite.TFLiteConverter.fromkerasmodel(strippedmodel)

converter.optimizations = [tf.lite.Optimize.DEFAULT]
prunedtflite = converter.convert()

print(f"Pruned model size: {len(prunedtflite) / 1024:.1f} KB")

// app/build.gradle
dependencies {
    implementation 'org.tensorflow:tensorflow-lite:2.15.0'
    implementation 'org.tensorflow:tensorflow-lite-support:0.4.4'
    implementation 'org.tensorflow:tensorflow-lite-gpu:2.15.0'
}

android {
    aaptOptions {
        noCompress "tflite"
    }
}

import org.tensorflow.lite.Interpreter
import org.tensorflow.lite.gpu.GpuDelegate
import java.io.FileInputStream
import java.nio.MappedByteBuffer
import java.nio.channels.FileChannel

class ImageClassifier(private val context: Context) {

    private var interpreter: Interpreter? = null
    private val inputSize = 224
    private val numClasses = 10
    private val labels = listOf(
        "airplane", "automobile", "bird", "cat", "deer",
        "dog", "frog", "horse", "ship", "truck"
    )

    fun initialize(useGpu: Boolean = false) {
        val options = Interpreter.Options()

        if (useGpu) {
            val gpuDelegate = GpuDelegate()
            options.addDelegate(gpuDelegate)
        }

        options.setNumThreads(4)

        val model = loadModelFile("modelint8.tflite")
        interpreter = Interpreter(model, options)
    }

    private fun loadModelFile(filename: String): MappedByteBuffer {
        val assetFileDescriptor = context.assets.openFd(filename)
        val inputStream = FileInputStream(assetFileDescriptor.fileDescriptor)
        val fileChannel = inputStream.channel
        val startOffset = assetFileDescriptor.startOffset
        val declaredLength = assetFileDescriptor.declaredLength
        return fileChannel.map(
            FileChannel.MapMode.READONLY,

            startOffset,
            declaredLength
        )
    }

    fun classify(bitmap: Bitmap): List> {
        val resized = Bitmap.createScaledBitmap(
            bitmap, inputSize, inputSize, true
        )

        // Prepare input tensor
        val input = Array(1) {
            Array(inputSize) {
                Array(inputSize) {
                    FloatArray(3)
                }
            }
        }

        for (y in 0 until inputSize) {
            for (x in 0 until inputSize) {
                val pixel = resized.getPixel(x, y)
                input[0][y][x][0] = ((pixel shr 16) and 0xFF) / 255.0f
                input[0][y][x][1] = ((pixel shr 8) and 0xFF) / 255.0f
                input[0][y][x][2] = (pixel and 0xFF) / 255.0f
            }
        }

        // Run inference
        val output = Array(1) { FloatArray(numClasses) }
        interpreter?.run(input, output)

        // Return sorted results
        return labels.zip(output[0].toList())
            .sortedByDescending { it.second }
    }

    fun close() {
        interpreter?.close()
    }
}

// Usage in Activity
class MainActivity : AppCompatActivity() {
    private lateinit var classifier: ImageClassifier

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        classifier = ImageClassifier(this)
        classifier.initialize(useGpu = true)
    }

    private fun classifyImage(bitmap: Bitmap) {
        val results = classifier.classify(bitmap)
        results.take(3).forEach { (label, confidence) ->
            Log.d("TFLite", "$label: ${confidence  100}%")
        }
    }

    override fun onDestroy() {
        super.onDestroy()
        classifier.close()
    }
}

# Convert TFLite model to CoreML for native iOS performance
import coremltools as ct

Option 1: Convert from TensorFlow directly to CoreML
coremlmodel = ct.convert(
    model,
    inputs=[ct.ImageType(
        name="input",
        shape=(1, 224, 224, 3),
        scale=1/255.0
    )],
    classifierconfig=ct.ClassifierConfig(

        ["airplane", "automobile", "bird", "cat", "deer",
         "dog", "frog", "horse", "ship", "truck"]
    )
)

coremlmodel.save("ImageClassifier.mlmodel")

Option 2: Use TFLite directly on iOS with TensorFlowLiteSwift

iOS Swift Code with TFLite

import TensorFlowLite

class ImageClassifier {
    private var interpreter: Interpreter?
    private let inputSize = 224

    init() {
        guard let modelPath = Bundle.main.path(
            forResource: "modelint8",

            ofType: "tflite"
        ) else {
            fatalError("Model not found")
        }

        var options = Interpreter.Options()
        options.threadCount = 4

        // Enable GPU delegate
        var gpuOptions = MetalDelegate.Options()
        gpuOptions.isPrecisionLossAllowed = true
        let metalDelegate = MetalDelegate(options: gpuOptions)

        do {
            interpreter = try Interpreter(
                modelPath: modelPath,
                options: options,
                delegates: [metalDelegate]
            )
            try interpreter?.allocateTensors()
        } catch {
            print("Failed to initialize: \(error)")
        }
    }

    func classify(pixelBuffer: CVPixelBuffer) -> [(String, Float)]? {
        guard let interpreter = interpreter else { return nil }

        do {
            let inputTensor = try interpreter.input(at: 0)
            let inputData = preprocessImage(
                pixelBuffer,
                width: inputSize,
                height: inputSize
            )

            try interpreter.copy(inputData, toInputAt: 0)
            try interpreter.invoke()

            let outputTensor = try interpreter.output(at: 0)
            let results = parseOutput(outputTensor.data)
            return results
        } catch {
            print("Inference error: \(error)")
            return nil
        }
    }
}

# Compile model for Edge TPU
First, ensure you have a fully INT8 quantized model

Install Edge TPU compiler:
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
echo "deb https://packages.cloud.google.com/apt coral-edgetpu-stable main" |
sudo tee /etc/apt/sources.list.d/coral-edgetpu.list
sudo apt update && sudo apt install edgetpu-compiler

Compile: edgetpucompiler modelint8.tflite


Run inference on Edge TPU
from pycoral.adapters import common
from pycoral.adapters import classify
from pycoral.utils.edgetpu import makeinterpreter

def runedgetpuinference(modelpath, image):
    """Run inference on a Coral Edge TPU."""
    # Create interpreter with Edge TPU delegate
    interpreter = makeinterpreter(modelpath)
    interpreter.allocatetensors()


    # Get input details
    inputdetails = interpreter.getinputdetails()[0]
    inputshape = inputdetails['shape']

    # Preprocess image
    imageresized = image.resize(

        (inputshape[2], inputshape[1])

    )
    inputdata = np.expanddims(imageresized, axis=0)

    # Handle quantization
    if inputdetails['dtype'] == np.uint8:

        inputscale, inputzeropoint = inputdetails['quantization']

        inputdata = (inputdata / inputscale + inputzeropoint)
        inputdata = inputdata.astype(np.uint8)

    # Run inference
    common.setinput(interpreter, inputdata)
    interpreter.invoke()

    # Get results
    classes = classify.getclasses(interpreter, topk=5)
    return classes


Benchmark Edge TPU vs CPU
import time

def benchmark(interpreter, inputdata, numruns=100):
    """Benchmark inference latency."""
    # Warmup
    for  in range(10):

        interpreter.invoke()

    times = []
    for  in range(numruns):

        start = time.perfcounter()
        common.setinput(interpreter, inputdata)
        interpreter.invoke()
        elapsed = (time.perfcounter() - start)  1000

        times.append(elapsed)

    return {
        'meanms': np.mean(times),
        'medianms': np.median(times),

        'p95ms': np.percentile(times, 95),
        'p99ms': np.percentile(times, 99),

    }

import time
import numpy as np

def benchmarktflitemodel(modelpath, inputshape,

                            numwarmup=10, numruns=100):

    """Comprehensive benchmark for a TFLite model."""
    import os

    modelsize = os.path.getsize(modelpath)


    interpreter = tf.lite.Interpreter(modelpath=modelpath)

    interpreter.allocatetensors()

    inputdetails = interpreter.getinputdetails()

    outputdetails = interpreter.getoutputdetails()

    # Generate random input
    inputdata = np.random.rand(inputshape).astype(
        inputdetails[0]['dtype']

    )

    # Warmup
    for  in range(numwarmup):

        interpreter.settensor(inputdetails[0]['index'], inputdata)
        interpreter.invoke()

    # Benchmark
    latencies = []
    for  in range(numruns):
        start = time.perfcounter()

        interpreter.settensor(inputdetails[0]['index'], inputdata)
        interpreter.invoke()
        end = time.perfcounter()

        latencies.append((end - start)  1000)


    results = {
        'modelsizekb': modelsize / 1024,
        'meanlatencyms': np.mean(latencies),
        'medianlatencyms': np.median(latencies),
        'p95latencyms': np.percentile(latencies, 95),
        'p99latencyms': np.percentile(latencies, 99),
        'minlatencyms': np.min(latencies),
        'maxlatencyms': np.max(latencies),
        'throughputfps': 1000.0 / np.mean(latencies),

    }

    return results


Compare all model variants
models = {
    'Original (FP32)': 'model.tflite',
    'Dynamic Quant': 'modeldynamicquant.tflite',

    'INT8': 'modelint8.tflite',
    'FP16': 'modelfp16.tflite',

    'QAT': 'modelqat.tflite',
}

print(f"{'Model':<20} {'Size (KB)':<12} {'Latency (ms)':<14} {'FPS':<8}")
print("-"  54)

for name, path in models.items():
    try:
        results = benchmarktflitemodel(
            path, inputshape=(1, 224, 224, 3)

        )
        print(f"{name:<20} {results['modelsizekb']:<12.1f} "

              f"{results['meanlatencyms']:<14.2f} "

              f"{results['throughputfps']:<8.1f}")
    except Exception as e:
        print(f"{name:<20} Error: {e}")

Using TFLite Benchmark Tool

# Download the benchmark tool
Android:
adb push benchmarkmodel /data/local/tmp/

adb push model.tflite /data/local/tmp/

Run benchmark on Android device
adb shell /data/local/tmp/benchmarkmodel \
--graph=/data/local/tmp/model.tflite \
--numthreads=4 \

--numruns=50 \
--warmupruns=10 \

--usegpu=true

Best Practices

Conclusion

TensorFlow Lite bridges the gap between training powerful models and running them where they matter most: on user devices. The conversion pipeline from TensorFlow and PyTorch is well-established, quantization techniques can reduce model size by 75% with minimal accuracy loss, and deployment paths exist for Android, iOS, and edge hardware. By combining post-training quantization for quick wins with QAT for maximum accuracy retention, and by benchmarking rigorously on target devices, you can deliver ML-powered features that work offline, respond instantly, and respect user privacy.