Deploying Deep Learning Models

Training a deep learning model is only half the challenge. To create real-world impact, models must be deployed — packaged, optimised, and served so that applications and users can make predictions in production.

---

From Training to Production

Stage	Description	Tools
Training	Train and validate the model	PyTorch, TensorFlow, GPUs
Export	Convert the model to a deployment-friendly format	ONNX, TorchScript, SavedModel
Optimise	Reduce model size and latency	Quantisation, pruning, distillation
Serve	Host the model behind an API or embed it	FastAPI, TorchServe, TF Serving
Monitor	Track performance, data drift, and errors	MLflow, Prometheus, Grafana

---

Exporting Models

TorchScript (PyTorch)

TorchScript converts a PyTorch model to a format that can run without Python.

import torch

# Method 1: Tracing (works for models without control flow)
model.eval()
example_input = torch.randn(1, 3, 224, 224)
traced_model = torch.jit.trace(model, example_input)
traced_model.save('model_traced.pt')

# Method 2: Scripting (handles control flow like if/else)
scripted_model = torch.jit.script(model)
scripted_model.save('model_scripted.pt')

# Load in any environment (no Python needed)
loaded = torch.jit.load('model_traced.pt')
output = loaded(example_input)