Advanced

FastAI Best Practices

Master the learning rate finder, mixed precision training, FastAI's callback system, model export, and deployment to production.

The Learning Rate Finder

The learning rate is the most important hyperparameter. FastAI's lr_find() systematically tests learning rates to help you find the optimal value:

Python

# Run the learning rate finder
learn.lr_find()

# This plots loss vs learning rate
# Pick a learning rate where:
# - Loss is still decreasing (not yet diverging)
# - Typically 1 order of magnitude before the minimum

# Use the suggested learning rate
learn.fit_one_cycle(5, 3e-3)

# For fine-tuning, use discriminative learning rates
learn.fine_tune(5, base_lr=1e-3, freeze_epochs=2)

Reading the LR Plot: Look for the steepest downward slope. The ideal learning rate is usually about 10x smaller than the rate at the minimum loss. For example, if the minimum is at 1e-1, try 1e-2 or 3e-3.

One-Cycle Training

FastAI's fit_one_cycle() implements the one-cycle policy: the learning rate warms up then decays, achieving faster convergence and better generalization than constant learning rates:

Python

# One-cycle: LR goes up then down, momentum goes down then up
learn.fit_one_cycle(
    n_epoch=10,
    lr_max=3e-3,
    div=25.0,         # Start LR = lr_max/25
    div_final=1e5,    # End LR = lr_max/1e5
    pct_start=0.3,    # 30% warmup, 70% decay
)

Mixed Precision Training

Python

# Enable mixed precision (fp16) - one line!
learn = vision_learner(dls, resnet50, metrics=accuracy).to_fp16()

# Train as usual - 2x faster with half the memory
learn.fine_tune(5)

Callbacks

FastAI's callback system lets you hook into every event in the training loop:

Python

# Built-in callbacks
learn.fit_one_cycle(5, cbs=[
    EarlyStoppingCallback(monitor='valid_loss', patience=3),
    SaveModelCallback(monitor='accuracy', fname='best_model'),
    CSVLogger(),  # Log metrics to CSV
])

# Custom callback
class PrintLossCallback(Callback):
    def after_batch(self):
        if self.iter % 100 == 0:
            print(f"Batch {self.iter}, Loss: {self.loss:.4f}")

Model Export and Deployment

Python

# Export the model (includes preprocessing pipeline)
learn.export('model.pkl')

# Load and use in production
learn_inf = load_learner('model.pkl')
pred, idx, probs = learn_inf.predict('path/to/image.jpg')

# Deploy as a simple web app with Gradio
import gradio as gr

def classify_image(img):
    pred, idx, probs = learn_inf.predict(img)
    return dict(zip(learn_inf.dls.vocab, map(float, probs)))

iface = gr.Interface(fn=classify_image, inputs=gr.Image(), outputs=gr.Label())
iface.launch()

Quick Reference

Technique	Code	Impact
LR Finder	`learn.lr_find()`	Find optimal learning rate
Mixed Precision	`learn.to_fp16()`	2x faster, half memory
Progressive Resizing	Start small, increase image size	Faster training, better results
Test Time Augmentation	`learn.tta()`	Better predictions at inference
Discriminative LR	`learn.fit(lr=slice(1e-5, 1e-3))`	Different LR per layer group
Gradient Accumulation	`GradientAccumulation(n_acc=4)`	Simulate larger batch sizes

Course Complete!

You now know how to use FastAI for vision, tabular, and NLP tasks. Continue exploring with Hugging Face Transformers for cutting-edge NLP, or dive deeper into PyTorch to understand what FastAI does under the hood.

Next Course: HF Transformers →

← NLP Course Overview →