140.140. Mixup Augmentation

Mixup Augmentation

Implement Mixup augmentation, a data augmentation technique that creates new training samples by taking **weighted linear combinations** of pairs of existing training images and their labels. Mixup encourages the model to produce linear interpolations between classes, leading to smoother decision boundaries and better generalization.

Formula:

x_mixed = lambda * x1 + (1 - lambda) * x2

y_mixed = lambda * y1 + (1 - lambda) * y2

Where:

x1, x2 = two training images

y1, y2 = their corresponding labels (one-hot encoded)

lambda = mixing ratio, sampled from Beta(alpha, alpha)

alpha = hyperparameter controlling mixing strength (typically 0.2-0.4)

Example:

Input:

image1 = [[1, 2], image2 = [[5, 6], ratio = 0.6

[3, 4]] [7, 8]]

Computation:

mixed = 0.6 * image1 + 0.4 * image2

= 0.6 * [[1,2],[3,4]] + 0.4 * [[5,6],[7,8]]

= [[0.6, 1.2], + [[2.0, 2.4],

[1.8, 2.4]] [2.8, 3.2]]

= [[2.6, 3.6],

[4.6, 5.6]]

If labels were: y1 = [1, 0, 0] (cat), y2 = [0, 1, 0] (dog)

y_mixed = 0.6 * [1,0,0] + 0.4 * [0,1,0] = [0.6, 0.4, 0.0]

**Explanation:** Mixup (Zhang et al., 2018) is a simple yet powerful regularization technique. By training on convex combinations of image pairs, the model learns smoother representations and becomes less prone to memorizing individual training examples. The soft labels (e.g., 60% cat + 40% dog) provide richer learning signals than hard labels. Mixup has been shown to improve model calibration, reduce adversarial vulnerability, and stabilize GAN training.

Constraints: