MNIST Example

A classic example of MNIST digit classification using phitodeep:

Imports

import numpy as np
from datasets import load_dataset

from phitodeep.model import SequentialBuilder
from phitodeep.loss import CategoricalCrossEntropy
from phitodeep.optimization.optimizers import Adam
from phitodeep.optimization.initialization import Xavier, InitType

Load MNIST dataset

Using the datasets library from Hugging Face, we can easily load the MNIST dataset:

train_dataset = load_dataset("ylecun/mnist", split="train")
test_dataset = load_dataset("ylecun/mnist", split="test")

X_train = train_dataset["image"]
y_train = train_dataset["label"]
X_test = test_dataset["image"]
y_test = test_dataset["label"]

X_train = np.array(X_train).astype(np.float32) / 255.0
y_train = np.array(y_train)
X_test = np.array(X_test).astype(np.float32) / 255.0
y_test = np.array(y_test)
print(X_train.shape, y_train.shape)

Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.

(60000, 28, 28) (60000,)

Model Definition

model = (
    SequentialBuilder()
    .flatten()
    .dense(784, 128, Xavier(InitType.NORMAL))
    .relu()
    .dense(128, 64, Xavier(InitType.NORMAL))
    .relu()
    .dense(64, 10, Xavier(InitType.NORMAL))
    .softmax()
    .optimizer(Adam())
    .loss(CategoricalCrossEntropy())
    .alpha(0.05)
    .epochs(5)
    .batch(64)
    .build()
)

model.summary()

Model Summary:
------------------------------------------------------------
Optimizer: Adam | Learning Rate: 0.05 | Batch Size: 64 
Epochs: 5 | Loss: CategoricalCrossEntropy
------------------------------------------------------------
Layer 0: FLATTEN   
Layer 1: DENSE      | Input: 784   Output: 128  
Layer 2: RELU      
Layer 3: DENSE      | Input: 128   Output: 64   
Layer 4: RELU      
Layer 5: DENSE      | Input: 64    Output: 10   
Layer 6: SOFTMAX   
------------------------------------------------------------

Training

First we check prediction accuracy before training, then we train the model for 5 epochs, and finally check the accuracy again to see the improvement.

y_pred = model.predict(X_test)
accuracy = np.mean(np.argmax(y_pred, axis=1) == y_test) * 100
print(f"Test Accuracy: {accuracy:.4f} %")

Test Accuracy: 8.5600 %

losses = model.train(X_train, y_train, X_test, y_test)

Epoch 0, Loss: 0.1575, Test Loss: 0.1850

Training complete.
------------------------------------------------------------
Starting Training Loss: 0.1575 | Starting Test Loss: 0.1850
Final Training Loss: 0.0950 | Final Test Loss: 0.1599
Training Loss Improvement: 0.0624 | Test Loss Improvement: 0.0251
------------------------------------------------------------

y_pred = model.predict(X_test)
accuracy = np.mean(np.argmax(y_pred, axis=1) == y_test) * 100
print(f"Test Accuracy: {accuracy:.4f} %")

Test Accuracy: 96.3300 %