Vq Pocket Lab

Verified project card

# VQ-Pocket

VQ-Pocket is a discrete-latent generative model for handwritten digits. A compact
VQ-VAE compresses each 8x8 image into sixteen tokens from a 32-entry visual
codebook. A class-conditional GRU then models those tokens autoregressively.

The codebook is initialized from warm-started encoder activations with k-means
before joint vector-quantized training. This avoids presenting a collapsed
codebook as a successful discrete representation. The report includes active-code
count, codebook perplexity, reconstruction fidelity, generation fidelity, and
unique token-sequence rate.

## Verified local result

The 14,193-parameter VQ-VAE used every one of its 32 codes and reached an
effective codebook perplexity of 22.72. Its held-out reconstruction MSE was
0.00673 with 95.93% frozen-judge fidelity. The 28,512-parameter conditional prior
generated 1,000 samples at 95.6% class fidelity with 99.6% unique code sequences.

## Reproduce

```bash
uv run python projects/vq-pocket/train.py
uv run pytest tests/test_vq_pocket.py
```

The model bundle contains both SafeTensors checkpoints and measured evaluation
artifacts. The Space displays decoded samples beside the 4x4 token map that
generated them. Local artifacts do not imply that the Hub repositories are live.

Evaluation snapshot

{
  "model": "VQ-Pocket",
  "method": "VQ-VAE with a class-conditional autoregressive latent-token prior",
  "autoencoder_parameters": 14193,
  "prior_parameters": 28512,
  "codebook_size": 32,
  "tokens_per_image": 16,
  "best_vq_epoch": 98,
  "best_prior_epoch": 178,
  "prior_training_nll": 0.7916229416096884,
  "reconstruction": {
    "pixel_mse": 0.006734587252140045,
    "judge_accuracy": 0.9592592592592593,
    "active_codes": 32,
    "codebook_size": 32,
    "codebook_perplexity": 22.71870231628418,
    "examples": 270
  },
  "generation": {
    "judge_accuracy": 0.9559999704360962,
    "judge_accuracy_by_class": {
      "0": 0.9800000190734863,
      "1": 0.9700000286102295,
      "2": 0.949999988079071,
      "3": 0.9599999785423279,
      "4": 0.9200000166893005,
      "5": 0.9700000286102295,
      "6": 0.9800000190734863,
      "7": 0.9800000190734863,
      "8": 0.9300000071525574,
      "9": 0.9200000166893005
    },
    "mean_pixel_variance_by_class": {
      "0": 0.021453002467751503,
      "1": 0.05069272592663765,
      "2": 0.038569375872612,
      "3": 0.03150773420929909,
      "4": 0.03649989888072014,
      "5": 0.03785640373826027,
      "6": 0.023341484367847443,
      "7": 0.035609740763902664,
      "8": 0.03689037263393402,
      "9": 0.03489069640636444
    },
    "unique_code_sequence_fraction_by_class": {
      "0": 0.99,
      "1": 1.0,
      "2": 1.0,
      "3": 1.0,
      "4": 1.0,
      "5": 1.0,
      "6": 0.98,
      "7": 1.0,
      "8": 0.99,
      "9": 1.0
    },
    "mean_unique_code_sequence_fraction": 0.9960000000000001,
    "samples": 1000,
    "sampling_temperature": 0.85
  },
  "judge": "Frozen Tiny Vision student, 98.52% real-image test accuracy"
}

Backed-up artifact tree

• README.md
• __pycache__/app.cpython-311.pyc
• __pycache__/model.cpython-311.pyc
• app.py
• artifacts/vq-pocket/code_prior.safetensors
• artifacts/vq-pocket/evaluation.json
• artifacts/vq-pocket/samples.png
• artifacts/vq-pocket/vq_vae.safetensors
• data/generated_token_sequences.parquet
• model.py
• requirements.txt
• train.py