Documentation

1. The core idea

Text-to-image diffusion models learn the mapping caption → pixels from millions of (image, caption) pairs. NNGen applies the same machinery one level up: it learns the mapping caption → neural network weights.

The “image” being denoised is the weight canvas: the parameter set of a fixed-size controller MLP,

    8 inputs → 32 tanh → 32 tanh → 8 outputs   (~1,608 parameters)

laid out as a 6×32×32 tensor (weight matrices as channels, biases in the first column, zeros padded, a validity mask marking real entries). A static mask is fed to the UNet as extra input channels; the training loss and the final decode are masked to valid entries. Weights are z-normalized per tensor with corpus statistics and clamped to ±4σ so the diffusion process operates in roughly [-1, 1] — the same regime as image pixels.

2. The synthetic corpus

There is no public dataset of (network, description) pairs, so NNGen manufactures its own:

1. For each task family, a batched, fully differentiable mini-environment simulates the task (chase, evasion, cart-pole, paddle-ball, leg animation, orbital flight, turtle-graphics drawing).
2. Hundreds of controller MLPs are trained in parallel through the dynamics with backprop-through-time (a short-to-full episode horizon ramp stabilizes training). A fresh environment seed every optimizer step prevents memorizing a single episode.
3. Every trained network is evaluated on a held-out episode; only successful ones join the corpus (quality is recorded and reflected in the caption).
4. Template-based English captions describe the behavior and its quality; a frozen MiniLM encoder caches token embeddings for the UNet's cross-attention.

For the drawing demo, the corpus contains eight shapes — circle, square, triangle, pentagon, star, spiral, line, arc — while hexagon and octagon are deliberately excluded from training. They exist only as evaluation targets: if the generator produces working hexagon-drawers, it has generalized the concept of “regular polygon” in weight space.

3. The diffusion model — standard techniques, novel target

NNGen uses the same techniques introduced for image generation, with one conceptual substitution: the data distribution being learned is not natural images but the weight space of competent controllers. Diffusion models only ever manipulate matrices of numbers — and a weight matrix is exactly that. No new generative machinery had to be invented; what changed is what the matrix means and how the corpus that teaches its meaning is produced.

Training loss is the masked v-prediction MSE; an EMA of the weights is kept for sampling. At generation time the UNet receives pure noise on the 6×32×32 canvas plus the prompt embedding, and removes noise step by step until a coherent set of network parameters emerges — then the canvas is decoded back into a standard nn.Linear MLP and executed.

4. Running the pipeline

pip install -r requirements.txt

# 1. mass-train controller networks for the drawing corpus
python -m dnn.corpus.train_controllers \
    --tasks draw_circle,draw_square,draw_triangle,draw_pentagon,draw_star,draw_spiral,draw_line,draw_arc \
    --per-task 300 --out data/corpus_draw

# 2. cache text embeddings
python -m dnn.corpus.embed_captions --corpus data/corpus_draw

# 3. train the generator
python -m dnn.train_diffusion --config mvp --corpus data/corpus_draw --run runs/draw_mvp

# 4. generate a network from a prompt (CLI)
python -m dnn.generate --prompt "..." --task draw_hexagon --ckpt runs/draw_mvp/ckpt.pt --out runs/demo

# 5. this website
python webapp/server.py --port 17656

5. HTTP API

All URLs are relative, so the site can be mounted behind a reverse proxy at any sub-path (e.g. nngen.agrillo.it).

6. Repository layout

dnn/
  controller.py        canonical MLP, weight canvas, normalization
  text_encoder.py      frozen MiniLM wrapper
  tasks/               mini-environments + draw shapes, registry
  corpus/              caption templates, batch trainer, storage, embeddings
  diffusion/           schedule, UNet, DDIM sampler + CFG, EMA
  train_diffusion.py   training loop (EMA, checkpoints, resume)
  generate.py          prompt → network → rollout → artifacts
  eval.py              generated vs teacher vs random benchmark
webapp/
  server.py            this website (stdlib HTTP server, port 17656)
  pages/, static/      pages and assets
tests/                 canvas round-trip, tasks, diffusion, e2e pipeline