DocumentationRogues' Gallery Competition
← Back to competition
In this guide
FROM FIRST SKETCH TO FIRST SUBMISSION

Building a competition entry

Your model draws the game's prompts. It is rendered through the same tracer and renderer the human drawings go through, then shown in the quiz. Your score is the share of players who failed to pick it out.

Quickstart

  1. Download the starter kit and unzip it.
  2. Pick a track: ONNX or model.py.
  3. Copy the matching example and replace its interior.
  4. Run the self-test until it says ACCEPTED, and look at the preview PNG.
  5. Sign in with GitHub and submit.
# from inside the unzipped kit
pip install numpy pillow safetensors onnxruntime
python competition/selftest.py --dir path/to/your/entry
The self-test is free; a submission is not. It runs the same checks the server runs, so if it passes locally it will not be rejected remotely. You get three submissions a day.

The contract

Every entry, on either track, accepts the same two inputs and returns the same one output. Nothing about what happens in between is constrained.

NameTypeShapeMeaning
prompt_embedfloat32(1, T, 3584) The caption, already embedded by a frozen text encoder. T varies with caption length, so do not hard-code it.
noisefloat32(1, 16, 64, 64) Your only source of randomness. Latent-shaped by convention; a model with no VAE may reshape or ignore the layout.
imagefloat32(512, 512) The drawing. 0.0 is black ink, 1.0 is white paper. (1, 512, 512) is also accepted and squeezed.

Why the embedding rather than the text

Every entrant is handed the same vector from the same frozen encoder. You do not ship a tokeniser and you do not ship a text encoder. That removes a large confound — whose text encoder was better — and keeps submissions small enough to validate quickly.

Why noise is passed in

So determinism is checkable. Your model is run twice on identical inputs and rejected if the two images differ. That is what lets a leaderboard placing be reproduced and audited later, rather than taken on trust.

Choosing a track

Track A — ONNXTrack B — model.py
You shipdrawer.onnx model.py + weights.safetensors
Built withanything that exports to ONNXanything Python runs
Runs asa frozen graph in onnxruntimeyour code, in a sealed container
Good forfeedforward models, unrolled loops runtime branching, custom sampling, anything awkward to export
Main frictiongetting the export to workno network, fixed package list

They score identically. Pick whichever gets your idea running.

Track A — ONNX

Export a frozen computation graph. The architecture travels inside the file, so any design is allowed — the runtime executes standard operators rather than your code.

import torch
from spec import EMBED_DIM, NOISE_SHAPE, IN_EMBED, IN_NOISE, OUT_IMAGE

model = MyDrawer().eval()
embed = torch.randn(1, 77, EMBED_DIM)
noise = torch.randn(*NOISE_SHAPE)

torch.onnx.export(
    model, (embed, noise), "drawer.onnx",
    input_names=[IN_EMBED, IN_NOISE], output_names=[OUT_IMAGE],
    # only the token axis varies; batch stays 1 so the contract is exact
    dynamic_axes={IN_EMBED: {1: "tokens"}},
    opset_version=20, dynamo=False,
)

Your sampling loop goes inside the graph

If you build a diffusion model, export the whole denoising loop, not one step. The simplest way is a plain Python for in forward() — tracing unrolls it into the graph. ONNX also has Loop/Scan, but unrolling is easier and usually faster.

Why the harness does not run the loop for you. A denoiser-shaped harness would privilege diffusion and make a GAN or an autoregressive stroke model a second-class citizen. Keeping the contract at one call per image means every architecture is expressed the same way. Inference budget is levelled by a wall-clock limit instead.

Export gotchas

Track B — model.py

Ship Python and weights. Your file must define a class named exactly Drawer that subclasses RoguesDrawer.

import numpy as np
from base_drawer import RoguesDrawer

class Drawer(RoguesDrawer):
    def load(self, weights):
        self.proj = weights["proj"]

    def draw(self, prompt_embed, noise):
        # ... your model ...
        return img  # (512, 512) float32 in [0,1], 0 = ink

Packaging your weights

from safetensors.numpy import save_file
save_file({"proj": proj, "bias": bias}, "weights.safetensors")

We read the file and hand you the arrays, so your code never parses an untrusted file and you never have to care where it was stored.

What the sandbox allows

AvailableNot available
numpy, torch (CPU), pillow, safetensors, the standard library network access of any kind, writing outside the output directory, installing packages at runtime
Fetching a checkpoint at runtime will not fail cleanly. With networking disabled the request hangs until the wall-clock budget expires, and the entry is rejected for being slow rather than for the real reason. Put everything in your weights file.

RoguesDrawer

The base class for track B, defined in base_drawer.py. Deliberately not an nn.Module: the interior is yours, and requiring a torch base class would exclude a perfectly valid JAX or NumPy entry.

load(weights)abstract

weights — dict[str, np.ndarray]. Every tensor from your safetensors file, keyed by the names you saved them under. Empty if you shipped no weights file.

Called exactly once, before any call to draw. Do setup here rather than in draw: the time budget is per image, and construction work repeated per call spends it for nothing.

draw(prompt_embed, noise) → np.ndarrayabstract

prompt_embed — float32 (1, T, 3584). T varies with caption length.
noise — float32 (1, 16, 64, 64). Your only entropy.
returns — float32 (512, 512) in [0, 1], 0 = black ink. (1, 512, 512) is accepted and squeezed.

Must be deterministic given identical arguments.

Class attributes

AttributeValue
EMBED_DIM3584
NOISE_SHAPE(1, 16, 64, 64)
IMAGE_SIZE512

spec constants

From spec.py. Import these rather than hard-coding numbers — if a limit ever changes, code that imported it keeps working.

ConstantValueMeaning
IMAGE_SIZE512canvas, square
INK, PAPER0.0, 1.0output polarity
EMBED_DIM3584text embedding width
NOISE_SHAPE(1, 16, 64, 64)entropy handed in
MAX_FILE_BYTES4 GBper submission
MAX_SECONDS_PER_IMAGE120.0wall clock, per image
MAX_OPSET21ONNX only
DETERMINISM_TOL1e-4max pixel drift between runs

Validation functions

You can call these yourself while developing; the server calls the same ones.

validate_signature(session) → list[Problem]

Checks an onnxruntime session's declared inputs and outputs against the contract. Runs before any inference, so a wrong signature costs milliseconds.

validate_graph(model) → list[Problem]

Static checks on an ONNX graph before it is executed: opset version, custom operator domains, forbidden operators.

check_output(array) → list[Problem]

Checks one generated canvas: shape, finite values, range, and the degenerate cases — a blank page and a solid page are both rejected.

Problem(field, detail)dataclass

One reason an entry cannot be scored. str(problem) gives "field: detail". An empty list means it passed.

Why blank and solid are rejected explicitly. A blank canvas scores perfectly on every sparsity metric this project uses, and a solid one satisfies every shape and range check. Both would otherwise pass as "valid" while being unusable.

Self-test

python competition/selftest.py --dir path/to/your/entry

Detects your track automatically from what is in the directory, runs every check the server runs, and writes preview.png next to your files.

Look at the preview. Every check here can pass on a picture no human would mistake for a doodle. The checks verify the format; only your eyes verify the drawing.

Validation stages

Ordered so that anything which can reject cheaply runs before anything expensive. A broken config is answered in about a second, before your weights are even downloaded.

  1. Files — the expected files are present and nothing else is.
  2. Contract — signature (ONNX) or class Drawer present (Python).
  3. Weights load — tensors match what the architecture expects.
  4. Draw — one image per competition prompt, each contract-checked.
  5. Determinism — a prompt is re-run and compared to its own earlier output.
  6. Trace and render — strokes extracted and redrawn through the game's renderer.

Rejection reasons

MessageWhat to do
no input named 'noise' Your forward() never reads it, so the exporter pruned it. Use the noise.
last dim must be 3584 Your embedding input is the wrong width. Import EMBED_DIM.
expected [1, 512, 512] Wrong output shape. Return the full canvas.
must lie in [0.0, 1.0] Clamp or sigmoid your output. Values outside the range are not clipped for you.
effectively blank / effectively solid Ink fraction under 0.002 or over 0.60. The model is not drawing.
identical inputs gave different images Something is sampling outside noise — dropout left on, or a global RNG.
opset N is newer than the runtime's 21 Re-export with opset_version=21 or lower.
custom modelling code is never loaded Your repo carries an auto_map. Use a whitelisted architecture or track B.
Ns per image exceeds the 120s limit Fewer sampling steps, or a smaller model. Check you are not fetching anything.

Quota and scoring

Three submissions per day

Counted per GitHub account, resetting at midnight UTC. Rejected entries do not count — only runs that reached the GPU and produced images. Losing a third of your day to a typo would punish exactly the people still learning the format.

How you are scored

Your images enter the quiz alongside human drawings. Your score is the share of players who failed to identify yours as machine-made — the fool rate.

Ranking uses the Wilson lower bound, not the raw rate. Picking the best of many noisy estimates selects for luck: an entry that happens to get a few favourable votes early would outrank a genuinely better one. Ranking on the pessimistic bound means an entry only climbs when even its unlucky estimate is high. Entries need 50 votes before they are ranked at all.