{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "# Rogues' Gallery: Conditional VAE\n**Your goal:** run a baseline, change one thing, and compare drawings using the same seeds.\n\nAn encoder compresses a drawing and a decoder reconstructs it. New drawings use a standard normal latent prior; reconstruction can be much stronger than new samples.\n\n**Scope:** this workshop generates four single-object categories at 32\u00d732. The full competition asks for 512\u00d7512 scene drawings conditioned on text. These checkpoints are classroom experiments, not upload-ready competition entries. Low-resolution previews do not use the game's final renderer.\n\n**Suggested 20-minute path:** setup and baseline (5 min), experiment (10 min), comparison and export (5 min). Runtime depends on your hardware; use the supplied weights for an immediate starting point.\n\nIn Colab choose **Runtime \u2192 Change runtime type \u2192 GPU**. CPU also works at reduced speed. Run cells in order. Dependencies are torch, NumPy, Pillow, matplotlib and safetensors. No API keys or paid inference services are needed.\n\n**Experimental baseline:** trained on binary images, but new samples can be weak. Use this to explore model limitations. This checkpoint is not marked visually approved.\n\n**Colab setup:** download this notebook and **vae-assets.zip**. Upload the notebook with File \u2192 Upload notebook. Select Runtime \u2192 Change runtime type \u2192 T4 GPU. Open the Files sidebar (folder icon on the left), click Upload, and select **vae-assets.zip**. Wait for the upload to finish, then run the cells in order. Keep the ZIP compressed.\n",
   "id": "cell-00"
  },
  {
   "cell_type": "code",
   "metadata": {
    "tags": [
     "setup"
    ]
   },
   "source": "import importlib.util, subprocess, sys\nmissing = [package for package, module in [('torch','torch'),('numpy','numpy'),('pillow','PIL'),('matplotlib','matplotlib'),('safetensors','safetensors')] if importlib.util.find_spec(module) is None]\nif missing:\n    subprocess.check_call([sys.executable, '-m', 'pip', 'install', *missing])\nfrom pathlib import Path\nASSETS = Path('rogues_classroom_assets')\nASSETS.mkdir(exist_ok=True)\n",
   "id": "cell-01",
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "metadata": {
    "tags": [
     "assets"
    ]
   },
   "source": "# Upload vae-assets.zip using Colab's Files sidebar (folder icon), then run this cell.\nimport hashlib, zipfile\nBASELINE_RECORD = {'kind': 'vae', 'phase': 'codec', 'steps': 60000, 'config': {'width': 64, 'latent': 64}, 'lr': 0.0005, 'beta': 1.0, 'batch': 64, 'device': 'mps', 'seed': 0, 'data_sha256': '92971541486989767b5a8d98805c5cdc86b030bc979ce021c1772ca37cdeafdc', 'raster_version': 'binary-direct-32-v3', 'source_sha256': '75484e2bdf88a8572948b33808770aba533ed9b8a7923319cc0f3707d4760e81', 'source_snapshot': 'runs/classroom-binary-large/vae/source/75484e2bdf88', 'visual_approved': False, 'ema_decay': None, 'diffusion_objective': None, 'total_training_steps': 60000, 'sampling': {'seed': 42, 'independent_seed': 2026, 'temperature': 0.65, 'sampling_steps': 60, 'threshold': 0.5}, 'training_phases': [{'from_step': 0, 'lr': 0.0005, 'beta': 0.2, 'batch': 64}, {'from_step': 40000, 'lr': 0.0005, 'beta': 1.0, 'batch': 64}]}\nasset_archive = Path('vae-assets.zip')\nif not asset_archive.exists():\n    raise FileNotFoundError(\"Upload vae-assets.zip in the Files sidebar, then run this cell again.\")\nif asset_archive.stat().st_size != 11311004:\n    raise ValueError('Asset upload is incomplete. Wait for the Files sidebar upload to finish, then run again.')\nwith zipfile.ZipFile(asset_archive) as archive:\n    for name, expected in {'data.npz': '92971541486989767b5a8d98805c5cdc86b030bc979ce021c1772ca37cdeafdc', 'baseline.safetensors': '8216f7baf8cfe12f1ef6f6a8c99681d335702624df79ffb85d5ded6883328f18'}.items():\n        content = archive.read(name)\n        if hashlib.sha256(content).hexdigest() != expected:\n            raise ValueError('Wrong asset archive: download the matching ZIP for this notebook.')\n        (ASSETS / name).write_bytes(content)\nprint('Data and pretrained weights loaded and verified.')\n",
   "id": "cell-02",
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "## 1. Data\nFour categories: apple, cat, fish, flower. We split original drawings before training. Held-out examples never enter the training batches. Input pixels are exactly 0 or 1: strokes render directly at 32\u00d732 without resizing, interpolation or antialiasing. Generated probability maps display at a fixed 0.5 threshold; the raw values remain available for diagnosis.\n\nThe next cell contains the supplied loader, training loop, samplers, and plotting utilities. You can inspect it, but you do not need to edit it.",
   "id": "cell-03"
  },
  {
   "cell_type": "code",
   "metadata": {
    "tags": [
     "utilities",
     "hide-input"
    ],
    "cellView": "form"
   },
   "source": "#@title Supplied setup (expand to inspect)\n\"\"\"Data, training and paired visual evaluation shared by classroom notebooks.\"\"\"\nimport copy\nimport math\nimport io\nimport json\nimport random\nimport time\nimport urllib.request\nfrom pathlib import Path\nimport numpy as np\nfrom PIL import Image, ImageDraw\nimport torch\nfrom torch.nn import functional as F\nimport matplotlib.pyplot as plt\n\nCATEGORIES=['apple','cat','fish','flower']\nBOS,SEP,EOS,PAD=32,33,34,35\nRASTER_VERSION = 'binary-direct-32-v3'\nDIFFUSION_OBJECTIVE = 'velocity-v1'\nGAN_OBJECTIVE = 'spectral-hinge-v1'\n\ndef seed_all(seed=0):\n    random.seed(seed); np.random.seed(seed); torch.manual_seed(seed)\n    if torch.cuda.is_available(): torch.cuda.manual_seed_all(seed)\n\ndef device_name():\n    return 'cuda' if torch.cuda.is_available() else ('mps' if torch.backends.mps.is_available() else 'cpu')\n\ndef prepare_data(source=None, per_category=500):\n    \"\"\"First recognized records, fixed split by original drawing; no augmentation leakage.\"\"\"\n    images=[]; labels=[]; sequences=[]; keys=[]\n    for label,category in enumerate(CATEGORIES):\n        path=Path(source)/f'{category}.ndjson' if source else None\n        if path and path.exists(): stream=path.open('rb')\n        else:\n            url='https://storage.googleapis.com/quickdraw_dataset/full/simplified/'+category+'.ndjson'\n            stream=urllib.request.urlopen(url,timeout=60)\n        count=0\n        with stream:\n            for line in stream:\n                row=json.loads(line)\n                if not row.get('recognized',True): continue\n                strokes=row['drawing']\n                image=Image.new('L',(32,32),0); draw=ImageDraw.Draw(image)\n                tokens=[BOS]\n                for xs,ys in strokes:\n                    points=list(zip(xs,ys))\n                    if len(points)<2: continue\n                    quantized=[]\n                    # Keep the original RDP vertices; round to the classroom grid.\n                    previous=None\n                    for x,y in points:\n                        xy=(min(31,max(0,round(x*31/255))),min(31,max(0,round(y*31/255))))\n                        if xy!=previous:\n                            tokens.extend(xy); quantized.append(xy); previous=xy\n                    if len(quantized)>1: draw.line(quantized,fill=255,width=1)\n                    elif len(quantized)==1: draw.point(quantized[0],fill=255)\n                    tokens.append(SEP)\n                tokens.append(EOS)\n                if len(tokens)>128: continue # never teach a truncated unfinished drawing\n                images.append(np.asarray(image,dtype=np.float32)/255)\n                labels.append(label); sequences.append(tokens+[PAD]*(128-len(tokens)))\n                keys.append(str(row.get('key_id',f'{category}-{count}')))\n                count+=1\n                if count>=per_category: break\n        if count<10: raise ValueError(f'Too few drawings for {category}: {count}')\n    rng=np.random.default_rng(1729); train=[]; val=[]\n    labels=np.asarray(labels)\n    for c in range(len(CATEGORIES)):\n        ids=np.flatnonzero(labels==c); rng.shuffle(ids); n=max(1,len(ids)//5)\n        val.extend(ids[:n]);train.extend(ids[n:])\n    return dict(images=np.asarray(images)[:,None],labels=labels,tokens=np.asarray(sequences),\n                train=np.asarray(train),val=np.asarray(val),keys=np.asarray(keys),raster_version=np.asarray(RASTER_VERSION))\n\ndef load_data(path):\n    with np.load(path,allow_pickle=False) as z: data={k:z[k] for k in z.files}\n    if str(data.get('raster_version','')) != RASTER_VERSION:\n        raise ValueError('Outdated antialiased dataset. Rebuild with prepare_data(); do not reuse old checkpoints.')\n    if not np.isin(data['images'], [0.,1.]).all():\n        raise ValueError('Training rasters must contain exactly 0 or 1, with no gray pixels.')\n    return data\n\ndef batch(data,batch_size,device,split='train'):\n    ids=np.random.choice(data[split],batch_size,replace=True)\n    return (torch.tensor(data['images'][ids],device=device),\n            torch.tensor(data['labels'][ids],dtype=torch.long,device=device),\n            torch.tensor(data['tokens'][ids],dtype=torch.long,device=device))\n\ndef schedule(device,steps=100):\n    # Cosine cumulative schedule reaches near-pure noise even at small T.\n    t=torch.linspace(0,1,steps+1,device=device)\n    cumulative=torch.cos((t+.008)/1.008*torch.pi/2).square()\n    cumulative=cumulative/cumulative[0]\n    beta=(1-cumulative[1:]/cumulative[:-1]).clamp(.0001,.999)\n    alpha=1-beta\n    return beta,alpha,alpha.cumprod(0)\n\ndef fit(model,kind,data,steps=200,lr=1e-3,batch_size=32,beta=1.0,seed=0,phase='codec',report_every=50,training_state=None,callback=None):\n    state = training_state if training_state is not None else {}\n    if not state: seed_all(seed)\n    device=next(model.parameters()).device; model.train()\n    optimizer=torch.optim.Adam(model.parameters(),lr=lr)\n    if 'optimizer' in state:\n        optimizer.load_state_dict(state['optimizer'])\n        for group in optimizer.param_groups: group['lr']=lr\n    start_step=state.get('step',0)\n    if kind=='gan':\n        gopt=torch.optim.Adam(list(model.generator.parameters())+list(model.label.parameters()),lr=lr,betas=(.5,.999))\n        dopt=torch.optim.Adam(list(model.discriminator.parameters())+list(model.dlabel.parameters()),lr=lr,betas=(.5,.999))\n        if 'gopt' in state:\n            gopt.load_state_dict(state['gopt']);dopt.load_state_dict(state['dopt'])\n            for opt in (gopt,dopt):\n                for group in opt.param_groups: group['lr']=lr\n    log=[]; start=time.perf_counter()\n    for step in range(steps):\n        x,y,tokens=batch(data,batch_size,device)\n        optimizer.zero_grad(set_to_none=True)\n        if kind=='vae':\n            if phase=='prior':\n                if model.latent_prior!='flow':raise ValueError('A normal-prior VAE has no separate prior training phase')\n                with torch.no_grad():\n                    mu,_=model.stats(torch.cat([model.encoder(x),model.label(y)],1)).chunk(2,1)\n                noise=torch.randn_like(mu);t=torch.rand(len(x),device=device)\n                z=(1-t[:,None])*noise+t[:,None]*mu\n                loss=F.mse_loss(model.prior_velocity(z,t,y),mu-noise)\n                model.prior_trained.fill_(True)\n            else:\n                recon,mu,lv=model(x,y)\n                rec=F.binary_cross_entropy_with_logits(recon,x,reduction='none').flatten(1).sum(1).mean()\n                kl=-.5*(1+lv-mu.square()-lv.exp()).sum(1).mean()\n                loss=rec+beta*kl\n        elif kind=='gan':\n            z=torch.randn(len(x),model.latent,device=device)\n            fake=model.generate(z,y)\n            dopt.zero_grad(set_to_none=True)\n            real_logits=model.discriminate(x,y);fake_logits=model.discriminate(fake.detach(),y)\n            dloss=F.relu(1-real_logits).mean()+F.relu(1+fake_logits).mean()\n            dloss.backward();dopt.step();gopt.zero_grad(set_to_none=True)\n            gl=model.discriminate(fake,y)\n            loss=-gl.mean()\n            loss.backward();gopt.step()\n        elif kind in ('diffusion','flow'):\n            clean=x*2-1;noise=torch.randn_like(clean)\n            if kind=='flow':\n                t=torch.rand(len(x),device=device);a=t[:,None,None,None]\n                prediction=model((1-a)*noise+a*clean,t,y)\n                loss=F.mse_loss(prediction,clean-noise)\n            else:\n                _,_,ab=schedule(device);index=torch.randint(len(ab),(len(x),),device=device)\n                a=ab[index,None,None,None];xt=a.sqrt()*clean+(1-a).sqrt()*noise\n                # Velocity prediction avoids dividing a noise-prediction error by\n                # almost-zero signal at the final cosine timestep.\n                target=a.sqrt()*noise-(1-a).sqrt()*clean\n                loss=F.mse_loss(model(xt,index.float()/(len(ab)-1),y),target)\n        elif kind=='stroke':\n            loss=F.cross_entropy(model(tokens[:,:-1],y).transpose(1,2),tokens[:,1:],ignore_index=PAD)\n        elif kind=='vq':\n            if phase=='prior':\n                with torch.no_grad(): _,ids=model.encode(x)\n                inp=torch.cat([torch.full_like(ids[:,:1],model.codes),ids[:,:-1]],1)\n                loss=F.cross_entropy(model.logits(inp,y).transpose(1,2),ids)\n            else:\n                recon,vq=model(x)\n                loss=F.binary_cross_entropy_with_logits(recon,x)+vq\n        else: raise ValueError(kind)\n        if not torch.isfinite(loss): raise RuntimeError('Non-finite loss. Try a lower learning rate.')\n        if kind!='gan':\n            loss.backward();torch.nn.utils.clip_grad_norm_(model.parameters(),1.0);optimizer.step()\n        log.append({'step':start_step+step+1,'loss':float(loss.detach()),'seconds':time.perf_counter()-start})\n        if kind=='gan':log[-1]['discriminator_loss']=float(dloss.detach())\n        if report_every and ((step+1)%report_every==0 or step==0):\n            print(f'{kind}/{phase} {start_step+step+1}/{start_step+steps}: loss {log[-1][\"loss\"]:.4f}, {log[-1][\"seconds\"]:.1f}s',flush=True)\n        if callback:\n            state.update(step=start_step+step+1,optimizer=optimizer.state_dict())\n            if kind=='gan':state.update(gopt=gopt.state_dict(),dopt=dopt.state_dict())\n            callback(model,state,log[-1])\n    state.update(step=start_step+steps,optimizer=optimizer.state_dict())\n    if kind=='gan':state.update(gopt=gopt.state_dict(),dopt=dopt.state_dict())\n    model.eval();return log\n\ndef tokens_to_images(tokens):\n    out=[]\n    for sequence in tokens:\n        im=Image.new('L',(32,32),0);d=ImageDraw.Draw(im);points=[];coords=[]\n        for t in sequence:\n            t=int(t)\n            if t==BOS:continue\n            if t in (SEP,EOS,PAD):\n                if len(points)>1:d.line(points,fill=255,width=1)\n                elif len(points)==1:d.point(points[0],fill=255)\n                points=[];coords=[]\n                if t in (EOS,PAD):break\n            else:\n                coords.append(t)\n                if len(coords)==2: points.append(tuple(coords));coords=[]\n        out.append(np.asarray(im,dtype=np.float32)/255)\n    return torch.tensor(np.asarray(out))[:,None]\n\n@torch.no_grad()\ndef sample(model,kind,labels,seed=42,temperature=.8,sampling_steps=30):\n    model.eval();device=next(model.parameters()).device\n    y=torch.as_tensor(labels,dtype=torch.long,device=device)\n    # A local RNG keeps preview sampling from changing future training batches.\n    gen=torch.Generator(device='cpu').manual_seed(seed)\n    def randn(shape):return torch.randn(shape,generator=gen).to(device)\n    if kind=='vae':\n        z=randn((len(y),model.latent))\n        if getattr(model,'latent_prior','normal')=='flow' and bool(model.prior_trained):\n            for i in range(sampling_steps):\n                t=torch.full((len(y),),i/sampling_steps,device=device)\n                z=z+model.prior_velocity(z,t,y)/sampling_steps\n        return model.decode(z,y).sigmoid().cpu()\n    if kind=='gan':return model.generate(randn((len(y),model.latent)),y).cpu()\n    if kind in ('flow','diffusion'):\n        x=randn((len(y),1,32,32))\n        if kind=='flow':\n            for i in range(sampling_steps):\n                t=torch.full((len(y),),i/sampling_steps,device=device)\n                x=x+model(x,t,y)/sampling_steps\n        else:\n            # Deterministic DDIM over the training schedule, fixed starting noise.\n            _,_,ab=schedule(device)\n            times=torch.linspace(len(ab)-1,0,sampling_steps,device=device).long().unique_consecutive()\n            for j,i in enumerate(times):\n                velocity=model(x,torch.full((len(y),),float(i)/(len(ab)-1),device=device),y)\n                clean=(ab[i].sqrt()*x-(1-ab[i]).sqrt()*velocity).clamp(-1,1)\n                eps=(1-ab[i]).sqrt()*x+ab[i].sqrt()*velocity\n                prev=ab[times[j+1]] if j+1<len(times) else torch.tensor(1.,device=device)\n                x=prev.sqrt()*clean+(1-prev).sqrt()*eps\n        return ((x.clamp(-1,1)+1)/2).cpu()\n    if kind=='stroke':\n        seq=torch.full((len(y),1),BOS,dtype=torch.long,device=device)\n        ended=torch.zeros(len(y),dtype=torch.bool,device=device)\n        parity=torch.zeros(len(y),dtype=torch.long,device=device)\n        for _ in range(126):\n            logits=model(seq,y)[:,-1]/max(.05,temperature)\n            logits[:,[BOS,PAD]]=-torch.inf\n            logits[parity==1,SEP]=-torch.inf;logits[parity==1,EOS]=-torch.inf\n            nxt=torch.multinomial(logits.softmax(-1).cpu(),1,generator=gen).squeeze(1).to(device)\n            nxt=torch.where(ended,torch.full_like(nxt,EOS),nxt)\n            ended|=nxt==EOS\n            parity=torch.where(nxt<32,1-parity,torch.zeros_like(parity))\n            seq=torch.cat([seq,nxt[:,None]],1)\n            if ended.all():break\n        seq=torch.cat([seq,torch.full_like(seq[:,:1],EOS)],1)\n        return tokens_to_images(seq.cpu().numpy())\n    if kind=='vq':\n        seq=torch.full((len(y),1),model.codes,dtype=torch.long,device=device)\n        for _ in range(model.grid_size**2):\n            p=(model.logits(seq,y)[:,-1]/max(.05,temperature)).softmax(-1)\n            seq=torch.cat([seq,torch.multinomial(p.cpu(),1,generator=gen).to(device)],1)\n        return model.decode_ids(seq[:,1:]).cpu()\n    raise ValueError(kind)\n\ndef grid(images,labels,title,path=None,binary=True,threshold=.5):\n    images=np.asarray(images).reshape(-1,32,32)\n    if binary: images=(images>=threshold).astype(np.float32)\n    cols=4;rows=math.ceil(len(images)/cols)\n    fig,axes=plt.subplots(rows,cols,figsize=(8,2.25*rows),squeeze=False)\n    for i,ax in enumerate(axes.flat):\n        ax.axis('off')\n        if i<len(images):\n            ax.imshow(1-images[i],cmap='gray',vmin=0,vmax=1,interpolation='nearest')\n            ax.set_title(CATEGORIES[int(labels[i])],fontsize=10)\n    fig.suptitle(title + (f' | binary threshold {threshold:.2f}' if binary else ' | raw probabilities'));fig.tight_layout()\n    if path:fig.savefig(path,dpi=140,bbox_inches='tight')\n    return fig\n\ndef compare(before,after,data,labels,title,path=None,threshold=.5):\n    \"\"\"Human / baseline / changed, same category and supplied sampling seed.\"\"\"\n    fig,axes=plt.subplots(3,len(labels),figsize=(2*len(labels),6),squeeze=False)\n    for col,label in enumerate(labels):\n        ids=data['val'][data['labels'][data['val']]==label]\n        human=data['images'][ids[0],0]\n        for row,im in enumerate([human,np.asarray(before[col,0]),np.asarray(after[col,0])]):\n            axes[row,col].imshow(1-(im>=threshold).astype(np.float32),cmap='gray',vmin=0,vmax=1,interpolation='nearest')\n            axes[row,col].set_xticks([]);axes[row,col].set_yticks([])\n            if col==0:axes[row,col].set_ylabel(['Human (held out)','Baseline','Your experiment'][row])\n        axes[0,col].set_title(CATEGORIES[label])\n    fig.suptitle(title);fig.tight_layout()\n    if path:fig.savefig(path,dpi=140,bbox_inches='tight')\n    return fig\n\ndef export_experiment(model,kind,config,history,path='my-experiment',model_source=''):\n    \"\"\"Portable research checkpoint; deliberately not a competition submission.\"\"\"\n    import zipfile\n    from safetensors.torch import save_file\n    out=Path(path);out.mkdir(exist_ok=True,parents=True)\n    save_file({k:v.detach().cpu().contiguous() for k,v in model.state_dict().items()},str(out/'weights.safetensors'))\n    (out/'config.json').write_text(json.dumps({'kind':kind,'architecture':config,'categories':CATEGORIES,'canvas':32,'polarity':'1=ink','competition_ready':False},indent=2))\n    (out/'history.json').write_text(json.dumps(history,indent=2))\n    (out/'model_source.py').write_text(model_source)\n    (out/'README.txt').write_text('Classroom research checkpoint. Category-conditioned 32px images; not a scene-prompt competition submission. Preserve code, configuration, weights and sampling seed together. Quick, Draw! data: Google Creative Lab, CC BY 4.0, https://github.com/googlecreativelab/quickdraw-dataset .\\n')\n    with zipfile.ZipFile(str(out)+'.zip','w',zipfile.ZIP_DEFLATED) as z:\n        for f in out.iterdir():\n            if f.is_file():z.write(f,f.name)\n    return str(out)+'.zip'\n\nRUNTIME_SOURCE = In[-1] if 'In' in globals() else ''\n",
   "id": "cell-04",
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": "torch.set_num_threads(2)\nseed_all(0)\ndevice = device_name()\nprint('Device:', device, '| PyTorch:', torch.__version__)\ndata = load_data(ASSETS/'data.npz')\nprint('Training:', len(data['train']), 'Validation:', len(data['val']))\nprint('Images:', data['images'].shape, 'Stroke tokens:', data['tokens'].shape)\nprint('Internal image polarity: 1 = ink, 0 = paper. Plotting reverses this for display.')\nids = [data['val'][data['labels'][data['val']] == c][0] for c in range(4)]\ngrid(data['images'][ids], list(range(4)), 'Held-out human reference');\n",
   "id": "cell-05",
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "## 2. Model: the layers you can edit\nTry beta = 0.2 versus 2.0. Compare reconstruction and generation separately.\n\nKeep input/output dimensions compatible. Architecture changes start from fresh weights. A changed architecture cannot load the original checkpoint.",
   "id": "cell-06"
  },
  {
   "cell_type": "code",
   "metadata": {
    "tags": [
     "baseline-definition",
     "hide-input"
    ],
    "cellView": "form"
   },
   "source": "#@title Supplied setup (expand to inspect)\n# Preserve the reviewed baseline architecture independently of your edits.\n_BASELINE_NAMESPACE = {}\nexec(\"import math\\nimport torch\\nfrom torch import nn\\nfrom torch.nn import functional as F\\n\\nclass CVAE(nn.Module):\\n    def __init__(self, classes=4, width=24, latent=24, latent_prior='normal', prior_width=256):\\n        super().__init__()\\n        self.latent = latent\\n        self.label = nn.Embedding(classes, 16)\\n        self.encoder = nn.Sequential(nn.Conv2d(1,width,4,2,1),nn.ReLU(),\\n            nn.Conv2d(width,width*2,4,2,1),nn.ReLU(),nn.Flatten())\\n        self.stats = nn.Linear(width*2*8*8+16,latent*2)\\n        self.fc = nn.Linear(latent+16,width*2*8*8)\\n        self.decoder = nn.Sequential(nn.ConvTranspose2d(width*2,width,4,2,1),nn.ReLU(),\\n            nn.ConvTranspose2d(width,1,4,2,1))\\n        self.width=width\\n        self.latent_prior=latent_prior\\n        if latent_prior=='flow':\\n            self.prior_net=nn.Sequential(nn.Linear(latent+17,prior_width),nn.SiLU(),\\n                nn.Linear(prior_width,prior_width),nn.SiLU(),nn.Linear(prior_width,prior_width),nn.SiLU(),\\n                nn.Linear(prior_width,latent))\\n            self.register_buffer('prior_trained',torch.tensor(False))\\n        elif latent_prior!='normal':raise ValueError('latent_prior must be normal or flow')\\n    def prior_velocity(self,z,t,y):\\n        return self.prior_net(torch.cat([z,t[:,None],self.label(y).detach()],1))\\n    def decode(self,z,y):\\n        h=self.fc(torch.cat([z,self.label(y)],1)).view(-1,self.width*2,8,8)\\n        return self.decoder(h)\\n    def forward(self,x,y):\\n        mu,logvar=self.stats(torch.cat([self.encoder(x),self.label(y)],1)).chunk(2,1)\\n        logvar=logvar.clamp(-10,8)\\n        z=mu+torch.randn_like(mu)*(0.5*logvar).exp()\\n        return self.decode(z,y),mu,logvar\", _BASELINE_NAMESPACE)\nBaselineModel = _BASELINE_NAMESPACE['CVAE']\n",
   "id": "cell-07",
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "metadata": {
    "tags": [
     "editable-model"
    ]
   },
   "source": "import math\nimport torch\nfrom torch import nn\nfrom torch.nn import functional as F\n\nclass CVAE(nn.Module):\n    def __init__(self, classes=4, width=24, latent=24, latent_prior='normal', prior_width=256):\n        super().__init__()\n        self.latent = latent\n        self.label = nn.Embedding(classes, 16)\n        self.encoder = nn.Sequential(nn.Conv2d(1,width,4,2,1),nn.ReLU(),\n            nn.Conv2d(width,width*2,4,2,1),nn.ReLU(),nn.Flatten())\n        self.stats = nn.Linear(width*2*8*8+16,latent*2)\n        self.fc = nn.Linear(latent+16,width*2*8*8)\n        self.decoder = nn.Sequential(nn.ConvTranspose2d(width*2,width,4,2,1),nn.ReLU(),\n            nn.ConvTranspose2d(width,1,4,2,1))\n        self.width=width\n        self.latent_prior=latent_prior\n        if latent_prior=='flow':\n            self.prior_net=nn.Sequential(nn.Linear(latent+17,prior_width),nn.SiLU(),\n                nn.Linear(prior_width,prior_width),nn.SiLU(),nn.Linear(prior_width,prior_width),nn.SiLU(),\n                nn.Linear(prior_width,latent))\n            self.register_buffer('prior_trained',torch.tensor(False))\n        elif latent_prior!='normal':raise ValueError('latent_prior must be normal or flow')\n    def prior_velocity(self,z,t,y):\n        return self.prior_net(torch.cat([z,t[:,None],self.label(y).detach()],1))\n    def decode(self,z,y):\n        h=self.fc(torch.cat([z,self.label(y)],1)).view(-1,self.width*2,8,8)\n        return self.decoder(h)\n    def forward(self,x,y):\n        mu,logvar=self.stats(torch.cat([self.encoder(x),self.label(y)],1)).chunk(2,1)\n        logvar=logvar.clamp(-10,8)\n        z=mu+torch.randn_like(mu)*(0.5*logvar).exp()\n        return self.decode(z,y),mu,logvar\n\n# Capture the actual edited cell for reproducible exports.\nMODEL_SOURCE = In[-1] if 'In' in globals() else ''\n",
   "id": "cell-08",
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": "KIND = 'vae'\nBASELINE_CONFIG = {'width': 64, 'latent': 64}\nCONFIG = dict(BASELINE_CONFIG)  # EDIT HERE: e.g. CONFIG['width'] = 48\n# Set True if you edited layer definitions above without changing CONFIG.\nARCHITECTURE_EDITED = False\nModel = CVAE\nseed_all(0)\nbaseline = BaselineModel(**BASELINE_CONFIG).to(device)\nfrom safetensors.torch import load_file\nif not (ASSETS/'baseline.safetensors').exists():\n    raise FileNotFoundError('Use the instructor notebook with its matching asset ZIP. A short untrained fallback is not a baseline.')\nbaseline.load_state_dict(load_file(str(ASSETS/'baseline.safetensors')))\nLABELS = [0, 1, 2, 3]\nSAMPLE_SEED = 42\nTEMPERATURE = 0.65\nSAMPLING_STEPS = 60\n# Temperature affects stroke/VQ sampling; sampling steps affect diffusion/flow and the VAE latent prior.\nbefore = sample(baseline, KIND, LABELS, seed=SAMPLE_SEED, temperature=TEMPERATURE, sampling_steps=SAMPLING_STEPS)\ngrid(before, LABELS, f'Baseline: seed {SAMPLE_SEED}, temperature {TEMPERATURE}, sampling steps {SAMPLING_STEPS}');\n",
   "id": "cell-09",
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "## 3. Fine-tune the supplied pretrained model\n**Default: continue from the supplied pretrained weights, not from scratch.** The next cell adds 100 training steps to a copy of that model. The curves below show only these additional steps; they do not include the original pretraining. Changing CONFIG or setting ARCHITECTURE_EDITED = True instead starts a new model from random weights.\n\nWrite your prediction before running. Change one setting first. Increasing model size may make learning slower within the same time budget.\n\n**Prediction:** I will change ___ because ___. I expect to see ___.\n\nThis cell resets the experiment from the same baseline every time. It does not silently continue from the previous experiment. The optimizer starts fresh, even when weights come from the baseline.",
   "id": "cell-10"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": "TRAIN_STEPS = 100  # EDIT HERE; reduce to 20 for a quick smoke run\nLEARNING_RATE = BASELINE_RECORD['lr']  # EDIT HERE: replace with a number, e.g. 0.0001\nBETA = BASELINE_RECORD['beta']  # EDIT HERE: VAE KL weight, e.g. 0.2 or 2.0\nseed_all(0)\nif CONFIG == BASELINE_CONFIG and not ARCHITECTURE_EDITED:\n    experiment = copy.deepcopy(baseline)\n    TRAINING_MODE = 'Fine-tuning supplied pretrained weights'\nelse:\n    print('Architecture changed: training a new model from scratch.')\n    experiment = Model(**CONFIG).to(device)\n    TRAINING_MODE = 'Training from scratch (changed architecture)'\nprint(TRAINING_MODE)\nprint('Additional steps in this experiment:', TRAIN_STEPS, '| optimizer starts fresh')\nprior_history = []\nhistory = fit(experiment, KIND, data, steps=TRAIN_STEPS, lr=LEARNING_RATE, beta=BETA)\nif KIND == 'vq' or (KIND == 'vae' and CONFIG.get('latent_prior') == 'flow'):\n    # Changing the encoder/codec changes its latent distribution, so train the prior again.\n    prior_history = fit(experiment, KIND, data, steps=TRAIN_STEPS, lr=LEARNING_RATE, phase='prior')\n",
   "id": "cell-11",
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "## 4. Monitor: look at the drawings\n**Learning curves start at experiment step 1, after loading the pretrained weights by default.** Faint lines show every raw loss; bold lines show a trailing 10-step moving average (using available steps at the start). Smoothing does not change training.\n\nUse the same categories, seed and sampling settings for both models. Loss scales differ between model families, so do not rank families by their loss values.\n\nCheck recognizability, variation, missing parts and blank or filled canvases. A low ink fraction can mean a blank failure, not a successful doodle.",
   "id": "cell-12"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": "# Recompute both models with identical current settings, even if you edited the sampler.\nbefore = sample(baseline, KIND, LABELS, seed=SAMPLE_SEED, temperature=TEMPERATURE, sampling_steps=SAMPLING_STEPS)\nafter = sample(experiment, KIND, LABELS, seed=SAMPLE_SEED, temperature=TEMPERATURE, sampling_steps=SAMPLING_STEPS)\ncompare(before, after, data, LABELS,\n        f'{KIND}: seed {SAMPLE_SEED}, temp {TEMPERATURE}, sample steps {SAMPLING_STEPS}',\n        'comparison.png');\nplt.show()\nfig, axes = plt.subplots(1, 2 if prior_history else 1,\n                         figsize=(12 if prior_history else 7, 3), squeeze=False)\nMOVING_AVERAGE_WINDOW = 10\n\ndef plot_loss(ax, records, key, label, color):\n    steps = np.asarray([r['step'] for r in records])\n    losses = np.asarray([r[key] for r in records], dtype=float)\n    cumulative = np.r_[0.0, np.cumsum(losses)]\n    end = np.arange(1, len(losses) + 1)\n    start = np.maximum(0, end - MOVING_AVERAGE_WINDOW)\n    smooth = (cumulative[end] - cumulative[start]) / (end - start)\n    ax.plot(steps, losses, color=color, alpha=0.25, linewidth=1, label=label + ' raw')\n    ax.plot(steps, smooth, color=color, linewidth=2,\n            label=f'{label}: trailing {MOVING_AVERAGE_WINDOW}-step mean')\n\nax = axes[0, 0]\nplot_loss(ax, history, 'loss', 'Generator' if KIND == 'gan' else 'Training', 'tab:blue')\nif KIND == 'gan':\n    plot_loss(ax, history, 'discriminator_loss', 'Discriminator', 'tab:orange')\nax.set(xlabel='Additional experiment steps', ylabel='Loss (family-specific)', title=TRAINING_MODE)\nax.legend(fontsize=8); ax.grid(alpha=0.25)\nif prior_history:\n    ax = axes[0, 1]\n    plot_loss(ax, prior_history, 'loss', 'Prior', 'tab:green')\n    ax.set(xlabel='Additional prior steps', ylabel='Prior loss', title='Latent/token prior \u2014 this experiment')\n    ax.legend(fontsize=8); ax.grid(alpha=0.25)\nfig.tight_layout(); fig.savefig('learning-curves.png', dpi=140, bbox_inches='tight')\nplt.show()\n# Several seeds reveal repetition that a single sample can hide.\nvariations = torch.cat([sample(experiment, KIND, LABELS, seed=s, temperature=TEMPERATURE,\n                              sampling_steps=SAMPLING_STEPS) for s in [42, 43, 44]])\ngrid(variations, LABELS * 3,\n     f'{KIND}: row seeds 42 / 43 / 44 | temp {TEMPERATURE}, sample steps {SAMPLING_STEPS}',\n     'sample-outputs.png');\nplt.show()\n",
   "id": "cell-13",
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "## 5. Reproducibility: save and explain\n**Observation:** What changed visually? Did the result match your prediction? What alternative explanation remains?\n\nSave the comparison, edited model source, configuration and weights. This export is a research bundle. The competition adapter, full-resolution renderer and text-conditioning integration are separate next steps.",
   "id": "cell-14"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": "experiment_config = {'architecture': CONFIG, 'train_steps': TRAIN_STEPS,\n                     'training_mode': TRAINING_MODE, 'moving_average_window': MOVING_AVERAGE_WINDOW,\n                     'learning_rate': LEARNING_RATE, 'beta': BETA, 'training_seed': 0,\n                     'sample_seed': SAMPLE_SEED, 'temperature': TEMPERATURE,\n                     'sampling_steps': SAMPLING_STEPS, 'torch_version': torch.__version__,\n                     'device': device, 'architecture_edited': ARCHITECTURE_EDITED,\n                     'baseline': globals().get('BASELINE_RECORD', 'local or trained in notebook')}\narchive = export_experiment(experiment, KIND, CONFIG, history, model_source=MODEL_SOURCE)\nPath('my-experiment/experiment.json').write_text(json.dumps(experiment_config, indent=2))\nPath('my-experiment/runtime.py').write_text(RUNTIME_SOURCE)\n# Include the visual evidence and full experiment settings in the final archive.\nimport zipfile\nwith zipfile.ZipFile(archive, 'a', zipfile.ZIP_DEFLATED) as z:\n    for figure in ['comparison.png', 'learning-curves.png', 'sample-outputs.png']:\n        z.write(figure, figure)\n    z.writestr('prior-history.json', json.dumps(prior_history, indent=2))\n    z.write('my-experiment/experiment.json', 'experiment.json')\n    z.write('my-experiment/runtime.py', 'runtime.py')\nprint('Saved:', archive)\ntry:\n    from google.colab import files\n    files.download(archive)\nexcept ImportError:\n    from IPython.display import FileLink, display\n    display(FileLink(archive))\n",
   "id": "cell-15",
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "## After class: the competition extension\nThe classroom model knows four categories. It does not understand arbitrary scene prompts. The competition needs text conditioning, 512\u00d7512 output with 0=ink, deterministic noise handling, and the shared renderer. Merely resizing a preview does not solve these requirements.\n\nPossible next experiments: more categories, better conditioning, a larger model, stroke-aware representation, or adaptation of a pretrained generator. Keep a held-out scene evaluation when moving beyond objects.\n\n**Data attribution:** Google Creative Lab, [Quick, Draw! dataset](https://github.com/googlecreativelab/quickdraw-dataset), CC BY 4.0. These are simplified, recognized drawings. Long stroke sequences are excluded consistently from both raster and stroke notebooks.\n\n**References:** [SketchRNN](https://arxiv.org/abs/1704.03477), [DDPM](https://arxiv.org/abs/2006.11239), [Flow Matching](https://arxiv.org/abs/2210.02747), [VQ-VAE](https://arxiv.org/abs/1711.00937). The small teaching implementations illustrate these families; they are not reproductions of the papers' architectures or results.",
   "id": "cell-16"
  },
  {
   "cell_type": "markdown",
   "id": "classroom-hand-in-intro",
   "metadata": {
    "tags": [
     "classroom-hand-in"
    ]
   },
   "source": "## Hand in your drawings for the class vote\nRun this after your experiment. Set **ENTRY_NAME** and **WHAT_CHANGED** below.\nThe cell generates 16 fresh drawings: four each of apple, cat, fish, and flower,\nusing the same four seeds for everyone. Submit the whole set, without picking\nonly your favorites. Pixels are black or white; no smoothing is applied.\n\nDownload **classroom-submission.json**, then open\n[the classroom page](https://competition.roguesgallery.io/classroom/), sign in\nwith GitHub, and upload it. You can replace your entry until the instructor\nstarts voting. Keep `my-experiment.zip` for your own code and weights.\n\nDuring voting, classmates see anonymous drawings mixed with held-out human\nexamples. They cannot vote on their own entry. The instructor reveals the\nresults and gallery at the end. Your GitHub handle and entry name appear with\nyour drawings in the class gallery after the reveal."
  },
  {
   "cell_type": "code",
   "id": "classroom-hand-in-export",
   "metadata": {
    "tags": [
     "classroom-hand-in"
    ]
   },
   "execution_count": null,
   "outputs": [],
   "source": "# Run this cell after your experiment. It exports drawings, not executable code.\nENTRY_NAME = \"My doodler\"  # Give your entry a recognizable name.\nWHAT_CHANGED = \"\"  # One or two sentences about your experiment.\n\nimport json\nfrom pathlib import Path\n\nsubmission_drawings = []\nsubmission_categories = ['apple', 'cat', 'fish', 'flower']\nfor submission_seed in [101, 202, 303, 404]:\n    submission_batch = sample(experiment, KIND, list(range(4)), seed=submission_seed,\n                              temperature=TEMPERATURE, sampling_steps=SAMPLING_STEPS)\n    submission_pixels = (submission_batch.detach().cpu().numpy()[:, 0] >= 0.5)\n    for category, pixels in zip(submission_categories, submission_pixels):\n        submission_drawings.append({'category': category, 'seed': submission_seed,\n                                    'pixels': ''.join('1' if p else '0' for p in pixels.flat)})\nsubmission = {'schema': 'rogues-classroom-v1', 'width': 32, 'height': 32,\n              'polarity': '1=ink', 'title': ENTRY_NAME, 'model': KIND,\n              'notes': WHAT_CHANGED, 'drawings': submission_drawings,\n              'sampling': {'temperature': TEMPERATURE, 'steps': SAMPLING_STEPS,\n                           'threshold': 0.5}}\nsubmission_path = Path('classroom-submission.json')\nsubmission_path.write_text(json.dumps(submission))\nprint('Ready: classroom-submission.json \u2014 16 drawings, four per category.')\nprint('Sign in and upload at https://competition.roguesgallery.io/classroom/')\ntry:\n    from google.colab import files\n    files.download(str(submission_path))\nexcept ImportError:\n    from IPython.display import FileLink, display\n    display(FileLink(str(submission_path)))\n"
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.12"
  },
  "colab": {
   "name": "vae.ipynb",
   "provenance": []
  },
  "accelerator": "GPU"
 },
 "nbformat": 4,
 "nbformat_minor": 5
}