MskProdComputing
Back to the topic

AI · 12.2

Diffusion Models and Multimodal AI

Section A: 10 multiple choice questions, 10 marks. Section B: 3 written questions, 14 marks. Total 24 marks.

Name

Class

Date

Mark  / 24

Section A

Circle the letter of the correct answer. One mark each.

  1. 1What does the forward process of a diffusion model do?1
    1. ARemoves noise from an image step by step
    2. BTurns a text prompt into embeddings
    3. CCompresses an image into a latent code
    4. DAdds noise to an image step by step until only noise remains
  2. 2What is the denoising network usually trained to predict?1
    1. AThe next pixel of the image
    2. BThe label of the image
    3. CThe noise that was added to the image
    4. DThe text prompt of the image
  3. 3Why does every generation run give a different image?1
    1. AThe network's weights change during generation
    2. BThe prompt is shuffled before each run
    3. CThe model picks a random training image
    4. DEach run starts from different random noise
  4. 4How does a text prompt influence the denoiser?1
    1. AIt chooses which training image to copy
    2. BThrough cross-attention from the image to the prompt's embeddings
    3. CIt is added to the final image as pixels
    4. DIt replaces the random starting noise
  5. 5What happens when the classifier-free guidance scale is raised?1
    1. AThe images become larger
    2. BGeneration needs no steps at all
    3. CImages follow the prompt more closely but vary less
    4. DImages ignore the prompt
  6. 6Why is latent diffusion faster than diffusion on pixels?1
    1. AIt denoises a small latent with far fewer numbers than the image
    2. BIt uses only one denoising step
    3. CIt skips the decoder
    4. DIt works on black-and-white images only
  7. 7How is CLIP trained?1
    1. AMatching pairs of images and captions are pulled together, others apart
    2. BBy generating images from captions
    3. CBy sorting images into 1,000 labelled classes
    4. DBy predicting the next word of each caption
  8. 8How can CLIP classify images into categories it was never trained on?1
    1. ABy guessing at random
    2. BBy picking the caption embedding closest to the image embedding
    3. CBy retraining the model on each new category first
    4. DBy asking people to label the images
  9. 9A 512 × 512 colour image holds how many numbers?1
    1. A1,536
    2. B16,384
    3. C262,144
    4. D786,432
  10. 10What does a C2PA content credential provide?1
    1. AFaster image generation
    2. BA signed record of how a file was made and edited
    3. CA guarantee that an image is beautiful
    4. DA copy of the training data

Section B

Answer in the space provided. The number of marks tells you how many separate points to make.

  1. 1ExplainExplain how a diffusion model is trained and how it generates a new image.6
  2. 2ExplainExplain how CLIP is trained and how it can be used to classify images without further training.4
  3. 3EvaluateEvaluate two ways of telling whether an image was made by AI.4

Mark scheme

Section A

1D2C3D4B5C6A7A8B9D10B

Section B

1 Explain how a diffusion model is trained and how it generates a new image. 6

One mark for each of the following, to a maximum of 6.

  • Forward process: noise is added to a training image over many steps
  • A random step is chosen and the image mixed with noise for that step
  • The network is trained to predict the noise that was added
  • Loss: squared difference between the real and predicted noise
  • Generation starts from pure random noise
  • The predicted noise is removed a little at a time over many steps until an image appears

Model answer. During training, a training image has noise added as the forward process would after a randomly chosen step t, so the image is partly destroyed. The network, usually a U-Net, is given the noisy image and the step number and trained to predict exactly what noise was added, with the loss being the squared difference between the real noise and its prediction. To generate a new image, the model starts from pure random noise and repeatedly predicts the noise present and removes a little of it, over many steps, so that an image gradually emerges; starting from different noise gives a different image.

2 Explain how CLIP is trained and how it can be used to classify images without further training. 4

One mark for each of the following, to a maximum of 4.

  • An image encoder and a text encoder map images and captions into one shared space
  • Trained on many image–caption pairs: matching pairs are pulled together, mismatched pairs pushed apart
  • To classify, captions such as 'a photo of a ...' are written for each category and embedded
  • The category whose caption embedding is most similar to the image embedding is chosen

Model answer. CLIP has an image encoder and a text encoder that map into one shared embedding space. It was trained on hundreds of millions of images with their captions using contrastive learning: in each batch, the true image–caption pairs are pushed to have high cosine similarity and every mismatched pair low. To classify an image into new categories, a caption such as "a photo of a fox" is written for each category and embedded, the image is embedded, and the category whose caption is most similar to the image is chosen, with no extra training: zero-shot classification.

3 Evaluate two ways of telling whether an image was made by AI. 4

One mark for each of the following, to a maximum of 4.

  • Detectors that guess from the pixels can be fooled and become outdated as generators improve
  • Watermarking hides a signal in generated content that software can detect
  • Watermarks only work if the generator adds them, and may be removed by editing
  • Provenance records such as C2PA credentials show how media was made, but can be stripped / need wide adoption

Model answer. One approach is a detector that looks at an image's pixels for signs of generation, but such detectors are unreliable: they make mistakes on new generators and can be fooled by small edits, so they become outdated as generators improve. A second approach is watermarking, where the generator hides a signal in its output that software can later detect, as SynthID does; this is far more reliable when present, but only works if the generator adds it, and heavy editing may remove it. Provenance records such as C2PA content credentials complement this by recording how a genuine photo was made and edited, but they need cameras and apps to adopt them, and metadata can be stripped, so no single method is enough on its own.