I thought this was great, and hilarious. Kudos to Opus 5, I thought it was the only one that came close to passing. Interestingly, I thought many of the failures drew the frog face OK, and they had some type of big blob for the jaw, so they knew "Hapsburg jaw" meant a protruding jaw, but it wasn't really connected to the frog face in any way that made sense.

Small side note, the first gemini-2.5-pro one totally reminded me of some sad faced meme or Pepe the frog from somewhere. Anyone know what I'm referring to, tried to find it.

• viciousvoxel

  today at 9:24 PM

  Perhaps you're thinking of Salad Fingers?

Opus 5 clearly frogmaxxed.

gemini-3.6-flash runs 2 and 3 responded best to the royal portrait context.

• fasterik

  today at 9:22 PM

  Arguably, a royal portrait is a misinterpretation of the prompt, since it's just asking for a specific facial feature. But I guess you could look at it as a bit of artistic license.

Hi all, the site is getting hugged to death, thank you, was not expecting this kind of warm response. I will be working to make this more reliable, in the meantime, sign up for my newsletter: https://www.jaymollica.com/blog/

also my favorite SVG was def the google/gemini-3.6-flash

edit: ok better now I think

• troupo

  today at 8:51 PM

  > also my favorite SVG was def the google/gemini-3.6-flash

  That looks like something from Machinarium or Robots :)

It's opus 5 > Kimi K3 > grok 4.5

That's a pretty good benchmark

Hopsburg Jaw

How do models approach SVG generation? In one version, I imagine them actually trying to reason about them as an LLM. In another, I imagine something closer to a GAN.

This is a strong benchmark! None of these could be remotely mistaken for human art. Opus 5 comes closest.

A friend’s favorite prompt is “Batman & Julia Child; in the kitchen laughing at a ham”. Sounds simple, but has been surprisingly tough.

• esaym

  today at 9:25 PM

  https://imgur.com/a/1BM8J1B

Can you also try the new deepseek v4 flash?

• thebigship

  today at 8:48 PM

  I will add it to next month's report!

  • leumon

    today at 9:34 PM

    Thank you!

Mine is any variations on mammoths in various situations, or anthropomorphic. Since mammoths are invariably majestically going from one place to another in any of the books, models have hard time imagining anything but that.

Also try a fantasy archer with a proper bow who is not brooding, sitting in a fantasy wood :)

For those who don’t know a Habsburg jaw also known as mandibular prognathism, it is a genetic condition characterized by a protruding lower jaw, which was notably prevalent among members of the Habsburg royal family due to their history of inbreeding. This condition often resulted in significant facial deformities and difficulties with eating and speaking.

I think this one has advantages over the “pelican riding a bicycle” one because it hinges on an anatomical feature that many models associate with royalty, “habsburg” being a lineage and “habsburg jaw” being an anatomical feature.

Seven of fourteen models silently imported royalty into a prompt that named only an anatomical feature. Two of them knew they were extrapolating ("because Habsburg") and did it anyway.

Mistral returned byte-identical output across separate calls.

Gemini narrates its work in 65 comments; Llama says nothing.

If you're deciding which model to trust with instructions, "how much does it embellish beyond what I asked" and "does it behave deterministically" are directly practical questions.

• n00bskoolbus

  today at 9:09 PM

  The identical pair from Mistral took me off guard. Many of the other models were so varied between the runs which is more what I would expect.

• HPsquared

  today at 8:54 PM

  Bite-identical?

  • thebigship

    today at 8:59 PM

    clearly I missed an amazing copy opportunity, thank you haha

  • today at 8:58 PM