I am growing tired of these pelicans posts every time a new model is published. Feels to me like low effort personal brand promotion. Just sharing my 2 cents.
+1. First thing I look for in a model announcement thread. I actually came across this one an hour ago and was sad there were no pelicans yet.
It's a decent heuristic because the better models generate better pelicans. That's all. Nobody sane is going to make a bet on a model based on a pelican. But it's cool, it's tradition by now, and it's a semblance of a good first impression for new models.
Thank you for posting them (as well as all your other AI takes). It's appreciated here!
I feel like the human brain massively overweights negative feedback over positive, and that's even after accounting for the fact that internet discourse tends to mostly surface negative comments (whereas the enjoyers stay silent). By default I have to try hard not to take it personally whenever someone leaves a negative comment about my work.
So just doing my bit to say I appreciate your commentary on so much of the fast-evolving AI landscape. Helps me orient :)
If enough people agreed with you, simonw's top level comment would be grey. It isn't. Not every comment is for every person, that is fine and normal. He didnt hijack a popular thread in here to make his post. He doesnt have a tin can in your face shouting from a soapbox that makes it hard to ignore. Flag or ignore are reasonable options that can be used.
People aren't supposed to upvote or downvote posts for these kinds of reasons here. Most people are downvoting far too much on this website, and it leads to significant echo-chamber dynamics that are worse than even reddit. The pelican test continuing to be taken seriously is a great example of that kind of echo chamber.
"He doesn't have a tin can in your face shouting from a soapbox that makes it hard to ignore."
He metaphorically does because people upvote his pelicans to the top and the ensuing comment threads are massive/bloated. Huge amounts of the readership of this website are lurkers who don't even know how to hide these giant posts. Look at how bloated this very thread is right now!
Also, a lot of people unironically are whining about him because of sour grapes. Pay them what Simon is likely making, give them as much mindshare/attention as Simon gets, and they wouldn't be so mad.
Anti-incumbency bias and anti-elitist attitudes are good actually.
> People aren't supposed to upvote or downvote posts for these kinds of reasons here.
Agreed, but scrolling past or hitting - were apparently off the table for the complainers. So flagging was yet another tool in their toolbelt and I bet a powerful one at that. If simonw's pelican posts routinely went dead from flagging, he would not make them. You know that, I know that.
> The pelican test continuing to be taken seriously is a great example of that kind of echo chamber.
You can try and support that argument if you like. But I would implore you to realize that it has been had many times recently and the other side does in fact find value and do not see it that way.
> He metaphorically does because people upvote his pelicans to the top and the ensuing comment threads are massive/bloated.
Users upvote the pelicans because they find it interesting. they arent paid trolls or simonw fanatics.
> Huge amounts of the readership of this website are lurkers who don't even know how to hide these giant posts.
They can learn... it's called hackernews. For those interested, that is what the [-] link is for above the comment. Use it and move on.
But, also, as said, you're an industry (and foss!) veteran, so I find it impossible to believe that you haven't had your fair share of baseless bullshit being thrown at you, and with that, you gaining a persona that will not be hit by that, because it clearly knows that it is in fact bullshit.
Unless of course it doesn't really know that with certainty.
As said, I would _love_ to give you the benefit of the doubt, because you might just have a stressful day or whatever, but content marketing is literally your whole thing by now.
It is impossible for me to do that with a clean conscience.
Your blog front page currently opens with
> Earlier this month I hosted a fireside chat session at the AI Engineer World’s Fair with Cat Wu and Thariq Shihipar from Anthropic’s Claude Code team.
That is not what "some rando foss maintainer we are morally obligated to be soft with" does.
I'm a professional blogger now. I still also work on open source software. I'm even fine being called an "influencer" (shudder), but I take offense to accusations of unethical behavior.
I think very hard about the ethics of what I'm doing and how I can best use my "platform" (shudder again) in as constructive a way as possible.
What do you want from him, seriously? Anyone who follows the scene knows that blogging about AI (and participating in conferences etc.) is what Simon does nowadays.
There's nothing shady here. The disclosure is front and center on his About page on his website.
He's not spamming you. It's one short link, sometimes a link to a first-impressions post. It's interesting and useful for me and the other commenters who keep upvoting his comments. Why are you so antagonistic?
At this point, it's kind of a hackernews thing. Simon posts them as a single comment in the relevant thread. It's okay for this place to have a little bit of a sense of community, and you can just ignore the comment.
But how else am I supposed to know when we've reached AGI, until I see an absolutely flawless pelican?
All of the pelicans so far have had really weird flaws / quirks so I am always a little interested to see how well these models perform at this task, since I've seen all the past pelicans and have some anchoring.
Seeing a truly flawless pelican would tell me that the model has true visual reasoning capabilities as well as good taste.
I agree to rednb that at this point it feels like rather obvious brand building, but also, I agree with you that some value is in it.
It does not feel all that authentic though, and it's good to react allergically to lack of authenticity. Bad for a lot of business models, but good for humanity.
Sorry mate, but you sound jealous in all these replies that the Pelican domain isn't your gig. The below is as labored as nitpicks ever get:
> It does not feel all that authentic though, and it's good to react allergically to lack of authenticity. Bad for a lot of business models, but good for humanity.
I thought the Gemini 3.5 Flash Lite response was quite telling myself. I personally like the Pelican SVG test, to me it is still a charming snapshot of model performance anecdata. No one would argue it's rigorous but I don't think it was ever intended to be.
I get people burning out on the pelican SVG test alongside the rest of the AI burnout, but I guess for myself I'm just choosing to keep enjoying it while I still can.
Not only does it give you a super easy-to-grok understanding of the model quality just by looking at the image, but when you compare tokens and costs (both input and output), you really get a good, simple COST x QUALITY evaluation across models.
Disagree. They're a nice tradition, but besides that, they're a useful way of eyeballing improvements. I realise labs are likely to be training for Pelicans - but if they're all training for them, the differences in the results are as indicative as they were before labs trained for them.
The 3.6 Flash pelican is just about the best I've seen.
Here is a different opinion. I’m always looking forward to see the pelican whenever a new model is released. It’s plain simple to understand, memorable, subtle enough in terms of details, and my favorite part is that you have been doing them consistently long enough for it to be useful for comparing almost everything with anything.
Your comment reads very pedantic with a hint of jealousy. The pelican and xbox controllers are great ways to see how well it can follow direction dealing with svg a difficult format for LLMs to use and testing their spatial vision awareness.
Yeah. It's something I can do myself in a couple seconds if I want, also on more varied SVG scenes. If this is going to be a benchmark people turn to I'd like to see more effort put into it than just a one-sentence prompt.
I'll split the difference. When it's a blog post there's usually an interesting observation or two, but if it's totally automated? Maybe just do the ones with a post.
A piece of the frame is missing between pedals and back wheel. The frame of the bike passes through the bird. It also puts a cap on the bird's head, and a fish in it's mouth.
The fish and the cap where always added when I asked an llm to improve it's first attempt.
This continues the trend in LLM progress of better=more stuff
Edit: I wonder if this is a function of the reasoning training, where more tokens/ stuff is rewarded.
https://tools.simonwillison.net/markdown-svg-renderer#url=ht...