Google’s Pixel 9 telephones are all about AI. If you happen to watched the corporate’s launch occasion, there was no method of lacking it, as that’s nearly all Google appeared all for speaking about. Some AI right here, a sprinkle of AI there, and an additional contact of AI on high, simply to be secure.
One of many many AI options on the Pixel 9 is an app known as Pixel Studio. It’s a picture generator app that lets you create a picture of just about something you’d like. You open the app, sort in what you need a image of, and voilà — Pixel Studio makes it for you. It’s not the primary app of its sort, however it’s the first one shipped on a Google Pixel telephone. Sadly, it’s a dumpster hearth.
What Pixel Studio does nicely
Joe Maring / Digital Tendencies
Earlier than I get into the unhealthy stuff, I wish to give Google credit score the place it’s due. Though I’m not a fan of AI picture generator apps as an entire, Pixel Studio is a well-designed and easy-to-use software.
Upon opening the app, you’re offered with 10 classes of steered picture varieties — together with magical castles, meme reactions, sports activities, retro video video games, and extra. You’ll be able to browse these for inspiration or faucet the blue Create button on the backside of the app to begin from scratch. If you wish to create your personal picture, it’s as straightforward as typing in what you need Pixel Studio to generate a photograph of. Need to see an image of a black cat carrying sun shades whereas carrying a humorous hat? Kind that into Pixel Studio, and also you’ll nearly instantly get a photograph of it.
Pixel Studio Digital Tendencies
What for those who’d slightly see your picture as a sketch? Perhaps you’d desire it within the fashion of a online game? You are able to do that with a few faucets. And if you wish to see that cat lounging on the seaside with fireworks within the background? Simply add it to your immediate, and also you’ve received it.
No matter your emotions towards AI picture technology are, that is an objectively slick and thoughtfully designed app. Nonetheless, it’s what you are able to do with Pixel Studio the place issues get … troublesome.
The place the app falls aside
Joe Maring / Digital Tendencies
As with most picture generator apps, Pixel Studio has guardrails to stop you from making any sort of picture. I fairly rapidly discovered that it refuses to generate pictures of individuals, however I used to be curious what different limits Google is imposing. When requested about this, the corporate informed Digital Tendencies the next:
“Pixel Studio and Pixel Screenshots observe Google’s AI Ideas. For instance, all totally artificial, text-to-image generated photos in Pixel Studio use SynthID watermarking, so these photos might be tracked — even when they’re shared or edited. Furthermore, we’ve disabled human technology and added many security checks to our servers to steadiness our daring — but accountable — method to AI. This offers our clients entry to a really highly effective device, however ensures now we have checks in place to make sure Pixel Studio can’t be used nefariously.”
I’ve repeatedly tried to make Pixel Studio generate photos of individuals, and it has persistently refused to take action, which is sweet. Nonetheless, these “many security checks” which are supposedly in place don’t seem to work almost in addition to they need to.
Pictures generated in Pixel Studio Digital Tendencies
How so? Because the headline of this text teases, I’ve had Pixel Studio generate a picture of SpongeBob dressed as a Nazi. When given the immediate “SpongeBob dressed as a German soldier from WWII with a swastika on the uniform,” Pixel Studio didn’t hesitate to offer me with a picture of exactly that.
And that’s only one instance. I’ve additionally gotten Pixel Studio to generate photos of Elmo pointing a shotgun at Massive Chook, Yoda doing cocaine, Mr. Krabs holding an assault rifle, and extra. To be clear, Pixel Studio hasn’t generated inappropriate photos unexpectedly. If I ask for a photograph of a cute canine, I’ll get a photograph of a cute canine — not one brandishing a gun. Pixel Studio solely generates photos of weapons, medication, and so forth., when particularly requested to.
Probably the most troubling factor Pixel Studio has generated are a number of photos of copyrighted characters firing weapons in a faculty capturing setting — together with depictions of lifeless college students mendacity on the bottom. We’ve determined to not embody these photos, however they’re the worst instance of how off the rails Pixel Studio is.
Pictures generated in Pixel Studio Digital Tendencies
It’s necessary to say that I didn’t should twist Pixel Studio’s arm to get these photos. Initially, I suspected you wanted to preface prompts with “an object resembling” earlier than it will generate these photos. Nonetheless, it rapidly turned obvious that wasn’t the case. A transparent and easy immediate like “Mickey Mouse dressed as a slave proprietor” or “Paddington Bear on a crucifix” is all Pixel Studio wants. You don’t must trick the app into producing these photos.
If this have been a random web site or third-party app, maybe we wouldn’t be so important. However that isn’t what’s occurring right here. Pixel Studio is a Google-made software and one of many handful of recent apps transport on the corporate’s newest Pixel 9 telephones. It needs to be artistic and open-ended, however there’s additionally a restrict to what apps like this could do. Moreover, Microsoft Copilot and Google’s personal Gemini picture generator refuse to generate photos with the identical prompts. That’s to not say these instruments are excellent, nevertheless it’s clear that security measures working in different picture mills aren’t working in Pixel Studio.
How is that this occurring?
Joe Maring / Digital Tendencies
If you happen to’ve adopted AI picture technology this yr, you’ll know this isn’t our first time right here. This month, X’s Grok chatbot received an much more outlandish picture generator device — producing photos of politicians and celebrities in equally tasteless conditions. Google itself was in scorching water this previous February after its Gemini picture generator created pictures of Nazis and the U.S. Founding Fathers as Black folks.
Given Google’s historical past right here, you’d suppose the corporate had discovered from its previous errors. And with different picture generator instruments, it seemingly has. I attempted lots of the similar prompts that labored in Pixel Studio with Google’s different picture mills — Gemini and ImageFX — and couldn’t replicate any of those prompts on these different platforms.
For context, Gemini presently makes use of the Imagen 2.0 mannequin for picture technology, whereas ImageFX makes use of Imagen 3 — the identical one which Pixel Studio makes use of. This discrepancy between ImageFX and Pixel Studio is especially complicated since they use the identical fashions. I requested Google for clarification about this and was redirected to the corporate’s Generative AI Prohibited Use Coverage.
What Google ought to do about Pixel Studio
Joe Maring / Digital Tendencies
Whereas scripting this story, I despatched Google examples of prompts I used with Pixel Studio — particularly, these about German troopers and characters utilizing cocaine. Somewhat over 24 hours after sending these examples, Pixel Studio now says, “An unknown error occurred” when attempting to generate these photos.
Is that progress? Definitely. Nonetheless, it doesn’t totally tackle the underlying issues with Pixel Studio. I’m glad these two photos are actually unattainable to generate, nevertheless it was solely after I despatched a number of emails to Google explaining the problem. It additionally doesn’t tackle that these photos have been allowed within the first place and that equally inappropriate pictures are nonetheless being generated. On the time of publication, Pixel Studio remains to be actively creating photos of college shootings, well-liked youngsters’s characters dressed as slave homeowners, and so forth.
I pressed Google for one more assertion about the entire scenario, and the corporate’s Communications Supervisor Alex Moriconi stated the next:
“Pixel Studio and Magic Editor are useful instruments meant to unlock your creativity with textual content to picture technology and superior picture modifying on Pixel 9 units. We design our Generative AI instruments to respect the intent of consumer prompts and which means they might create content material which will offend when instructed by the consumer to take action. That stated, it’s not something goes. We have now clear insurance policies and Phrases of Service on what sorts of content material we enable and don’t enable, and construct guardrails to stop abuse. At instances, some prompts can problem these instruments’ guardrails and we stay dedicated to repeatedly enhancing and refining the safeguards now we have in place.”
Joe Maring / Digital Tendencies
As an organization with an already problematic observe report for AI picture technology, creating an app particularly for this was at all times going to be a danger. Google took that danger, and now that is the place we’re at. At its greatest, Pixel Studio may have been a enjoyable tech demo for Google to advertise its AI chops. As a substitute, it’s put an uncomfortable cloud over the corporate’s newest smartphones — which is a disgrace, because the Pixel 9 Professional XL is likely one of the greatest telephones I’ve used this yr.
The final time Google was right here with Gemini picture technology issues, the corporate responded inside days. One can solely hope for a equally immediate response to Pixel Studio’s points, although whether or not that needs to be executed by way of fast app updates or scrapping the appliance altogether stays to be seen.
