Apple's Genmoji Might Be a Prompt Engineering Training Tool in Disguise

If a image is price a thousand phrases, what is the worth of an AI-generated emoji?

I’ve been giving that a lot of thought since Apple unveiled Genmoji at its builders’ convention on June 10. The new emoji maker, due this fall in the subsequent model of its iOS working system software program, will use a generative AI engine to let iPhone customers describe an emoji with a few phrases — Apple’s examples included “smiley relaxing wearing cucumber slices” and “squirrel DJ.” Apple’s iOS 18 software program then creates the digital icon for you in seconds, so you may share it in your messages and as stickers. 

Many predict that Genmoji, supplied as a part of a new set of Apple Intelligence options for the iPhone, Mac and iPad, will result in an explosion of customized pictograms you should use in addition to the greater than 3,700 emoji that the Unicode consortium has standardized and that we’re fairly acquainted with (😉👋✔️). Keith Broni, editor in chief of the emoji dictionary Emojipedia, informed The Washington Post that Genmoji will “open the floodgates” of area of interest emoji creation.

I’ve little doubt Apple customers will attempt to outdo themselves by dreaming up authentic emoji which might be cool (💀), artistic (💡) , intelligent (🤓), weird (🪱), obscure (🤺), foolish or cheesy (💩) or some play on all that. 

But whether or not an AI-powered emoji generator is nice or unhealthy is up for debate, provided that Apple’s prior makes an attempt to inject some enjoyable into its Message app acquired a meh response — keep in mind 2018’s personalised “Memojis” and animated “Animojis?” And whereas your customized emoji may appear intelligent to you, will your recipient truly know what a squirrel DJ represents? 

“By adding the potential for unlimited, unhinged emojis, the company could springboard AI emojis to new digital-culture heights. Or it could fail to read the room and change how we communicate for the worse,” Tatum Hunter wrote in The Washington Post.

Fair level. Will Genmojis usher in the daybreak of emoji confusion and Mehmojis? I truthfully do not know.

But I’ve a completely different considered this new emoji maker: Might Genmoji be a easy, nonthreatening solution to get individuals who have not performed round with gen AI to develop into comfy writing “prompts,” the time period used to explain what we ask of AI-powered chatbots?

Given that Apple is a client tech behemoth that is actually good at mainstreaming new expertise and getting individuals to purchase into new methods of doing issues, and provided that there are thousands and thousands of individuals utilizing the iPhone day by day, no matter options it introduces have the potential to disrupt the patron market — and our relationship with tech.

That’s why I believed as an alternative of simply spawning variations on eggplants, unicorns and trolls, Genmoji could possibly be a rudimentary coaching device for a new technology of immediate engineers. 

The Mac’s Puzzle and ‘mouse coordination’

Before you write me off as loopy, think about The Puzzle desk accent included in the unique working system software program for the Macintosh 128K pc when it launched in January 1984. Designed by Apple’s creative and resourceful software program engineer Andy Hertzfeld, The Puzzle injected a little bit of enjoyable into the entire thought of proudly owning a private pc at a time when old-school IBM dominated the PC market. 

But The Puzzle’s true contribution could also be that it helped individuals perceive the way to use the Mac’s groundbreaking graphical person interface and aided them to develop into extra comfy utilizing its novel enter gadget, the mouse.

Hertzfeld informed me that was a part of the thought behind it, once I interviewed him for the fifth anniversary of the Mac whereas I used to be working as a reporter for MacWEEK. I’m glad to say he is documented The Puzzle’s origin story on Folklore.org, a web site that options over 100 tales concerning the early historical past of Apple and the Mac written by the individuals who had been there.

“Desk accessories were usually utilities, like the calculator or the alarm clock, but I thought that we should also have a game or two, to show that the Macintosh was fun, too,” Hertzfeld says. “I decided to write a ’15 number puzzle,’ where there are fifteen numbered tiles in a four by four space that must be arranged in sequential order.

“If you click on on a tile subsequent to the empty house, it slides into that house,” he explains. “It was a enjoyable solution to waste time and construct up your mouse coordination.”

Clever, right? Early Mac users were just moving tiles around and playing a game, but what actually happened is that they got comfortable navigating Apple’s GUI and using a mouse, which helped popularize it as a computer input device. (By the way, Hertzfeld also recounts how The Puzzle almost wasn’t included in the OS because it took up too much disk space. So he rewrote the code and got it down to 600 bytes. Read his essay if you’re a Mac fan.)

Anyway, when you consider how a little game helped prepare the planet for the mouse as an input device, it’s not a stretch to view Genmoji as a way to get people thinking about writing prompts. I mean, dreaming up original emoji certainly sounds like a fun way to waste time.

Prompts, avadoas and baby daikon

To write successful prompts, you need to provide enough specifics to get a satisfying answer within the first few asks so you’re not frustrated by the AI’s early outputs. 

I know this because I’ve spent the past year understanding the implications and use cases for gen AI. I also studied over 55 prompt guides, like this one from OpenAI and this one from Adobe, and watched hours and hours of YouTube videos to create a primer for the CNET team that boils my learning down to 15 tips on how to master the art of the prompt. (You can get a copy by signing up for my weekly AI newsletter at CNET’s AI Atlas.)

The trick is to make sure your prompt is specific. Instead of asking for a “blue star” and hoping for the best, try asking for a “a five-pointed, shiny aqua blue star in the type of a graffiti artist.” Or maybe you’re into cat emoji (no judgments here); instead of a “dancing cat,” you might ask for a “moonwalking calico cat sporting aviator sun shades and Converse Chuck Taylors in black.” 

Genmoji, Apple says, will allow you to turn photos of friends, family and I presume enemies into emoji as well. Maybe they’ll let us AI-ify that photo and turn it into a black-and-white, Ansel Adam-esque image or a Kodachrome colorized masterpiece.

Or maybe the Genmoji toolbox will be robust enough to allow us to riff off ideas like OpenAI’s text-to-image experiments with its Dall-E chatbot using fruits and vegetables, including the “avocado chair” and the “child daikon radish in a tutu strolling a canine.” 

Genmoji as a learning environment?

Again, what you’ll be able to ask Genmoji depends on what training data and capabilities Apple builds into the text-to-image tool. With Memoji, Apple provided a deck of characters, and then allowed us to customize features like skin tone, hairstyle and eyes.

When it comes to Genmoji, there aren’t that many details yet about how it will work.

I asked Apple if anyone on their design team, focused on user interface and human factors design, had considered Genmoji as a way to train people to talk to a gen AI chatbot. An Apple spokeswoman referred me to its press release and the Genmoji demo shared during the keynote at the Worldwide Developers Conference. 

There’s also an 11-minute explainer for developers that describes how they can “convey expression to your app with Genmoji.” In it, Apple input experience engineer Aaron Hurley shares the difference between Unicode emoji and Genmoji.

“Traditional emojis aren’t truly photos however are as an alternative a standardized listing of Unicode characters despatched as plain textual content and it is as much as the viewing gadget to render the suitable picture in its personal font, identical to some other textual content content material,” Hurley says. “Personalized photos like Genmoji, alternatively, are distinctive, rasterized bitmaps that may’t be described by a Unicode textual content character.” 

He adds that Genmoji “are picture glyphs” and developers can add them to their apps via a new application programming interface called NSAdaptiveImageGlyph that supports a standard image format, with a square aspect ratio that can be created in multiple resolutions and augmented with metadata, such as content description. 

OK. But I still don’t know if Apple was as clever as Hertzfeld in seeing beyond the fun factor when they came up with Genmoji. 

So I asked a user experience expert what he thought about Genmoji as a nonthreatening, easy way to get average people, some of whom may have not engaged with a chatbot before, to get comfortable talking with an AI tool.

Caleb Sponheim, a user experience specialist with the Nielsen Norman Group and a former computational neuroscientist, told me my idea isn’t that crazy. It could be a way for “Apple to sort of create a low-stakes, low-cost atmosphere for individuals to have a little enjoyable,” he said. But, he added, it all depends on how capable and user friendly the tool is.

“If we’re taking Apple’s presentation at face worth [and that] it is going to work at the very least in addition to they’re saying, then it is going to be an entrance level for individuals to expertise and begin working with generative AI in a picture technology context,” Sponheim said.

But, “it is as much as Apple and their implementation about how usable it’s and the way pleasant of an expertise it’s,” he added. “Because if it is not, if it is difficult or irritating, it is going to flip into one other iMessage characteristic that will get hidden behind the keyboard.”

Even if Apple does create a truly inventive emoji maker, that doesn’t mean people will necessarily enjoy creating with it. A problem with popular AI text-to-image editors today, including OpenAI’s Dall-E and Midjourney, is that they don’t provide good feedback about what’s working and what’s not with your prompts, Sponheim said. The lack of a feedback loop would be frustrating to users trying to figure out how to create an emoji of a relaxing apricot with cucumber slices over its eyes instead of a smiley face if the system doesn’t have any apricot images.

“AI will not be exempt from fundamental interface pointers about the way to make a good, usable expertise,” Sponheim said.

We’ll just have to wait and see how much AI intelligence Apple builds into Apple Intelligence. In the meantime, if you’re excited about the prospects of creating your own emoji, start brainstorming. Genmoji may be just a fun way to waste time — or it could put you ahead of the class in mastering the art of the prompt. 🤷 

Editors’ note: CNET used an AI engine to help create several dozen stories, which are labeled accordingly. The note you’re reading is attached to articles that deal substantively with the topic of AI but are created entirely by our expert editors and writers. For more, see our AI policy.