Apple held its annual WWDC event this week in California, and the company delivered on the topic everyone has been waiting to hear about from the brand: AI. How will Apple integrate AI into its products in a bigger way, when will this happen, and what will it look like? The answer is Apple Intelligence, and a partnership with OpenAI that will see deeper ChatGPT integration across Apple devices as well.
What is Apple Intelligence?
Apple Intelligence is Apple’s name for its AI system that works across iPhone, iPad, and Mac with iOS 18, iPadOS 18, and Mac Sequoia, respectively. While many of the other upgrades that come with these three OS updates will be available on older devices as well, Apple Intelligence specifically will work with fewer of them. For iPhone, you will only be able to use Apple Intelligent on iPhone 15 Pro and 15 Pro Max (as well as future devices) and for iPad and Mac, it’s only accessible on models with M-series chips. Apple Intelligence will be available in English beta this Fall.
What does it do? Like other AI systems, Apple Intelligence focuses on generative models that understand personal context to deliver more useful and relevant assistance across various apps and services. It can better understand natural language commands both spoken and written, generate unique images leveraging your photos and specific commands, take different intelligent actions across apps, and use context to provide more detailed and useful information as you perform different tasks.
What about privacy? Apple Intelligence works through Private Cloud Compute. It predominantly uses on-device processing, only switching to larger, server-based models that run on Apple silicon servers when more context is needed to reply to queries or perform certain actions. Apple says the data is never retained nor exposed, and independent experts can inspect the code to verify privacy.
How Can You Use Apple Intelligence?
Apple Intelligence can be used across a variety of apps on various Apple devices. Here are some of the key ways you can use it.
Writing Tools
Apple Intelligence includes Writing Tools to help you rewrite, proofread, and summarize text in Mail, Notes, Pages, and other compatible third-party apps. With Rewrite, for example, you can take what you have written and adjust the tone to make it more professional, friendly, or concise. This could be used while writing a cover letter for a job or sending a cheekier invitation to friends for a summer barbecue. Proofread, naturally, checks for grammar, word choice, spelling, and sentence structure and suggests edits as well, along with an explanation for why these edits are being suggested. You can review, accept, or keep your writing as-is. Summarize will take selected text and recap it for you in a simple to read, concise paragraph with a bullet list of key points if necessary, or even a table or list.
Mail and Notifications
If you use the Mail app, you’ll be able to see Priority Messages at the top of your inbox, which highlights urgent e-mails. These might include details for a dinner party that night or a boarding pass for your upcoming flight. Previews will also no longer show the first line of an e-mail but will intelligently display a summary of the contents so you can get the gist before actually opening the message. Smart Reply, meanwhile, will suggest a quick response that identifies questions in the message and includes relevant answers. There’s now also the option to schedule an e-mail or message.
This extends to Notifications as well, with priority ones being shown at the top of the stack along with summaries of key details. With a new Focus mode, only notifications that require immediate attention will show up.
Recording a Call
If you’re recording a call with the other person or group’s permission, you can have it automatically transcribed with key points summarized. Imagine doing this while on a call with your cell phone carrier where a rep is running through all the plan options that you want to review and get back to them about; or a call about an event. You can remember who is doing what and what you need to take care of without scribbling notes the entire time.
Image Playground
Image Playground works in Notes, too, so you can make yours more visually engaging. It can modify rough sketches into something that looks more professional, or create an image in a blank spot based on the copy that surrounds it. It works in Keynote, Freeform, Pages, and third-party apps that use the new API as well.
Genmoji
Building on Image Playground is Genmoji, which will allow you to create your own original versions of Memojis, both of yourself and others. Use a person’s photo and type a description, like that you want to see your mom as a superhero or your son as a basketball player. Send the creation in messages, save it as a sticker to use later, or use it as a Tapback reaction (when you press and hold a message to reply with an expressive emoji).
New Features in Photos
The Photos app will be entirely revamped to make viewing photos more seamless in one continuous scroll. From an AI perspective, Search will also be improved. You can use natural language to search for photos not just from a specific time, place, or person, but drilling down to specifics like “Christine on a beach in a dress.” With videos, you can even search for specific moments in clips like when Patrick hit the home run.
Apple’s answer to Magic Eraser in Google Pixel devices and Object Eraser in Samsung Galaxy phones is called Clean Up. Like those tools, you can use it to easily highlight a distracting object or person from the background of a photo and have it removed while AI fills in the rest.
Memories are also editable using natural language descriptions to choose what you want to include and the device will choose the best photos and videos that fit your description. The images will be turned into a story with chapters based on themes and arranged with a narrative arc. I can already see using this to create Memories of my workout journey through the years, for example. Making this even more immersive, Apple Intelligence will suggest a fitting song from Apple Music to accompany the slideshow.
Siri Improvements
Siri is also getting a drastic overhaul with Apple Intelligence. It will have richer language understanding capabilities and provide more contextually relevant and personal replies. It can maintain context from one query to the next, so if you ask about a venue, for example, then say “how long will it take to get there?” it will know where “there” is.
Instead of the pulsing circle at the bottom of the phone, Siri will also now appear as a glowing light around the edge of the screen. Plus, it can continue to use voice commands as well as typed ones, and you can switch back and forth based on what feels most comfortable at the time (like instead of yelling “Hey Siri” in the middle of a crowded subway station.
Siri now offers user support so you can ask it how to do certain things on the device (including many of the new features in the latest OSs), like how to schedule a Message. Its contextual understanding flows across apps and services, so you can add a new address a friend sends in iMessage directly to their contact card using Siri. Or you can ask Siri to send today’s workout photo to your Fitness Gals group chat. You could even ask it to do things like play a song a friend mentioned without having to confirm if it was sent via e-mail or iMessage; or to get your spouse’s flight information and it will pull it up instantly.
What About ChatGPT?
How will ChatGPT work? Currently, you can access the ChatGPT via the dedicated app on iPhone. But this new integration runs much deeper. When a question can’t be answered or a document or image requires more context than Apple Intelligence can give, you will be asked if you’d like to hand it off to ChatGPT? If you agree, ChatGPT will chime in to help. It might be to provide a recipe with ingredients you have in your kitchen, or give more detail about a concept you want to learn about.
ChatGPT will be available with the aforementioned Writing Tools to help you generate content as well. You could ask Siri to create bedtime story to read with your child that features puppies, unicorns, and a cute little farm and it will ask if you want to get that with the help of ChatGPT.
When using ChatGPT on Apple devices, powered by GPT-4o, IP addresses are obscured and OpenAI says it won’t store requests. You can use it for free without an account, or if you already have a paid account, you can leverage the added features for which you have access.
Apple Intelligence Will Continue to Get Smarter
Many of the features in Apple Intelligence are late to the game, far behind other smartphones, tablets, and computers that have been using AI for some time now. But it isn’t a race, and if the features work swimmingly, Apple users will be overjoyed to finally join the party.
AI will become an integral part of the Apple experience, and with the brand’s products already so tightly integrated, there’s a lot it will be able to help with to make your daily tasks more convenient, quick, easy, and personal.
-30-
More on Apple
