Foundation Models framework
Tap into the power of Apple Intelligence
You now have direct access to the on-device foundation model at the core of Apple Intelligence so you can build experiences that are smart, private, and work without internet connectivity through the Foundation Models framework. With native support for Swift, you can tap into the model with as few as three lines of code. You can use this framework to power intelligent features in your app, with model capabilities, such as text extraction, summarization, and more. Features like guided generation, tool calling, and more are built into the framework, making it easier than ever to implement intelligent experiences right into your existing apps and games.
Visual intelligence
Seamlessly explore the content on your iPhone screen
Visual intelligence now extends to the screen, in addition to the iPhone camera. People can search, take action, and ask questions about the content on their screen, by pressing the same buttons used to take a screenshot.
With App Intents, you can integrate search capabilities from your apps into visual intelligence. This allows people to search visually across your app for the content on their iPhone screen, or for physical objects in their surroundings.
Shortcuts
Help people create more powerful workflows
Now people can build actions that combine in powerful ways with new intelligent actions in the Shortcuts app. With App Intents, your app’s actions and entities can work with features like Writing Tools, Image Playground, and a new “Use Model” action that lets people tap directly into Apple Intelligence models on-device or with Private Cloud Compute to provide responses that feed into the rest of their shortcut.
Genmoji
The perfect emoji does exist
Now you can enhance your app to let people make the perfect emoji to express themselves in the moment while using your app. New ways to make Genmoji include the option to mix together favorite emoji and Genmoji with descriptions to create something brand new, and people will have more control over their Genmoji that are inspired by family and friends.
Genmoji is automatically supported as stickers in your app when you use system text controls, and there are APIs for you to render them with custom text engines.
Image Playground
Image generation capabilities right in your apps
With Image Playground, people can create unique images using concepts, descriptions, and people from their Photos library. Now, they have even more control over images inspired by people and the option to tap into more styles with ChatGPT.
When you use the Image Playground API you can support these new image creation capabilities right from your apps. And with the Image Creator API you can programmatically create images within your app’s experience using our on-device model.
Writing Tools
Transform how you communicate
Writing Tools are available system-wide, and help people rewrite, proofread, and summarize text. If you’re using any of the standard UI frameworks to render text fields, your app will automatically have the ability to use Writing Tools, or you can adopt APIs to provide your users access to these capabilities in your app.
