Everything announced at Google I/O 2024, including Gemini AI, Project Astra, Android 15 and more

At the end of I/O, Google’s annual developer conference at the Shoreline Amphitheater in Mountain View, Google CEO Sundar Pichai revealed that the company said “AI” 121 times. That was essentially the gist of Google’s two-hour keynote — bringing AI into every Google app and service used by more than two billion people around the world. Here are all the major updates that Google announced at the event.

Gemini 1.5 Flash and updates to Gemini 1.5 Pro

Google has announced a brand new AI model called Gemini 1.5 Flash, which it says is optimized for speed and efficiency. Flash sits between the Gemini 1.5 Pro and the Gemini 1.5 Nano, which is the company’s smallest model that runs natively on a device. Google said it created Flash because developers wanted a lighter and cheaper model than Gemini Pro for building AI-powered apps and services while preserving some of the things like a long context window of one million tokens that sets the Gemini Pro apart from competing models. Later this year, Google will double Gemini’s context window to two million tokens, meaning it will be able to process two hours of video, 22 hours of audio, more than 60,000 lines of code, or more than 1.4 million words simultaneously.

Project Astra

Google showed off Project Astra, an early version of an AI-powered universal assistant that Google DeepMind CEO Demis Hassabis said is Google’s version of an AI agent “that can be useful in everyday life.”

In a video that Google says was shot in one shot, an Astra user walks around Google’s London office holding his phone and pointing the camera at various things — a speaker, some code on a whiteboard and through a window — and there a natural conversation with the app about what it looks like. In one of the most impressive moments in the video, she correctly tells the user where she left her glasses before, without the user ever picking up the glasses.

The video ends with a twist – when the user finds and wears the missing glasses, we learn that they have a built-in camera system and can use Project Astra to seamlessly have a conversation with the user, perhaps indicating that Google may be working on a competitor to Meta’s Ray Ban smart glasses.

Ask Google Photos

Google Photos was already smart when it came to searching for specific images or videos, but with AI, Google is taking things to the next level. If you’re a Google One subscriber in the US, you’ll be able to ask Google Photos a tricky question like “show me the best photo from every national park I’ve visited” when the feature rolls out in the next few months. Google Photos will use GPS information as well as its own judgment of what is “best” to provide you with options. You can also ask Google Photos to generate captions to post the photos on social media.

Veo and Imagen 3

Google’s new AI-powered media creation engines are called Veo and Imagen 3. Veo is Google’s answer to OpenAI’s Sora. It can produce “high-quality” 1080p videos that can last “over a minute,” Google said, and can understand cinematic concepts like timelapse.

Imagen 3, meanwhile, is a text-to-image generator that Google claims handles text better than its previous version, Imagen 2. The result is the company’s highest-quality text-to-image model with an “incredible level of detail” for “photorealistic , realistic images” and fewer artifacts – essentially pitting it against OpenAI’s DALLE-3.

Big updates to Google Search

Google is making big changes to the way search works. Most of the updates announced today, like the ability to ask really complex questions (“Find the best yoga or pilates studios in Boston and show details about their introductory offers and walk-in times from Beacon Hill.”) and using Search to schedule of food and vacations won’t be available unless you opt in to Search Labs, the company’s platform that lets people try out experimental features.

But a big new feature that Google calls AI Overviews, which the company has been testing for a year, is finally rolling out to millions of people in the US. Google Search will now present AI-generated answers on default results, and the company says it will roll out the feature to more than one billion users worldwide by the end of the year.

Gemini on Android

Google has integrated Gemini directly into Android. When Android 15 is released later this year, Gemini will know about the app, image or video you’re running, and you’ll be able to pull it up as an overlay and ask it contextual questions. Where does that leave Google Assistant, which already does this? Who knows! Google didn’t bring it up at all during today’s keynote.

There were also a bunch of other updates. Google said it will add digital watermarks to AI-generated video and text, make Gemini available in the sidebar in Gmail and Docs, power a virtual AI teammate in Workspace, listen to phone calls and detect if you’ve been scammed in real time time and much more.

Catch up on all the news from Google I/O 2024 right now here!