By now, you may have gotten familiar with Google Gemini. No matter which smartphone or laptop you’re using, you probably notice Gemini search results pop up when you search for something via Google. You may have been leveraging features like image generation and writing help. Now, Gemini 2.0 is here with more AI features for you to enjoy for both productivity and convenience. From native intelligence tools to upgrades to the Android experience, here’s a rundown of what you’ll find in the new model.
Gemini 2.0 Flash
This experimental version of the 2.0 models can create or edit images or generate text in different tones. This includes multimodal reasoning, long context understanding, and the ability to follow and plan using complex instructions. It’s available to Gemini Advanced and Devs users in AI Studio and Vertex AI with plans for a wider rollout in January along with compatibility with more Google products.
Project Astra
The Project Astra research prototype involves enhancements to the AI universal assistant. Powered by Gemini 2.0, it allows for more natural conversations and efficient retrieval of information using images, videos, and voice, all of which combine into a timeline. The latest version will include better dialogue so it can converse in multiple and mixed languages, with the ability to better understand accents and uncommon words. It can use Google Search, Lens, and Map so you get a virtual assistant in more applications. Its memory has been improved with up to 10 minutes of in-session memory, affording better personalization. There’s also improved latency. It’s currently being tested with those in the trusted tester program.
Project Mariner
Focused on improving how people interact with the web, Project Mariner combines Gemini’s multimodal understanding capabilities with web interaction to automate tasks. The early research prototype, which was built with Gemini 2.0, can understand and reason with information that appears on your browser screen. This includes pixels and web elements like text, code, images, and forms. It then uses this information via an experimental Chrome extension to complete various tasks on your behalf. While Google notes that Project Mariner isn’t always accurate and it works slowly, it is improving. Currently, it can type, scroll, or click in an active tab but it will require your human final confirmation before completing sensitive actions, like filling out a form or making a purchase. It’s being used by trusted testers right now.
Project Jules
Designed specifically for developers, Project Jules is an experimental AI-powered code agent to which you can offload tasks, like resolving bugs and coding challenges. The developer can give it direction and supervision and Jules can tackle an issue at hand by developing a plan and executing it. It’s currently available to a group of trusted testers.
Deep Research
Deep Research is a new capability for Gemini Advanced users that uses AI to explore complex topics on your behalf. It then provides findings in a comprehensive report. Ideal for students, you can enter a question and Deep Research will create a multi-step research plan for you that you can revise or approve. It then analyzes relevant information from across the web to provide you with a variety of sources from which to use. It basically handles the task of research for you so you can get down to the nitty-gritty and start reading, researching, and selecting which sources you wish to use or discard. It also generates a report of the key findings, ideal for someone studying complex topics. All the information is provided with links to original sources so you can learn more. You can also ask Gemini follow-up questions as you continue on your research journey. It is available in English on desktop and mobile web and will be available in the mobile app in early 2025. Go to “Gemini 1.5 Pro with Deep Search” to get started and enter a research question to try it.
AI Features in Android
There are new AI-powered features for Android users that enhance accessibility, creativity, and productivity. Among these are detailed image descriptions, intelligent notetaking capabilities, seamless file transfers via QR codes, and improved document scanning.
Expressive Captions: This captures the emotion while someone is speaking, including volume and tone, to interpret everything from a whisper to a groan or a cheer. It will appear in streaming, social, and video messages, only on Android devices.
Image Q&A in Lookout: For those who are blind or have low vision, there’s richer, more helpful image descriptions.
Emoji Kitchen: Mix together new pizza-themed sticker combos for fun emojis you can share with friends and family.
QR Codes in Quick Share: You can more easily transfer photos, videos, and documents using QR codes that the other person can scan for secure transfer. You don’t even need to add the person as a contact, verify devices, or change share settings.
Auto-Enhancements in Google Drive: Everything from receipts to documents and ID cards can be optimized when scanned, improving things like white balance and contrast and removing shadows and blur without any manual editing. Just take a photo of the item and it will be saved in a cleaner version for your accounting or records.
Gemini in More Apps: You’ll now find Gemini across more apps through new Extensions. There’s a Spotify Extension, for example, that helps you find favourite songs and discover new playlists. You can use your default phone and messaging apps to make calls or send messages. You can also do things like set alarms and control device settings and your camera. Soon, Gemini will also work for controlling compatible smart home devices that can be accessed from your Google Account, and to get more information about places in Maps.
Pixel Drop
The latest Pixel Drop includes more personalization featured powered by AI, including more tailored responses via the new saved info feature, smart call screening via Gemini Nano on Pixel, auto-organized screenshots, and enhanced audio. Gemini Live is also now available in more languages.
