Google Unveils Next-Generation Model: Gemini 1.5 193

Just two months after launching Gemini (which Google hopes to bring to the pinnacle of the AI industry), the company has already announced a new version. Google today introduced Gemini 1.5 and is making it available to developers and enterprise users ahead of a full consumer release.

Gemini 1.5 features a context understanding capability of 1 million tokens, making it the most powerful to date! Gemini 1.5 possesses cross-modal understanding and reasoning: it is capable of highly complex understanding and reasoning across text, code, images, audio, and video. It allows for the analysis of 1 hour of video, 11 hours of audio, over 30,000 lines of code, or over 700,000 words of text.

Introduction to Gemini 1.5:

1. Performance Improvement: Gemini 1.5 demonstrates significant performance improvements across multiple dimensions, particularly achieving a major breakthrough in its ability to handle long-context information, being able to continuously process up to 1 million tokens, possessing the longest context window of any large foundation model to date.

2. Efficient Architecture: Gemini 1.5 is designed using a new Mixture of Experts (MoE) architecture, making its training and serving more efficient. At the same time, it reduces computational requirements while maintaining quality similar to the 1.0 Ultra model.

When tested on a panel of text, code, image, audio, and video evaluations, Gemini 1.5 Pro outperformed Gemini 1.0 Pro in LLM benchmarks with a score of 87%. And it performs roughly comparably to the largest model, 1.0 Ultra.

3. In-Context Learning Ability: This feature allows Gemini 1.5 to process and analyze large amounts of information, such as 1 hour of video, 11 hours of audio, a codebase of over 30,000 lines of code, or over 700,000 words of text.

When Gemini 1.5 Pro was given a grammar manual on the Kalamang language, it was able to learn new skills from the information provided, and was able to perform translation tasks from English to Kalamang at a translation level similar to that of a human learning from the same content.

Kalamang is a niche language with fewer than 200 speakers worldwide!

This achievement highlights Gemini 1.5 Pro's ability to handle rare or niche languages, and also demonstrates its strong performance in understanding and applying long, complex textual information. This capability has important implications for applications in language learning, natural language processing, and cross-cultural communication.

4. Cross-Modal Understanding and Reasoning: Gemini 1.5 is capable of highly complex understanding and reasoning across different modalities of content (including text, code, images, audio, and video), for example, analyzing a 44-minute silent film and accurately capturing plot points and details.

5. Enhanced Performance: When tested on a comprehensive panel of text, code, image, audio, and video evaluations, it is capable of highly complex understanding and reasoning across text, code, images, audio, and video. Gemini 1.5 can precisely analyze the plot points in a 44-minute silent film, demonstrating a deep understanding of different media content.

How Gemini 1.5 Works:

MoE Technology: By dividing the model into smaller "expert" neural networks and selectively activating the most relevant expert paths based on the input type, MoE technology greatly improves the model's efficiency. Through a series of machine learning innovations, Gemini 1.5 is able to process and analyze unprecedented amounts of information, delivering new capabilities for developers and enterprise customers to help their models and applications. This long-context window capability is particularly suited to application scenarios that require processing large amounts of data and complex contexts, such as in-depth analysis of large documents, video content understanding, complex code analysis, and multimodal data fusion.

Google is currently engaged in an intense race to build the best artificial intelligence tools, as businesses around the world are trying to figure out their AI strategies and whether to sign developer agreements with OpenAI, Google, or other companies. On the very same day, just 2 hours after Google released Gemini Pro 1.5, OpenAI announced its text-to-video model: Sora, which aims to allow web users to generate high-quality AI videos through text prompts alone.