# Gemini 1.5 and the million-token memory

URL: https://ai-pulse.aglaganova.com/blog/gemini-1-5-and-the-million-token-memory
Published: 2026-10-09
Author: Roman Aglaganov, Full-stack developer
Category: Models
Region: Global
Tags: benchmarks

> In 2024, Google showed a model that could read a whole book, an hour of video or a full codebase in one go. It even learned a language with fewer than 200 speakers from a grammar book.

![Typographic cover: 1,000,000 tokens of context, a whole book read at once](https://ai-pulse.aglaganova.com/api/media/file/gemini15-1.png)

Every language model has a *context window*: the amount of text it can look at in one go, measured in tokens, the word-pieces models read. For years these windows were small. Early ChatGPT could hold a few thousand tokens, roughly a long essay, and would forget the start of a conversation if you went on too long.

## One million tokens

In February 2024, Google announced Gemini 1.5 Pro with a context window of up to one million tokens, up from 32,000 in Gemini 1.0. In Google’s words, that is about an hour of video, 11 hours of audio, a codebase of more than 30,000 lines or over 700,000 words, all at once. In research, they tested up to 10 million.

## Finding the needle

A big window is useless if the model cannot find things in it. The standard test is called *Needle in a Haystack*: hide one specific fact somewhere in a huge block of text, then ask about it. Gemini 1.5 Pro found the needle 99% of the time. It is the AI equivalent of reading the whole phone book and remembering one number on page 4,000.

## Learning Kalamang

The most striking result was in the technical report. Kalamang is a language with fewer than 200 speakers worldwide, so there is next to nothing about it in a model’s usual training data. Researchers gave the model a grammar manual for the language in its context window, and it learned to translate English into Kalamang at a level similar to a person who had studied the same material. It did not know the language before; it learned it from the book, on the spot.

**Why it matters:** long context changed how people use AI. Instead of chopping documents into pieces, you can now give a model the whole contract, the whole codebase or the whole film and ask questions. Today million-token windows are common across the big labs.

## Sources

- [Google: Our next-generation model, Gemini 1.5](https://blog.google/technology/ai/google-gemini-next-generation-model-february-2024/)
- [arXiv: Gemini 1.5 technical report](https://arxiv.org/abs/2403.05530)