Google's BERT to Gemini 4: What Actually Changed, in Plain English
Published · 4 min read
By Cong Le · Pocket Grove
Original research brief: . Sources reviewed: .
Quick answer: Google invented the Transformer that every modern chatbot uses, and then moved through understanding search queries (BERT), chatting (LaMDA and Bard), seeing and hearing everything at once (Gemini 1.0), reading huge inputs (Gemini 1.5, 1M tokens), using tools on its own (Gemini 2.0), thinking first (Gemini 2.5), to coding agents (Gemini 3 and Gemini 4 Argon).
This is the Google version of our GPT-2 to GPT-6 explainer. Each step links to Google's own posts or docs.
2017–2018: the Transformer and BERT
Google researchers published Attention Is All You Need in 2017. The "T" in GPT stands for that Transformer.
BERT (2018) used it to understand text rather than write it, and Google used it to improve Search.
How you used it: you didn't, directly. Your searches just got better.
2021–2023: LaMDA, Bard and PaLM 2
LaMDA was a model trained for open-ended conversation. In February 2023 it powered Bard, Google's first public chatbot. A few months later PaLM 2 took over, bringing better reasoning, coding and languages.
How you used it: a chatbot, Google's answer to ChatGPT.
December 2023: Gemini 1.0, built for every kind of input
Gemini was built to be natively multimodal: trained on text, images, audio and video together from the start, rather than having vision bolted on later. It came in three sizes, Ultra, Pro and Nano (small enough for phones). Bard was renamed Gemini soon after.
How you used it: show it anything, ask about it.
2024: Gemini 1.5 and 2.0, huge inputs and tools
Gemini 1.5 (February 2024) was about size of input: a 1 million token context window, later 2 million. That's enough for a codebase, several books, or an hour of video.
Gemini 2.0 (December 2024) was Google's "agentic era" launch. The model could call tools itself, such as Google Search and running code.
How you used it: dump everything in, let it look things up.
2025: Gemini 2.5 and 3, thinking and coding agents
Gemini 2.5 (March 2025) made Gemini a "thinking model" that reasons before answering, with a thinking budget developers could set.
Gemini 3 (November 2025) was pitched as Google's strongest model for agents and coding. It launched alongside the Antigravity agent coding platform, with a Deep Think mode. In the API, simpler thinking levels replaced exact budgets. Gemini 3 Flash followed in December as the fast default.
2026: faster Flash models and Gemini 4
Gemini 3.5 Flash (May) and 3.6 Flash (July) kept making the fast tier cheaper and more efficient. Older preview models were switched off or quietly redirected to newer ones, which is worth knowing if you build on the API (see the changelog).
Gemini 4 Argon became available on 13 October 2026, described by Google as its most capable model for complex coding, agent, maths and science tasks. Its context window is still 1M tokens. Google has also announced Gemini 4 Carbon for long engineering and security work, but it isn't public yet.
How you use it: pick a thinking level, let the agent work.
What this means for you
- Context stopped growing at around 1M tokens. Since Gemini 1.5, the gains have come from thinking, tools and price, not bigger windows. Read context engineering explained for why stuffing the window isn't free.
- The thinking level is your main dial. Turn it up for hard problems and down for speed and cost.
- Pin your model IDs. Google retires and redirects models quickly. Check the changelog before you assume yesterday's model is still the one answering.
Compare with OpenAI's GPT-2 to GPT-6 and Anthropic's Claude 1 to Claude 5.
Frequently asked questions
- What is Google's newest Gemini model?
- As of November 2026, Gemini 4 Argon is Google's newest generally available model, released on 13 October 2026 with a 1 million token context window. Google has also announced Gemini 4 Carbon, but it is only in limited pre-release testing.
- What does natively multimodal mean for Gemini?
- Google built Gemini 1.0 to be trained on text, images, audio and video together from the start, rather than adding image support to a text model afterwards.
- When did Gemini get a 1 million token context window?
- Gemini 1.5 introduced a 1 million token window in February 2024, later extended to 2 million. Since then Google has focused on reasoning, tools and price rather than bigger windows.
- What are Gemini thinking levels?
- Gemini 2.5 let developers set an exact thinking budget. From Gemini 3, the API uses simpler thinking levels instead, so you choose roughly how hard the model should think.
Apps from the studio
All appsThese practices come from shipping Pocket Grove's active apps. If you came here looking for something to install, start with one of these.
Related guides
Claude 1 to Claude 5: What Actually Changed, in Plain English
Anthropic's Claude from 2023 to 2026 explained simply: 100K context, vision, computer use, extended thinking, Claude Code, 1M context and effort.
GPT-2 to GPT-6: What Actually Changed, in Plain English
GPT-2 to GPT-6 Astra explained simply: how OpenAI models went from finishing text to reasoning, using tools and taking corrections mid-task.
AGENTS.md vs CLAUDE.md: What They Are and How to Use Both
AGENTS.md vs CLAUDE.md explained: what each file is, how Codex and Claude Code load them, which wins, and how to keep one shared file with a symlink or import.