Skip to content
Pocket Grove
Developer blog

Google's BERT to Gemini 4: What Actually Changed, in Plain English

Published · 4 min read

By · Pocket Grove

Original research brief: . Sources reviewed: .

Quick answer: Google invented the Transformer that every modern chatbot uses, and then moved through understanding search queries (BERT), chatting (LaMDA and Bard), seeing and hearing everything at once (Gemini 1.0), reading huge inputs (Gemini 1.5, 1M tokens), using tools on its own (Gemini 2.0), thinking first (Gemini 2.5), to coding agents (Gemini 3 and Gemini 4 Argon).

This is the Google version of our GPT-2 to GPT-6 explainer. Each step links to Google's own posts or docs.

2017–2018: the Transformer and BERT

Google researchers published Attention Is All You Need in 2017. The "T" in GPT stands for that Transformer.

BERT (2018) used it to understand text rather than write it, and Google used it to improve Search.

How you used it: you didn't, directly. Your searches just got better.

2021–2023: LaMDA, Bard and PaLM 2

LaMDA was a model trained for open-ended conversation. In February 2023 it powered Bard, Google's first public chatbot. A few months later PaLM 2 took over, bringing better reasoning, coding and languages.

How you used it: a chatbot, Google's answer to ChatGPT.

December 2023: Gemini 1.0, built for every kind of input

Gemini was built to be natively multimodal: trained on text, images, audio and video together from the start, rather than having vision bolted on later. It came in three sizes, Ultra, Pro and Nano (small enough for phones). Bard was renamed Gemini soon after.

How you used it: show it anything, ask about it.

2024: Gemini 1.5 and 2.0, huge inputs and tools

Gemini 1.5 (February 2024) was about size of input: a 1 million token context window, later 2 million. That's enough for a codebase, several books, or an hour of video.

Gemini 2.0 (December 2024) was Google's "agentic era" launch. The model could call tools itself, such as Google Search and running code.

How you used it: dump everything in, let it look things up.

2025: Gemini 2.5 and 3, thinking and coding agents

Gemini 2.5 (March 2025) made Gemini a "thinking model" that reasons before answering, with a thinking budget developers could set.

Gemini 3 (November 2025) was pitched as Google's strongest model for agents and coding. It launched alongside the Antigravity agent coding platform, with a Deep Think mode. In the API, simpler thinking levels replaced exact budgets. Gemini 3 Flash followed in December as the fast default.

2026: faster Flash models and Gemini 4

Gemini 3.5 Flash (May) and 3.6 Flash (July) kept making the fast tier cheaper and more efficient. Older preview models were switched off or quietly redirected to newer ones, which is worth knowing if you build on the API (see the changelog).

Gemini 4 Argon became available on 13 October 2026, described by Google as its most capable model for complex coding, agent, maths and science tasks. Its context window is still 1M tokens. Google has also announced Gemini 4 Carbon for long engineering and security work, but it isn't public yet.

How you use it: pick a thinking level, let the agent work.

What this means for you

  1. Context stopped growing at around 1M tokens. Since Gemini 1.5, the gains have come from thinking, tools and price, not bigger windows. Read context engineering explained for why stuffing the window isn't free.
  2. The thinking level is your main dial. Turn it up for hard problems and down for speed and cost.
  3. Pin your model IDs. Google retires and redirects models quickly. Check the changelog before you assume yesterday's model is still the one answering.

Compare with OpenAI's GPT-2 to GPT-6 and Anthropic's Claude 1 to Claude 5.

Frequently asked questions

What is Google's newest Gemini model?
As of November 2026, Gemini 4 Argon is Google's newest generally available model, released on 13 October 2026 with a 1 million token context window. Google has also announced Gemini 4 Carbon, but it is only in limited pre-release testing.
What does natively multimodal mean for Gemini?
Google built Gemini 1.0 to be trained on text, images, audio and video together from the start, rather than adding image support to a text model afterwards.
When did Gemini get a 1 million token context window?
Gemini 1.5 introduced a 1 million token window in February 2024, later extended to 2 million. Since then Google has focused on reasoning, tools and price rather than bigger windows.
What are Gemini thinking levels?
Gemini 2.5 let developers set an exact thinking budget. From Gemini 3, the API uses simpler thinking levels instead, so you choose roughly how hard the model should think.

Apps from the studio

All apps

These practices come from shipping Pocket Grove's active apps. If you came here looking for something to install, start with one of these.

Related guides