--- title: "What an AI image edit actually changes" subtitle: "Measuring the pixel drift caused by a single image edit." date: "2026-04-18" tags: ["research", "AI Image", "Deepdive"] --- Every time I ask an AI to edit a photo of me, the result doesn't quite look like me. The difference is hard to name. Most people would miss it, but close friends and family notice. I wanted to find out what changed and whether I could prevent it. All edits use Gemini 3.1 Flash Image Preview. I wanted to run the same matrix against OpenAI's gpt-image-2 but didn't have access to it at the time of writing. I'll update when that changes. ## Methodology The setup is simple. Take a source image and send it to the model with one prompt. Compute the per-pixel RGB distance between the input and output, then render it as a heatmap that runs from black through blue, cyan, yellow, and red. Brighter pixels changed more. This is a case study, not a benchmark. Unless noted otherwise, each condition is one generation. Some outputs came back at a different resolution and were resized to match the input. Most generated files contain JPEG data and, like all Gemini-generated images, include a [SynthID watermark](https://ai.google.dev/gemini-api/docs/image-generation). The heatmaps therefore measure the full output pipeline, including encoding and resizing, not just changes made during generation. **[VisualizationsPreview component]** A `TabbedImage` showing two views of the same edit. The prompt removes the bowl of lemons. - Heatmap, showing the magnitude of each pixel change: `/research/image-edit-drift/gemini/remove_lemons/simple/heatmap_compare.png`. - Colour diff, showing the direction of each pixel change: `/research/image-edit-drift/gemini/remove_lemons/simple/color_diff.png`. Caption: "Two views of the same edit. The heatmap shows how much each pixel changed. The colour diff shows the direction of the change." In the figures below, ignore the brightest patch. That's the intended edit, where the lemons used to be or where the new plate of olives went. It's supposed to change. The interesting part is everything the model wasn't asked to touch. A note on scale: every heatmap in this post (except the colour grid further down) is normalised against the same global ceiling, so brightness means the same thing across cells. If one image looks brighter than another, it actually drifted more. OK. So what does drift look like on a normal edit? ## The baseline AI-generated source (a balcony lunch produced by Gemini) and a one-line prompt: > Remove the bowl of lemons from the table. Click **Show changes** to overlay the heatmap. **[ImageCompare component]** Interactive before/after slider with optional difference-heatmap overlay. - Before, **Original**: `/research/image-edit-drift/gemini/remove_lemons/bare/original.png` - After, **Lemons removed**: `/research/image-edit-drift/gemini/remove_lemons/bare/gemini.png` - Overlay, **Change heatmap**: `/research/image-edit-drift/gemini/remove_lemons/bare/heatmap_compare.png` Caption: "Drag the divider to compare. Toggle Show changes to overlay the per-pixel diff. The bright patch on the table is the intended edit. Everything else is drift." The lemons are gone. At a glance, the rest of the photo looks the same. Open it fullscreen, pick one spot, and drag the slider back and forth. The differences become visible. The basil has more texture. The brick on the railing is redder. These changes are easy to miss on a quick scroll, but hard to unsee once you spot them. The heatmap shows that most of the frame shifted slightly. The largest unwanted changes follow edges and contours in the basil, table, plants, and railing. Most of that drift is in the foreground. The background is more stable. That is how to read every figure in this post. The intended edit is the bright patch. Everything else is drift. The next test was simple. Tell the model not to change anything else and see whether it listens. ## Does prompt strictness help? Same removal, same source, three prompts. Only the preservation instruction changes. **Bare.** Just the action. > Remove the bowl of lemons from the table. **Simple.** The action plus one extra sentence. > Remove the bowl of lemons from the table. Don't change anything else. **Aggressive.** A long instruction naming every object that should stay the same. > Remove the ceramic bowl of lemons from the table. Do not alter any other part of the image in any way. The wine bottle, wine glasses, bread, cutting board, napkin, table, chairs, railing, basil plant, ivy, buildings, cliffs, sea, sky, sun, lighting, shadows, colors, composition, and camera angle must remain pixel-identical to the original. Only the bowl of lemons should be gone, with the table surface behind it plausibly filled in. If naming what to preserve helps, the aggressive heatmap should be visibly tighter than the bare one. **[PromptStrengthStrip component]** A `HeatmapStrip` comparing three prompt strengths on the same edit (remove the bowl of lemons): - **Bare**, action only: `/research/image-edit-drift/gemini/remove_lemons/bare/gemini.png` + heatmap `/research/image-edit-drift/gemini/remove_lemons/bare/heatmap_compare.png` - **Simple**, action plus a preservation request: `/research/image-edit-drift/gemini/remove_lemons/simple/gemini.png` + heatmap `/research/image-edit-drift/gemini/remove_lemons/simple/heatmap_compare.png` - **Aggressive**, full preservation list: `/research/image-edit-drift/gemini/remove_lemons/aggressive/gemini.png` + heatmap `/research/image-edit-drift/gemini/remove_lemons/aggressive/heatmap_compare.png` Caption: "Same edit, three prompt strengths. The drift pattern is roughly the same in each. Listing more objects to preserve had no visible effect." It isn't. In these three runs, all three drift by about the same amount and in about the same places. Listing what to preserve did not visibly change how the model treated those parts of the image. This test found no visible effect from prompt strictness. The next variable was the edit itself. ## Does the edit type matter? Would a removal cause more drift than an addition or replacement? Three edit types on the same source with the simple prompt: - **Remove** the bowl of lemons. - **Add** a small white plate of green olives next to the bread. - **Replace** the white wine with red wine. **[EditTypeStrip component]** A `HeatmapStrip` comparing three edit types on the same source image (each cell shows the model output and a per-pixel-difference heatmap): - **Remove**, bowl of lemons: `/research/image-edit-drift/gemini/remove_lemons/simple/gemini.png` + heatmap `/research/image-edit-drift/gemini/remove_lemons/simple/heatmap_compare.png` - **Add**, plate of olives: `/research/image-edit-drift/gemini/add_olives/simple/gemini.png` + heatmap `/research/image-edit-drift/gemini/add_olives/simple/heatmap_compare.png` - **Replace**, white wine with red: `/research/image-edit-drift/gemini/replace_wine/simple/gemini.png` + heatmap `/research/image-edit-drift/gemini/replace_wine/simple/heatmap_compare.png` Caption: "Same source and prompt shape, three edit types. Every heatmap except the colour grid uses the same ceiling, so brightness is comparable across figures." All three edits produce widespread changes outside the apparent target area. Their whole-image RGB distances differ, but that number includes the intended edit, and those regions are not the same size. One run per edit is not enough to say whether edit type affects off-target drift. What about the source image? Maybe AI-generated photos are easier to repaint, and a real photo or a flat illustration would hold its ground better. ## Does the source image matter? Perhaps the model handles its own images better than real photos or images from another model. A flat illustration might also leave less room for small, unintended changes. I ran the same prompt shape on three more sources: a real photograph with the bread basket removed, a flat-shaded illustration with the lemons removed, and a GPT-generated image with the lemons removed. The last one tests whether Gemini treats its own output differently from another model's. **[SourceTypeStrip component]** A `HeatmapStrip` comparing pixel drift across four source-image types (same model, same simple prompt): - **Base**, remove bowl of lemons: `/research/image-edit-drift/gemini/remove_lemons/simple/gemini.png` + heatmap `/research/image-edit-drift/gemini/remove_lemons/simple/heatmap_compare.png` - **Real photo**, remove bread basket: `/research/image-edit-drift/real/remove_bread/simple/gemini.png` + heatmap `/research/image-edit-drift/real/remove_bread/simple/heatmap_compare.png` - **Illustration**, remove lemons: `/research/image-edit-drift/drawing/remove_lemons/simple/gemini.png` + heatmap `/research/image-edit-drift/drawing/remove_lemons/simple/heatmap_compare.png` - **GPT-generated**, remove lemons: `/research/image-edit-drift/gpt/remove_lemons/simple/gemini.png` + heatmap `/research/image-edit-drift/gpt/remove_lemons/simple/heatmap_compare.png` Caption: "Same model and prompt, four source types. These heatmaps use the same ceiling, so brightness is comparable across figures." This is the first test that shows a clear difference between examples. Every source drifts, but not by the same amount. The illustration drifts the least. Its small palette and flat colours may give the model fewer details and textures to change. The drift is still visible, just weaker than in the other sources. The real photograph drifts slightly more than the baseline. It also contains finer detail and camera artifacts that the model can change. This test does not isolate which of those properties matters. The GPT-generated source also drifts more than the baseline, though its mean RGB distance is lower than the real photo's. These four sources came back with different amounts of drift. But the scenes, resolutions, and edit targets also differ, so this test does not isolate source type as the cause. None came back clean. Each source still contains details the model could reinterpret. What happens when the source has none? ## The colour grid Six flat-colour squares with no gaps between them and no texture. The simplest possible edit. > Change the green square to pink. Don't change anything else. Every square interior has one intended colour. **[ImageCompare component]** Interactive before/after slider with optional difference-heatmap overlay. - Before, **Original**: `/research/image-edit-drift/simple/change_green_to_pink/original.png` - After, **Green → pink**: `/research/image-edit-drift/simple/change_green_to_pink/gemini.png` - Overlay, **Change heatmap**: `/research/image-edit-drift/simple/change_green_to_pink/heatmap.png` Caption: "Six flat colours, one intended change. This heatmap is normalised to its own maximum to make small off-target changes visible. Compare brightness only within this figure." The green square is now pink. The other five are almost identical to the original. Their interiors have slight colour drift, but most of the visible damage follows the edges. The once-hard borders between squares are now softer. Some of that edge softness can come from JPEG encoding and resizing. Even with a simple source, the returned image differs across the whole frame. This test cannot separate generation changes from encoding and resizing, but the API output has no pixel-exact "leave this alone" mode. So if every generation changes the whole image slightly, what happens when you chain two of them? ## Does drift compound? If every call regenerates the whole image, chaining calls should stack the damage. I tested it. Ask for one edit. Then in the same conversation, on top of the previous output, ask for a second unrelated edit. - **Call 1:** Remove the bowl of lemons. - **Call 2:** (same conversation) Add a small white plate of green olives next to the bread. Each cell below shows the output of one call. The heatmap on each cell is *incremental*: call 1's heatmap is drift relative to the original; call 2's is drift relative to call 1's output. We're looking at the dose each call adds, not cumulative damage. **[CompoundingStrip component]** A `HeatmapStrip` showing two sequential edits in the same conversation. Each cell's heatmap is incremental (each call vs the previous step's output): - **Call 1**, remove lemons: `/research/image-edit-drift/gemini/remove_lemons/roundtrip/step1_remove_lemons.png` + heatmap `/research/image-edit-drift/gemini/remove_lemons/roundtrip/original_vs_step1/heatmap_compare.png` - **Call 2**, add olives in the same conversation: `/research/image-edit-drift/gemini/remove_lemons/roundtrip/step2_add_olives.png` + heatmap `/research/image-edit-drift/gemini/remove_lemons/roundtrip/step1_vs_step2/heatmap_compare.png` Caption: "Two sequential edits in one conversation. Each heatmap shows the change from the previous image: call 1 vs the original, then call 2 vs call 1." Both calls produce changes across the whole frame. Mean per-pixel RGB distance is 10.1 on call 1 and 8.0 on call 2. The second call still adds a substantial amount of drift. So drift compounds in this sequence. Cumulative drift after both calls is 14.4. That is larger than either step alone but smaller than their sum because RGB distances are not additive and changes can align or cancel. The final image is 43% farther from the original than it was after the first call. But how similar is the drift between calls? ## Same prompt, two calls We ran the same removal twice. Same source, same prompt, same model, same parameters. Then computed the heatmap directly between the two outputs. No original involved. The diff is between Run 1 and Run 2 of the same edit. **[ImageCompare component]** Interactive before/after slider with optional difference-heatmap overlay. - Before, **Run 1**: `/research/image-edit-drift/gemini/remove_lemons/simple/gemini.png` - After, **Run 2**: `/research/image-edit-drift/gemini/remove_lemons/simple/gemini-v2.png` - Overlay, **Difference between runs**: `/research/image-edit-drift/gemini/remove_lemons/simple/heatmap_v1_v2_compare.png` Caption: "Two outputs of the identical prompt. Heatmap is on the same shared ceiling as the rest of the post, so the relative dimness here is meaningful: two runs differ from each other less than either differs from the original." The two runs aren't identical, but they're close. Their mean per-pixel RGB distance is 4.6, about half the distance from the original to either run. The outputs are stochastic but tightly clustered in this pair. The heatmap is noticeably dimmer than the earlier comparisons between the original and an output. The interesting part is *where* they differ. The brightest patch covers the actual edit, the part of the table where the bowl used to be. That area forces the model to invent pixels that weren't visible in the source. There are many plausible versions of the hidden table, and the model picks a different one each time. Outside the edit zone, the two runs are more similar to each other than either is to the original. The drift is real, but correlated across calls. These results show that the API does not enforce pixel identity outside the requested edit. They do not tell us whether the model uses masks or other constraints internally. ## What I take from this A few practical conclusions if you're building with these models. If pixel retention matters most, avoid chaining calls when the edits can be described reliably in one prompt. Fewer generations mean fewer opportunities for drift. Complex edits may still work better as a conversation. If you need pixel-exact retention, do it outside the model. Mask the edit region, run the model, composite the new pixels onto the original. The API does not guarantee unchanged pixels, no matter how hard you ask. In this test, elaborating the preservation prompt did not help. Bare and aggressive prompts produced the same amount of drift here. A "don't change anything else" sentence can clarify your intent, but this test found no evidence that it protects unchanged pixels. Future models may contain edits better, but this one still changes pixels outside the target area. I want to test a masked workflow next: isolate the edit region, generate it with enough margin for shadows and blending, then composite it onto the original. That should work well for local edits. It may also explain why my face never quite comes back the same. That is the next thing I want to test directly. > One thing worth noting: Every edit in this post is localized. Remove a bowl. Add a plate. Swap a colour. Whole-scene edits, such as changing the background or lighting, are a different question. I haven't tested them here.