ETL Idempotency

Check whether a pipeline can be re-run without doubling its output, and what to key on if not.

When to use this

Reach for this when the situation above is the one you're actually in. It does one job; anything broader belongs to another skill.

Steps

1. Read what you were given before changing anything — the code, the query, the document, whichever applies here.
2. Work out what's actually wrong. State it in one sentence before proposing a fix, so the fix has something to answer to.
3. Make the smallest change that answers it.
4. Say what you changed and what you deliberately left alone.

Output

A short report: what you found, where (file and line where that applies), and the change you'd make. Rank by what breaking would cost, not by how easy each one is to fix.

Out of scope

Anything that isn't data. Say so and stop rather than wandering into a neighbouring problem.