Two checks before you use a field as an identifier

Before a field becomes a key:

1. Is it unique? No collisions.
2. Is it meant to identify? Does the field exist to point at the target?

Check ① without ② and you'll key on a search field like tags.

Check ② without ① and a legitimate identifier will bite you through truncation or parent-sharing.

Field used	What broke
External SKU field	① — stored truncated at 20 chars. Duplicates in the 40% range
Indexing by product name	①② — state files mixed, an empty husk claimed the slot first
Product-name matching	② — three incidents in a row
Platform tags	② — long-tail search field. One tag misfiled a whole product line
Progress file name	① — another script used the same name with a different format → ~200 items lost
Category name	② — partial-match false positives. Fixed by pinning to leaf ID
Two fields on one record	① — the key and the URL pointed at different targets
Mail sender domain	② — it is the sending infrastructure's domain, so every store shares one value. Second-stage check on the From display name

Rules

• Never match on a single field without cross-checking.
• Eyeball a sample of every match result. A plan with dozens of mismatches nearly passed because "the names looked right."
• Before creating a progress file, check it isn't taken: grep -rn "filename" *.py
• If one record has two fields pointing at the same target, compare them. They can disagree with no error.

Prefix lookup on a truncated key: adopt only when unique

The external SKU is cut at 20 characters, so the original key can't be found and you fall back to a prefix search. Within one brand's 300 items, 42 collided on the first 20 characters. If two or more candidates match, give up and hand off to the similarity fallback. Unlinked beats mislinked.

Measure link rate against a fixed denominator (what is registered). The map total keeps growing because other scripts keep adding to it; measured against the total, progress looks like nothing.

Extension and declared MIME are not evidence of format

PNG bytes uploaded with image/jpeg declared: upload 400. The filename and the declared type are labels a human attached; they fail check ②. Judge the format by magic bytes, convert to what the receiver accepts, then upload.

Repair procedure when two fields on one record disagree

46 of 1,928 records had a key and a URL pointing at different products — a direct path to ordering the wrong item.

1. The side registered in a separate index is canonical — if the key exists in the SKU index, trust the key.
2. Regenerate the other side (the URL) from the canonical one.
3. Validate that both fields name the same target at insert time. Cheaper than reconciling afterwards.

The reverse of "different things, same name" — the same thing got two names

.job_seen and .jobs_seen both existed. One plural s split the processing history across two files: what A recorded B didn't know, so B reprocessed; what B recorded A didn't know, so A never sent. No error. It isn't a typo — it is a separate file.

• State-file paths live in one constant. Never scatter them as string literals.
• Two similar names are themselves the signal — singular/plural, underscore or not, an abbreviation.

Exactly the mirror of the progress-file row above: that was different things sharing a name; this is one thing with two names. Both come from not treating the name as a contract.

---

---