Keys for the same thing drift apart in two ways. Fix one and the other stays.
| Direction | Cause | Symptom |
|---|---|---|
| different to same key | truncation cuts the distinguishing suffix | overwritten; only the last survives |
| same to different key | re-registration appends a suffix | the lookup never matches at all |
B0XXXXXXXX -> B0XXXXXXXXAB -> ...R / RR / RRRFix
75% matched after applying this (6,657 of 8,858).
★ Some never match even with expansion — they were minted under a different scheme entirely. If you do not write the counterpart key at minting time, there is no recovering it later.
The first direction in practice — the overwrite shows up as "unindexed"
The external SKU is the source handle truncated to 38 characters, so variants collapse onto one key. index[key] = id keeps only the last one; the rest are unindexed forever. They didn't fail processing — there was no slot for them in the index.
key → id is 1:N. A dict that assumes 1:1 is the wrong structure → keep a separate id → key reverse index and let the forward side hold a list.★ If "unindexed: N" doesn't shrink round after round, suspect key collision, not processing failure. A stalled metric can mean "nowhere to land", not "the work didn't happen". The job exits clean every time; only the number refuses to move.
Truncation length differs per system:
| Where | Truncated at |
|---|---|
| Marketplace A, stored external SKU | 20 chars |
| Local index key (source handle) | 38 chars |
| Marketplace B, seller management code | 30-char limit |
Knowing "it gets truncated" is not enough. The cut-off length defines the collision set. The same data cut at different points per channel collides differently on each.
★★★ And the remedy is not to shorten but to reorder
Option names truncated at 25 characters left every option with the same name.
"Standard (Single Passport) / Green" ← first 25 chars identical
"Green / Standard" ← the distinguishing part moved up front★★★ Truncation always discards the tail. So put what distinguishes up front. Before shortening the string or appending a hash, look at the order — reordering alone keeps a name a human can still read.
★ And the common convention points exactly the wrong way. Naming that leads with a shared prefix (spec name, brand, category) reads well and truncates worst. Don't use the same naming scheme for fields cut at the front and fields cut at the back.
Key expansion must be applied at every lookup site
Put the suffix/prefix expansion into five of six batches and the sixth stays unmatched forever, on its own. "The fix is decided" and "the fix is in every call site" are different facts. grep every call to the lookup and count them before calling it done.