Symptom
A scorer read pages and rated trustworthiness. Positive signals +1, risk signals -2. On a run, three long-standing, entirely legitimate counterparties came back at -4 — the highest risk band.
Cause
Rate limiting meant the pages were never read at all. Yet the score landed at the negative floor rather than at zero. The asymmetry in the rules is why.
| On a failed fetch | |
|---|---|
Positive rule (has an email address, +1) | unconfirmed → 0, neutral |
Risk rule (has no email address, -2) | "absent" holds, so it fires → penalty |
★★★ Positive rules key off presence; risk rules key off absence. So one failed fetch forfeits the credit and triggers the penalty at the same time. The single fact "we could not check the email" loses +1 and takes -2 — a three-point swing. With several signals it compounds.
★★★ An event that should be neutral acts as a conviction. And it fails in the most plausible-looking direction — a top-risk verdict looks like the scorer did its job, not like it never read anything.
Fix
Carry a fetched flag and separate "could not check" from "not present."
★ When fetched is false, do not evaluate the penalty rules at all. Not a zeroed score, not a neutral value — withhold the verdict and give it a fourth outcome: UNKNOWN. With only safe/caution/risk available, failure must leak into one of the three.
if not fetched:
return Verdict.UNKNOWN # not a zero score. not a grade.★ And remove the cause too — here, spacing the requests. Fixing the scorer and fixing the collector are separate jobs, and you need both.
★★★ This was the third repeat
Two others are already in this collection — a failed fetch read as "out of stock", and a blocked search read as "zero results." Here a block was read as "dangerous."
★★★ That is not coincidence. Scorers are usually built to look for evidence of normality — so failing to obtain evidence produces a negative verdict automatically. Every time you write a new classifier, ask one question: "what does this return when the input could not be read?" Most classifiers never define that path, and an undefined path falls to the worst grade.
How to verify
★★ Run the scorer with the network cut. Every item should come back UNKNOWN. If even one comes back graded, that rule is penalizing absence. A single assertion catches it.
assert all(v is Verdict.UNKNOWN for v in run_all(offline=True))★ Also check which way the false positives point. A rule that treats size as a risk signal (too many distinct suppliers) will flag every legitimately large counterparty. Look at how both tails of the normal distribution land on your rules.
★★★ And the signal itself may not be a signal — format validation is not attribution
+1 if an email address is present, implemented as a regex, was awarding points to these:
| Address found | What it actually was |
|---|---|
back-in-stock@notifyboost.net | injected by a third-party app |
support@storefront.com | a platform template default |
example@mail.com | a placeholder |
★★★ "there is an email on the page" and "this business has an email" are different propositions. A regex only checks shape. Whether the value you found actually belongs to the subject is a separate check — here: the domain must match, or the address must contain the brand name.
★★★ Worse is the direction of the error. All three appear more often on the suspicious subjects. The more a site was left as a stock template, the more likely the platform defaults and placeholders survive untouched. So this rule hands bonus points to the least trustworthy subjects. The false positives are not random — they point backwards.
★★ When you design a signal, also ask "how easy is this to fake?" Anything easy to forge or accidentally inherit must not be a positive signal. Use it only as a penalty, or attach an attribution check before it earns credit.
★★ Set thresholds from normal samples
The safe line was set at 3 to fit one malicious sample scoring -7. A long-standing legitimate counterparty then fetched successfully and still scored 0 — "caution." Not a fetch problem; the threshold itself was wrong.
★ Look at the score distribution of your normal population before drawing the line. A threshold fitted to a single bad case cuts the good ones. And you usually have exactly one failure case but as many normal cases as you like — calibrate from the side with enough material to see a distribution.
Measuring 10 legitimate subjects gave 1–7 (median 5); the one malicious sample scored -6. A gap of 7. Yet the threshold sat at 3 — not the middle of the gap (-2.5) but well inside the normal distribution. One legitimate subject fell below the line and another landed exactly on it.
★★★ Even so, "the gap is 7, so it's stable" is not a claim you can make — there is one malicious sample
Ten normal, one malicious. ★★★ One point tells you nothing about a distribution. The second bad actor could score -1. "Calibrate from the normal side" being right does not mean you know the malicious side.
★★ So don't force a verdict on the region you have no data for — add a band. Put a borderline — human check before proceeding band in the empty zone between the two distributions, and the classifier commits only where it has evidence and hands off where it doesn't. Same reasoning as UNKNOWN above: don't attach a grade where you have no confidence. Recalibrate once a few malicious samples have accumulated.
★★★ The fetched flag belongs on the signal, not on the subject
Once you drop the bonus and keep only a weak penalty for absence, that penalty becomes entirely dependent on fetched being accurate. And real collection is not all-or-nothing — the main page loads but the contact page fails, so fetched=True while that particular signal's evidence was never seen. → penalty. The bug you fixed comes back in partial-failure form.
★★ Each signal must carry its own "did I actually read my evidence?" — fetched_email, fetched_address. A single subject-level flag rounds partial failure up to full success.
★★★ Removing a bias means re-deriving the thresholds — the correction shifts the scale, not the separation
Re-measured after splitting the flag per signal:
| Lowest normal | Malicious | Gap | |
|---|---|---|---|
| Before | 1 | -6 | 7 |
| After | 2 | -5 | 7 |
★★★ The gap is unchanged; both sides moved up together. The unfair penalty applied equally to legitimate and malicious subjects — a failed fetch does not discriminate.
★★ So removing the false positive did not improve discrimination. What improved is what the score means: it now reflects only evidence actually read. ★★★ But the threshold was fitted to the old scale — change the scoring and leave the threshold, and the threshold quietly comes to mean something else. Touch the score computation, always re-derive the threshold.
★★★ Never pin a threshold to the minimum
After recalibration the safe line was set at 2 — and the lowest legitimate sample was exactly 2. The same shape as the previous round, where the minimum landed right on the line.
★★★ The minimum is a statistic that keeps falling as the sample grows. Ten legitimate samples bottoming out at 2 means the eleventh could be 1, not that 2 is the floor of normal. Pin the line to the minimum and it slides down every time you add data — that is an observation, not a threshold.
★ Use a quantile instead. Find the 10th–20th percentile of the normal population and leave headroom below it.
★ And bound the cost of being wrong with bands. If falling below the line moves a subject one band down (a light extra check) rather than straight to a block, a slightly wrong threshold costs little. The finer the bands, the less sensitive you are to threshold error.
★★★ Confirmed — growing the sample from 10 to 40 pushed the minimum back down
The normal-population distribution, re-measured at 40 samples:
{1:1, 2:4, 3:5, 4:7, 5:10, 6:10, 7:3} n=40
min 1 · p10 2 · p20 3 · median 5 · max 7In the previous round, fixing the bias raised the minimum from 1 to 2, and the safe line was pinned there. Grow the sample and something else takes the 1 slot. The predicted "the eleventh could be 1" happened exactly as stated.
★★★ The minimum is an observation that keeps falling as the sample grows, not the floor of the distribution. And at 40 samples that 1 is still a single item — so you still cannot claim to know the floor. The minimum is a statistic that never stabilizes, no matter how much data you add.
★ A quantile removes the wobble. Set the line at something like p10 − 1 — one step below a quantile — and it barely moves as the sample grows.
★★★ And rather than trying to get the threshold right, make being wrong cheap
In that distribution the recommended line is p10 − 1 = 1, but the line actually in use is one step stricter, 2. As a result 2.5% of the normal population (one item) falls below it.
★★ That is acceptable because falling below means a "light extra check" band, not a block. The cost of a false positive is bounded. ★★★ A bounded cost is more robust than an exact threshold — design standing in for precision.
★ Decide which way you want to be wrong before drawing the line. Here, the cost of double-checking a legitimate subject < the cost of letting a malicious one through, so the line errs strict. Without that decision, a threshold is just a number.
★★★ Follow-up measurement — tripling the sample left the quantiles unchanged
A full sweep took the normal population from 40 to 121.
| n=40 | n=121 | |
|---|---|---|
| min | 1 | 1 |
| p10 | 2 | 2 |
| median | 5 | 5 |
| max | 7 | 7 |
★★★ All four statistics held. The quantile-based choice is now empirically justified. The line pinned to p10 did not move across a 3× sample — in contrast to pinning it to the minimum, which shifted every round.
★★ Read the stable minimum carefully. It fell from 10→40 and held from 40→121. That is weak evidence that 1 is near the true floor, not evidence that the minimum is a stable statistic. The strong evidence is the median holding across 3× the data.
⚠️ ★★★ But "zero flagged as high risk" is not evidence of detection power
All 121 are normal samples. There is still exactly one malicious sample, and it sits on the blocklist, so it never reaches the classifier.
★★★ Growing only the normal side confirms "it doesn't cut the good ones." It confirms nothing about "it catches the bad ones." False positives were validated against 121 cases; false negatives were validated against zero.
★★ Detection-power samples only appear when something goes wrong — miss the moment and it is gone for good. → Every time something goes on the blocklist, persist its score and raw signals as of that moment. You cannot reconstruct them later: the subject is already gone or changed. Until a few malicious samples accumulate, the margin on the safe side of your threshold is hope, not a validated number.
⚠️ ★★★ But the early return you just added makes that recording impossible
Blocks now return early at the top, so for anything on the blocklist the signals below are never computed at all. The observations you most need are structurally absent for exactly the subjects you need them from.
★★★ An early return preserves the verdict at the cost of the observation. Another instance of this collection's a fix creates the next trap pattern — the move that fixed the precedence bug immediately blocked the data collection.
★ Blocks still belong at the top. What has to change is when you record. Immediately before adding something to the blocklist, compute the full signal set once with the block suspended, and store it. Early return on the verdict path; full computation on the observation path — keep the two separate.
★★★ And the moment the first malicious sample existed, what looked perfect fell apart
Recorded, that one case scored -3. The blocking band is below -4. The only known malicious case does not classify as high risk.
Tracing back, that value had been climbing all along.
| Point | Malicious sample | Lowest normal | Gap |
|---|---|---|---|
| Initial | -7 | 1 | 8 |
| After removing the fake positive signal | -6 | ||
| After splitting the flag per signal | -5 | 2 | 7 |
| Now | -3 | 1 | 4 |
★★★ Every correction that reduced false positives also raised the malicious score. Removing an unfair penalty stops it from unfairly docking legitimate subjects — and stops it from rightly docking malicious ones too. Earlier this chapter said "a correction shifts the scale, not the separation"; this time the separation itself shrank — from 7 to 4.
★★★ None of this is visible from normal samples alone. "Zero high-risk" across a full sweep looked flawless, and one malicious sample exposed it instantly.
★★ So keep a regression test — every time you touch the scoring logic, re-score the known malicious samples and fail the build if any score went up. As it stands, the side effect surfaced weeks later. A single malicious fixture in the test suite would have caught it on the spot.
★ And the fix is not the threshold. Lowering it raises false positives — the shrinking separation originates in the risk signals. Stop the separation trend before re-deriving any threshold.
★★★ A regression baseline must store the population, not just the number
The baseline file recorded separation: 5, and the problem appeared immediately. That 5 is against a 5-item control group; against the full population it is 4 — the control group's minimum is 2, the full population's is 1.
★★★ Store only the number and the comparison breaks the moment the control group changes — with no way to notice it broke. A regression test asks "is this worse than last time?", and if you don't know what last time measured, the question doesn't parse. → Record the value, the population definition, and the measurement time together.
⚠️ ★★★ And a frozen control group invites overfitting
If the control group omits the bottom of the distribution, separation is overstated. "Nothing got docked" passes far too easily.
★★★ Worse, a frozen control group eventually gets tuned against. The moment the test becomes the target, it stops being a test.
★ Compute separation from a quantile of the full distribution, not the control group's minimum. As shown above, quantiles didn't move across a 3× sample — exactly the property a baseline needs. Keep the small control group as a fast check, and make the pass/fail call against the full distribution.
★★★ A value obtained under special conditions is only a value if the conditions are stored with it
Once you split the observation path, one subject carries two values — BLOCKED from the verdict path, -3 (middle band) from the observation path. The discrepancy is intentional, but once stored, nothing says which one is "the" value. Months later, -3 reads as "this was mid-band, so why was it blocked?"
{"score": -3, "band": "borderline",
"blocks_suspended": true, "observed_at": "...", "purpose": "detection_sample"}★★ Without blocks_suspended: true, that number will be mistaken for a verdict. ★ Don't write observations into the same field as verdicts — same field, and they eventually mix regardless of the flag.
★ Generalized: "not needed for the verdict, so don't compute it" and "not needed for the verdict, so don't record it" are different statements. A value irrelevant to the verdict may be exactly what training, auditing, or debugging needs.
★ The same shape is everywhere — a benchmark run with non-default settings, a response time measured with the cache cleared, a row count collected with the rate limit lifted. Omit the conditions and an incomparable number takes the canonical slot.
⚠️ ★★ And a field name must not contradict the population
The baseline recorded "population": "full p10 (n=121)" alongside "normal_low": 2. That 2 is the p10, not the minimum — the full minimum is 1.
★★★ The description string says p10; the field name reads as low. Humans read the string; code reads the field name. When they disagree, the code wins, and it wins quietly. → normal_p10. Put the population in the name, not in the prose.
⚠️ ★★★ "Comparison deferred" becomes the way the test gets switched off
Once the baseline records a population, a natural rule follows — "if the population differs, defer the comparison and don't count it as a failure." The rule is right. It is the same reasoning as UNKNOWN above, and regression tests get a third value too: pass / fail / incomparable.
★★★ But if "deferred" quietly tallies as a pass, the regression test is neutralized. Change the population once and the test stays "deferred" forever, raising nothing the entire time. The easiest way to disable the test becomes "adjust the population definition slightly" — and it happens without any intent to do so.
★★ Report deferrals in a different color from passes, and after N consecutive deferrals (say 3) escalate to a failure and alert. ★ When deferring, also print what has to match for comparison to resume — without that, deferrals are simply left alone.
★★★ A safeguard creates the path by which it switches itself off. Another entry in this collection's a fix creates the next trap family — except this time the trap points at the safeguard itself.
★★ Five checks before adding a signal
★ If 1 or 2 holds, never use it as a positive. Without 3, use it only as a penalty. ★★ If 4 points backwards the signal is worse than nothing — random false positives dilute as the sample grows; backwards ones get worse. They barely move aggregate accuracy metrics and fail exactly where failing matters most.