1U[k]
Local training
Each hospital trains on its own images and returns only a model update. Standard federated learning - runs today.
02 · The method
How EARN decides whose update counts.
EARN - Earned, Audited, Rare-class-aware aggregation - was our proposal. It is now built and unit-tested step by step, but its foundation - reading from each update which hospitals hold a disease - came in under the bar on its check and its one retry. So EARN is not validated. We ran it once more on an oracle signal, clearly labelled, to see whether the rest of the mechanism would have worked.
EARN reads each hospital's "evidence" for each disease. Our check of that signal came back inverted: mean Spearman -0.21 against a bar of 0.7, and -0.46 on the one allowed retry. The project therefore switched to Fallback F1, the attack study.
One round
Six steps, every training round.
Every step is implemented in src/federated/earn.py and unit-tested. The evidence step is the one that came back negative.
The core idea
Trust peers less when fewer peers can check.
Checking a claim needs someone able to check it. When many hospitals hold a disease, peers can contradict a false update. When few do, the only evidence left is the hospital's own track record. EARN blends the two smoothly, with no hard cutoff.
The blend
p(c) = clip((n(c) - 1) / 4, 0, 1)
a(k,c) = p(c) · agree_with_peers
+ (1-p(c)) · agree_with_own_historyn(c) is how many hospitals hold disease c. p(c) is how much the peer check can be trusted for it.
| n(c) | p(c) | Checked by | Our diseases |
|---|---|---|---|
| 1 | 0.00 | Own locked history only | constructed split S2 |
| 2 | 0.25 | Mostly history | Squamous cell carcinoma |
| 3 | 0.50 | Half peers, half history | Basal cell carcinoma, Actinic keratosis, Vascular lesion |
| 4 | 0.75 | Mostly peers | Dermatofibroma |
| 5-6 | 1.00 | Peers only | Melanoma, Melanocytic nevus, Benign keratosis |
holder counts are measured (≥ 20 training images); rare diseases highlighted. The blend itself is the proposal.
Trust and weighting
Slow to earn, quick to lose.
Trust update
+0.1 or ×0.5
Agree with the check and trust rises by a small fixed step. Disagree once and it halves.
Weighting
≤ 50% per row
Each rare-disease row gets FedAvg weight plus trust × evidence share. No hospital may hold more than half of any row.
Safe default
T = 0 → FedAvg
With all trust at zero, EARN is clipped FedAvg - except that the 50% cap still applies, which trims centre 0's 53% share. The design doc's "exactly FedAvg" could not hold alongside the cap; we kept the cap and say so.
Exploratory · oracle evidenceNot a validated result
If the signal had worked, would EARN?
We replaced the measured signal with an oracle: every honest hospital's true image counts, which an attacker can fake exactly as it fakes a reported count. Everything else is EARN as designed, with three ablations and four sensitivity settings. Gate G2 cannot be evaluated while the measured signal sits under its bar; below is the same test on the oracle, reported as exploratory.
S1 · natural
- Under A1, rare F1 within 0.05 of no attack✓
- Under A2, rare F1 within 0.05 of no attack✕
- No attack: within 0.02 of Camp A✓
- No attack: above FedAvg✓
- Centre 2 trust ≥ 0.8 by round 15✕
rare F1: no attack 0.596 · A1 0.568 · A2 0.541 · FedAvg 0.592
S2 · specialist
- Under A1, rare F1 within 0.05 of no attack✕
- Under A2, rare F1 within 0.05 of no attack✕
- No attack: within 0.02 of Camp A✓
- No attack: above FedAvg✓
- Centre 2 trust ≥ 0.8 by round 15✓
rare F1: no attack 0.555 · A1 0.383 · A2 0.000 · FedAvg 0.490
EARN and its ablations, specialist split S2
Rare macro-F1, mean ± sd over 3 seeds. Oracle evidence throughout.
Where it stalls on S1
Honest hospitals' rare-disease updates barely point the same way as each other in a round, so the peer check turns them down too. Centre 2's trust at round 15 is only 0.04 (dermatofibroma) and 0.19 (vascular lesion) - EARN ends up close to clipped FedAvg.
Where it stalls on S2
With one holder, only the history check runs - and it rewards being consistent, not being right. The A1 attacker is consistent from round 1, earns full trust and captures 7.5x its fair weight. Peer-only checking ("no blend") holds it to 1.06x, but then zeroes the honest specialist's trust too.