A second dataset, and what it exposed.
The six centres on this page are constructed, not real.
DermaCon-IN was collected across three hospitals, but the public release does not record which hospital took which photograph. So a real hospital split cannot be recovered from it. The six centres below are age bands — a split we built, and one we label as built everywhere it appears. Fed-ISIC2019 remains the only dataset in this project with real hospitals, and the two are reported side by side, never pooled.
Indian skin, and a much longer tail.
Skin tone
74%
of its images are Fitzpatrick 4 to 6, and not one is Fitzpatrick 1 or 2. ISIC 2019, which every other number on this site rests on, sits at the opposite end. That is the reason a second dataset was worth the work rather than a box to tick.
- FST 448%
- FST 326%
- FST 524%
- FST 62%
The rare pair
52
Neoplasms and tumors
72
Keratanisation Disorders
Picked by the same rule used on Fed-ISIC2019 — below the head class's share divided by 25 — not chosen by hand.
Classes used
- Infectious Disorders2,227
- Inflammatory Disorders1,592
- Pigmentary Disorders831
- Skin Appendages Disorders480
- Other skin disorders159
- Keratanisation Disorders72
- Neoplasms and tumors52
5,413 images used. The 37 labelled “No Definite Diagnosis” are dropped: it is not a disease.
Six age bands, one of them lopsided.
| Constructed centre | Training images | Rare cases | Rare share ÷ size share |
|---|---|---|---|
| age 0 - 10 | 416 | 5 | 0.54× |
| age 10 - 20 | 500 | 6 | 0.53× |
| age 20 - 40 | 1,738 | 24 | 0.62× |
| age 40 - 60 | 1,356 | 39 | 1.28× |
| age 60 - 80the specialist | 344 | 19 | 2.46× |
| age 80 - 100 | 14 | 5 | 15.92× |
The 60 - 80 band is the specialist: 8% of the training images, 2.46× its share of the rare cases. An older cohort carrying more neoplasms is clinically coherent rather than an artefact we arranged. The last band reaches a far higher ratio on only 14 training images, which is too few to mean anything — it is shown rather than quietly dropped.
A second split, patient id hashed into six groups - the near-IID control, is run alongside as a control: with no specialist to find, a method that claims to find one is misfiring.
One cleared, one did not.
DG0 · is the baseline worth studying?
0.408
balanced accuracy over 7 classes · bar was 0.40
Cleared — by 0.008. That is thin enough to say out loud rather than round away. fedavg, 40 rounds, seed 42, with the threshold written down before the run.
DG2 · does EARN help here?
2 of 3
conditions met — so it did not clear
- metD1 rare macro-F1 not worse than FedAvg by more than the margin
- not metD1 specialist weight share on rare rows above its size share
- metD0 within the margin of FedAvg (no effect expected)
The threshold was calibrated to the first dataset.
EARN only listens to a centre about a disease once that centre holds at least 20 images of it. On Fed-ISIC2019, four of six hospitals clear that for the rare classes. Here, only 2 of six does — the specialist holds 11 of one rare class and 8 of the other, so it counts as a holder of neither. With no holders, trust and evidence never engage and EARN quietly becomes capped FedAvg.
An absolute image count is the wrong unit when the rarest class has 52 images instead of hundreds. Swapping it for a threshold relative to the round's own evidence restores the intended behaviour: the specialist goes from 0.068 of the rare rows — below its 0.079 size share — to 0.142.
That change was made after seeing the failure and knowing its cause, so it is post-hoc, and it does not change the DG2 verdict. It has since been tested on Fed-ISIC2019 under a pre-registered adversarial check — and rejected. A threshold set at “above this round's average evidence” is one a liar can clear by lying, because the claim is the evidence: under the sudden-expert attack, a centre holding no rare images at all went from 4.8% of the rare rows to 20.7%. EARN keeps the absolute threshold, and this page's finding stands as one about units rather than a defect with a known fix.
Rare-class F1, mean of 3 seeds
| Method | Balanced accuracy | Rare-class F1 | Specialist's rare-row weight |
|---|---|---|---|
| FedAvg | 0.4217 ± 0.0127 | 0.1557 ± 0.0208 | 0.079 / 0.079 size |
| Camp A (evidence weighting) | 0.4377 ± 0.0097 | 0.1633 ± 0.0161 | 0.137 / 0.079 size |
| EARN, threshold as pre-registered | 0.4181 ± 0.0165 | 0.1478 ± 0.0261 | 0.068 / 0.079 size |
| EARN, scale-relative threshold (post-hoc)post-hoc | 0.4307 ± 0.0085 | 0.1714 ± 0.0125 | 0.142 / 0.079 size |
Two claims have to be kept apart here. The weighting change is solid: ±0.003 across seeds, because the weight comes from the evidence and the trust schedule rather than from training noise. The accuracy gain is not: 0.1714 against FedAvg's 0.1557 is a gap smaller than either standard deviation. On three seeds that cannot be separated from noise, and we do not claim it.
Where this data came from.
DermaCon-IN — Madarkar et al., NeurIPS 2025 Datasets & Benchmarks. three tertiary-care hospitals and affiliated clinics, North Karnataka. Licensed CC BY-NC-SA 4.0: non-commercial, share-alike, attribution required. Images are smartphone clinical photographs, not dermoscopy, so the model on the live demo does not apply to them.
doi:10.7910/DVN/W7OUZM