NDB-UFES Data Organizer

This repository rebuilds the dataset behind the P-NDB-UFES oral histopathology patches. It ties every patch to the parent picture it was cut from, keeps related pictures and patients together, and publishes folds that do not leak between patients.

What is here

• Every patch traced to its image. Each of the 3,763 P-NDB-UFES patches is tied to the parent picture it was cut from, found by exact pixel matching. Most of those pictures are NDB-UFES images; the others come from internal datasets that are described in About the Data.
• Cases and case groups. Pictures and patients that share evidence are tied into case groups, and a case group is never split across folds.
• Six folds. Balanced at once on the patch label, the NDB-UFES diagnosis of the picture, gender, age group and three morphology clusters. See New Folds for the numbers.
• Nothing relabeled. Where the patch label and the picture's diagnosis disagree, the disagreement is recorded, not corrected. A disagreement is not necessarily bad: a picture labeled OSCC can also contain dysplasia, so a with-dysplasia patch in it is expected. The problem is the other direction, a picture labeled with dysplasia that has an OSCC patch.

Download

The tables are small CSV files in this repository:

• Fold assignments: the fold of every patch. Use this file to train and evaluate.
• Patch table: label, parent picture, position and checks for every patch.
• Picture table: one row per parent picture.
• Case table: the masked cases and their case groups.

Every column is explained in the data dictionary. The images themselves are not published here: the patches are in P-NDB-UFES and the origin images in NDB-UFES (Mendeley Data). Some parent images come from an internal dataset, SAB, which is private; see About the Data.

Using the folds

• Use fold_assignments.csv instead of a random patch-level split. Patches of one case group are always in one fold, so patches of the same patient never sit on both sides of an evaluation.
• Each fold is held out in turn. There is no separate train/test split.
• Skip the three patches with an empty fold: they are exact duplicates of another patch, which stays in the folds.
• The label to predict is patch_diagnosis. ndb_diagnosis belongs to the whole picture and is context, not a target.

Where to look

• To see how the four datasets differ and how they relate, read About the Data.
• For the factsheet of NDB-UFES, read Factsheet: NDB-UFES.
• For the factsheet of P-NDB-UFES, read Factsheet: P-NDB-UFES.
• To see how patches and pictures were matched, and to open the atlases, read Matching and Atlases.
• For the old fold configuration and how contaminated it was, read Old Folds and Contamination.
• For the new fold configuration, read New Folds.
• For every column and value of the tables, read the Data Dictionary.
• To install, preview the wiki and rebuild the dataset, read Setup and Installation and Reproducibility.

AI-Assisted Documentation Note

This wiki was prepared with the help of AI tools, following instructions written by a human. Pages are reviewed and validated by a human before release.