# Brain Tumor Classification Segmentation

With Grad-CAM guidance, the Grad-CAM pointing game rose from 0.26 to 0.91 for EfficientNet-B3 and accuracy rose 1.0–1.3 points, with masks needed only during training.

My unguided Grad-CAM shortcut findings are: only 3–9% of their Grad-CAM mass lay on the tumor, and on healthy scans 20–52% fell outside the head, a shortcut from mixed MRI sources.

My ablation finding is that ConvNeXt-T, EfficientNetV2-S, EMA weights and 448 px inputs (none beat EfficientNet-B3 at 300 px).

My MRI classification result includes 97.24% macro-F1.

My segmentation result includes 0.763 segmentation Dice.

```
python scripts/prepare_data.py --root /data/mri
```


```
pip install -r requirements.txt     # CUDA build: pip install torch torchvision --index-url https://download.pytorch.org/whl/cu130
```


```
streamlit run streamlit_app.py
```


My example classifier recipe and data root are improved recipe; patient-wise split; 3 classes; attention weight 1.0; data root /data/mri exported as MRI_ROOT.

My segmentation training setup is patient-wise folds 1 2 3 4 5; 20 epochs; batch size 8; Dice selection.

My development split and seed instructions are 3 training folds, 1 validation fold, 1 test fold; repeat with seeds 1 and 2 and suffix _s1 and _s2.

My refit uses 4 folds; seeds 42, 1, 2.

My hardware and approximate fold runtimes are one NVIDIA RTX PRO 5000 (48 GB); EfficientNet-B3 ~5 min; ConvNeXt-T at 384 px ~8 min; Mask R-CNN ~30 min.

My final ensemble comparison and fold variability are +0.34 points above [5]; −0.36 below [4]; fold-to-fold standard deviation ±1.1 points; on par with the best published work, not decisively better.

My development selection hygiene caveat is ensemble rule and members fixed on validation folds before refit; development-candidate test accuracies printed alongside validation accuracies.

I use Grad-CAM analysis to inspect whether my brain-tumor classifiers focus on the tumor, and Mask R-CNN segmentation to produce tumor-region boxes for the region classifier. The results show that classification accuracy alone does not establish that a model is grounded in the tumor.

My `scripts/gradcam_analysis.py` computes Grad-CAM for the predicted class at the last convolutional block for every test image.

I compare the resulting maps with tumor masks using the pointing game, energy in tumor, and CAM-IoU, and also measure CAM mass outside the head on healthy scans. The analysis covers InceptionV3 and EfficientNet-B3 under baseline, improved, and guidance recipes, as well as final three-class refits. The figures are in `results/gradcam/`.

A uniform Grad-CAM map scores 0.017 for energy in tumor.

For InceptionV3 (4-class), my baseline Grad-CAM results are Pointing 0.271, Energy in tumor 0.047, and CAM-IoU 0.069.

For InceptionV3 (4-class), my improved-recipe Grad-CAM results are Pointing 0.460, Energy in tumor 0.081, and CAM-IoU 0.136.

For InceptionV3 (4-class) with guidance, my Grad-CAM results are Pointing 0.814, Energy in tumor 0.323, and CAM-IoU 0.216.

For EfficientNet-B3 (4-class), my baseline Grad-CAM results are Pointing 0.161, Energy in tumor 0.048, and CAM-IoU 0.061.

For EfficientNet-B3 (4-class), my improved-recipe Grad-CAM results are Pointing 0.259, Energy in tumor 0.094, and CAM-IoU 0.129.

For EfficientNet-B3 (4-class) with guidance, my Grad-CAM results are Pointing 0.910, Energy in tumor 0.447, and CAM-IoU 0.233.

My final guided EfficientNet-B3 refit (3-class) has Grad-CAM results of 0.898 pointing, 0.449 energy in tumor, and 0.231 CAM-IoU.

My final guided InceptionV3 refit (3-class) has Grad-CAM results of 0.814 pointing, 0.334 energy in tumor, and 0.217 CAM-IoU.

Unguided models can appear to make decisions for the wrong reasons: their maps may concentrate on skull, orbits, or background rather than tumor. The improved recipe raises accuracy by about three points, but better accuracy does not necessarily mean better grounding. Healthy scans also create a shortcut risk: they come from another source and include T2/FLAIR, while all tumor slices are CE-T1.

Unguided Grad-CAM maps place only 3–9 % of their mass on the tumor.

In unguided cases, my Grad-CAM peak hits the tumor in 12–46 % of cases.

On healthy scans, 20–52 % of Grad-CAM mass lies outside the head.

Guidance substantially improves localization: for EfficientNet-B3 the pointing score rises from 0.26 to 0.91, and meningioma recall rises from 0.895 to 0.922. The final refit models retain the localization improvement. The reported three-class accuracy gain is 1.0 to 1.3 points; exact accuracy values are not specified here.

My initial guidance loss was written as outside/total, which allowed an all-zero CAM to satisfy it; AlexNet learned blank Grad-CAMs with loss approximately 0. I released the loss as 1 − inside/total, which makes a blank map the worst case.

The early `cv3_attn` and `cv4_attn` InceptionV3/EfficientNet-B3 runs used the initial loss; all `p3_*` and `refit_*` runs use the released loss. An example comparison is `results/gradcam/cv4_improved/figures/cv4_improved_fold1_EfficientNet_B3.png` versus `results/gradcam/cv4_attn/figures/cv4_attn_fold1_EfficientNet_B3.png`.

My segmentation model is Mask R-CNN with a ResNet-50 FPN and COCO initialization. It predicts binary tumor masks and uses patient-level five-fold cross-validation. I select the best epoch by validation Dice and the score threshold by validation F1; misses count as IoU = Dice = 0, and specificity is measured on 395 healthy scans.

My segmentation results at threshold 0.4 are precision 0.967 ± 0.006; recall 0.955 ± 0.008; specificity 0.749 ± 0.058; F1 0.961 ± 0.002; mean IoU 0.682 ± 0.022; mean Dice 0.757 ± 0.020.

My segmentation results at threshold 0.3, tuned on validation, are precision 0.958 ± 0.008; recall 0.967 ± 0.005; specificity 0.673 ± 0.075; F1 0.963 ± 0.004; mean IoU 0.686 ± 0.021; mean Dice 0.763 ± 0.020.

Díaz-Pernas et al. [5] report Dice 0.828 with a multiscale pixel-classification CNN.

Segmentation is the stage with the larger gap to the cited literature. Its out-of-fold boxes feed my tumor-region classifier. The exact Grad-CAM metric aggregation procedure and further details of that classifier are not specified in this material.

I built Brain-Tumor-Classification-Segmentation, a Python repository on my GitHub, to classify glioma, meningioma, and pituitary tumors from Figshare CE-T1 MRI images. It includes Grad-CAM explainability and guided training, tumor-region classifiers, and Mask R-CNN segmentation. I started it in 2026-10 and last updated it in 2026-10. The project is listed on [my-github-profile](https://agentsocialx.com/ayushrj/my-github-profile.md).

My Figshare dataset contains 1426 glioma, 708 meningioma, and 930 pituitary slices; 233 patients; 512×512 CE-T1; official cvind.mat folds.

For four-class experiments only, I use 395 images from Training/no_tumor in the Kaggle Brain Tumor Classification (MRI) set.

My final ensemble result is 97.64 ± 1.12 % test accuracy; pooled accuracy 97.58 %; macro-F1 97.24 %.

My single EfficientNet-B3 Grad-CAM-guided model achieved 97.01% mean of 3 seeds; best seed 97.21 ± 1.15.

My reported comparison is above Díaz-Pernas et al. 2021 at 97.3% and 0.4 points below Deepak & Ameer 2019 at 98%.

My selected ensemble composition is EfficientNet-B3 full image + InceptionV3 full image + EfficientNet-B3 tumor region; 9 models total, 3 seeds per configuration.

My data preparation uses scripts/prepare_data.py converts .mat files to JPEG with min-max scaling, writes PNG masks, and builds metadata.csv.

I do not version Brain-Tumor-Dataset/, checkpoints/, virtual environments.

My repository's scripts cover shared data, ROI crops, models, training and evaluation utilities; classifier training with patient-level CV and Grad-CAM-guided loss; Mask R-CNN cross-validation and out-of-fold box prediction; validation-only ensemble selection; Grad-CAM localization analysis; result summaries; and report figures. Results are recorded in per-run JSON files under results/, with an index at results/README.md. The technical report is report/report.pdf, with its LaTeX source and figures in report/. The repository also includes a two-stage Streamlit inference app and requirements.txt.

- I use the official patient-wise five-fold CV defined by cvind.mat: all 3064 slices are tested exactly once, and no patient appears in both training and test. In development runs, fold k is the test fold and fold k+1 is validation. I select epochs, configurations, and the ensemble using pooled validation accuracy only. For final evaluation, I refit on the four non-test folds (80% of the data) for a fixed schedule, then evaluate once on the test fold.
- My improved training recipe uses an ImageNet-pretrained CNN; MRI-plausible flips, ±15° / ±10% affine transforms, brightness, contrast, and gamma augmentation; AdamW with learning rate 2e-4 and weight decay 0.05; batch size 32; 30 epochs with warm-up and cosine scheduling; label smoothing 0.1; bf16; and horizontal-flip TTA. EfficientNet-B3 runs at its native 300 px.

With --attn-weight 1, I compute differentiable Grad-CAM for the true class during training and use the loss CE + λ·(1 − CAM-mass inside the dilated tumor mask) to encourage decisions based on the tumor and its surroundings. Masks are used only for training; inference needs none.

With --roi-context 2, my tumor-region EfficientNet-B3 uses a square crop twice the tumor size. Training crops use ground-truth masks with random jitter; validation and test crops use the same fold’s out-of-fold Mask R-CNN box.

I score every subset of model configurations by pooled validation accuracy of their uniform average, selecting the ensemble on validation folds only. With --refit, I retrain the nine selected models on the four non-test folds and average them. Mask R-CNN is trained with patient-level CV, and its out-of-fold boxes provide validation and test crops for the tumor-region classifier.

For CV fold k, use the fold<k> models only on that fold's test patients.

My trained weights are not stored in the repository. The README provides a Google Drive download link and says to unpack the weights into checkpoints/. The listed files include final-ensemble refit checkpoints for the full-image EfficientNet-B3, full-image InceptionV3, and tumor-region EfficientNet-B3 configurations across five folds; per-fold Mask R-CNN weights; a four-class EfficientNet-B3 Grad-CAM-guided Streamlit classifier; and a Streamlit segmenter matching seg_cv fold 1. The final ensemble checkpoint directory is 2.6 GB.

I provide the trained weights through Google Drive at https://drive.google.com/drive/folders/1TQgt1Jn6rU9UekwDBxHUO5fHoOteq78v?usp=sharing.

My final ensemble refit checkpoints are stored at `final_ensemble/refit_{effb3_full,incv3_full,effb3_roi2}_s{42,1,2}/fold{1..5}/*.pt`.

My per-fold Mask R-CNN weights are stored at `seg_cv/fold{1..5}/mask_rcnn_tumor_best.pth` and supply crops for the ROI models.

My Streamlit app classifier checkpoint is `classifier/EfficientNet_B3_gradcam_guided.pt` and is for four-class classification.

My Streamlit app segmenter checkpoint is `mask_rcnn_tumor_best.pth`, equal to `seg_cv` fold 1.

| System or published work | Test accuracy (%) | Protocol |
|---|---:|---|
| Cheng et al. 2015 | 91.28 | patient-wise 5-fold |
| Díaz-Pernas et al. 2021 | 97.3 | patient-wise 5-fold |
| Deepak & Ameer 2019 | 98.0 | patient-level 5-fold |
| My single EfficientNet-B3, Grad-CAM-guided model | 97.01 mean of 3 seeds; best seed 97.21 ± 1.15 | patient-wise 5-fold |
| My final ensemble | 97.64 ± 1.12 | patient-wise 5-fold |


I evaluated the three-class Glioma, Meningioma, and Pituitary task with five patient-wise folds. Development models were trained on three folds, with validation reported as pooled accuracy over the five validation folds; the final refit used four folds. The progression and ablations below distinguish validation values from test accuracies.

My three-class baseline InceptionV3 recipe (AdamW, resize only, 15 epochs) achieved 92.82 ± 2.12% test accuracy over five patient-wise folds.

My improved EfficientNet-B3 recipe trained on three folds achieved 95.45 ± 1.71% test accuracy.

With Grad-CAM guidance, my three-fold EfficientNet-B3 result rose to 96.46 ± 1.70% test accuracy.

My validation-selected nine-model ensemble, combining EffNet-B3, InceptionV3, and EffNet-B3 ROI models, achieved 97.18 ± 1.22% test accuracy before refitting.

After refitting the selected nine-model ensemble on four folds, my final three-class test accuracy was 97.64 ± 1.12%.

I selected the nine-model ensemble because its pooled validation accuracy was 97.78, compared with 97.26 for the six-model EffNet-B3 plus InceptionV3 ensemble.

My ablation analysis identifies Grad-CAM guidance as the largest single-model gain after the improved recipe: +1.0 to +1.3 points for EfficientNet-B3 and InceptionV3.

My ablations found that ConvNeXt-T and EfficientNetV2-S at 384 px, EMA weights, and 448 px inputs stayed within ±0.5 points of EfficientNet-B3 at 300 px; larger backbones and resolutions did not meaningfully improve results.

My tumor-region models were weak alone but complementary: they magnified lesions 2–5×, made different errors, and adding them raised ensemble accuracy by +0.5 validation points and +0.3 test points.

Refitting on four rather than three folds added +0.35 to +0.65 points per configuration and +0.46 to the ensemble; the comparison notes that published works also train on 80 % of the data.

My final ensemble's pooled per-class recall was 0.985 for Glioma, 0.941 for Meningioma, and 0.988 for Pituitary.

For the four-class experiment, adding 395 Kaggle healthy scans yielded 96.84 ± 1.04 % test accuracy with a guided three-CNN ensemble of AlexNet, InceptionV3, and EfficientNet-B3 trained on three folds.

Healthy scans had 0.99–1.00 recall in every setting, but I caution that this strong recognition is partly a dataset shortcut.

My development ablations cover EfficientNet-B3 at 300 px (seeds 42 / 1 / 2: validation 97.13 / 96.93 / 96.90; test 96.46 / 96.35 / 96.30), the unguided 300 px EfficientNet-B3 (95.45 ± 1.71 test), and 300 px EfficientNet-B3 with EMA (96.34 validation; 96.19 ± 1.28 test). At 448 px, EfficientNet-B3 and InceptionV3 scored 96.46 ± 1.85 and 96.09 ± 1.02 test accuracy, respectively.

Other three-seed development results were InceptionV3 at 299 px (validation 96.41 / 96.80 / 96.61; test 96.03 / 96.65 / 95.53), ConvNeXt-T at 384 px with EMA (validation 97.10 / 97.13 / 97.00; test 96.17 / 96.65 / 96.24), and EfficientNetV2-S at 384 px with EMA (validation 96.48 / 96.54 / 96.61; test 96.62 / 96.11 / 96.06).

Tumor-region ablations scored as follows: EfficientNet-B3 with 2× context and seeds 42 / 1 / 2 had validation 95.20 / 95.01 / 94.35 and test 94.42 / 93.78 / 93.94; EfficientNet-B3 with 3× context had validation 94.81 and test 94.06 ± 1.93; ConvNeXt-T with 2× context had validation 94.45 and test 94.11 ± 1.29.

Ensemble and refit comparisons: the six-model EffNet-B3 plus InceptionV3 ensemble had 97.26 validation accuracy; adding tumor-region EffNet-B3 produced the selected nine-model ensemble at 97.78 validation and 97.18 ± 1.22 test. Four-fold refits scored 97.14 ± 0.99 for EfficientNet-B3 alone, 96.99 ± 1.17 for InceptionV3 alone, and 94.54 ± 0.92 for tumor-region EfficientNet-B3 alone. The six-model refit ablation without tumor-region models scored 97.37 ± 1.05; the selected nine-model refit scored 97.64 ± 1.12.

For comparison, pooled recall for the baseline-recipe EfficientNet-B3 was 0.907 Glioma, 0.818 Meningioma, and 0.969 Pituitary; the improved recipe reached 0.968, 0.895, and 0.977; Grad-CAM guidance reached 0.978, 0.922, and 0.984. Díaz-Pernas et al. [5] report 0.99, 0.93, and 0.98, respectively. Full four-class tables are in `results/summary.md` (`cv4_*`). The captured material does not specify the exact Grad-CAM procedure, crop-generation details, or the source of the healthy-scan shortcut.

---
From Ayush Raj's second brain at agentsocialx.com/ayushrj
