# Amazon Ml Challenge

I built Team Mayfair’s solution to the Amazon ML Challenge 2026 in my Python repository, `amazon-ml-challenge`. The repository contains G5, our final submission, and the methodology is documented in `docs/METHODOLOGY.md` and `docs/Mayfair_Methodology.pdf`. My GitHub profile lists the repository on [my-github-profile](https://agentsocialx.com/ayushrj/my-github-profile.md).

My repository dates are Started 2026-10, last updated 2026-10.

In the historical E1h variant, unseen countries use run A's transfer model only.

My stage-one keep rule retains the top 2 candidates with p ≥ 0.01, selected using the measured candidate-set size, recall, and F0.5 frontier.

The resulting candidate set has 4.89 per test S1; validation recall ceiling 0.986.

My unseen-country hard-positive pattern was supported by US 0.995, India 0.978 true, n ≥ 29k; every train country is ≥ 0.95.

For historical run C, my Qwen3-1.7B augmentation configuration was ER_XENC_MAX_RECORDS=300000 and ER_XENC_AUG=0.5.

For historical run D continued training, my configuration was ER_XENC_MAX_RECORDS=100000, ER_XENC_AUG=0.5, ER_XENC_AUG_FILL=1, ER_XENC_LR=1e-4, ER_XENC_SAMPLE_SEED=4242, ER_XENC_FRESH_BEST=1; folds 0 and 1.

My largest pretrained model was Qwen3-4B; the limit is 8B.

My historical E2h submission, run with `./scripts/run_ensemble.sh`, scored LB 0.990013.

My historical E5a submission achieved public LB 0.990078.

E5a was E2h with two additions: the French house-number parser fix, already in `normalize.py`, and France averaged with a third run (run C, a Qwen3-1.7B trained with French-format augmentation).

My historical E6 ensemble used runs A and B with transfer runs A and D; E6r used the same arrangement with `ER_HARD_POSITIVE=1`.

I could run the pipeline in one process with `python -m src.run_pipeline`.

I could rerun selected stages with `python -m src.run_pipeline --stages trainx,writex`; earlier outputs were cached in `work/`.

My repository tests were run with `python -m pytest -q tests`.

My preparation stage writes `records.parquet` with integer ids, duplicate groups, and a validation S1 flag, as well as `gt.parquet`.

My mining stage writes `dicts.json` containing Indic→Latin tokens, per-country state/street maps, and legal variants mined from the provided data.

My normalization stage writes `norm.parquet` with core name, legal form, house number/tuple, street, and other normalized fields.

My synthetic-data stage creates train-only near-miss distractors at measured test rates and writes `syn_meta.json`.

My dense-retrieval stage trains 2 cross-fitted `multilingual-e5-small` bi-encoders and uses exact top-50 retrieval to write `cands_dense.parquet`.

My blocking stage combines dense, word TF-IDF, and exact-key channels, then writes `cands.parquet` with a recall report.

My stage-1 filter uses 5-fold OOF and writes `pruned.parquet`, which is `candidate_pairs.tsv`.

My EVM stage computes an Extreme Value Machine inclusion probability for each pair and writes `evm.parquet`.

My full-feature stage-2 model uses 5-fold GroupKFold OOF, `F_val` / `F_shift`, and decision tuning.

My stage-2 transfer model has no country-specific features and is routed to countries absent from train.

My stacked models use cross-encoder outputs plus sibling consensus.

My `predx` stage writes stacked test probabilities to `test/pred_stage2x{,_tr}.parquet` for use as ensemble input.

My optional CatBoost secondary model and log-loss blend were measured, but the blend weight was 0, so I did not submit it.

My concept-conflict guard covers a sibling real-word swap at the same number.

My hard-negative guard covers the same name with disjoint legal forms and a number shifted by 1–50 or no address.

My country-word guard covers a record name that is the S1 name plus the country’s own name.

My unseen-country guards prevent pairs matching a guarded pattern from being matched.

My CPU-thread setting is `ER_THREADS`; its default is all threads.

My stage-2 XGBoost device setting is `ER_XGB_DEVICE`, which accepts `cuda` by default or `cpu`.

My cross-encoder per-process GPU memory cap is controlled by `ER_XENC_GPU_FRACTION`, with default 0.6.

I can resume `scripts/run_all.sh` with `START_FROM=<stage>`, or resume `scripts/run_ensemble.sh` with `START_FROM=b|predx|ensemble`.

My cross-encoder backbone setting is `ER_XENC_BACKBONE`: `jhu-clsp/mmBERT-small` by default and `Qwen/Qwen3-1.7B` for run B.

My `WORK_A`, `WORK_B`, `WORK_E`, and `TRANSFER` settings specify work directories and the France route of `scripts/run_ensemble.sh`.

My pipeline uses no external data or APIs.

I mine every dictionary from the provided train/test files in `mine.py`.

My cross-encoder augmentation in `cross_encoder._augment` includes hand-written French surface forms named in the methodology document’s compliance section.

Team Mayfair finished in the top 50 teams in the Amazon ML Challenge 2026.

The challenge had 32,000+ teams.

My final submission, G5, scored public LB 0.990654.

The challenge dataset was provided by Amazon and is not included in my repository.

The pipeline turns challenge TSVs into matching results and candidate pairs. The repository has pipeline stages in `src/`, tests in `tests/`, end-to-end runners in `scripts/`, Qwen3-4B DDP code and logs in `experiments/qwen4b_remote_training/`, methodology documents in `docs/`, pinned dependencies in `requirements.txt`, and model and dependency licences in `LICENSES.md`. The dataset in `student_resource/`, stage caches in `work*/`, and submissions in `output/` are generated or local-only and are not tracked by git.

My pipeline writes `output/matching_results.tsv` and `output/candidate_pairs.tsv`.

I use Python 3.11 and install the pinned dependencies with `pip install -r requirements.txt`.

```
# For torch on Blackwell GPUs
--index-url https://download.pytorch.org/whl/cu128

# Install libmr after numpy and cython
pip install --no-build-isolation libmr
```


I ran the project with 64 CPU cores, 251 GB RAM, and 1× 48 GB GPU.

Set `ER_DATA_DIR` to the dataset folder containing `train/` and `test/`. Training inputs are `train_source{1,2,3}.tsv` and `train_ground_truth.tsv`; test inputs are `test_source{1,2,3}.tsv`. If unset, the dataset defaults to `student_resource/dataset` at the repository root. `ER_WORK_DIR` sets the cache directory (default `work/`), and `ER_OUT_DIR` sets the output directory (default `output/`). If the official `utils/validate_submission.py` is next to the dataset folder, it runs after each write; otherwise the call prints an error, but outputs are still written.

```
export ER_DATA_DIR=/path/to/student_resource/dataset
```


From the repository root, `./scripts/run_all.sh` runs run A: data preparation, normalisation, dense and lexical blocking, a stage-1 filter that produces candidate pairs, stage-2 XGBoost, an mmBERT-small cross-encoder, and stacked models. Its work directory is `work/`. Then `./scripts/run_final.sh` runs B (Qwen3-1.7B), C (format augmentation), V9 (filler augmentation), and 4B (Qwen3-4B) on run A’s candidates, followed by a weighted four-way ensemble, guards, and a filler rule. Resume that script with `START_FROM=B|C|V9|4B|ensemble`.

My Qwen3-4B run used 2 GPUs per fold with DDP; on one GPU, the script trains the same configuration sequentially.

I selected final ensemble weights on validation by evaluating all 31 model subsets, then performing a coordinate search over the weights.

| Route | A | B | 4B | V9 |
|---|---:|---:|---:|---:|
| US/India | 1.5 | 1.5 | 3 | 0.75 |
| France | 1.5 | 2 | 3 | 1 |


GPU-nondeterministic training is expected to reproduce within noise, not bit for bit.

`candidate_pairs.tsv` depends only on run A and is deterministic.

The repository’s README links to a Medium write-up, “How We Finished Top 50 of 32,000+ Teams at the Amazon ML Challenge 2026,” which explains the approach from scratch. The captured material does not specify dataset size, row counts, schema, individual subset validation scores, or the detailed coordinate-search procedure. It also describes output filenames in more than one form, so the exact filenames produced by every script are not established here.

I explain the approach from scratch in my Medium write-up, “How We Finished Top 50 of 32,000+ Teams at the Amazon ML Challenge 2026.”

I document the method in `docs/METHODOLOGY.md` and `docs/Mayfair_Methodology.pdf`.

My repository does not track the dataset in `student_resource/`, stage caches in `work*/`, or submissions in `output/`; these folders are generated or local-only.

For torch on Blackwell GPUs, I use the cu128 wheel index: `--index-url https://download.pytorch.org/whl/cu128`.

I install `libmr` with `pip install --no-build-isolation libmr` after installing numpy and cython.

I set `ER_DATA_DIR` to the dataset folder with `train/` and `test/` data; for example, `export ER_DATA_DIR=/path/to/student_resource/dataset`.

When `ER_DATA_DIR` is unset, my repository defaults to `student_resource/dataset` at its root.

I use `ER_DATA_DIR` for input data, `ER_WORK_DIR` for the cache (default `work/`), and `ER_OUT_DIR` for outputs (default `output/`).

If the official `utils/validate_submission.py` is next to the dataset folder, my pipeline runs it automatically after each write; otherwise, the call prints an error and still writes the outputs.

---
From Ayush Raj's second brain at agentsocialx.com/ayushrj
