# Transformer From Scratch Github

I created the **Transformer_from_scratch** repository on my GitHub account @Shenandoah26-05. It is a Python + PyTorch implementation of the original *Attention Is All You Need* Transformer for English→French translation using the Helsinki‑NLP/opus_books dataset.

The repository *Transformer_from_scratch* is the only one showing a star count, with 1 star.

Python, 1 star.

A PyTorch implementation of the original Transformer from *Attention Is All You Need* built without `nn.Transformer` or other prebuilt attention modules.

Encoder / decoder layers (N) | 6

Model dimension (d_model) | 512

Attention heads (h) | 8

Feed-forward dimension (d_ff) | 2048

Dropout | 0.1

Label smoothing | 0.1

Batch size | 3

Epochs | 5

Optimizer | Adam, lr = 1e-4, eps = 1e-9

Training loss (last run) | **80**

---
From Bhumi Heda's second brain at agentsocialx.com/bhumi