I process documents using the pipeline: PDF → extract text (page‑by‑page, reading‑order aware) → segment into sentences (spaCy) → embed every sentence (sentence‑transformers) → align sentences across documents (Hungarian algorithm on cosine similarity) → classify each aligned pair: unchanged / trivial / substantive (NER‑based entity diff) → unmatched sentences = added / removed → summarize + risk‑score.
Running the tool locally requires Python 3.10+, a clean virtual environment (python3 -m venv docdiff‑env), and the dependencies from requirements.txt. After installing the spaCy English model, I start the UI with streamlit run app.py at http://localhost:8501.
To avoid a macOS segmentation fault caused by competing OpenMP runtimes, I set the environment variables OMP_NUM_THREADS=1 and KMP_DUPLICATE_LIB_OK=TRUE before importing torch or spaCy, as noted in app.py.
Deploying the repository to GitHub involves git init, adding files, committing with the message "Initial commit: AI Document Comparator", adding the remote URL, renaming the branch to main, and pushing with a Personal Access Token that has repo scope.
The NLP‑Document‑updates repository is owned by @aditiraj0129‑netizen.
It is implemented in Python.
Project work started 2026-09 and the last update was also 2026-09.
A live demo is available at https://nlp-document-updates-8amtvfzp32fwx43vfarwb7.streamlit.app/.
The tool provides three main functions: Document comparison, Document insights, and Smart Insert. Comparison aligns sentences across two PDFs and classifies each pair as Unchanged, Modified‑trivial, Modified‑substantive, Added, or Removed. Insights generate an extractive summary and key terms for a single PDF. Smart Insert finds the most semantically related sentence(s) and inserts a new clause with clear markup.