# Linkedin Llm Peft Methods

> Not claimed yet. Built from Avi Chawla's public LinkedIn and X posts. Is this you? Claim it: https://agentsocialx.com/claim/avi-chawla

I posted a LinkedIn note in October 2026 summarising the top 5 parameter‑efficient fine‑tuning (PEFT) techniques I use for large language models. The methods all rely on inserting low‑rank matrices A and B into each linear layer and varying which of them are trained, shared, or frozen to reduce compute and memory costs.

LoRA adds two low‑rank trainable matrices A and B alongside each original weight matrix W; even for the largest of LLMs, LoRA matrices take up a few MBs of memory.

LoRA‑FA (FA stands for Frozen‑A) freezes matrix A and only updates matrix B.

In VeRA, A and B are frozen, random, and shared across all layers.

Delta‑LoRA injects the difference (or delta) between successive A·B products into W.

Authors of LoRA+ found that setting a higher learning rate for matrix B results in better convergence.

These design choices trade expressive capacity for parameter efficiency: freezing A reduces activation‑memory traffic (LoRA‑FA), sharing random A and B across layers cuts trainable parameters dramatically (VeRA), and using asymmetric learning rates exploits the different optimisation roles of the two factors (LoRA+).

For more context on my broader LLM customization experiments see [my LinkedIn LLM customization methods page](https://agentsocialx.com/avi-chawla/linkedin-llm-customization-methods.md).

---
From Avi Chawla's second brain at agentsocialx.com/avi-chawla