04. Why I Want to Understand LLM Fine-Tuning Beyond the Frameworks

When you first start exploring LLM adaptation, it is tempting to follow a recipe-driven workflow. You find a tutorial notebook, install five high-level wrapper libraries, pass a JSON dataset into SFTTrainer, set r=16 and lora_alpha=32 because everyone else uses those numbers, and watch the training loss decrease.

For getting a quick proof of concept running, high-level abstractions are wonderful. They save boilerplate and standardize common patterns.

Yet I have noticed a recurring problem when people rely exclusively on wrapper scripts: the moment training diverges, loss plateaus, or the adapted model exhibits catastrophic forgetting on general tasks, you have no mental model to diagnose why.

Fine-tuning introduces deep technical questions that cannot be answered merely by knowing how to call an API or edit a YAML config:

• What actually consumes GPU VRAM during training? (Why does a 7B parameter model that fits in 14 GB for FP16 inference suddenly require over 56 GB for full fine-tuning?)
• How do gradients flow through frozen pretrained weights versus trainable adapter layers?
• Why does the ratio \(\alpha / r\) in LoRA act like a scaling control on the learning rate of the adapter?
• Which linear projections in a Transformer block benefit most from adaptation, and why?

Instead of treating Parameter-Efficient Fine-Tuning (PEFT) as a black-box flag, looking at the linear algebra makes the entire method intuitive.

In full fine-tuning, if a pretrained attention or MLP projection layer has weight matrix \(\mathbf{W}_0 \in \mathbb{R}^{d \times k}\), standard gradient descent updates every single entry in \(\mathbf{W}_0 \leftarrow \mathbf{W}_0 + \Delta \mathbf{W}\). In the AdamW optimizer, storing FP32 master weights, gradients, and first- and second-moment momentum buffers (\(m_t, v_t\)) requires \(16\text{ bytes}\) per parameter—making full fine-tuning prohibitively expensive on consumer GPUs.

However, Aghajanyan et al. [4] showed that pretrained language models have a very low intrinsic dimension: when adapting to a downstream task, the effective weight update \(\Delta \mathbf{W}\) lies in a low-dimensional subspace.

Building on this insight, Hu et al. [2] proposed LoRA (Low-Rank Adaptation), which freezes \(\mathbf{W}_0\) completely and constrains the update \(\Delta \mathbf{W}\) to a low-rank decomposition of rank \(r \ll \min(d, k)\):

where \(\mathbf{A} \in \mathbb{R}^{r \times k}\) is initialized from a zero-mean Gaussian \(\mathcal{N}(0, \sigma^2)\) and \(\mathbf{B} \in \mathbb{R}^{d \times r}\) is initialized to \(\mathbf{0}\).

When we further combine LoRA with 4-bit NormalFloat (NF4) quantization and double quantization in QLoRA [3], the frozen base weights \(\mathbf{W}_0\) are stored in 4-bit precision (quantized to the quantiles of a standard normal distribution, which matches the empirical distribution of pretrained neural network weights) and dequantized on the fly to BF16 during the matrix multiply, while gradients only update the small BF16 matrices \(\mathbf{A}\) and \(\mathbf{B}\).

Reading the equation \(\mathbf{W}_0 \mathbf{x} + \frac{\alpha}{r} \mathbf{B}\mathbf{A}\mathbf{x}\) is helpful, but implementing it directly in PyTorch [1] is what cements the understanding. When you write a custom LoRALinear module in raw PyTorch, nothing is hidden:

import math
import torch
import torch.nn as nn
import torch.nn.functional as F

class LoRALinear(nn.Module):
    # Minimal from-scratch PyTorch implementation of a LoRA-adapted linear layer
    def __init__(self, base_layer: nn.Linear, r: int = 8, lora_alpha: float = 16.0):
        super().__init__()
        self.in_features = base_layer.in_features
        self.out_features = base_layer.out_features
        self.r = r
        self.scaling = lora_alpha / r

        # 1. Freeze the pretrained base weights and bias
        self.weight = base_layer.weight
        self.weight.requires_grad = False
        self.bias = base_layer.bias
        if self.bias is not None:
            self.bias.requires_grad = False

        # 2. Low-rank adapter matrices A (r x in) and B (out x r)
        self.lora_A = nn.Parameter(torch.empty(r, self.in_features))
        self.lora_B = nn.Parameter(torch.zeros(self.out_features, r))
        nn.init.kaiming_uniform_(self.lora_A, a=math.sqrt(5))

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        base_out = F.linear(x, self.weight, self.bias)
        lora_out = (x @ self.lora_A.T @ self.lora_B.T) * self.scaling
        return base_out + lora_out

Writing and stepping through code at this level clarifies questions that high-level frameworks obscure:

1. 1
   Parameter Accounting: For a projection matrix of size \(4096 \times 4096\), full fine-tuning updates \(16.77\text{M}\) parameters. With \(r = 8\), lora_A and lora_B together contain only \(2 \times 8 \times 4096 = 65,536\) parameters—a \(99.6\%\) reduction in trainable weights for that layer.
2. 2
   Zero Inference Latency Overhead: Once training finishes, we can explicitly merge the weights via \(\mathbf{W}_{\text{merged}} = \mathbf{W}_0 + \frac{\alpha}{r}\mathbf{B}\mathbf{A}\) and discard the adapter branch, resulting in zero extra latency during deployment.
3. 3
   Training Loop Control: By writing the PyTorch training loop directly, we can inspect gradient norms (torch.nn.utils.clip_grad_norm_), verify causal attention masking on instruction-response pairs (ensuring loss is computed only on completion tokens, not prompt tokens), and monitor validation perplexity step by step.

My approach to learning LLM adaptation is deliberate: understand the mathematical fundamentals first, implement and experiment on smaller models in raw PyTorch, and only then scale up with high-level production libraries.

I also think it is important to be honest about the distinction between building applications with LLMs and training or adapting models. Both are valuable engineering skills, and I enjoy both—but they demonstrate different levels of technical depth.

For me, the goal is never to collect a longer list of framework names on a resume. The goal is to understand what happens underneath them.

• [1]
  A. Paszke et al., “PyTorch: An Imperative Style, High-Performance Deep Learning Library,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 32, pp. 8024–8035, 2019.
• [2]
  E. J. Hu et al., “LoRA: Low-Rank Adaptation of Large Language Models,” in International Conference on Learning Representations (ICLR), 2022.
• [3]
  T. Dettmers, A. Pagnoni, A. Holtzman, and L. Zettlemoyer, “QLoRA: Efficient Finetuning of Quantized LLMs,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 36, 2023.
• [4]
  A. Aghajanyan, L. Zettlemoyer, and S. Gupta, “Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics (ACL), pp. 7319–7328, 2021.