# LoRA Desk

> Check a LoRA or QLoRA supervised fine-tuning config before training. Paste Unsloth or PEFT + TRL Python, an Axolotl
> YAML, JSON or command-line flags. The page reads every adapter and trainer knob with the line it sits on - without
> running anything - checks it against the reference recipe (LoRA Without Regret), works out the adapter's trainable
> parameters and a per-GPU memory floor from the base model's shape, and then, as a metered run, reviews the config
> knob by knob or rewrites it with every fix applied. Built on the SkillSafe skill @wshobson/lora-qlora-recipes
> (wshobson/agents, MIT). Not affiliated with Unsloth, Hugging Face, Axolotl or any model maker. LoRA here means low-rank adaptation of models; this is not about, or affiliated with, the LoRa radio technology.

https://lora-desk.skillsafe.ai/

## What is free
- Reading: keyword arguments, assignments, YAML and JSON keys and --flags under every name a knob goes by (r or lora_r, per_device_train_batch_size or micro_batch_size, max_length, max_seq_length or sequence_len, use_gradient_checkpointing or gradient_checkpointing ...), variables followed back to their values, commented-out lines ignored, Jupyter notebooks read from their code cells. Framework (Unsloth + TRL, PEFT + TRL, Axolotl) and method (LoRA, QLoRA, 8-bit, none) detected.
- The recipe, per the source skill: all-linear target modules (q, k, v, o, gate, up, down - the MLP matters most); lora_alpha = 2 x r; rank 16-32 for general SFT, up to about 256 at scale, 1-32 for RL adapters; rsLoRA only at r >= 32; learning rate 2e-4 for QLoRA, 1e-4 conservative, 5e-5 very conservative (about 10x full fine-tuning); effective batch (per-device x accumulation x GPUs) under 32; bf16 on BF16 hardware, never fp16; LoRA first, QLoRA when memory forces it; on DGX Spark, QLoRA can OOM before bf16 LoRA.
- Unsloth: lora_dropout 0, bias "none", use_gradient_checkpointing "unsloth", adamw_8bit, a fixed random_state, padding_free=False passed explicitly, no messages-shaped path with assistant_only_loss=True (2026.7.x), attn_implementation silently dropped. Current TRL: processing_class not tokenizer=, max_length on SFTConfig (renamed from max_seq_length), dataset_text_field on SFTConfig.
- Numbers: the effective batch; trainable adapter parameters (r x (inputs + outputs) per targeted projection per layer) and their share of the model; a per-GPU memory floor (frozen base in bf16, NF4 or 8-bit, the fp32 adapter with gradients and Adam or 8-bit Adam state, 1 GB CUDA context; activations extra) against the chosen card. Shapes for Llama 3.1/3.2/3.3, Mistral 7B, Qwen2.5, Qwen3 and Gemma 2 from each model's published config.
- Findings at fix, check or info with a code and the knob; a findings CSV; three samples whose review and rewrite replay at no cost.

## The metered lanes
Two tasks on gpt-terra, metered per run, routed by the `task` field. Input string fields: `task` (review or rewrite), `target` (same, unsloth, peft-trl or axolotl), `setup` (framework, method, model shape, GPU, task type, dataset size), `config` (the paste; a long one keeps its head and tail with a marked cut), `facts` (every knob as set with its line and the recipe's value, the computed numbers, the expected verdict, every finding) and `review` (an earlier review's knob lines, or none).
- `review` replies with `verdict` (ready, adjust or rethink - rethink when the method itself is wrong), `summary`, `knobs` (knob, now, recipe, level fix/check/ok, why), `risks` (looks_like, cause, fix), `keep` and `next`. The page holds the verdict to its own rule, requires every fix-level finding to be taken up and checks quoted values against the config.
- `rewrite` replies with `framework`, the whole corrected `config`, `changes` (knob, from, to, why), `unresolved` and `note`. The page scans the rewrite with the same checks, requires the same framework and base model, and checks every claimed value is in the config; it shows a line diff.
- Handoff: the review's verdict and knob lines go to the rewrite as its `review` field.

## Links
- API tutorial: https://lora-desk.skillsafe.ai/api.html
- Source skill: https://skillsafe.ai/skill/@wshobson/lora-qlora-recipes