Run Your Own AI at Home for the Price of a Gaming PC

Here’s a fun bit of irony for you. The AI boom is the reason a gaming PC costs so much right now, and a gaming PC is also the cheapest ticket to running your own AI. The same data center buildout that’s soaking up the world’s memory chips has pushed a basic 32GB kit of DDR5 to about $450, according to Tom’s Hardware, more than double what the same kit cost a year earlier. So yes, the robots are making your next build more expensive. Hmmm.

Still, if you want to run a local LLM at home, you don’t need a server rack, a colo contract or a second mortgage. You need a box that looks a lot like the one a teenager would build to play shooters on, a few free pieces of software, and about an hour of your time. That’s the whole trip from lowercase point a, to UPPERCASE POINT B.

This post is the hands-on companion to my AI rent vs. own vs. hybrid TCO model. That one does the math for organizations. This one is the garage version.

Quick note before we start: every price below is approximate as of late October 2026, and the market is moving fast (mostly up). I’ve linked where each number came from so you can check today’s price before you buy anything.

What “running AI at home” actually means

When you type into ChatGPT, your words travel to someone else’s data center, a very expensive GPU does the math, and the answer streams back to you. Running a model locally just moves that GPU into your house. You download an open model (the weights, basically one big file of numbers), load it into memory, and your own hardware generates every token.

If you want the deeper version of how that works, I wrote it up in The Machines Were Always Trying to Talk. For today, you only need one idea from it: the model has to live somewhere fast while it’s thinking.

And “somewhere fast” means your graphics card’s memory.

VRAM is king

If you remember one thing from this post, make it this. The single spec that decides which models you can run is how much memory (VRAM) is on your GPU. Not the core count, not the boost clock, not the RGB. Memory.

Here’s why. Every time the model produces a token, it has to read through its weights. If the whole model fits in VRAM, that happens at full speed. If it doesn’t fit, tools like Ollama will split it between the GPU and your regular system RAM, and the part sitting in system RAM runs a lot slower. Ollama will even tell you how the split landed; ollama ps shows something like “100% GPU” when everything fits, or a CPU/GPU split such as “48%/52%” when it doesn’t (that example comes straight from Ollama’s FAQ). Think of it like a turbo car stuck in traffic: all that horsepower, and you’re still crawling.

The second spec, once the model fits, is memory bandwidth. That’s how fast the card can shovel those weights to the compute cores, and it’s a big part of why one 24GB card can feel quicker than another 24GB card. A used RTX 3090, for example, is rated at 936 GB/s.

Quantization, or how a 70 billion parameter model fits in your house

Models are described by parameter count: 8B, 14B, 32B, 70B and so on. At full 16-bit precision, each parameter takes two bytes, so a 70B model is enormous. Quantization stores those numbers with fewer bits. Q8 uses roughly 8 bits per weight, and Q4 uses roughly 4 (the popular “Q4_K_M” flavor mixes in a little more precision where it matters). You give up a little quality and get a much smaller file.

Don’t take my word for the sizes. These come straight from the download sizes on Ollama’s model library for Qwen3 and Llama 3.3:

Model	Q4_K_M	Q8_0	FP16
Qwen3 8B	5.2 GB	8.9 GB	16 GB
Qwen3 14B	9.3 GB	16 GB	30 GB
Qwen3 32B	20 GB	not listed	not listed
Llama 3.3 70B	43 GB	75 GB	141 GB

The rough rule of thumb that falls out of that table: at Q4, budget a little over half a gigabyte per billion parameters; at Q8, a little over one gigabyte; at full precision, two. Then leave headroom, because the model file isn’t the only thing in memory. The conversation itself (the context window) takes space too, and Ollama’s context length docs say plainly that a larger context “will increase the amount of memory required to run a model.”

So, translated into hardware:

• 8GB card: 7B and 8B models at Q4, comfortably. That’s it.
• 16GB card: 14B at Q4 with breathing room, or 8B at Q8.
• 24GB card: the 30B to 32B class at Q4. This is the sweet spot a lot of home tinkerers chase.
• 70B at Q4: around 43GB before context. That’s two 24GB cards, a lot of system RAM and patience, or a Mac with plenty of unified memory.

One more wrinkle worth knowing: some newer models are “mixture of experts” designs that only use a slice of their parameters for each token. Qwen3’s 30B-A3B is one; its Q4 file is 19GB, so it still needs the memory, but it tends to run quicker than a dense model the same size.

System RAM is the supporting cast

System RAM matters less than VRAM, but it isn’t nothing. It’s where models get staged while loading, it’s the overflow lot when a model doesn’t fit on the GPU, and it’s what everything else on the machine (your browser, Docker, Open WebUI) is using in the meantime. I’d call 32GB the floor for a dedicated AI box and 64GB comfortable.

Here’s the painful part. Tom’s Hardware’s RAM price index showed the cheapest 32GB DDR5-6000 kit at about $447 and the cheapest 64GB kit at about $989 as of October 22. TrendForce’s October 21 spot price update describes DDR5 supply as tight with suppliers “clearly reluctant to sell.” Buy what you need, not what would be nice to have.

The Apple Silicon loophole

Macs play by different rules. On Apple Silicon, the CPU and GPU share one pool of unified memory, so a Mac with 64GB of memory can hand most of it to a model. There’s no separate VRAM wall to hit. Ollama supports Apple GPUs through Metal, per its hardware support page, and LM Studio runs Apple’s MLX engine alongside llama.cpp.

The current lineup, straight from Apple’s September 2026 announcement:

• Mac mini with M6: starts at $899 with 16GB, configurable to 32GB, up to 170GB/s of memory bandwidth.
• Mac mini with M5 Pro: starts at $1,699 with 24GB, configurable to 64GB, with 307GB/s of memory bandwidth.
• Mac Studio with M5 Max: starts at $2,499, up to 128GB of unified memory.
• Mac Studio with M5 Ultra: starts at $5,499 and goes up to 512GB, per Apple’s Mac Studio press release. That’s “run a frontier-sized open model on your desk” territory, at a price that is very much not a gaming PC.

The trade-off is that bandwidth. A Mac mini’s 307GB/s is roughly a third of a used 3090’s 936GB/s, so for models that fit on both, the NVIDIA card will usually generate faster. Where the Mac wins is capacity per dollar on big models, plus it’s small, quiet and sips power. Apple’s memory upgrades are priced like Apple’s memory upgrades, so price the exact configuration you want before you fall in love with it.

Two sample builds (approximate, late October 2026)

To be clear, these are sample parts lists for you to price out, not rigs I’ve built or benchmarked. Prices are the best current figures I could verify from the linked trackers, and they will have moved by the time you read this.

The 16GB daily driver

• GPU: NVIDIA RTX 5060 Ti 16GB, about $765 new (launch MSRP was $429, so you can see where the market went). NVIDIA rates it at 180W with a 600W recommended system power supply.
• CPU: AMD Ryzen 7 9700X, about $290. The CPU barely matters for GPU inference; this is just a sensible midrange pick.
• RAM: 32GB DDR5-6000, about $447.
• Storage: 2TB NVMe, median around $370 across in-stock US drives, with cheaper ones out there. Models are big files and you will collect them like baseball cards.
• Board, power supply and case: your call. I didn’t verify prices for these, so add them on top.

That’s roughly $1,870 for the four priced parts. It runs 8B models at Q8 and 14B models at Q4 entirely on the GPU, which covers a surprising amount of daily use.

The 24GB sweet spot

• GPU: a used RTX 3090 24GB. Fair asking prices were about $1,456 to $1,558 as of October 20, which is about the same money as a new 16GB RTX 5080. If you’re comfortable on Linux, a used Radeon RX 7900 XTX gets you the same 24GB for around $1,050, and Ollama supports it through ROCm.
• CPU: the same Ryzen 7 9700X is plenty.
• RAM: 64GB DDR5-6000, about $989.
• Storage: the same 2TB NVMe, or more.
• Power supply: size it for a card with a 350W rating, per GPUDojo’s 3090 spec sheet.

Call it $3,100 or so for the priced parts with a 3090. A used card is five or six years old, usually has no warranty, and may have spent its youth mining crypto; buy from a seller with a return policy, and test it the day it arrives.

What about the RTX 5090 and its 32GB? NVIDIA’s MSRP was $1,999, but on October 22 PC Guide found major retail listings at $8,499.99 and $8,999.99, with other models out of stock. It also draws 575W on its own. If you have to ask, it’s not your build. (It’s not mine either.)

The software stack (all free)

You’ve got four names to know, and they stack together nicely.

Ollama is the easy button. It downloads models, runs them on your GPU, and exposes a local API on port 11434. Under the hood it uses llama.cpp, per Ollama’s own README. If you only install one thing, install this.

LM Studio is the friendly desktop app for folks who’d rather click than type. You browse and download models inside the app and chat with them in a window. Its site says it’s powered by MLX and llama.cpp under the hood.

llama.cpp is the engine room, the open source project that most of this ecosystem runs on. Use it directly when you want every knob, or when you want a lean server with no extras.

Open WebUI gives you a ChatGPT-style web interface in your browser that talks to Ollama. Chat history, model switching, document uploads, multiple users for the family. It runs nicely in Docker.

Quick start: your first local model in about 15 minutes

Time to experiment! These commands come from the official docs for each project. I’m using Linux and macOS syntax; Windows has its own one-line installers on the same pages.

Step 1: Install Ollama. On Linux or macOS, the official install is one line:

curl -fsSL https://ollama.com/install.sh | sh

Step 2: Pull and run a model sized for your card. On a 16GB card, start with Qwen3 14B (about 9.3GB at Q4). On an 8GB card, use the 8B version instead.

ollama run qwen3:14b

The first run downloads the model, then drops you at a prompt. Type something. Watch your own GPU answer. Pretty rad, right?

Step 3: Check whether it actually fit in VRAM. Open a second terminal while the model is loaded:

ollama ps

You want to see “100% GPU” in the Processor column. If you see a CPU/GPU split, drop to a smaller model or a smaller quantization.

Step 4: Talk to it from code. Ollama’s API listens on localhost by default:

curl http://localhost:11434/api/chat -d '{
  "model": "qwen3:14b",
  "messages": [{"role": "user", "content": "Explain VLANs to a five year old."}],
  "stream": false
}'

Step 5: Add the nice web interface. With Docker installed, this is the command from Open WebUI’s README for when Ollama runs on the same machine:

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

Then browse to http://localhost:3000, create your admin account, and pick your model from the dropdown.

Step 6, optional: Give it more room to think. Ollama sets a default context size based on your VRAM (4K tokens on cards under 24GB, per its docs), which is short for long documents or coding tools. You can raise it when you start the server, as long as you have the memory to back it up:

OLLAMA_CONTEXT_LENGTH=32768 ollama serve

Prefer to skip Ollama and go straight to the metal? The llama.cpp README now has its own installer and pulls models straight from Hugging Face, with llama cli -hf for a chat session and llama serve -hf for an OpenAI-compatible API server.

What it’s good at, and where the cloud still wins

Let me be clear: a 14B model on a 16GB card is not going to out-reason the biggest frontier models from OpenAI, Anthropic or Google. Those run on clusters with far more memory than anything you’ll own, and they’re tuned for the hardest problems. If you need deep multi-step reasoning, huge context, current web knowledge, or top-tier coding help on a gnarly codebase, the cloud is still the better tool.

But “smaller” isn’t “useless.” Not even close. Local models are genuinely good at a lot of everyday work:

• Summarizing and rewriting your own documents, notes and email drafts.
• Answering questions about files you’d never paste into a public chatbot.
• Drafting, brainstorming and first-pass editing.
• Coding helpers, scripts and config explanations (check the output; this is true of every model).
• Running all night on a batch job without a meter running.
• Working when your internet is down.

The weak spots are real too. Smaller models make things up more often, so verify anything that matters. Knowledge stops at the training date unless you hook in search or documents. And you are now the IT department, which means updates, drivers and the occasional “why won’t this load” evening are on you.

Power, noise and the electric bill

Here’s the part people forget until the first bill shows up. A GPU only draws near its rated power when it’s actually generating, and it idles far lower, but let’s do the worst-case math anyway.

Say you run a 350W card (the 3090) flat out for two hours a day. That’s 0.7 kWh a day, or about 21 kWh a month. At the US average residential price of about 18.16 cents per kWh for August 2026, per the EIA, that’s about $3.81 a month for the GPU. A 180W card doing the same work is about half that. Your rate will differ, so plug in the number from your own bill.

Noise is harder to put a number on, and I won’t pretend otherwise. Big air-cooled cards spin their fans up hard under sustained load, which is fine in a basement or closet and less fine on the desk next to your ear. If quiet is the priority, that’s a point in the Mac’s favor. And whatever you build, give it airflow; heat is the thing that quietly shortens the life of hardware.

Privacy: the whole point, if you do it right

This is the biggest reason I think more folks should try it. Ollama’s FAQ answers it directly: “Ollama runs locally. We don’t see your prompts or data when you run locally.” Your medical questions, your tax documents, your half-baked business idea all stay on your hardware.

Two caveats, because hope isn’t a strategy:

• Ollama also offers cloud-hosted models. Those are processed on their servers, by definition. If you want to be sure nothing leaves the box, the FAQ documents an OLLAMA_NO_CLOUD=1 setting that turns cloud models off.
• Mind your network exposure. Ollama binds to 127.0.0.1 by default, meaning only the machine itself can reach it. You can change that with OLLAMA_HOST, but don’t open it to the internet. If you want to reach it from your phone away from home, put it behind a VPN; I do exactly that for my family’s streaming setup with WireGuard, which I wrote about in my Raspberry Pi Plex workbench post.

So should you rent or own?

For one person or one household, the math is usually simple. If you only ask an AI a handful of things a week, a monthly chatbot subscription will probably beat a $2,000 build for a long, long time. If you’d use it daily, care a lot about privacy, or just love tinkering, the box starts to make sense, and you get a gaming PC in the deal (why not, right?).

For a business, it gets a lot more interesting, because utilization, staffing, power and depreciation swing the answer hard. That’s exactly what my free AI rent vs. own vs. hybrid TCO model is built for. Same question, bigger spreadsheet.

Your homework

If you already own a gaming PC with a decent NVIDIA or AMD card, you’re one command away from trying this tonight. Install Ollama, pull a model that fits your VRAM, and ask it something you’d never type into a public chatbot. Then run ollama ps and see how it landed.

No gaming PC? Try a small model on the computer you have. It’ll be slower, and that’s fine. You’ll learn more in one evening of breaking it than in a month of reading about it.

Figure it out. Then tell me what you ran and how it did.

Keep reading

• AI Rent vs. Own vs. Hybrid: A Free TCO Decision Model in Excel. The business-sized version of this decision, with the full cost math.
• The Machines Were Always Trying to Talk. The Hardware Finally Let Them. Why GPUs and memory are what made LLMs possible in the first place.
• How to Build Your Own Private Family Streaming Service. More workbench self-hosting, three Raspberry Pi 5s at a time.