If you're talking about 3.3 70B Q4, any PC with 64 GB RAM could run it. | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

wmf on Jan 29, 2025 | parent | context | favorite | on: Complete hardware and software setup for running D...

If you're talking about 3.3 70B Q4, any PC with 64 GB RAM could run it.

mrbonner on Jan 29, 2025 [–]

I can use RTX to speed the inference up. My budget is up to $2500.

wmf on Jan 29, 2025 | [–]

Maybe you could run some layers on a 3090; I'm not sure how much speedup it would give.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact