Tagged "consumer-gpu"
-
Running a Private AI Brain on Windows PC as Alternative to Cloud Services
-
Powerful AI Search Engine Built on Single GeForce RTX 5090
-
Ditching Paid AI Services: Building Self-Hosted LLM Solutions as ChatGPT, Claude, and Gemini Alternatives
-
Rust Project Perspectives on AI
-
Qwen 3.5 122B Uncensored (Aggressive) Released with New K_P Quantisations
-
Nvidia Nemotron Cascade 2 30B Emerges as Powerful Alternative to Qwen Models
-
Developer Builds Fully Local Multi-Agent System Using vLLM and Parallel Inference
-
Llama 8B Matches 70B Performance on Multi-Hop QA Using Structured Prompting
-
ik_llama.cpp Fork Delivers 26x Faster Prompt Processing on Qwen 3.5 27B
-
Why You Should Use Both ChatGPT and Local LLMs: A Practical Hybrid Approach
-
Careless Whisper – Personal Local Speech to Text
-
AI Playground for Developers Built in Vite and Python
-
Qwen 3.5 397B emerges as top-performing local coding model
-
Multi-Token Prediction support coming to MLX-LM for Qwen 3.5
-
Apple M5 Max 128GB real-world performance benchmarks for local inference
-
Local AI Coding Assistant: Free Cursor Alternative with VS Code, Ollama & Continue
-
DeepSeek R1 RTX 4090 vs Apple M3 Max: Benchmark & Performance Guide
-
Build a $1,500 AI Server with DeepSeek-R1 on RTX 4090
-
Qwen 3.5 Emerges as Top Performer for Local Deployment with Extensive Quantization Options
-
Community Converges on Optimal KV Cache Quantization Strategies for Qwen 3.5 Models
-
Repurpose Old GPUs as Dedicated AI Inference Accelerators
-
NVIDIA Nemotron Cascade 2 30B Delivers 120B-Class Performance in Compact Form Factor
-
NVIDIA Nemotron 3 Nano 4B Enables On-Device Inference Directly in Web Browsers via WebGPU
-
Llamafile 0.10 Released with GPU Support and Rebuilt Core
-
Meet Sarvam Edge: India's AI Model That Runs on Phones and Laptops With No Internet
-
Tether's QVAC Introduces Cross-Platform Bitnet LoRA Framework for On-Device AI Training
-
Unsloth Studio: Open-Source Web UI for Training and Running LLMs Locally
-
Snapdragon 8 Elite Gen 5 Hands the Galaxy S26 the AI Upgrade We've Been Waiting For
-
MiniMax-M2.7: New Compact Model Announced for Local Deployment
-
Mamba 3: State Space Model Architecture Optimized for Inference
-
I Switched to a Local LLM for These 5 Tasks and the Cloud Version Hasn't Been Worth It Since
-
Custom GPU Multiplexer Achieves 0.3ms Model Switching on Legacy Hardware
-
Run LLMs Locally with Llama.cpp
-
I Ran Local LLMs on a 'Dead' GPU, and the Results Surprised Me
-
Qwen 3.5 4B Outperforms Nvidia Nemotron 3 4B in Local Benchmarks
-
Mistral Small 4 119B Released with NVFP4 Quantisation Support
-
Mistral Releases Small 4 Open-Source Model Under Apache 2.0
-
Kimi Introduces Attention Residuals: 1.25x Compute Performance at <2% Overhead
-
OpenClaw Isn't the Only Raspberry Pi AI Tool—Here Are 4 Others You Can Try This Week
-
OmniCoder-9B: Efficient Coding Model for 8GB GPUs
-
This External GPU Enclosure Tries to Break Cloud Dependence for Local AI Inference
-
Dictare – Open-source Voice Layer for AI Coding Agents (100% Local)
-
AMD Declares 'AI on the PC Has Crossed an Important Line' – Agent Computers as Next Breakthrough
-
Nvidia's Nemotron 3 Super: Understanding the Significance for Local LLM Deployment
-
Running Qwen3.5-27B Across Multiple GPUs Over LAN Achieves Practical Speed for Local Inference
-
Startup Transforms Mac Mini Into Full-Powered AI Inference System With External GPU
-
Two Local Models Prove Competitive Enough to Replace ChatGPT, Gemini, and Copilot
-
India's Mobile-First AI Strategy Could Accelerate Local Inference Adoption in Emerging Markets
-
Hybrid AI Desktop Layer Combining DOM-Automation and API-Integrations
-
Open-Source GreenBoost Driver Augments NVIDIA GPU VRAM With System RAM and NVMe Storage
-
AMD Launches Agent System Optimized for Local AI Inference With Ryzen and Radeon
-
Achieving 2000 Tokens Per Second with QWEN 3.5 27B on RTX-5090
-
P-EAGLE: Faster LLM Inference with Parallel Speculative Decoding in vLLM
-
Local Manga Translator: Production LLM Pipeline with YOLO, OCR, and Inpainting
-
Best Local LLM Models 2026: Developer Comparison
-
3-Path Agent Memory: 8 KB Recurrent State vs. 156 MB KV Cache at 10K Tokens
-
Linux 7.0 AMDGPU Fixing Idle Power Issue For RDNA4 GPUs After Compute Workloads
-
How to Install OpenClaw with Ollama (Step-by-Step Tutorial)
-
Show HN: VmExit – An Experiment in AI-Native Computing
-
Sarvam Open-Sources 30B and 105B Reasoning Models
-
Qwodel – An Open-Source Unified Pipeline for LLM Quantization
-
Quantization Explained: Q4_K_M vs AWQ vs FP16 for Local LLMs
-
Nvidia Pushes Jetson as Edge Hub for Open AI Models
-
Nvidia Releases Nemotron 3 Super: 120B MoE Model for Local Deployment
-
Apple M5 Max 128GB Benchmark Results for Local LLM Inference
-
The $1,500 Local AI Setup: DeepSeek-R1 on Consumer Hardware
-
Local AI Coding Assistant: Complete VS Code + Ollama + Continue Setup
-
Cutile.jl Brings Nvidia CUDA Tile-Based Programming to Julia
-
Experiment: 0.8B Model Self-Improvement on MacBook Air Yields Surprising Results
-
Texas Instruments Launches NPU-Powered MCUs for Low-Power Edge AI
-
Sarvam Open-Sources 30B and 105B Reasoning Models
-
Qwen 3.5-35B Uncensored GGUF Models Now Available
-
Llama.cpp Celebrates Major Milestone: From Leak to Industry Standard
-
Qwen 3.5 Ultra-Compact Models Enable On-Device AI from Watches to Gaming
-
HP OMEN MAX 16 Review: Is Local AI on a Laptop Viable in 2026?
-
Fine-Tuned Qwen SLMs (0.6–8B) Demonstrate Competitive Performance Against Frontier LLMs on Specialized Tasks
-
Strix Halo (Ryzen AI Max+ 395) Achieves Strong Local Inference Performance with ROCm 7.2
-
Sarvam Open-Sources 30B and 105B Reasoning Models
-
Qwen 3.5 Family Benchmark Comparison Shows Strong Performance Across Smaller Models
-
Qwen 3.5 Derestricted Model Available for Local Deployment
-
When Running Ollama on Your PC for Local AI, One Thing Matters More Than Most
-
Nemotron 9B Powers Large-Scale Local Inference: Patent Classification and Real-Time Applications
-
Gyro-Claw – Secure Execution Runtime for AI Agents
-
Engram – Open-Source Persistent Memory for AI Agents
-
Qwen 3.5 27B Achieves Strong Local Inference Performance
-
Mistral AI Prepares Workflows Integration for Le Chat
-
Llama.cpp Prompt Processing Optimization: Ubatch Size Configuration Guide
-
ETH Zurich Research Challenges Context-Length Assumptions in LLM Agents
-
Apple Launches MacBook Neo with A18 Pro Chip for Affordable Local AI Inference
-
Windows 11 Notepad Gets On-Device AI Text Generation Without Subscription
-
Alibaba Releases Qwen 3.5 AI Model with On-Device AI Support
-
The Emerging Role of SRAM-Centric Chips in AI Inference
-
Final Qwen3.5 Unsloth GGUF Update with Improved Size/Quality Tradeoffs
-
Alibaba Releases Qwen 3.5 AI Model with On-Device AI Support
-
Kakao Launches Kanana AI for On-Device Schedule and Recommendation Management
-
Qwen 3.5-35B-A3B Achieves 37.8% on SWE-bench Verified Hard
-
Qwen 3.5-27B Q4 Quantization Comparison and Analysis
-
On-Device AI Laptop Lineups Become Standard Across Major Manufacturers
-
AMD Launches Copilot+ Desktop Chips to Compete in On-Device AI Market
-
VibeWhisper – macOS Voice-to-Text with 100% Local Processing Option
-
Qwen 3.5 0.8B Running in Browser with WebGPU via Transformers.js
-
Intel Arc Pro B70 Workstation GPU Confirmed via vLLM AI Release Notes
-
AMD Ryzen AI 400 Series Desktop Processors Launch with Integrated 60 TOPS NPU
-
Local LLM Performance Improvements: A Year of Progress Since DeepSeek R1 Moment
-
Jan Releases Code-Tuned 4B Model for Efficient Local Code Generation and Development Tasks
-
HP ZBook Ultra 14 G1a Workstation Reclaims Local AI Workflows for Professionals
-
Browser Use vs. Claude Computer Use: Comparing Agent Automation Frameworks
-
Apple Neural Engine Reverse-Engineered for Local Model Training on Mac Mini M4
-
Qwen 3.5-35B-A3B Emerges as Efficient Daily Driver, Replacing 120B Models
-
Nummi – AI Companion with Memory and Daily Guidance
-
4 Free Tools to Run Powerful AI on Your PC Without a Subscription
-
Apple Intelligence, Galaxy AI, Gemini: Why Your AI-Powered Phone Is Worth Repairing
-
Unsloth Dynamic 2.0 GGUFs
-
Qwen3.5-35B Unsloth Dynamic GGUFs Achieve SOTA Across Nearly All Quantisation Levels
-
Qwen3.5-35B RTX 5080 Experiments Confirm KV q8_0 as Free Lunch, Q4_K_M Remains Optimal
-
Qwen 3.5-35B Unsloth Dynamic GGUFs Achieve SOTA Quantisation Benchmarks
-
Qwen 3.5-27B Demonstrates Exceptional Performance with Thoughtful Prompt Engineering
-
On-Device AI in Mobile Apps: What Should Run on the Phone vs the Cloud (A 2026 Decision Guide)
-
The ML.energy Leaderboard
-
LLmFit: Terminal Tool for Right-Sizing LLM Models to Your Hardware
-
LLmFit: One-Command Hardware-Aware Model Selection Across 497 Models and 133 Providers
-
Krasis: Hybrid CPU/GPU MoE Runtime Achieves 3,324 Tokens/Second Prefill on RTX 5080
-
Krasis Hybrid MoE Runtime Achieves 3,324 tok/s Prefill on Single RTX 5080
-
Accuracy vs. Speed in Local LLMs: Finding Your Sweet Spot
-
5 Useful Docker Containers for Agentic Developers
-
Show HN: Caret – Tab to Complete at Any App on Your Mac
-
Qwen 3.5 MoE Delivers 100K Context Window at 40+ TPS on RTX 5060 Ti
-
Qwen 3.5 Underperforms on Hard Coding Tasks—APEX Benchmark Analysis
-
Qwen3.5 122B Achieves 25 tok/s on 72GB VRAM Setup
-
Researchers Develop Persistent Memory System for Local LLMs—No RAG Required
-
DeepSeek Releases DualPath: Addressing Storage Bandwidth Bottlenecks in Agentic Inference
-
DeepSeek Paper – DualPath: Breaking the Bandwidth Bottleneck in LLM Inference
-
The Complete Developer's Guide to Running LLMs Locally: From Ollama to Production
-
Qwen3.5-35B-A3B Emerges as Game-Changer for Agentic Coding Tasks
-
Qwen3.5-27B Identified as Sweet Spot for Mid-Range Local Deployment
-
PyTorch Foundation Announces New Members as Agentic AI Demand Grows
-
Show HN: Pluckr – LLM-Powered HTML Scraper That Caches Selectors and Auto-Heals
-
Mirai Announces $10M to Advance On-Device AI Performance for Consumer Devices
-
Show HN: 100% LLM Accuracy–No Fine-Tuning, JSON Only
-
Advanced Quantization Techniques Show Surprising Performance Gains Over Standard Methods
-
How AI is Redefining Price and Performance in Modern Laptops
-
Enterprise Infrastructure Guide: Running Local LLMs for 70-150 Developers
-
Breaking the Speed Limit: Strategies for 17k Tokens/Sec Local Inference
-
South Korea to Launch $687 Million Project to Develop On-Device AI Semiconductors
-
Qwen3's Voice Embeddings Enable Local Voice Cloning and Mathematical Voice Manipulation
-
Custom Portable Workstation Optimized for Local AI Inference Builds
-
Open-Source Framework Achieves Gemini 3 Deep Think Level Performance Through Local Model Scaffolding
-
Nvidia Could Launch Its First Laptops With Its Own Processors
-
Local GPT-OSS 20B Model Demonstrates Practical Agentic Capabilities
-
A Tool to Tell You What LLMs Can Run on Your Machine
-
Open-Source llama.cpp Finds Long-Term Home at Hugging Face
-
GPT-OSS 20B Demonstrates Practical Agentic Capabilities Running Fully Locally
-
GLM-5 Becomes Top Open-Weights Model on Extended NYT Connections Benchmark
-
Elastic Introduces Best-in-Class Embedding Models for High Performance Semantic Search
-
Breaking the Speed Limit: Strategies for 17k Tokens/Sec Local Inference
-
Yet Another Fix Coming for Older AMD GPUs on Linux – Thanks to Valve Developer
-
Ouro 2.6B Thinking Model GGUFs Released with Q8_0 and Q4_K_M Quantization
-
O-TITANS: Orthogonal LoRA Framework for Gemma 3 with Google TITANS Memory Architecture
-
At India AI Impact Summit, Intel Showcases AI PCs and Cost-Efficient Frugal AI
-
Strix Halo Performance Benchmarks: Minimax M2.5, Step 3.5 Flash, Qwen3 Coder
-
Qwen3 Coder Next Remains Effective at Aggressive Quantization Levels
-
[Release] Ouro-2.6B-Thinking: ByteDance's Recurrent Model Now Runnable Locally
-
At India AI Impact Summit, Intel Showcases Its AI PCs and Cost-Efficient Frugal AI
-
GGML.AI Acquired by Hugging Face
-
Qwen3 Coder Next 8FP Demonstrates Exceptional Long-Context Performance on 128GB System
-
PaddleOCR-VL Now Integrated into llama.cpp for Multilingual OCR
-
NVIDIA Releases Dynamo v0.9.0: Infrastructure Overhaul With FlashIndexer and Multi-Modal Support
-
Mirai Secures $10M to Optimize On-Device AI Amid Cloud Cost Surge
-
Free ASIC-Accelerated Llama 3.1 8B Inference at 16,000 Tokens/Second
-
AI Integration in Sublime Text: Practical Local LLM Editor Enhancement
-
Enhanced Quantization Visualization Methods for Understanding LLM Compression Trade-offs
-
LayerScale Launches Inference Engine Faster Than vLLM, SGLang, and TRT-LLM
-
Kitten TTS V0.8 Released: State-of-the-Art Super-Tiny Text-to-Speech Model Under 25MB
-
Hardware Economics Shift: DDR5 RDIMM Pricing Now Comparable to GPUs for Local Inference
-
Qwen3-Next 80B MoE Achieves 39 Tokens/Second on RTX 5070/5060 Ti Dual-GPU Setup
-
Qwen 3.5-397B-A17B Now Available for Local Inference with Aggressive Quantisation
-
High Bandwidth Flash Memory Could Alleviate VRAM Constraints in Local LLM Inference
-
Cohere Releases Tiny Aya: Efficient 3.3B Multilingual Model for 70+ Languages
-
ASUS Zenbook 14 Launches in India with AI-Capable Hardware, Starting at Rs 1,15,990
-
Ask HN: What is the best bang for buck budget AI coding?
-
GPU-Accelerated DataFrame Library for Local Inference Workloads
-
Alibaba Unveils Major AI Model Upgrade Ahead of DeepSeek Release
-
NVIDIA's Dynamic Memory Sparsification Cuts LLM Inference Costs by 8x
-
MiniMax-M2.5 230B MoE Model Released with GGUF Support for Local Deployment
-
LLaDA2.1 Introduces Token Editing for Massive Speed Gains in Local Inference
-
GPT-OSS 20B Now Runs 100% Locally in Browser via WebGPU
-
GNOME's AI Assistant Newelle Adds llama.cpp Support and Command Execution
-
Context Management Identified as Real Bottleneck in AI-Assisted Coding
-
Ring-1T-2.5 Released with SOTA Deep Thinking Performance
-
The Future of AI Slop Is Constraints - Implications for Local Models
-
Samsung's REAM: Alternative Model Compression Technique
-
Running Mistral-7B on Intel NPU Achieves 12.6 Tokens/Second
-
GLM-5 Released: 744B Parameter MoE Model Targeting Complex Tasks
-
I Tried a Claude Code Rival That's Local, Open Source, and Completely Free
-
NAS System Achieves 18 tok/s with 80B LLM Using Only Integrated Graphics
-
Nanbeige4.1-3B: A Small General Model that Reasons, Aligns, and Acts
-
Carmack Proposes Using Long Fiber Lines as L2 Cache for Streaming AI Data
-
Community Member Builds 144GB VRAM Local LLM Powerhouse