Tagged "inference-frameworks"
- Ask HN: What are you using for LLM inference in production?
- Open-Weight AI on Kubernetes: Comparing vLLM and KubeAI for Local Deployment
- The 5 Coolest Open-Source Projects I've Discovered in 2026
- Apple's MacBook Lineup Overhaul Features M7 Chip for Enhanced Local AI
- vLLM vs Ollama 2026: 793 vs 41 TPS Performance Benchmark
- DeepSeek's Flagship V4 Pro Model Drops to 75% Lower Pricing, Increasing Competitive Pressure on Local Inference Economics
- Self-Hosting LLMs Reveals Local AI Has a Friction Problem, Not a Quality Problem
- AI/ML Benchmark Tool for Local LLM Inference and XGBoost Training
- Cotypist – AI Autocomplete for Mac
- Researcher Discovers 221 Bugs in vLLM Stemming From Single Root Cause
- MiniMax M2.7 Is Now Open Source
- Intel Arc Pro B70 32GB Achieves 12 Tokens/Sec on Qwen 3.5-27B
- Ollama is Still the Easiest Way to Start Local LLMs, But It's the Worst Way to Keep Running Them
- NVIDIA and Google Optimize Gemma 4 AI Models for Local RTX Deployment
- GPUs vs. TPUs: Decoding the Powerhouses of AI
- NVIDIA Accelerates Gemma 4 for Local Agentic AI on RTX GPUs
- VRAM Optimization Technique Cuts Gemma 4 Memory Usage by 3x
- Researcher Successfully Runs Local LLMs on Legacy "Dead" GPU With Surprising Results
- Ditching Paid AI Services: Building Self-Hosted LLM Solutions as ChatGPT, Claude, and Gemini Alternatives
- Qwen 3.5 Emerges as Top Performer for Local Deployment with Extensive Quantization Options
- Kimi Introduces Attention Residuals: 1.25x Compute Performance at <2% Overhead
- FreeBSD 14.4 Released: Implications for Local LLM Deployment
- Community Survey: AI Content Automation Stacks in 2026
- M5 Max and M5 Ultra Chipsets Demonstrate Significant Bandwidth Improvements for Local LLM Inference
- How to Run Your Own Local LLM — 2026 Edition
- AMD Expands Ryzen AI 400 Series Portfolio for Consumer and Enterprise AI PC Options
- Qwen3.5-27B Identified as Sweet Spot for Mid-Range Local Deployment
- ByteDance Releases Seed2.0 LLM with Complex Real-World Task Improvements