Tiny-vLLM logo

Tiny-vLLM

Tiny-vLLM is an educational high-performance LLM inference engine built from scratch in C++ and CUDA. Created by Jakub Maczan, it implements the core features of production inference servers including KV cache, continuous batching, PagedAttention, and FlashAttention-like online softmax. The repository doubles as a comprehensive course where developers learn to build each component step by step, making it both a working inference engine and an invaluable teaching resource. Already supporting Llama 3.2 1B Instruct with full CUDA kernel computation, it has garnered massive attention on Hacker News with 187 points and significant community interest. Ideal for ML engineers, researchers, and educators who want to deeply understand LLM inference internals.

Reader rating

No ratings yet

Visit website

You might also like

Related tools

View all
codemap favicon
codemap
No ratings yet

codemap is an MIT-licensed project brain for AI coding tools that gives LLMs instant architectural context from your codebase without burning tokens. It generates a fast tree/context view, dependency flow, dependency blast-radius analysis, and a layered handoff format for cross-agent continuation, then exposes everything through a JSON context bundle and an MCP server compatible with Claude Code and Codex. A built-in Codex plugin and community skill registry make it easy to install and share. Developers use codemap to onboard agents to large repos in seconds, keep session continuity across handoffs, and scope the impact of a change before running it.

View details
Ollama favicon
Ollama
No ratings yet

Ollama is a local AI platform for running, managing, and sharing open models on your own machine or private infrastructure. It makes it easy to pull models, serve them through an API, and integrate local inference into developer workflows without relying on a fully managed cloud stack. Teams use Ollama for privacy-sensitive assistants, internal tools, offline experimentation, and rapid testing of open-weight models across laptops, workstations, and servers. It is especially useful for developers, operators, and AI builders who want quick setup with less operational overhead. What makes Ollama distinctive is how approachable it is: it packages model runtime, distribution, and deployment into a streamlined experience that helps people get productive with local AI in minutes instead of spending days on configuration.

View details
FileForge Finder favicon
FileForge Finder
No ratings yet

FileForge Finder is an AI-powered local file search utility that optimizes search results for developer workflows. It uses natural language processing to understand query intent and prioritize relevant files, code snippets, and documentation. The tool integrates with popular IDEs and terminals to provide instant, context-aware file retrieval, reducing time spent navigating complex project structures. It supports multiple file formats and offers advanced filtering by content type, modification date, and relevance.

View details

From the blog

Related articles

View all
Branded HungryMinded cover showing the phrase Default Wins for an article about AI models becoming tool defaults.
July 28, 2026 · 7 min read

The AI Model War Is Moving Into Your Tools

Kimi K3 shows why AI competition is shifting from benchmark wins to default integrations inside coding tools and inference platforms…

Branded HungryMinded cover reading Workflow Beats Benchmarks for an article about Claude Opus 5 and AI coding tools.
July 26, 2026 · 7 min read

Claude Opus 5 Shows Where the Model Race Is Going

Claude Opus 5 and Cursor show why AI competition is shifting from raw benchmarks to tools that sit inside real work…

Branded HungryMinded cover reading The Open Model You Can Actually Run, with a subtitle about hardware deciding who wins when Kimi K3 tops the chart.
July 21, 2026 · 7 min read

The Best Open Model Might Be the One You Can Actually Run

Kimi K3 tops the Frontend Code Arena, but free-to-download does not mean possible-to-run. The hardware math, not the leaderboard, decides who gets to use the next open model…