Getting Started

Getting Started with LLMIR

Welcome to LLMIR! This section helps you get started with the LLMIR compiler infrastructure for LLM inference optimization.

pip install llmir

import llmir
config = llmir.KVCacheConfig(num_layers=32, num_heads=32, head_dim=128)
cache = llmir.PagedKVCache(config)

See the User Guide for installation options, use cases, and API examples. See Developer Guide for building from source and C++ dialect.

Resource	Description
User Guide	Installation, use cases, API examples, repo layout
Developer Guide	Build from source, benchmarks, project structure
Architecture	KV cache, quantization, distributed deployment
Performance Evaluation	Benchmark results, verification
Publications	ICCD 2025 paper, performance highlights
FAQ	Common questions