MNN is a blazing fast, lightweight deep learning framework, battle-tested by business-critical use cases in Alibaba. Full multimodal LLM Android App:[MNN-LLM-Android](./apps/Android/MnnLlmChat/README.md)
High-speed Large Language Model Serving for Local Deployment
QOwnNotes is a plain-text file notepad and todo-list manager with Markdown support and Nextcloud / ownCloud integration.
Connect home devices into a powerful cluster to accelerate LLM inference. More devices means faster inference.
INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model
A free and open source, self hosted Ai based live meeting note taker and minutes summary generator that can completely run in your Local device (Mac OS and windows OS Support added. Working on adding linux support soon)
A @ClickHouse fork that supports high-performance vector search and full-text search.
A highly optimized LLM inference acceleration engine for Llama and its variants.
a lightweight LLM model inference framework
RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.
🤘 TT-NN operator library, and TT-Metalium low level kernel programming model.
Pure C++ implementation of several models for real-time chatting on your computer (CPU & GPU)
A high-performance inference system for large language models, designed for production environments.
校招、秋招、春招、实习好项目，带你从零动手实现支持LLama2/3和Qwen2.5的大模型推理框架。
Yet Another Language Model: LLM inference in C++/CUDA, no libraries except for I/O
CPU inference for the DeepSeek family of large language models in pure C++
Add a description, image, and links to the llm topic page so that developers can more easily learn about it.
To associate your repository with the llm topic, visit your repo's landing page and select "manage topics."