Zehao Huang

A high-throughput and memory-efficient inference and serving engine for LLMs

Generative AI Examples is a collection of GenAI examples such as ChatQnA, Copilot, which illustrate the pipeline capabilities of the Open Platform for Enterprise AI (OPEA) project.

GenAI components at micro-service level; GenAI service composer to create mega-service

Model Zoo for Intel® Architecture: contains Intel optimizations for running deep learning workloads on Intel® Xeon® Scalable processors

Semantic cache for LLMs. Fully integrated with LangChain and llama_index.

A command-line tool to render Jinja templates for great good

Intel® Low Precision Optimization Tool, targeting to provide a unified low precision inference interface cross different deep learning frameworks, and support auto-tune with specified accuracy criterion to find out best quantized model.