2026-06-09 –, Poster Island C
With the rapid evolution of RISC-V extensions such as Vector, Matrix, and other custom instructions, RISC-V platforms are becoming capable of executing modern AI models. However, achieving high-performance deployment while maintaining a unified software stack across diverse extensions remains a key challenge. This paper introduces Buddy Compiler, an end-to-end AI compiler designed to provide a unified interface for AI model integration, multi-level compilation optimizations, and extensible code generation targeting diverse RISC-V extensions. Buddy Compiler adopts a multi-level architecture consisting of a frontend, middle-section, and backend, enabling reusable high-level optimizations while supporting specialized backends for RISC-V architectures. The frontend provides a graph infrastructure that interfaces with mainstream AI frameworks and converts imported models into a unified representation expressed with high-level MLIR dialects. The middle-section is built on MLIR and performs multi-level compilation optimizations, including operator fusion, memory access optimization, and vectorization. The backend implements dedicated MLIR dialects for RISC-V extensions, such as RVV, IME, AME, and Gemmini, and performs target-specific code generation. Through multi-level compilation, Buddy Compiler and its runtime system enable efficient deployment of AI models on RISC-V platforms, achieving performance comparable to manually optimized implementations such as llama.cpp.
Hongbin Zhang is a postdoctoral researcher at the Institute of Software, Chinese Academy of Sciences (ISCAS). He received his Ph.D. degree from the University of Chinese Academy of Sciences. His research focuses on AI system software and compiler technology. He leads the RuyiAI system software team at ISCAS. He launched the Buddy Compiler community and has served as a community mentor for the LFX RISC-V Mentorship and OSPP open-source programs. He is a member of the Technical Steering Committee (TSC) of RISC-V International and serves on the Governing Board of CHIPS Alliance.