At the 2024 European LLVM Developers’ Meeting, Mehdi Amini and Jeff Nui presented their keynote talk “How Slow is MLIR” 1. This presented micro-benchmarks for key operations in the MLIR compiler, such as traversing the IR and creating operations. However, the code for the benchmarks and instructions for running them are not easily available online alongside the talk. This blog post addresses this gap in the provision.
The benchmarks
By asking the in the LLVM discord and spelunking around GitHub, I found the benchmarks discussed in this talk are available on a branch of Mehdi’s fork of LLVM available here, with a diff with the main branch available here.
Running the benchmarks
To run the benchmarks, first clone the repository at the correct branch:
1git clone -b benchmarks https://github.com/joker-eph/llvm-project/
Then, built the appropriate CMake target:
1mkdir llvm-project/build
2cd llvm-project/build
3cmake -G Ninja ../llvm \
4 -DLLVM_ENABLE_PROJECTS=mlir \
5 -DLLVM_TARGETS_TO_BUILD="host" \
6 -DLLVM_ENABLE_BENCHMARKS=ON \
7 -DCMAKE_BUILD_TYPE=Release \
8 -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++
9cmake --build . --target MLIR_IR_Benchmark
Finally, run the benchmarks:
1./tools/mlir/unittests/Benchmarks/MLIR_IR_Benchmark