linux
-
Useful AMD GPU Profiling tools for Linux (CLI/AI)
Below is a list of Linux CLI tools for profiling AI inference engines on Linux. I use these for optimising Kernels. Kernel-Level Profiling Command Description Install rocprof Legacy kernel profiler.
-
How to Reduce AMD R9700 AI PRO GPU Power Usage by 17% for AI Inference with a Dockerised Controller
Running four AMD Radeon AI PRO R9700 GPUs at stock 300W each for local AI inference is wasteful. Like NVIDIA GPUs, AI inference workloads are memory-bandwidth bound, not compute bound.
-
Dual AMD R9700 Setup on ASUS X99-E WS/USB 3.1
Running two AMD Radeon AI PRO R9700 cards (gfx1201, RDNA4, 32GB each) on an ageing ASUS X99-E WS/USB 3.1 board is possible, but the platform fights you at
-
Layer Split Model Parallelism on Hybrid AMD NVIDIA AI Servers using Vulkan and Llama CPP
Running a single large language model across GPUs from two different vendors is not something the tooling expects you to do. CUDA is NVIDIA only. ROCm is AMD only. The
-
How to Reduce NVIDIA GPU Power Usage and Clock Speeds for AI Inference with a Dockerised Controller
Running multiple GPUs for local AI inference at stock power settings is wasteful. Consumer cards like the RTX 3090 draw 350W by default, but AI inference workloads are typically memory-bandwidth