On my machine I’m running opensuse tumbleweed and has the amdgpu driver installed. I use it for gaming and recently I’ve become interested in running LLMs. So I would like to keep a balance of both without compromising too much on performance.
I know that there are proprietary drivers for AMD cards but I’m hesitant to install it as I’ve heard that it performs less efficiently in games when compared to the open source driver.
I’m mainly confused about this ROCM thing. Is it not included with the opensource amdgpu drivers ? Or is it available as a separate package?
So what driver to use ?
Or perhaps, is it possible to run oogabooga or stable diffusion within a distrobox container (with the proprietary drivers) and still keep using the open source gpu drivers for the Host operating system.
You must log in or register to comment.
llama.cpp (and ollama) has AMD support through ROCm and also now Vulkan.
To do general purpose GPU calculations on AMD hardware you need a GPU that is supported by ROCm (AMD’s equivalent to CUDA). Most of the gaming GPUs are not.
There is a list here but be aware that that is for the latest rocm version, some tools might still use older versions with different supported devices.
Has that changed recently? I’ve ran ROCm successfully on an RX6800. I seem to recall that was supported, the host OS (Arch) was not.
When I tried it maybe a year or so ago there were four supported chipset in that version (5.4.2 I think) of rocm but I don’t remember which card models those were since they were only specified in that internal chip name. Mine wasn’t supported at the time (5700XT)
No, GFX1030 is still supported.
Llama.cpp supports OpenCL as well and performs better than rocm in my limited experience. That should work on basically any GPU.
This link is misleading. For example, the Radeon RX6800 IS supported because it is the same chip as one of the Radeon Pros. GFX1030. Many others are too…though support does not go very far back.
Latest ollama has support for AMD GPUs. I had to compile from source to make it pick up the GPU on my system.
Look into llamafile. It makes things so easy.
Local LLMs usually run using CPU only
I want to use software like stable diffusion and oogabooga. Don’t they utilize GPU ?
Ooga Booga is just a webui for local text gen LLMs, that usually only utilise CPU
SD, on the other hand, does utilise GPU. I haven’t tried it but I don’t see why it won’t run with the open source AMD drivers, they are good. Just try it yourself, everything is free and you would only lose some time in the worst case scenario
You absolutely can use GPU. Try the auto installer and select AMD when the option shows up. The documentation should have more info.
All the ones I’ve used so far are able to use the GPU but it has to be enabled in the app settings. I mostly use LM Studio and it flies on my nvidia 3060. Doesn’t seem to have options for AMD GPUs though, unless I’m mistaken.
