So, the kind people at AMD and Digital ocean granted me some credit to figure out stuff on their cloud.
I've been wondering, for a while, how to setup those cards, which are super hard to come by and totally out of my developer home setup budget.
In any case, with the credit, I've been able to figure out how to do things. I'm gonna write a couple of articles detailing how to use these awesome cards for differnet things. For now, let's just start with the basics:
Setting up a development environment
The first thing we need to do is to setup the dev environment. For now, I'm playing around with two GPU enabled droplets. Here's how to create them quickly:
Install doctl
- Requirements:
- curl
- jq
- tar
- gzip
First things first, we install doctl:
# install/update it
curl -Ls $( curl -Ls https://api.github.com/repos/digitalocean/doctl/releases/latest | jq -r '.assets | .[] | select(.name | endswith("linux-amd64.tar.gz")) | .browser_download_url') | tar -C ~/bin -f - -xvz
This way, in one step, we can easily install/update doctl in ~/bin. This works fine in fedora-based distros.
If you don't have it, just mkdir ~/bin.
Register doctl
Now, just register:
# initialize
doctl auth init --context=amdcloud
# configure the context as default (for now)
doctl auth auth switch --context amdcloud
Nota
Better check the official documentation, which is: https://docs.digitalocean.com/reference/doctl/ at the time.
Creating the GPU-enabled droplets
We have the tools we need.
Now, we're gonna proceed with the actual droplet creation and configuration.
droplet
So, let's create our first GPU droplet then!
doctl compute droplet create \
--api-url https://api-amd.digitalocean.com \
--enable-ipv6 \
--image fedora-42-x64 \
--region atl1 \
--size gpu-mi300x1-192gb-devcloud \
--ssh-keys=49228960 \
--tag-names 'amdgpu,fedora' \
0.fc42.gpu.evalinux.net
What do you need to change here?
- IPv6:
- No need for this one. I just like using it.
- tag-names:
- These are arbitrary. Just set your own or none at all.
- ssh-keys:
- Set up your own keys, get their ID and set them.
Now, this is, pretty much, a regular dropplet creation command. The thing is it's using a custom --api-url. This is important. Otherwise, you won't get it running.
That was the easy part. The really easy part is configuring ollama to work with AMD's MI300X GPUs. Here we go!
De-cripple
First things first. DigialOcean, for some reason, has a crippled version of Fedora up there.
We need to fix it:
# upgrade
dnf -y upgrade
dnf -y distro-sync --allowerasing
dnf -y autoremove
# install stuff
dnf -y remove nano
dnf -y install @core amd-gpu-firmware amd-ucode-firmware amdsmi bash-completion btop firewalld \
kernel kernel-core kernel-modules kernel-modules-core kernel-modules-extra \
lshw lspci python3-torch radeontop "rocm*" rpmconf tmux tuned vim \
vim-default-editor
# update configs
rpmconf -a
# set the right profile
tuned-adm profile accelerator-performance
# relabel and reboot
fixfiles onboot
reboot
With these commands we:
- Installed many of the utilities and firmware that were missing in Fedora. Even the kernel modules.
- Updated any configuration that required updating.
- Set tuned to the appropriate profile.
- Fixed SELinux contexts and rebooted.
I get why DigitalOcean "cleans up" the Fedora image for regular use. You don't need all that stuff in the image if you're gonna setup wordpress or some static HTML website. But we're not doing that, so... ;D
Configure
OK, this is the super easy part:
# install ollama
curl -fsSL https://ollama.com/install.sh | sh
# format and mount scratch disk
mkfs.btrfs -f /dev/vdc
# setup the directory where we will store the models (scratch disk)
mkdir -p /usr/local/share/models
echo '/dev/vdc /usr/local/share/models btrfs defaults 1 1' >> /etc/fstab
systemctl daemon-reload
mount -a
mkdir -p /usr/local/share/models/ollama
# fix perms
chmod 2770 /usr/local/share/models/ollama
chown ollama:ollama /usr/local/share/models/ollama
# edit the ollama unit
mkdir -p /etc/systemd/system/ollama.service.d
# configure ollama for AMD MI300X
# reference: https://rocm.docs.amd.com/en/latest/reference/gpu-arch-specs.html
cat << EOF > /etc/systemd/system/ollama.service.d/override.conf
[Service]
# listen everywhere
Environment="OLLAMA_HOST=0.0.0.0"
# put stuff in the scratch disk
Environment="OLLAMA_MODELS=/usr/local/share/models/ollama"
# many threads
Environment="OLLAMA_NUM_THREADS=$( nproc )"
# MI300X
Environment="HSA_OVERRIDE_GFX_VERSION=9.4.2"
Environment="PYTORCH_ROCM_ARCH=gfx942"
EOF
## restart the unit
systemctl daemon-reload
systemctl restart ollama
# firewall
# totally insecure; we're not using SSL
#firewall-cmd --permanent --add-port=11434/tcp
#firewall-cmd --reload
# get some models
for m in gemma3:27b llama4:maverick qwen3:235b qwen3-coder:latest; do ollama pull $m; done
¡PELIGRO!
I left the firewall part commented. We're not using SSL so it's dangerous to use the models using an ssh tunnel. You can try it out but do it knowlingly. It's a risk.
There you go! Once you run that, you'll have ollama installed, with a few awesome models to try out.
Testing things out
# check if ollama is running
systemctl status ollama
# list models
ollama ls
# run one
ollama run llama4:maverick
# ask it to check your logs
ollama run llama4:maverick "Check the logs for errors and propose some fixes, please! " << <( journalctl -b )
And remember, have fun with it!
Here's a referral link if you wanna try DigitalOcean (and to support me ;D): https://m.do.co/c/5ac5243c4061
Nota
If you wanna test out the GPUs, you need to apply for it here: https://www.amd.com/en/developer/resources/cloud-access/amd-developer-cloud.html
SSH tunnel
If you'd like to map that ollama port locally so that you can run qwen-code or something similar, just:
ssh -L 11434:localhost:11434 user@your-gpu-droplet
This way, all your requests from localhost will arrive at the GPU-enabled droplet in a secure way.
Acknowledgements
- Thanks to AMD for giving me credit to test out the cloud.
- Thanks to DigitalOcean for giving me more credit for the testing and the support.