Say you want to run an open model, for an experiment, a benchmark, or to skip an API bill, and the GPUs you have are on a cluster. Getting it going means a batch script, a vLLM that matches the cluster's CUDA, finding which node the job landed on, a free port, and a tunnel back to your laptop. Every time.
sllm does all of that. It sizes a job to the model's memory, sends it to whichever of
your clusters will start it soonest (using rclust),
waits until the model is loaded, and forwards a port through your SSH connection. After that,
anything that speaks the OpenAI API can use it:
sllm prepare qwen-3-8b # start (or reuse) a job and wait for the modelsllm proxy # http://localhost:8000/v1
from openai import OpenAI client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused")client.chat.completions.create(model="qwen-3-8b", messages=[{"role": "user", "content": "Hello!"}])
If your clusters need a two-factor login, you log in once with sllm connect, and
every later command reuses that connection.
Install
uv tool install git+https://github.com/narunraman/slurm-llmrclust config # describe your clusterssllm init # where models and jobs live on each onesllm setup # install vLLM on the cluster, once
Settings live in ~/.config/slurm-llm/. sllm config validate checks them.
Use
sllm connect # log in to each cluster oncesllm prepare qwen-3-8b # serve a modelsllm jobs # what's running wheresllm cancel <job-id> # stop it