Run a vLLM Server on HF Jobs in One Command
Hugging Face run a vLLM Server on HF Jobs in One Command
Once it’s up, you can query it from your laptop, a notebook, or anywhere else. It’s the quickest way to stand up a model for tests, evals, or batch generation. (If you’re after a managed, production-ready service instead, that’s what Inference Endpoints are for — more on when to pick which at the end.) hf jobs run is docker run for HF infrastructure. We use the official vllm/vllm-openai image, ask for a GPU with –flavor, and expose vLLM’s port with –expose: hf jobs run –flavor a10g-large –expose 8000 –timeout 2h vllm/vllm-openai:latest vllm serve Qwen/Qwen3-4B –host 0.0.0.0 –port 8000 –expose 8000 routes the container’s port through HF’s public jobs proxy (see the Serve Models guide for the full reference). The command prints the URL your server is reachable at: ✓ Job started id: 6a381ca1953ed90bfb947332 url: https://huggingface.co/jobs/qgallouedec/6a381ca1953ed90bfb947332 Hint: Exposed ports are reachable at (requires an HF token with read access to the job): https://6a381ca1953ed90bfb947332–8000.hf.jobs 6a381ca1953ed90bfb947332 is your job ID. Keep track of it, we’ll need it. We’ll use <job_id> as a placeholder for it in the rest of the post. Give it a couple of minutes to download weights and boot. When the logs show Application startup complete, you’re live. vLLM speaks the OpenAI API, and every request just needs your HF token as a bearer token. The quickest way to hit it is curl: curl https://<job_id>–8000.hf.jobs/v1/chat/completions -H “Authorization: Bearer $(hf auth token)” -H “Content-Type: application/json” -d ‘{ “model”: “Qwen/Qwen3-4B”, “messages”: [{“role”: “user”, “content”: “Hello!”}], “chat_template_kwargs”: {“enable_thinking”: false} }’ which returns the usual OpenAI-style JSON, with choices[0].message.content holding “Hello! How can I assist you today? 😊”. Or, from Python, point the OpenAI client at the exposed URL and pass the token as the API key: from huggingface_hub import get_token from openai import OpenAI client = OpenAI( base_url=”https://<job_id>–8000.hf.jobs/v1″, api_key=get_token(), ) resp = client.chat.completions.create( model=”Qwen/Qwen3-4B”, messages=[{“role”: “user”, “content”: “Hello!”}], extra_body={“chat_template_kwargs”: {“enable_thinking”: False}}, ) print(resp.choices[0].message.content) Hello! How can I assist you today?
