NevTan Cloud is the developer cloud where apps and AI live together, with one platform, one bill, and one identity. If you want to run a Llama model without standing up your own GPU cluster, you have three routes, and picking the right one before you start saves you a rebuild later.
This guide covers all three: calling a Llama model NevTan already hosts, launching a managed model server on a GPU you choose, and running your own serving stack on a raw GPU instance over SSH. By the end you will know which path fits your workload and how to get an answer out of a Llama model today.
Three ways to run Llama on NevTan Cloud
Hosted inference — call
meta-llama/Meta-Llama-3-8B-InstructorMeta-Llama-3-70B-Instructthrough the OpenAI-compatible API. No GPU to launch, billed per token.Model server — pick a model and a GPU offer, launch, and chat with it from the Playground. NevTan runs the serving stack. Billed per GPU-hour.
Raw GPU instance — launch a GPU from the catalog, SSH in, and run vLLM, TGI, or anything else yourself. Full control, billed per GPU-hour.
Start at the top of that list and move down only when you need what the next option gives you.
Which path should you take?
If you need… | Reach for | Billing |
|---|---|---|
A Llama chat model with zero infrastructure | Per token | |
To serve a specific open-weights model or your own fine-tune, without writing serving code | Per GPU-hour | |
Root access for a custom inference stack, training loop, or arbitrary tooling | Per GPU-hour | |
To adapt Llama to your own data first | See fine-tuning pricing |
The key difference is control versus convenience. A model server hands you a running model with nothing to configure. A raw instance hands you a Linux box with a GPU in it and gets out of your way.
One detail worth knowing up front: the hosted catalog models are called through the OpenAI-compatible API using a provider/model id. The console Playground is a separate chat tester that talks to models you deploy, meaning a model server or a fine-tuned model. They are different surfaces, so do not go looking for catalog models in the Playground model list.
Path A: Call a hosted Llama model (fastest)
If a Llama model NevTan already runs covers your use case, this is done in minutes and there is nothing to monitor or shut down.
Step 1 — Create an API key
Generate a key from the console. See API keys for scoping and rotation.
Step 2 — Find your model in the catalog
Open the model catalog and filter by modality. Each entry shows context length, status, and per-token pricing. The Llama entries available are:
meta-llama/Meta-Llama-3-8B-Instruct— chat, fastmeta-llama/Meta-Llama-3-70B-Instruct— chat, capable
Check the Status column before you build around a model. Most show Available immediately; a model can briefly show Starting right after it is added or when demand causes capacity to scale up.
Step 3 — Call it
The chat completions endpoint is OpenAI-compatible, so existing SDK code usually works after changing the base URL and model id. There is also a single-prompt mode for simpler completions.
When this path wins: spiky or low-volume traffic, prototypes, and anything where leaving a GPU running would be wasteful. You pay for what you generate rather than for uptime.
Path B: Launch a model server (a few clicks)
Choose this when you want a specific model that is not in the hosted catalog, or when you want to serve your own fine-tuned checkpoint.
Step 1 — Choose a model
Pick the model to serve: an open-weights model, or one of your own fine-tunes produced by fine-tuning.
Step 2 — Choose a GPU offer
Pick an offer from the catalog to run the server on, sized for the model you chose. The catalog searches a global compute network and ranks results by price and availability, and every offer shows its current per-hour rate. Filter by hardware, GPU count, and budget.
Step 3 — Launch and watch it load
The console shows live launch progress and logs while the model loads onto the GPU. Large models take longer to load than small ones, since the weights have to be fetched and placed in VRAM before the server reports ready.
Step 4 — Chat with it
Once the server reports ready, open it in the Playground and start chatting. There is no separate integration step. A model server behaves like any other model in the Playground model selector.
Step 5 — Manage and clean up
The Model Servers list shows every server you have running, its status, and its logs. Delete a server when you are done with it to stop billing for the GPU underneath it. The hourly rate accrues whether or not the server is handling requests.
Pro tip: Run your evaluation prompts in the Playground before wiring the model into anything. The catalog tells you what a model is; the Playground tells you how it behaves on your actual prompts.
Path C: Run your own serving stack on a GPU instance
Choose this when you need control the managed options do not give you: a custom vLLM configuration, a specific quantization, a non-standard runtime, or a training loop alongside inference.
Step 1 — Register your SSH key first
This is the step people get wrong. Instances only accept SSH keys already registered to your account under SSH keys. An instance that is already running will not pick up a key added afterward. If you launch without a registered key, the box boots with no way in and you have to destroy and relaunch it.
Step 2 — Pick an offer and an image
From the catalog, click Launch on the offer you want, then choose the container image the instance should boot. A general-purpose image with CUDA and your usual tooling is a reasonable default for interactive work. If you are running a fixed job, bake it into a custom image instead.
Step 3 — Connect over SSH
Once the instance reports running, its entry in the Instances list shows the SSH host and port. These are per-instance and are not fixed across launches, so re-check them every time.
Once connected you have root access on the box, the same as any bare Linux GPU host. Install packages, mount data, and run long jobs inside a terminal multiplexer so they survive a dropped connection.
Step 4 — Serve Llama with vLLM
Install vLLM and start an OpenAI-compatible server. Bind to 0.0.0.0 rather than the default loopback address, or the server will not be reachable from outside the container. Use --tensor-parallel-size matching your GPU count when sharding a larger model across multiple GPUs.
Step 5 — Clean up when you are done
Destroying an instance from the Instances list stops the running GPU immediately so billing ends. Nothing survives past the instance. Files live on that instance's disk only, so copy off checkpoints, logs, and outputs before you stop or destroy it.
Sizing a GPU offer for Llama
There is no universally correct GPU here. Work from memory requirements, because VRAM is the constraint that decides whether the model runs at all.
A rough starting point: a model needs roughly 2 bytes per parameter at FP16, before activations and KV cache. So an 8B model needs around 16 GB for weights alone, and a 70B model needs around 140 GB, which is more than a single 80 GB GPU holds at full precision.
Three things change that math:
Quantization. Running 8B at 4-bit drops weights to roughly 4–5 GB, which fits comfortably on a much smaller card. This is the biggest lever available.
Context length. The KV cache grows with both context length and concurrency. A configuration that works at 8K context can run out of memory at 128K with several concurrent users.
Tensor parallelism. Splitting a 70B model across multiple GPUs is how you serve it when one card cannot hold it, at the cost of inter-GPU communication overhead.
Always leave headroom. A configuration that exactly fills VRAM will fail the first time someone sends a longer prompt than you tested with.
Our complete guide to choosing the right GPU for your AI project covers the sizing framework in more depth.
How billing actually works
GPU pricing on NevTan Cloud is dynamic. There are no fixed GPU price tables. The catalog shows the current per-hour price for each machine based on real-time hardware availability, and that price can change between visits. You pay the rate shown on the offer at the moment you launch it, metered hourly and deducted from your credit balance.
That leads to a straightforward rule:
GPU instances and model servers bill per hour they are running, whether or not they are handling work. This pays off for steady, predictable load.
Hosted inference bills per token, with no instance to leave running. This is usually cheaper for spiky or low-volume use against a model NevTan already hosts.
Before budgeting, check the live per-hour price on the offer and the per-token rates for the catalog model you are comparing against. GPU availability and pricing vary by region and over time, so confirm the current rate rather than working from a number you saw last month.
Common mistakes
1. Launching before registering an SSH key. The instance boots with no way in and has to be destroyed and relaunched. Register the key first.
2. Leaving instances and model servers running. The hourly rate keeps accruing until you stop or destroy them. A forgotten model server is the single most common source of surprise spend. Delete servers you are finished with.
3. Assuming files persist. Instance storage does not survive the instance. Copy checkpoints and outputs off before you destroy anything.
4. Setting context length to the model maximum. Running at the full supported context consumes far more VRAM than most applications need. Measure your actual prompt lengths first.
5. Hardcoding tokens in a repository. Store credentials as environment variables rather than in code, and rotate them. See environment variables and API keys.
6. Reaching for a GPU when hosted inference would do. If a Llama model in the catalog covers the use case, per-token billing with no instance to babysit is almost always the cheaper and simpler answer.
7. Picking a GPU offer without checking region. Availability differs by location. See regions and availability.
FAQ
Which Llama models can I run on NevTan Cloud?
The hosted inference catalog includes meta-llama/Meta-Llama-3-8B-Instruct and meta-llama/Meta-Llama-3-70B-Instruct, callable through the OpenAI-compatible API. For any other Llama variant, launch a model server with the model you want, or run it yourself on a GPU instance.
Do I need a GPU to run Llama models?
Not if you use hosted inference, which runs on NevTan's infrastructure and bills per token. You need a GPU when you want to serve a specific model yourself, run a fine-tuned checkpoint, or control the serving stack.
Which Llama model should I start with?
Start with Llama 3 8B Instruct. It handles summarization, classification, and chat well, costs less per token, and responds faster. Move to 70B only after you have measured a quality gap on your specific task.
How much does it cost to deploy a Llama model?
GPU instances and model servers are billed at the per-hour rate shown live on each catalog offer, metered hourly against your credit balance. Hosted inference is billed per token. Because GPU pricing is dynamic, check the current rate on the offer before you launch rather than relying on a published figure.
Can I serve my own fine-tuned Llama model?
Yes. Run a fine-tuning job to produce the model artifact, then either deploy the model or launch a model server pointed at that checkpoint and chat with it from the Playground.
How do I stop being billed for a Llama deployment?
Delete the model server from the Model Servers list, or destroy the instance from the Instances list. Destroying stops the running GPU immediately so billing ends. Hosted inference has nothing to stop, since you are only charged per token generated.
What happens to my files when an instance is destroyed?
They are gone. Instance disks do not persist past the instance. Copy anything you need off the box first, or write outputs to object storage as the job runs.
Is my data safe?
NevTan Cloud is SOC 2 Ready and GDPR aligned. Store credentials as encrypted environment variables rather than in code, register SSH keys per account, enable two-factor authentication, and review audit logs. For specifics on controls and compliance posture, see the Trust Center and Security pages.
Get started
Running Llama on NevTan Cloud comes down to one decision: how much of the stack do you want to own? Call a hosted model and own none of it. Launch a model server and own the model choice but not the serving. Launch a GPU instance and own all of it.
Most teams should start with hosted inference, move to a model server when they need a model the catalog does not carry, and only reach for a raw instance when they need control that the managed paths do not offer.
Start free and send your first prompt to a Llama model today, or browse the GPU catalog to see what is available right now.
