GPU rental for 70B LLMs — ASUS GX10
Request ASUS Ascent GX10 to test a 70B model in your application. One node has 128 GB of unified CPU and GPU memory. This lets you consider quantized LLMs that do not fit entirely within 24 GB of VRAM. Start with the model format, context window and target workload.
Budgeting memory for 70B weights
For weights alone, estimate 70 billion parameters × bit width / 8. That gives about 35 GB at 4 bits, 70 GB at 8 bits and 140 GB at 16 bits, using decimal units. Quantization formats add metadata. For 70B in FP16, weights alone already exceed a node’s 128 GB.
Size context for your actual workload
Long conversations and concurrent requests increase the KV cache. Add runtime, system and application memory, then measure peak use at the required input length. For RAG, use real documents and retrieval chunk sizes: a short demonstration prompt does not represent the service’s working load.
What to test on ASUS GX10
Choose an engine that supports the model format, ARM64 and GB10. Record the container version, quantization settings and prompt set. Measure time to first token, response speed and behavior with concurrent users. Compare identical prompts and settings to determine the cost of a useful result.
Choose a configuration and rental period
Start with one ASUS system to evaluate the model. A pair is useful for two independent tests or a distributed workload with a suitable runtime; node memory is not automatically pooled. Request your dates and duration. Current rates and the duration calculator are on the ASUS GX10 configuration page.
ASUS GX10 rental rate
One hourly rate applies to the selected rental duration. The calculator multiplies it by the full rental period.
The rate is for one ASUS Ascent GX10 with 128 GB of memory shared by the CPU and GPU.
Choose ASUS for your model
Share the model, quantization, context and dates. Choose one node or a pair and agree on a configuration for your workload.
Models for ASUS GX10ASUS GX10 rental
Before you rent
Will every 70B model fit?
Memory use depends on weight format, context, KV cache and runtime. The weights estimate is a starting point. Validate the final configuration with the selected model.
Can one GX10 run 70B in FP16?
A 16-bit representation of 70B weights needs about 140 GB before overhead. For a single 128 GB node, consider suitable quantization and the full memory budget.
Can I use Ollama or vLLM?
Specify the engine, version and model. ARM64, GB10 and weight-format support need checking, followed by a trial run.
How do I prepare a rental request?
Include the model link, quantization, maximum context, concurrency and rental dates. Add a sample input and the criterion you will use to evaluate the result.