Choosing your configuration
Compare GPU memory, software compatibility and workload requirements.
Does the workload fit in memory?
Start with your model and maximum expected load. Our ASUS Ascent GX10 nodes each have 128 GB of shared CPU/GPU memory, with some used by the system. Include context, concurrent requests and intermediate data. The weights file size is only a starting point.
Does your stack run on GX10?
Check the container architecture, engine support for the GPU and quantization format. Include any custom CUDA extensions in your compatibility test. Measure performance instead of assuming results from a different GPU will transfer.
Do you need a second node?
Choose two nodes for a supported distributed setup or two independent workloads. If you need one large process, first check whether the application can split its work across nodes. Attach your launch configuration to make the discussion specific.
Need GPU rental for this workload?
Provide your model and preferred dates. We will confirm the configuration, rate, availability and connection method before the rental starts.
More guides
Prepare for GPU rental and your first run