instances

An instance is a containerised GPU sandbox running on a DGX Spark host. Each instance has a name, a single attached SSH key, and a chosen GPU profile. Click new instance in the dashboard header to launch one.

gpu types

• NVIDIA DGX Spark GB10 — single node, 128 GB unified memory. $0.65 / hour
• 2× NVIDIA DGX Spark GB10 — dual node interconnect, 256 GB unified memory. $1.50 / hour capacity not online yet — selecting this gpu enters the queue automatically. See 2× dgx spark for how the cluster works once you are in.

lifecycle

Once an instance is launched it stays around until you delete it. The card shows the live container status (running, exited, unknown) and exposes two actions:

• restart — cycles the container without touching the billing session. Useful when something inside hangs. Takes 10–15 seconds; the badge flips to restarting and the SSH block hides until the new sshd is ready.
• delete — removes the container and ends the billing session. The name becomes reusable after delete. Stopping is not a thing in the dashboard — only restart and delete are exposed, because stopping a container without ending the session would leave it billing for nothing. See billing for details.

ssh access

When an instance is running, the card shows a ssh user@<hostname>.ssh.enverge.dev snippet — click to copy. SSH terminates at a relay that forwards onto the actual container; the first connection after create or restart can take a minute while the tunnel settles.

If the connection is refused right after a restart, wait a moment and retry. If you see host key verification failed, run ssh-keygen -R <hostname> on your machine and try again.

docker inside your instance

Docker is pre-installed and already running — run your own containers inside the instance with no setup. Plain builds and runs work normally.

To give a container the GPU, use the nvidia runtime plus the NVIDIA_VISIBLE_DEVICES environment variable:

docker run --rm --runtime nvidia -e NVIDIA_VISIBLE_DEVICES=all \
  nvcr.io/nvidia/cuda:13.0.0-base-ubuntu24.04 nvidia-smi

2× dgx spark cluster

A 2× DGX Spark instance is a two-node GPU cluster behind a single SSH endpoint. You land on node0 (the head). A second Spark (node1, the worker) is attached over a ~200 Gb/s RoCE fabric. You do not SSH to the worker directly — you launch a distributed job from node0 and it uses both GPUs.

memory model

The tier advertises 256 GB unified memory — that is aggregate across two 128 GB pools joined by the fabric, not one flat address space. No single process or tensor can exceed ~128 GB. Use model parallelism or sharding to spread work across both nodes.

pre-configured on your instance

• NCCL over RoCE — both ConnectX-7 rails are wired for RDMA (/etc/nccl.conf). NCCL auto-discovers the fabric; no manual NCCL_IB_* exports needed.
• VLLM_HOST_IP — preset to this node’s rail IP (from /etc/profile.d/enverge-cluster.sh) so vLLM’s multi-node control plane advertises the correct address instead of the default-route interface.
• worker SSH — passwordless SSH from this container to the worker’s container over the rail is already set up for launchers that spawn remote ranks (check /etc/enverge/cluster.md on the instance for this VM’s rail IPs and worker address).

running a multi-node job

Launch rank 0 here and rank 1 on the worker over the rail. Any NCCL-based stack works once the workload is sharded. The shape is:

torchrun --nnodes=2 --nproc_per_node=1 --node_rank=0 \
         --rdzv_endpoint=<node0-rail-ip>:29400 your_job.py

your_job.py is your distributed script — nothing is pre-installed to run for you. Set --node_rank=1 on the worker, or let Ray / mpirun spawn the remote rank via the pre-configured rail SSH. Replace <node0-rail-ip> with this instance’s rail-2 address (see /etc/enverge/cluster.md for the filled-in template, or ip -4 addr show enP2p1s0f0np0).

unified memory on gb10

shared filesystem

There is no cluster-wide shared filesystem yet. Model weights, datasets, and code must be present on each node (or downloaded separately). Plan for N× download size when pulling large checkpoints.

the queue

We have limited Spark capacity. When you click launch instance and the GPU type you picked has no free capacity (or, for the 2× Spark, isn’t online yet), you’ll see the enter queue offer instead of an error.

how it works

• Queue order is first-in, first-out per GPU type. Each GPU type (1× Spark, 2× Spark) has its own independent queue.
• You may have one open entry per GPU type. You can be queued for the 1× Spark and the 2× Spark at the same time, but not twice for the same type.
• When capacity opens (someone else deletes their instance), the head of that GPU’s queue is launched automatically with the name and SSH key you picked when entering.
• You’ll receive an email when your instance is ready. You can also see the new instance immediately in the dashboard on next refresh.
• To leave a queue, click exit queue on the queued card. No charge.

where it shows up

Queued entries render as dashed cards in the same grid as live instances, with a queued badge. Past queue activity (including the wait duration and whether you got an instance or cancelled) lives in usage, under queue history.

ssh keys

You add SSH keys directly inside the create panel — there’s no separate keys page. The picker is a dropdown of your named keys with two links: + add new key to reveal an inline name + public-key form, and delete selected to remove the currently-chosen key.

• Names are unique per account. Pick something memorable (macbook, work-laptop) so you can tell them apart on the dropdown.
• Paste the public key only — never the private one. The hints under the textarea show the right command for macOS, Windows, and Ubuntu.
• A key cannot be deleted while it’s linked to a live instance. Delete the instance first.
• Keys are written to the instance’s ~/.ssh/authorized_keys at create time. Removing the key from your account does not retroactively revoke access on already-running instances.

billing

Billing is per-minute, metered, and settled via Stripe. You add a card before launching your first instance and we charge based on actual session minutes.

payment method

Until a card is on file, launch instance and restart are gated and a red banner appears at the top of the dashboard. Set the card up from the banner, or from the billing section in your user menu.

On a team, only the owner adds or manages the card. Members see read-only “billing managed by {team}” copy instead.

queue and billing

Entering the queue does not require a card and does not incur charges. We only ask for a card once an instance is actually launching on your behalf — either because you clicked launch instance and capacity was available, or because the queue drained in your favour.

teams

A team lets several people share one payment method. Everyone launches their own instances; usage is tracked per person and billed to the team. Open team from the user menu.

roles

• Owner — creates the team, manages billing, sends invites, sees each member’s usage, and can remove members or revoke pending invites.
• Member — launches instances billed to the team. Cannot add a personal payment method. Sees their own usage in view usage; the owner sees everyone’s totals on the team page.

create a team

Any signed-in user without an active team can create one from team in the user menu. You become the owner. If you already had solo billing, it moves to the team — you are not billed twice.

invite a member

• Owner enters an email on the team page and clicks send invite.
• Invitee receives an email with an accept link.
• New users sign up first (the invite authorizes that email), then accept. Existing users sign in, then accept.
• Once active, the member can launch instances as soon as the team has a payment method on file.

limits

• One active team per user.
• One payment method per team.
• Removing a member (or revoking a pending invite) takes effect immediately. Removed members fall back to solo billing if they set up their own card.

usage history

Open view usage from the user menu to see what you’ve run and queued. Team members see only their own sessions and queue history; the team owner sees per-member totals on the team page.

• Sessions table — one row per create→delete cycle. Columns: name, instance id, started, ended, duration. Live sessions show ● active in the ended column.
• Queue history — one row per past queue entry, with how long you waited and the outcome (launched if an instance was created for you, cancelled if you exited).
• Both tables show 15 rows per page with prev/next paging. Refresh in the page header pulls the latest from the database.

account

• Sign in with email + password, or use the send magic link button to skip the password and authenticate through your inbox.
• Forgot or never set a password? Click the small link at the bottom of the sign-in card — you’ll get an email to set a new password.
• Sign out is in the user menu under your email at the top right.