AWS EC2¶

Renting GPU capacity from your own AWS account.

Your hardware, your IAM, AWS datacenters, SOC 2. No third-party acknowledgement and no per-job peer-host opt-in.

Quotas come first¶

GPU quotas default to zero on every AWS account. Until an increase is granted, no GPU instance can launch at all.

On-demand and spot quotas are independent. Request increases before first use; a first-ever GPU request can take days and may be partially granted.

Spot is often not cheaper¶

Observed GPU spot discounts run 3–40%, not the advertised "up to 90%". Current-gen parts (L4, L40S, Blackwell) barely discount because demand is saturated. Compare before enabling.

Account setup¶

1. A dedicated account, or at least a dedicated quota¶

Orphan reconciliation terminates instances tagged krakenhashes:managed that the database does not recognise. The tag is applied by KrakenHashes and enforced by IAM, so the blast radius is bounded — but the GPU quota is shared with everything else in the account, so a dedicated one makes the quota a real ceiling rather than a shared allowance.

2. Networking¶

The worker needs outbound connectivity only — it dials your VPN and the provider control plane. Nothing connects to it.

A NAT gateway costs more per hour than some of the smaller GPU instances, so a public subnet is usually the cheaper choice for ephemeral workers.

3. The provisioner IAM principal¶

Create a dedicated IAM user (or role) — the reference deployment uses a user named krakenhashes-provisioner — and attach the policy below. Generate an access key for it and paste the key id and secret into the provider form.

IAM policy¶

This is the policy the reference deployment actually runs, published as-is. It is what drove the paid end-to-end launch and the zone picker. Replace <ACCOUNT_ID> and <REGION>.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "RunInstancesOnResourcesItDoesNotCreate",
      "Effect": "Allow",
      "Action": "ec2:RunInstances",
      "Resource": [
        "arn:aws:ec2:<REGION>::image/*",
        "arn:aws:ec2:<REGION>:<ACCOUNT_ID>:subnet/*",
        "arn:aws:ec2:<REGION>:<ACCOUNT_ID>:security-group/*"
      ]
    },
    {
      "Sid": "RunInstancesOnResourcesItCreates",
      "Effect": "Allow",
      "Action": "ec2:RunInstances",
      "Resource": [
        "arn:aws:ec2:<REGION>:<ACCOUNT_ID>:instance/*",
        "arn:aws:ec2:<REGION>:<ACCOUNT_ID>:volume/*",
        "arn:aws:ec2:<REGION>:<ACCOUNT_ID>:network-interface/*",
        "arn:aws:ec2:<REGION>:<ACCOUNT_ID>:spot-instances-request/*"
      ],
      "Condition": {
        "StringEquals": {
          "aws:RequestTag/krakenhashes:managed": "true"
        }
      }
    },
    {
      "Sid": "TagOnlyAtCreation",
      "Effect": "Allow",
      "Action": "ec2:CreateTags",
      "Resource": [
        "arn:aws:ec2:<REGION>:<ACCOUNT_ID>:instance/*",
        "arn:aws:ec2:<REGION>:<ACCOUNT_ID>:volume/*",
        "arn:aws:ec2:<REGION>:<ACCOUNT_ID>:network-interface/*",
        "arn:aws:ec2:<REGION>:<ACCOUNT_ID>:spot-instances-request/*"
      ],
      "Condition": {
        "StringEquals": {
          "ec2:CreateAction": "RunInstances"
        }
      }
    },
    {
      "Sid": "TerminateOnlyWhatItCreated",
      "Effect": "Allow",
      "Action": "ec2:TerminateInstances",
      "Resource": "arn:aws:ec2:<REGION>:<ACCOUNT_ID>:instance/*",
      "Condition": {
        "StringEquals": {
          "aws:ResourceTag/krakenhashes:managed": "true"
        }
      }
    },
    {
      "Sid": "DescribeIsAllOrNothing",
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeInstances",
        "ec2:DescribeInstanceTypes",
        "ec2:DescribeInstanceTypeOfferings",
        "ec2:DescribeImages",
        "ec2:DescribeSubnets",
        "ec2:DescribeSecurityGroups",
        "ec2:DescribeVpcs",
        "ec2:DescribeAvailabilityZones",
        "ec2:DescribeSpotPriceHistory",
        "ec2:DescribeSpotInstanceRequests",
        "ec2:DescribeTags",
        "ec2:DescribeVolumes",
        "ec2:GetConsoleOutput",
        "ec2:GetSpotPlacementScores"
      ],
      "Resource": "*"
    },
    {
      "Sid": "ResolveTheDeepLearningAMI",
      "Effect": "Allow",
      "Action": [
        "ssm:GetParameter",
        "ssm:GetParameters"
      ],
      "Resource": "arn:aws:ssm:<REGION>::parameter/aws/service/deeplearning/*"
    },
    {
      "Sid": "PreflightQuotaAndIdentity",
      "Effect": "Allow",
      "Action": [
        "servicequotas:GetServiceQuota",
        "servicequotas:GetAWSDefaultServiceQuota",
        "sts:GetCallerIdentity"
      ],
      "Resource": "*"
    }
  ]
}

Why it is shaped that way¶

Each statement exists because the obvious simplification breaks it.

What is required and what merely degrades¶

Not every action is load-bearing. Tightening the policy costs information rather than function in several places, and knowing which is which saves a debugging session.

Credentials¶

Supplied as a JSON document in the credentials field. The admin form builds this for you from two inputs.

{
  "access_key_id": "AKIA...",
  "secret_access_key": "..."
}

session_token is also accepted for STS temporary credentials, but has no field in the admin form — a config using it has to be POSTed to the API directly.

Credentials are AES-256-GCM encrypted at rest and never returned by the API; the config exposes only a has_credentials boolean. Leaving the field blank on an edit keeps what is stored, so changing a rate ceiling cannot silently blank your key.

Settings¶

{
  "region": "us-east-1",
  "security_group_ids": ["sg-..."],
  "ami_ssm_parameter": "/aws/service/deeplearning/ami/x86_64/base-oss-nvidia-driver-gpu-ubuntu-22.04/latest/ami-id",
  "use_spot": false,
  "instance_type_rates": { "g4dn.xlarge": 53, "g6e.xlarge": 187 },
  "instance_type_gpus": {
    "g4dn.xlarge": { "gpu_model": "T4",   "gpu_count": 1, "vram_gb_per_gpu": 16 },
    "g6e.xlarge":  { "gpu_model": "L40S", "gpu_count": 1, "vram_gb_per_gpu": 48 }
  },
  "root_volume_gb": 100,
  "ebs_cents_per_gb_month": 8,
  "zones": [
    { "zone": "us-east-1a", "zone_id": "use1-az1", "subnet_id": "subnet-...", "instance_types": [] },
    { "zone": "us-east-1b", "zone_id": "use1-az2", "subnet_id": "subnet-...", "instance_types": ["g4dn.xlarge"] }
  ]
}

instance_type_rates is in cents per hour and is operator-supplied rather than resolved from the AWS Pricing API: that API needs exact capacitystatus/preInstalledSw/tenancy filters or it silently returns the wrong SKU.

Zones: why one subnet is not enough¶

EC2 spot capacity is a property of the (instance type, availability zone) pair. g4dn.xlarge being exhausted in us-east-2b says nothing about g4dn.xlarge in us-east-2c — they are different physical inventories. So the number of independent chances a launch gets is:

capacity pools = (zones you allow) x (instance types you allow in them)

A config with one subnet_id and one instance type has exactly one pool. When it is empty, provisioning simply stops: the launch retry loop walks its candidate list, but every candidate resolves to the same inventory, so the second refusal was implied by the first. Measured on a real deployment: fifteen consecutive InsufficientInstanceCapacity refusals against one pool, then a success within two scheduler cycles of widening to three instance types across two zones. Nothing about the account, quota or region changed.

Each entry in zones is one placement. subnet_id is the only required field — a launch takes a subnet, not a zone. An empty instance_types means "every type in instance_type_rates", which is what you want by default and what keeps the selection from silently narrowing when you price a new instance type later. Narrow it per zone only when you know a particular card is never obtainable there.

zone_id (use2-az1) is the stable physical identifier; zone (us-east-2a) is an alias AWS shuffles per account, so your us-east-2a and another account's are usually different datacentres. Capacity APIs report the zone ID, which is why both are stored.

When zones is set, the top-level subnet_id is ignored. It remains supported on its own for existing configs and as the fallback when no zones are selected.

Picking zones from the UI¶

Admin → Settings → Cloud Provisioning → Providers → (edit an AWS provider) → Availability zones and instance types.

"Check availability" (GET /api/admin/cloud/providers/{id}/capacity) reads live data and spends nothing — it is read-only EC2 describe calls, with the spot price and placement-score lookups made only when use_spot is set. It renders a zone × instance-type grid where each ticked cell is one capacity pool, with the running pool count underneath.

Read the columns in this order of trustworthiness:

The placement score is AWS's own relative ranking of a pool, and it is worth less than it looks: every us-east-2 zone scored 1/10 on a live account minutes before a g4dn.xlarge launch in one of them succeeded on the first attempt. It is used to decide which pool to try first and for nothing else — it never removes a pool from the candidate list, and gating on it would have refused a launch that worked. If your IAM role lacks ec2:GetSpotPlacementScores, you lose the ordering and nothing else; the screen says so rather than failing.

The spot price is shown next to your configured rate because the two drift and only one is real. Reservations are denominated in your configured figure, so pricing above the live rate merely over-reserves; pricing below it under-reserves against a cap you were told was hard. One deployment had g4dn.xlarge priced at 53c while spot was quoting 27c.

The screen reads the saved provider config, so save before checking: an unsaved form's credentials and region are not visible to the backend.

Verifying¶

Run the pre-flight from the provider page. For AWS it reports, separately:

• whether credentials resolve (sts:GetCallerIdentity)
• the applied or default quota, and current usage
• exactly which IAM permissions are missing, via a real DryRun

Anything inconclusive is treated as failure, not success: "unknown" is not "allowed".

Then rent one cheap instance with a short TTL and confirm teardown. See Verifying before you spend for the $0 rehearsal that should come first.

Related¶

• Cloud GPU Providers — budgets, provisioning rules, trust tiers
• Cloud Agent VPN — the VPN credential every provider needs
• Cloud GPU Provisioning — design rationale, spot request details