GPU Infrastructure Engineer

About PolarGrid

PolarGrid is an early-stage Canadian company that runs managed AI inference. A customer brings a model or a serving image. We run it on dedicated GPUs behind one endpoint, serve each request from the nearest site, and move traffic to the next nearest site when one fails.

Our goal is to make AI inference fast, reliable, and secure for customers running production workloads.

The Role

This role is responsible for scaling PolarGrid's inference network from a handful of sites to a geographically distributed fleet of GPU clusters. The GPU Infrastructure Engineer will own how new sites and nodes come online, from provisioning and burn-in through validation and a staged move into production. They will define when a site is ready for customer traffic, and make sure each new site joins the network without hurting latency or reliability.

A core responsibility is running the global routing layer that sends each request to the best place to serve it. The GPU Infrastructure Engineer will work with live signals such as network latency, capacity and health, and tune routing against real traffic to keep latency low and predictable.

The role also owns reliability across the network. The GPU Infrastructure Engineer will set and track availability targets, build the health checks and automation that detect and route around failing hardware and models, keep recovery fast, and test failure handling regularly.

The GPU Infrastructure Engineer will build the observability, release and network systems needed to run a distributed GPU fleet without operations work growing with every new site. This includes end-to-end tracing and latency alerting, safe staged rollouts, network configuration as code, and security and DDoS protection at the edge.

Requirements

• Experience operating distributed systems across multiple sites or regions with high availability targets
• Strong networking fundamentals: BGP, DNS, TCP/TLS and load balancing
• Hands-on experience running bare-metal GPU fleets: provisioning, burn-in, firmware and node lifecycle
• InfiniBand or RDMA networking experience
• Experience running LLM inference engines such as vLLM, SGLang or TensorRT-LLM, including multi-node serving, cold starts and TTFT
• Deep observability skills focused on p95/p99 latency and gray failures, not just uptime
• Experience designing failover, graceful degradation and automated recovery
• Strong automation and infrastructure-as-code mindset, including CI/CD for infrastructure
• Ability to plan capacity on constrained, uneven hardware supply
• Experience building internal tooling or control planes, ideally in Go or Rust

Nice to Have

- DDoS mitigation experience

- Running DNS infrastructure at scale

• Hardware security: TPM, measured boot or confidential computing
• Real-time streaming inference, including voice and audio
• Operating remote datacenter sites with on-site technicians
• Certificate automation across many domains

- Deterministic simulation testing

Originally posted on Himalayas