Senior AI Compute Infrastructure Engineer

About the role

Kraken is creating a dedicated AI Compute and Infrastructure team to power the next generation of model training, inference, evaluation, and experimentation across its exchange. The team owns the GPU and accelerator infrastructure layer, ensuring fast, reliable, and cost‑effective AI workloads. You will work closely with AI/ML researchers, platform engineers, security, and product teams to make Kraken’s AI ambitions a reality.

Key responsibilities

• Own and operate GPU and accelerator clusters for training, inference, evaluation, and experimentation, including drivers, runtimes, kernels, device plugins, and workload isolation.
• Design infrastructure that enables teams to run models locally on GPUs, reducing reliance on external providers and controlling compute costs.
• Build and improve scheduling, orchestration, placement, quota management, and utilization systems across heterogeneous accelerator environments.
• Optimize inference pipelines for latency, throughput, reliability, and memory efficiency using frameworks such as vLLM, Triton Inference Server, TensorRT, or equivalent.
• Partner with ML engineers and researchers to remove bottlenecks in training, batch inference, online inference, deployment, and production debugging.
• Develop observability for GPU utilization, memory pressure, queue depth, token throughput, request latency, failed workloads, capacity pressure, and spend.
• Drive reliability, incident response, alerting, runbooks, and post‑incident improvements for always‑on AI compute infrastructure.

Required profile

• 5+ years of infrastructure engineering experience with significant focus on GPU compute, ML infrastructure, distributed systems, or large‑scale production platforms.
• Hands‑on experience operating GPU or accelerator‑backed clusters in production, including scheduling, orchestration, utilization monitoring, and cost optimization.
• Strong systems engineering fundamentals across Linux, networking, storage, containers, Kubernetes, distributed runtimes, and production debugging.
• Proficiency in Python for automation, tooling, debugging, and operational workflows.
• Track record of optimizing compute costs while maintaining performance, reliability, and availability expectations.

Required skills

• GPU/accelerator cluster management
• Linux system administration
• Kubernetes orchestration
• Python scripting and automation
• ML serving frameworks (vLLM, Triton Inference Server, TensorRT, TorchServe, KServe, Ray Serve)
• Distributed systems and scheduling
• Observability tools (metrics, logs, traces, dashboards)
• Cost‑optimization and capacity planning