Principal Cloud Platform Engineer

About this role

SambaNova is a leader in next-generation AI infrastructure, delivering a full-stack inference platform for customers worldwide. At the core of SambaNova's technology is the RDU (Reconfigurable Dataflow Unit) - a chip built on a dataflow architecture rather than the traditional GPU model. Its decode performance is especially strong for agentic workloads like multi-turn agents, code generation, and long-running applications. RDUs are packaged into SambaRack, rack-scale hardware that lets customers deploy state-of-the-art models with better performance, greater energy efficiency, and faster time to value. About the Team The Cloud Platform team runs SambaNova's production inference service. We handle capacity planning, deployment, monitoring, and incident response across regions in the United States, Asia, Europe, and Latin America. About the Role As a Principal Cloud Platform Engineer, you will design and build the software that deploys, schedules, and scales model workloads across our shared accelerator hardware, staying hands-on in the code while leading the technical design. You will set uptime and latency targets for our inference endpoints and share in the on-call rotation that keeps them running. Responsibilities Shared ownership of the production inference service across regions, covering availability, latency, performance, change management, and capacity planning Designing and building platform services, Kubernetes operators, and controllers that deploy, schedule, and scale model workloads Designing scheduling for shared accelerator capacity, including quotas, priority, and fairness between customers Rolling out upgrades without disrupting running workloads, and defining how the service recovers when a node or worker fails Designing platform security, including tenant isolation, workload identity, and secrets management Standing up new regions with infrastructure as code in Terraform and Ansible, and building CI/CD pipelines for model and service deployments Building monitoring and alerting in Prometheus, Grafana, and Datadog, and defining SLOs and SLIs that drive reliability work Finding and eliminating performance bottlenecks, and designing auto-scaling for variable inference loads Leading incident response and postmortems as part of a shared primary/secondary on-call rotation Forecasting infrastructure needs and working with finance to manage cloud spend Required Qualifications B.S. in Computer Science, Computer Engineering, or related field 8+ years of experience in Site Reliability Engineering, platform engineering, or distributed systems roles Experience supporting a large-scale, customer-facing service in a public cloud environment (AWS, GCP, Azure) Strong programming skills in Python, Go, Rust, or Java, with recent hands-on coding Experience designing and building a platform service, control plane, or scheduler used by multiple teams or customers Kubernetes experience such as writing operators or controllers, running cluster upgrades, or configuring the scheduler Experience with multi-tenant systems, including resource quotas, isolation, and fair sharing of capacity Experience designing for failure, including retries, idempotency, and safe rollouts Deep understanding of monitoring and observability principles and tools (e.g., Prometheus, Grafana, ELK Stack, Datadog) Experience with Infrastructure as Code (e.g., Terraform, CloudFormation) Experience with CI/CD principles and tools (e.g., Jenkins, GitHub Actions, ArgoCD) Strong Linux/Unix system administration fundamentals Preferred Qualifications Experience in a hybrid environment bridging cloud and on-premise/data center infrastructure Direct experience supporting ML/AI inference services in production Knowledge of model serving frameworks like vLLM, SGLang, or Ray, including batching and KV cache management Familiarity with GPU-accelerated computing and optimizing workloads for NVIDIA GPUs for purposes of mapping to RDUs Experience with batch or job scheduling systems such as Slurm, Volcano, or Kueue Experience with Kubernetes internals such as etcd, control plane high availability, or the scheduler Understanding of MLOps principles and practices Experience managing and tuning databases (SQL or NoSQL) and caching systems (Redis, Memcached) Base Salary Range: Base Pay Range $210,000 - $280,000 USD Submission Guidelines Please note that in order to be considered an applicant for any position at SambaNova Systems, you must submit an application form for each position for which you believe you are qualified. EEO Policy SambaNova Systems is an Equal Opportunity/Affirmative Action Employer. All qualified applicants will receive consideration for employment without regard basis of age (40 and over), color, disability, gender identity, genetic information, marital status, military or vet

The employer's own description, as posted on their site. Apply on SambaNova's site ↗