Senior AI Research Engineer - GPU Systems & Kernels

Datadog, Inc.
  • Full-time
  • New York, United States
Development Research

As a Research Engineer on Datadog AI Research’s Frontier Foundations team, you will help make large-scale model training and inference as fast and efficient as possible on available hardware. You will build and optimize GPU kernels and address memory and communication bottlenecks, including those affecting large Mixture-of-Experts models. You will work with Research Scientists and contribute to infrastructure supporting Datadog’s AI research in cloud observability and security.

Requirements

  • Deep expertise in at least one area: GPU kernel development and optimization, low-latency communication and collective operations, or large-scale training infrastructure.
  • Ability to reason about hardware constraints such as memory, bandwidth, and interconnect topology, identify bottlenecks, and explain how to verify them.
  • Proficiency in a systems language such as C++, Rust, or CUDA C, and in Python.
  • Strong debugging and profiling skills, including experience with tools such as Nsight.
  • Ability to communicate design and performance trade-offs clearly to technical and non-technical audiences.
  • Write, tune, and debug GPU kernels using CUDA, Triton, or similar technologies for major LLM training and inference operations.
  • Optimize for recent NVIDIA GPUs, including memory hierarchy, tensor cores, and low-precision formats such as FP8 and FP4.
  • Reduce distributed communication costs, including all-to-all and all-reduce, and overlap communication with computation across NVLink and nodes.
  • Profile end-to-end training and rollout workloads, determine whether they are compute-, memory-, or communication-bound, and quantify optimization headroom.
  • Build benchmarks, profiling tools, and regression tests; collaborate with Research Scientists to implement new model architectures efficiently.
  • Explore techniques such as low-precision training and serving, speculative decoding, and fused or persistent kernels.
  • Experience with Mixture-of-Experts models, expert parallelism, sparse inference, NCCL or NVSHMEM, communication/computation overlap, Triton or CUTLASS, distributed training frameworks, GPU-intensive systems, or open-source ML and systems projects is an advantage.

Additional information

Datadog is an observability and security platform bringing visibility across applications, infrastructure, data, models, and security. The role is marked hybrid. Datadog welcomes applicants from all backgrounds and encourages people to apply even if they do not meet every qualification yet. The company is an equal opportunity employer; accessibility assistance is available through its accommodation request form. Application information is handled under Datadog’s Applicant and Candidate Privacy Notice, and its AI guidelines are available for candidates to review.

More jobs within the Development and Research categories

APG Asset Management

  • Full-time
  • Heerlen, Netherlands

Salary: 5533.19 - 7143.86 EUR P1M

Development Research

Companies hiring AI talent

From research labs to fast-growing startups, these employers are building with Artificial Intelligence and recruiting on OnlyAIJobs.