NiNI Labs · Remote (worldwide) · Full-time

Inference Systems Engineer

Build the runtime that schedules, serves and scales agent workloads for NiNI Cloud, measured against live traffic from the gateway.

Apply for this role Opens an email to careers@nicrron.ai

About the role

NiNI Labs is the applied research lab in the family. Its first service, NiNI Cloud, is serverless inference for AI agents: a pipeline is imported as a graph of model and non-model stages, and the platform schedules, scales and traces it.

You will own pieces of that runtime: placement of requests across heterogeneous accelerators, scale-to-zero and resume without losing state, KV and prompt caches, and the tracing that attributes cost to every stage. The proving ground is real: the Nicrron gateway and the Ninoo agents generate the workloads, so ideas are judged on tail latency and dollars per useful token.

What you’ll do

01Design and ship the scheduler and serving path for agent pipelines on NiNI Cloud.
02Make stages stateful and cheap: durable state across long runs, resume after scale-to-zero, cache reuse across turns.
03Instrument everything: per-stage traces, cost attribution and the dashboards the lab and customers read.
04Run experiments against gateway traffic and write up what held and what did not.
05Turn findings into changes in the Nicrron gateway's routing and caching.

What we look for

Strong systems background: Rust, Go or C++ in production, and comfortable reading CUDA or kernel-level code.
Experience serving models or running inference infrastructure: vLLM, TensorRT-LLM, SGLang or your own.
You measure before you optimize and can explain a p99 regression from a trace.
Comfortable owning a component from design through on-call.

Nice to have

Published work or open-source contributions in serving, scheduling or distributed systems.
Experience with accelerator fabrics, RDMA or topology-aware placement.
Familiarity with agent frameworks and what makes their traffic bursty and branching.

Apply

Send a short note and links.

The button opens an email to careers@nicrron.ai with the subject line filled in. Tell us why this role and point us at one thing you have shipped. We reply to every application within a week.

Apply for this role

Other open roles