Inference Engineer

Company hidden

San Francisco Remote Full-time Posted 11d ago

THE ROLE You'll own the cost and performance of our inference stack. Your work will determine how efficiently we serve models as workloads, traffic, and hardware change. You'll work closely with the engineers operating the serving fleet while owning the core performance levers: caching, batching, quantization, decoding, and kernel-level optimization. Success means improving throughput and latency without compromising reliability or model quality. RESPONSIBILITIES * Improve throughput, cost, and tail latency through KV-cache management, continuous batching, speculative decoding, and…

See the company and apply

Unlock to see who’s hiring, read the full description and apply on the company site.

Checking access…

See how well you match this job

Upload your resume and we’ll score your fit for this role and 6 similar roles — then tailor your CV to it with AI. Free, no credit card.

Check your match

Similar jobs

Explore related jobs

Want more roles like this? Browse fresh jobs or tailor your resume with AI.