Software Engineer at Fireworks AI
May 2025 - Current | Redwood City, CA
Joining Fireworks AI as a Software Engineer, working on cutting-edge AI infrastructure and model serving technologies.
Key Responsibilities:
- Part of Forbes AI 50 startup, working closely with pytorch team who are powering the AI revolution. [Video]
- Automatically provision the latest GPUs across 10+ clouds and 15+ regions for high availability, consistent performance, and seamless scaling—so you can focus on building.
- Building and optimizing high-performance inference infrastructure for large language models (akin to vLLM and SGLang)
- Developing scalable solutions for model deployment and serving at scale with capacity planning and scaling strategies, cost analysis.
- Working with state-of-the-art AI/ML frameworks and training/inference pipelines & dev envs setup best practices
- Contributing to the platform that powers AI applications for thousands of developers.
Technology Stack:
AI/ML: PyTorch, In house Inference engine powered by FireOptimzer
Infrastructure: Kubernetes, Docker, GPU orchestration, Lots of terraform, Helmfiles, charts, gRPC, REST, Protobufs
Languages: Python, Go, C++, Bit of Mojo (tinkering for fun)
Cloud: Multi-cloud deployment, scheduling strategies and dynamic traffic scaling , monitoring, observability [otel, prometheus, grafana, GCP dashboards-logs-metrics-alerts, custom slack bot alerts, etc] (resource sharing for 150K+ AI developer Platform and dedicated enterprise,on-demand & servless customers)
About Fireworks AI:
Fireworks AI is revolutionizing how developers build and deploy generative AI applications. The platform provides the fastest and most reliable API for running and fine-tuning open-source models, making advanced AI capabilities accessible to developers worldwide.
More details on the grind soon...
Back to Timeline