AI inference on GPU Kubernetes
Running GPU-backed Kubernetes clusters that serve BHASHINI's speech-to-text, text-to-speech, and translation models as containerised microservices on India's sovereign government cloud.
Currently keeping GPUs busy at Bhashini
Lead DevOps Engineer | AI Infrastructure | AWS Solutions Architect | MBA
Rishabh Gupta
I build the quiet parts behind reliable software — including the GPU platforms that AI models actually run on.
AWS, Kubernetes, GitOps, CI/CD, and observability — currently running GPU-backed clusters that serve speech and translation AI models for a national platform.
- Noida, Uttar Pradesh
- rgrishabh@icloud.com
- +91 70075 54304
0Years shipping
0Users served
0Uptime held
0Faster releases
0MTTR, down from 45
Impact
Production highlights
Measurable results from AI inference platforms, cloud infrastructure, fintech systems, and enterprise engineering tooling.
GitOps release automation
Architected Jenkins and ArgoCD pipelines across 5 microservices, reducing deployment cycle time by approximately 60% and removing 90% of manual release intervention.
Kubernetes platform governance
Managed production clusters at 10M+ user scale, using namespace governance and resource quotas to reduce infrastructure over-provisioning by 30% while sustaining 99.9% uptime.
Observability and incident response
Built Prometheus, Grafana, and ELK observability, cutting detection time by 50% and reducing MTTR from 45 minutes to under 15 minutes.
Enterprise engineering systems
Administered GitHub Enterprise, Jira, and Confluence Data Center for 30,000+ engineers with 99.95% platform availability and zero-downtime upgrade cycles.
AI-assisted operations
Use LLM assistants day to day to scaffold Terraform and Ansible, triage noisy log output, draft RCAs, and turn tribal knowledge into runbooks — every generated change reviewed before it reaches a cluster.
AI platform
Infrastructure for AI workloads
AI models are only useful once something keeps them running. That part is DevOps — GPU capacity, scheduling, rollouts, and the signals that tell you an inference service is degrading before users do.
Current work — Bhashini, Govt. of India
Language AI as a digital public good
BHASHINI is India's national language platform under MeitY: speech-to-text, text-to-speech, and translation across Indian languages, exposed as APIs that other government and public services build on. I run the Kubernetes layer those models are served from, on Yotta's sovereign Government Community Cloud.
The operational problem is not the model — it is GPU capacity that is scarce and expensive, inference pods that behave nothing like stateless web services, and a release process that has to ship model containers without dropping live traffic.
- Kubernetes
- GPU node pools
- ASR / TTS / NMT
- Yotta sovereign cloud
- Docker
- ArgoCD
- Jenkins
- Prometheus
- Grafana
- Models Speech-to-text, text-to-speech, and translation services, packaged as containers
- Serving Kubernetes deployments, rolling updates, readiness gates on model warm-up
- Compute GPU-backed node pools with labels, taints, and per-namespace resource quotas
- Platform Yotta Government Community Cloud — data stays on sovereign infrastructure
- Signals Prometheus and Grafana on request latency, throughput, pod health, and node saturation
01 — Capacity
GPU-aware Kubernetes
Scheduling inference workloads onto GPU nodes: node labelling and taints so only GPU workloads land on GPU hardware, resource requests and quotas per namespace, and placement rules that stop one team's batch job from starving a live service.
02 — Delivery
Release pipelines for model containers
Model images are large, slow to pull, and slow to warm up. Pipelines are built around that: registry hygiene, image promotion between environments, staged rollouts, and health gates that wait for a model to actually be ready rather than for the process to merely start.
03 — Leverage
AI-assisted operations
LLM assistants are part of my daily loop: scaffolding Terraform and Ansible, reading through noisy log dumps during triage, drafting RCAs from incident timelines, and converting fixes into runbooks. Useful for the first draft — never for the last word. Generated changes get reviewed like any other diff.
- IaC scaffolding
- Log triage
- RCA drafting
- Runbook generation
- Script generation
Technical skills
Tools I work with
A production-heavy toolkit across AI workload infrastructure, AWS architecture, delivery automation, containers, infrastructure as code, monitoring, incident management, and Linux operations.
AI infrastructure and GPU ops
- GPU node pools
- GPU scheduling
- Node taints and labels
- Resource quotas
- Model container delivery
- ASR / TTS / NMT serving
- Inference autoscaling
- Yotta Cloud
AI-assisted engineering
- LLM-assisted IaC
- Log triage
- RCA drafting
- Runbook generation
- Prompt-driven scripting
- Claude
- GitHub Copilot
Cloud and infrastructure
- AWS
- EC2
- RDS
- S3
- VPC
- Route 53
- ACM
- CloudFront
- ELB
- IAM
- ECR
- API Gateway
- CloudWatch
- Secrets Manager
- Inspector
- Security Hub
- GuardDuty
DevOps and CI/CD
- Jenkins
- GitHub Actions
- ArgoCD
- JFrog Artifactory
- Git
- GitHub
- Gradle
- Maven
GitOps and platform delivery
- Argo CD
- Kustomize
- Helm
- Jenkins multibranch
- Jenkins Config as Code
- Kaniko
- containerd
- Private registries
- Progressive rollouts
Edge, TLS and DNS
- NGINX reverse proxy
- TLS termination
- Let’s Encrypt / certbot
- Automated renewal
- HSTS
- DNS operations
- WebSocket and gRPC-Web proxying
Business and finance
- MBA — Finance & IT
- Cloud cost optimisation
- Capacity planning
- Vendor evaluation
- Stakeholder communication
Containers and IaC
- Docker
- Kubernetes
- Terraform
- Ansible
- Linux
- Ubuntu
- CentOS
Monitoring and incidents
- Prometheus
- Grafana
- Grafana Incident
- ELK Stack
- New Relic
- AppDynamics
- Zabbix
- PRTG
- AlertSite
- Opsgenie
Professional experience
Recent platform and operations work
-
Tarento Group
|
Client:
Bhashini (Govt. of India)
Senior Software Engineer, DevOps
- Deploying and operating GPU-backed Kubernetes clusters on Yotta Cloud (Government Community Cloud) that serve BHASHINI's speech-to-text, text-to-speech, and translation models as containerised microservices.
- Managing GPU capacity for inference workloads through node labelling and taints, resource requests and limits, and per-namespace quotas, so live inference services are not starved by competing workloads.
- Automating releases for AI model containers with CI/CD pipelines built around large images and slow model warm-up — registry hygiene, staged rollouts, and readiness gates that wait for a model to be genuinely serving before shifting traffic.
- Running platform observability across inference latency, throughput, pod health, and node saturation with Prometheus and Grafana, so degradation is caught before it reaches API consumers.
- Supporting cloud infrastructure for Bhashini, a digital public good under the Ministry of Electronics and IT (MeitY), Government of India, where language services are consumed by other public-sector platforms.
-
Persistent Systems
|
Client:
Intuit
Lead Software Engineer (DevOps)
- Architected Jenkins and ArgoCD pipelines across 5 microservices, cutting deployment cycle time by approximately 60% and eliminating 90% of manual release interventions.
- Managed Kubernetes clusters serving 10M+ users, applying namespace governance and resource quotas that reduced over-provisioning by 30% while sustaining 99.9% uptime.
- Built Prometheus, Grafana, and ELK observability to cut incident detection time by 50% and reduce MTTR from 45 minutes to under 15 minutes.
- Led 24/7 on-call response, kept average P1 response under 10 minutes, and authored RCAs that reduced repeat incidents by 40% quarter over quarter.
- Administered GitHub Enterprise, Jira, and Confluence Data Center for 30,000+ engineers with 99.95% availability and zero-downtime upgrades across 3 major version cycles.
-
System Support Engineer
- Hardened AWS security for a PCI-DSS-sensitive payment platform with IAM RBAC, least-privilege policies, Secrets Manager rotation, AWS Inspector, Security Hub, and GuardDuty.
- Reduced critical security findings by 65% within the first quarter while partnering with engineering and SRE teams to resolve production incidents and maintain SLA adherence.
- Designed an Active-Passive RDS setup with synchronized standby for disaster recovery, zero data loss failover, and clean dev/prod separation.
-
Freelance DevOps Engineer
- Hosted a production frontend on S3 and CloudFront, improving global page load times by 40% through edge caching and custom cache-control policies.
- Provisioned private-subnet RDS MySQL and PostgreSQL with daily snapshots, parameter tuning, and security group hardening for a recoverable production database layer.
- Authored 5+ SOPs for deployment, environment setup, and rollback workflows so the client team could execute releases independently.
-
Codewalla Software
Development
|
Client:
1Huddle
Junior DevOps Engineer
- Built and maintained Jenkins pipelines for 8+ application deployments, reducing manual deployment effort by 70% and cutting average release time from 3 hours to under 45 minutes.
- Containerized legacy applications and authored Terraform and Ansible modules for AWS provisioning, reducing environment setup from 2 days to under 2 hours.
- Implemented Prometheus and Grafana monitoring for 3 production services, establishing SLI baselines that detected 90% of performance anomalies before user impact.
-
Site Operations Engineer - Intern
- Triaged 20+ daily alerts via Zabbix and AWS CloudWatch on a 24/7 rotation, escalating P1/P2 incidents within a sub-30-minute acknowledgement SLA.
- Developed shell scripts for routine health checks across 15+ servers, reducing manual monitoring effort by 40%.
- Resolved 15+ operational tickets per week across scheduled job failures, access issues, and service disruptions.
Built and running
Things I ship outside work hours
A self-hosted platform I designed, built, and operate end to end — the same discipline as day job work, with nobody else to hand the pager to.
Product · self-hosted
InfraSight
Endpoint and SSL certificate monitoring: a FastAPI service and an independent check worker sharing one image, Postgres for history, Redis for rate limiting, and a React dashboard. Incidents only open after a failure is confirmed from a second network vantage point, so a bad path from one host does not page anyone at 3am.
- FastAPI
- PostgreSQL
- Redis
- React
- Alembic
- Playwright
Platform · GitOps
Single-node Kubernetes platform
Jenkins, Argo CD, the Kubernetes Dashboard and InfraSight running on one small cloud box behind NGINX, each on its own hostname with automated TLS. Manifests live in a Git repo as a Kustomize base plus per-environment overlays; Argo CD reconciles, so a rollback is a revert rather than an archaeology exercise.
- Argo CD
- Kustomize
- Helm
- NGINX
- Let’s Encrypt
- containerd
Delivery · CI/CD
Multibranch pipeline with a GitOps handoff
A Jenkins multibranch pipeline builds container images and deploys by committing the new tag to the manifest repo — CI never touches the cluster. Main goes to production; every other branch gets its own namespace and overlay, generated on first build.
- Jenkins
- JCasC
- Multibranch
- Kaniko
- Private registry
Background
Certification and education
Certification
AWS Solutions Architect – Associate
SAA-C03
VPC design, IAM, RDS, disaster recovery, and production hardening — the same areas I work in daily.
View on CredlyEducation
-
MBA, Finance and Information Technology2026
-
B.Tech, Computer Science and Engineering2022
-
Intermediate (12th)2018
Contact
Open to DevOps, platform, and AI infrastructure roles.
Best fit: teams that need hands-on ownership across GPU and AI workload infrastructure, AWS, Kubernetes operations, CI/CD automation, observability, and incident response.