Engineering Manager

About the Role

We are looking for an experienced and technically strong Engineering Manager - AI Platform to lead the engineering team responsible for building scalable AI/ML platforms and infrastructure.

In this role, you will be responsible for the technical direction, architecture, delivery, reliability, and continuous improvement of AI platform capabilities that enable engineering, data science, and product teams to build and deploy AI-powered solutions at scale.

You will work closely with Product, Data Science, Data Engineering, DevOps, Security, Infrastructure, and Business teams to build reliable, secure, scalable, and cost-efficient AI/ML platforms.

The ideal candidate has strong experience in software engineering and engineering management, along with hands-on experience in AI/ML platforms, MLOps, cloud infrastructure, distributed systems, and Generative AI technologies. The candidate should be comfortable making architectural decisions, mentoring engineers, managing delivery, and working hands-on when required.

What You'll Do

Engineering Leadership & People Management

Lead, mentor, and manage a team of software engineers working on AI/ML platform capabilities.

Define engineering goals, technical direction, and execution plans for the team.

Drive hiring, onboarding, performance management, career development, and succession planning.

Establish a culture of engineering excellence, ownership, collaboration, innovation, and continuous learning.

Conduct regular one-on-ones, provide technical and career guidance, and support engineers in their professional development.

Build a high-performing engineering team capable of delivering reliable and scalable AI platform solutions.

Identify skill gaps and create development plans to strengthen technical capabilities within the team.

AI Platform Engineering

Define and build scalable platforms supporting the complete AI/ML lifecycle.

Design infrastructure for model development, experimentation, training, evaluation, deployment, monitoring, and lifecycle management.

Build reusable APIs, SDKs, services, and platform components that enable product and data science teams to consume AI capabilities.

Develop and maintain scalable model serving and inference infrastructure.

Support machine learning workloads across batch and real-time inference environments.

Build systems for model versioning, experimentation, deployment, rollback, and monitoring.

Improve the reliability, scalability, performance, and cost efficiency of AI workloads.

MLOps & LLMOps

Establish and improve MLOps capabilities for model development and production deployment.

Build CI/CD pipelines for machine learning models and AI applications.

Implement automated model testing, validation, deployment, monitoring, and rollback mechanisms.

Develop LLMOps capabilities supporting Generative AI applications.

Build infrastructure for LLM application development, evaluation, deployment, and monitoring.

Support prompt management, model evaluation, model versioning, and inference monitoring.

Establish processes for tracking model performance, latency, availability, usage, and cost.

Generative AI & LLM Platform

Build scalable infrastructure supporting LLMs, Generative AI, RAG, vector search, embeddings, and AI agents.

Design and develop reusable platform components for GenAI applications.

Integrate and manage multiple foundation models and model providers.

Build frameworks for prompt management, experimentation, evaluation, and observability.

Implement mechanisms to monitor token usage, inference latency, model quality, and infrastructure costs.

Establish appropriate security controls, guardrails, and governance mechanisms for GenAI applications.

Work with Product and Data Science teams to accelerate the development and deployment of AI-powered products.

Architecture & System Design

Define technical architecture and design scalable distributed systems for AI/ML workloads.

Lead architecture reviews and make technology and design decisions.

Design highly available, fault-tolerant, secure, and observable systems.

Identify architectural bottlenecks and drive modernization initiatives.

Ensure platform components follow engineering standards for scalability, maintainability, security, and performance.

Evaluate new technologies, frameworks, and infrastructure solutions relevant to AI/ML engineering.

Balance short-term delivery requirements with long-term platform scalability and maintainability.

Cloud & Infrastructure

Design and operate AI/ML infrastructure across AWS, GCP, Azure, or equivalent cloud environments.

Work closely with DevOps and Infrastructure teams on compute, storage, networking, security, and deployment architecture.

Optimize cloud infrastructure and AI workloads for performance and cost.

Support GPU-based workloads and scalable model inference infrastructure.

Implement infrastructure-as-code and automated deployment practices.

Ensure appropriate monitoring, logging, alerting, and observability across AI platform components.

Platform Reliability & Observability

Define and maintain reliability, availability, scalability, and performance standards for the AI platform.

Establish monitoring and observability for models, services, infrastructure, and AI applications.

Define and track SLIs, SLOs, SLAs, and operational metrics.

Lead incident management and root-cause analysis for platform issues.

Implement proactive measures to reduce system failures and improve platform resilience.

Drive capacity planning and performance optimization.

Security, Governance & Responsible AI

Work closely with Security and Compliance teams to implement appropriate security controls for AI platforms.

Ensure secure handling of sensitive and regulated data used by AI/ML systems.

Implement access controls, authentication, authorization, secrets management, and data protection mechanisms.

Support model governance, auditability, lineage, and explainability requirements.

Establish appropriate controls around LLM usage, data exposure, prompt injection, model misuse, and sensitive information leakage.

Ensure AI platform architecture aligns with organizational security and compliance requirements.

Cross-Functional Collaboration

Partner with Product, Data Science, Data Engineering, Security, DevOps, Infrastructure, and Business teams.

Translate business requirements into scalable technical solutions.

Work with stakeholders to define priorities, roadmap, milestones, and delivery plans.

Communicate technical architecture, risks, dependencies, and delivery status to senior leadership.

Identify opportunities where AI/ML can improve customer experience, operational efficiency, risk management, and business outcomes.

Engineering Excellence

Establish best practices around coding standards, testing, code reviews, CI/CD, observability, and documentation.

Drive automation across development, testing, deployment, and operational processes.

Conduct architecture and code reviews to ensure engineering quality.

Improve development velocity and engineering productivity through reusable platform components and tooling.

Promote a culture of continuous improvement and technical innovation.

What We're Looking For

Educational Background

Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, Artificial Intelligence, Machine Learning, or a related technical field.

Relevant certifications in cloud, AI/ML, or software architecture are an added advantage.

Professional Experience

10+ years of experience in software engineering, platform engineering, or related technical roles.

3+ years of experience managing engineering teams or leading large-scale technical initiatives.

Strong experience building and operating distributed systems and cloud-native platforms.

Hands-on experience with AI/ML platforms, MLOps, ML infrastructure, or AI engineering.

Experience managing and mentoring software engineering teams.

Experience delivering large-scale, production-grade technology platforms.

Strong experience working with Product, Data Science, DevOps, Security, and Infrastructure teams.

Technical Expertise

Strong programming experience in Python, Java, Go, Scala, or equivalent languages.

Strong understanding of software architecture, distributed systems, microservices, APIs, and event-driven architectures.

Experience with cloud platforms such as AWS, GCP, or Azure.

Strong knowledge of Docker, Kubernetes, CI/CD, Infrastructure as Code, and observability.

Experience with ML frameworks such as PyTorch, TensorFlow, or equivalent.

Experience with ML lifecycle and MLOps platforms such as MLflow, Kubeflow, Airflow, SageMaker, Vertex AI, or equivalent.

Experience designing and operating scalable model serving and inference systems.

Strong understanding of databases, data pipelines, distributed computing, and cloud infrastructure.

Experience with LLMs, Generative AI, RAG, embeddings, vector databases, prompt engineering, or AI agents.

Strong understanding of system design, scalability, reliability, security, and performance optimization.

Nice to Have

AI/ML & Generative AI

• Experience building an AI/ML platform from the ground up.

• Experience with LLMOps and production-grade Generative AI systems.

• Experience with RAG architectures, vector databases, embeddings, and semantic search.

• Experience with AI agents and agent orchestration frameworks.

• Experience with model evaluation and AI quality measurement frameworks.

• Experience optimizing GPU utilization and model inference performance.

Cloud & Infrastructure

Strong experience with AWS, GCP, or Azure AI/ML services.

Experience with Kubernetes-based ML infrastructure.

Experience with GPU clusters and distributed model training.

Experience with Terraform or other Infrastructure-as-Code technologies.

Experience with Kafka or other event-streaming platforms.

FinTech & Security

Experience working in FinTech, banking, lending, payments, or other regulated industries.

Understanding of data privacy, information security, and regulatory requirements.

Experience building AI systems handling sensitive financial or customer data.

Familiarity with responsible AI, model governance, and AI risk management.

Engineering Management

Experience managing teams of 8–20+ engineers.

Experience hiring and building engineering teams.

Experience working with senior engineering and business leadership.

Strong stakeholder management and communication skills.

Tools & Technologies

Programming: Python, Java, Go, Scala

AI/ML: PyTorch, TensorFlow, MLflow, Kubeflow, SageMaker, Vertex AI

Generative AI: LLMs, RAG, Vector Databases, Embeddings, AI Agents, Prompt Management

Cloud: AWS, GCP, Azure

Infrastructure: Kubernetes, Docker, Terraform, Infrastructure as Code

Data & Streaming: Kafka, Spark, Airflow, SQL

Databases: PostgreSQL, MySQL, MongoDB, Redis, Vector Databases

Observability: Prometheus, Grafana, OpenTelemetry, CloudWatch, Datadog, or equivalent

CI/CD: GitHub Actions, Jenkins, GitLab CI, ArgoCD, or equivalent

Security: IAM, Secrets Management, Encryption, API Security, Data Protection

Architecture: Microservices, Distributed Systems, Event-Driven Architecture, REST APIs, gRPC

Role Details

Position: Engineering Manager - AI Platform

Employment Type: Full-Time

Experience Level: Senior Management / Engineering Leadership (10+ Years)

Location: Bengaluru

Work Model: On-site

Recruitment Fraud Warning & Disclaimer

Krazybee is committed to maintaining a transparent, fair, and secure recruitment process. We have become aware of fraudulent individuals and entities falsely claiming to represent Krazybee and offering fake employment opportunities.

Please note that Krazybee never asks candidates to pay any money for registration, training, certifications, security deposits, or any other purpose at any stage of the recruitment process. Candidates are advised to verify the sender's email address before responding to any recruitment-related communication. Krazybee does not conduct recruitment through unofficial email addresses or request payments.

If you receive a suspicious job offer, interview invitation, or recruitment-related communication claiming to be from Krazybee, do not make any payment or share sensitive personal or financial information. We encourage you to verify the authenticity of the communication through the official careers page or appropriate company channels.

Disclaimer: Krazybee shall not be responsible for any loss, damage, or claims arising from fraudulent job offers or recruitment communications made by unauthorized individuals or entities falsely representing the company. Candidates should exercise due diligence and verify the authenticity of all recruitment-related communications before taking any action.

This job description is intended to outline the general nature and key responsibilities of the position. It is not intended to be an exhaustive list of all duties, responsibilities, or qualifications associated with the role. The responsibilities and qualifications described may be subject to change, and other duties may be assigned as needed.

Data Utilization Disclaimer

By applying for this position, you acknowledge and agree that any personal data you provide may be used for recruitment and employment purposes. The data collected will be stored and processed in accordance with our privacy policy and applicable data protection laws. Your information will only be shared with relevant internal stakeholders and will not be disclosed to third parties without your consent.