Principal Data Engineer - Databricks

By clicking the “Apply” button, I understand that my employment application process with Takeda will commence and that the information I provide in my application will be processed in line with Takeda’s Privacy Notice and Terms of Use. I further attest that all information I submit in my employment application is true to the best of my knowledge.

Job Description

The Future Begins Here

At Takeda, we are leading digital evolution and global transformation. By building innovative solutions and future-ready capabilities, we are meeting the need of patients, our people, and the planet.

Bengaluru, the city, which is India’s epicenter of Innovation, has been selected to be home to Takeda’s recently launched Innovation Capability Center. We invite you to join our digital transformation journey. In this role, you will have the opportunity to boost your skills and become the heart of an innovative engine that is contributing to global impact and improvement.

At Takeda’s ICC we Unite in Diversity

Takeda is committed to creating an inclusive and collaborative workplace, where individuals are recognized for their backgrounds and abilities they bring to our company. We are continuously improving our collaborators journey in Takeda, and we welcome applications from all qualified candidates. Here, you will feel welcomed, respected, and valued as an important contributor to our diverse team.

The Opportunity

The Principal Data Engineer - Databricks is responsible for designing, building, and delivering Databricks platform capabilities and solutions to enable Takeda’s Data & Digital Transformation, paving the way for an AI-powered intelligent platform. This role requires a deep understanding of Databricks architecture, cloud computing, and modern data platform strategies to drive efficiency, scalability, and innovation.

A Databricks Platform Architect creates an end-to-end vision for Enterprise Data Platforms, integrating evolving technology and business needs into a cohesive, user-centric framework. A product-centric mindset, focus on reducing time to market, and ability to develop reusable frameworks and automation patterns are essential.

The architect will collaborate with fellow architects, product owners, and engineering teams to define the roadmap for Databricks infrastructure, automation, governance, and patterns supporting complex data use cases aligned with Takeda’s Data & Digital strategy.

Responsibilities

• Develop and execute a strategic roadmap for Databricks platform capabilities, ensuring alignment with Takeda’s data transformation initiatives.

• Design and implement scalable, serverless, and well-architected Databricks solutions to support data products, AI/ML workloads, and analytics use cases.

• Enable data access, governance, and self-service analytics through Unity Catalog, Delta Sharing, and data mesh principles.

• Define reusable frameworks, automation patterns, and DevOps practices for infrastructure as code (IaC), CI/CD, and monitoring.

• Drive data platform observability, implementing Databricks Lakehouse Monitoring and cost optimization strategies.

• Establish and enforce data quality, compliance, security, and privacy policies, aligning with HIPAA, SOX, and GxP requirements.

• Collaborate with Data Engineers, Scientists, and Analysts to optimize workflows and performance within Databricks.

• Develop integration strategies with Informatica, AWS services (S3, Glue, Lambda, IAM), and streaming technologies like Kafka.

• Lead Proof of Concepts (PoCs) and innovation initiatives in Generative AI, LLMs, and Databricks AI/ML capabilities.

• Partner with vendors and service providers to evaluate new tools and technologies that enhance Databricks capabilities.

• Define KPIs and dashboards for monitoring Databricks platform health, performance, and cost

Skills and Qualifications

• Bachelor’s degree in Computer Science, Engineering, or a related field with 8+ years of relevant experience; Master’s degree with 6+ years of experience.

• 12+ years of hands-on experience in enterprise/data platform architecture, strategy, and solution development.

• 5+ years of experience in the pharmaceutical/life sciences industry (preferred), including technical architecture, data management, and AI/ML integration.

Core Technical Expertise:

• Databricks Expert: 5+ years of experience in Databricks, including Unity Catalog, Delta Lake, Delta Sharing, Photon, MLflow, and Data Engineering/Science workloads.

• Cloud Platforms & Infrastructure: Deep expertise in AWS (preferred) with experience in GCP/Azure being a plus. Must have experience with serverless architectures, security, identity management, and governance (IAM, AD, encryption, privacy policies).

• Data & AI/ML Platforms: Strong knowledge of GenAI, LLMs, MLOps, Deep Learning, Data Science, and Model Deployment. Experience with Dataiku, DataRobot, Hugging Face, and AI frameworks is a plus.

• Infrastructure Automation & DevOps: 3+ years of experience implementing IaC (Terraform preferred, CloudFormation, Ansible, Chef) and CI/CD pipelines for data platforms.

• Data Management & Engineering: Hands-on experience with Data Lakes, Data Mesh, Data Fabric, Streaming (Kafka), ETL/ELT, Data Warehousing, Advanced Analytics, GraphQL, and Orchestration (Airflow, Tidal, databricks workflow).

• Programming & Automation: Strong coding skills in Python, Spark, SQL, Shell scripting; experience with GitHub Actions, Ansible, and Terraform for platform automation is required.

• Security, Governance & Compliance: Deep knowledge of Unity Catalog, data privacy (HIPAA, GDPR, GxP, SOX), access control, encryption, auditing, FinOps, and observability (Splunk, AppDynamics, Wiz, SonarCloud).

Enterprise Architecture & Leadership:

• Familiarity with TOGAF (preferred), Zachman, or other EA frameworks.

• Experience designing self-service data platforms, modern API-based architectures, and enterprise governance frameworks.

• Ability to translate complex business problems into scalable, high-performing architectures.

• Strong stakeholder management, presentation, and leadership skills, with the ability to align cross-functional teams on technical decisions.

Added Value:

• Databricks Certifications

• AWS Certifications

BENEFITS

It is our priority to provide competitive compensation and a benefit package that bridges your personal life with your professional career. Amongst our benefits are:

• Competitive Salary + Performance Annual Bonus

• Flexible work environment, including hybrid working

• Comprehensive Healthcare Insurance Plans for self, spouse, and children

• Group Term Life Insurance and Group Accident Insurance programs

• Health & Wellness programs including annual health screening, weekly health sessions for employees.

• Employee Assistance Program

• 3 days of leave every year for Voluntary Service in additional to Humanitarian Leaves

• Broad Variety of learning platforms

• Diversity, Equity, and Inclusion Programs

• Reimbursements – Home Internet & Mobile Phone

• Employee Referral Program

• Leaves – Paternity Leave (4 Weeks) , Maternity Leave (up to 26 weeks), Bereavement Leave (5 calendar days)

ABOUT ICC IN TAKEDA:

• Takeda is leading a digital revolution. We’re not just transforming our company; we’re improving the lives of millions of patients who rely on our medicines every day.

• As an organization, we are committed to our cloud-driven business transformation and believe the ICCs are the catalysts of change for our global organization.

Locations

Worker Type

Worker Sub-Type

Time Type