Free cookie consent management tool by TermsFeed Principal Data Engineer - Databricks | Antal Tech Jobs
Back to Jobs
8 Weeks ago

Principal Data Engineer - Databricks

decor
Chennai, Tamil Nadu, India
Information Technology
Full-Time
Takeda Pharmaceutical

Overview

By clicking the “Apply” button, I understand that my employment application process with Takeda will commence and that the information I provide in my application will be processed in line with Takeda’s Privacy Notice and Terms of Use. I further attest that all information I submit in my employment application is true to the best of my knowledge.

Job Description

The Future Begins Here

At Takeda, we are leading digital evolution and global transformation. By building innovative solutions and future-ready capabilities, we are meeting the need of patients, our people, and the planet.

Bengaluru, the city, which is India’s epicenter of Innovation, has been selected to be home to Takeda’s recently launched Innovation Capability Center. We invite you to join our digital transformation journey. In this role, you will have the opportunity to boost your skills and become the heart of an innovative engine that is contributing to global impact and improvement.

At Takeda’s ICC we Unite in Diversity

Takeda is committed to creating an inclusive and collaborative workplace, where individuals are recognized for their backgrounds and abilities they bring to our company. We are continuously improving our collaborators journey in Takeda, and we welcome applications from all qualified candidates. Here, you will feel welcomed, respected, and valued as an important contributor to our diverse team.

The Opportunity

The Principal Data Engineer - Databricks is responsible for designing, building, and delivering Databricks platform capabilities and solutions to enable Takeda’s Data & Digital Transformation, paving the way for an AI-powered intelligent platform. This role requires a deep understanding of Databricks architecture, cloud computing, and modern data platform strategies to drive efficiency, scalability, and innovation.

A Databricks Platform Architect creates an end-to-end vision for Enterprise Data Platforms, integrating evolving technology and business needs into a cohesive, user-centric framework. A product-centric mindset, focus on reducing time to market, and ability to develop reusable frameworks and automation patterns are essential.

The architect will collaborate with fellow architects, product owners, and engineering teams to define the roadmap for Databricks infrastructure, automation, governance, and patterns supporting complex data use cases aligned with Takeda’s Data & Digital strategy.

Responsibilities

  • Develop and execute a strategic roadmap for Databricks platform capabilities, ensuring alignment with Takeda’s data transformation initiatives.

  • Design and implement scalable, serverless, and well-architected Databricks solutions to support data products, AI/ML workloads, and analytics use cases.

  • Enable data access, governance, and self-service analytics through Unity Catalog, Delta Sharing, and data mesh principles.

  • Define reusable frameworks, automation patterns, and DevOps practices for infrastructure as code (IaC), CI/CD, and monitoring.

  • Drive data platform observability, implementing Databricks Lakehouse Monitoring and cost optimization strategies.

  • Establish and enforce data quality, compliance, security, and privacy policies, aligning with HIPAA, SOX, and GxP requirements.

  • Collaborate with Data Engineers, Scientists, and Analysts to optimize workflows and performance within Databricks.

  • Develop integration strategies with Informatica, AWS services (S3, Glue, Lambda, IAM), and streaming technologies like Kafka.

  • Lead Proof of Concepts (PoCs) and innovation initiatives in Generative AI, LLMs, and Databricks AI/ML capabilities.

  • Partner with vendors and service providers to evaluate new tools and technologies that enhance Databricks capabilities.

  • Define KPIs and dashboards for monitoring Databricks platform health, performance, and cost

Skills and Qualifications

  • Bachelor’s degree in Computer Science, Engineering, or a related field with 8+ years of relevant experience; Master’s degree with 6+ years of experience.

  • 12+ years of hands-on experience in enterprise/data platform architecture, strategy, and solution development.

  • 5+ years of experience in the pharmaceutical/life sciences industry (preferred), including technical architecture, data management, and AI/ML integration.

Core Technical Expertise:

  • Databricks Expert: 5+ years of experience in Databricks, including Unity Catalog, Delta Lake, Delta Sharing, Photon, MLflow, and Data Engineering/Science workloads.

  • Cloud Platforms & Infrastructure: Deep expertise in AWS (preferred) with experience in GCP/Azure being a plus. Must have experience with serverless architectures, security, identity management, and governance (IAM, AD, encryption, privacy policies).

  • Data & AI/ML Platforms: Strong knowledge of GenAI, LLMs, MLOps, Deep Learning, Data Science, and Model Deployment. Experience with Dataiku, DataRobot, Hugging Face, and AI frameworks is a plus.

  • Infrastructure Automation & DevOps: 3+ years of experience implementing IaC (Terraform preferred, CloudFormation, Ansible, Chef) and CI/CD pipelines for data platforms.

  • Data Management & Engineering: Hands-on experience with Data Lakes, Data Mesh, Data Fabric, Streaming (Kafka), ETL/ELT, Data Warehousing, Advanced Analytics, GraphQL, and Orchestration (Airflow, Tidal, databricks workflow).

  • Programming & Automation: Strong coding skills in Python, Spark, SQL, Shell scripting; experience with GitHub Actions, Ansible, and Terraform for platform automation is required.

  • Security, Governance & Compliance: Deep knowledge of Unity Catalog, data privacy (HIPAA, GDPR, GxP, SOX), access control, encryption, auditing, FinOps, and observability (Splunk, AppDynamics, Wiz, SonarCloud).

Enterprise Architecture & Leadership:

  • Familiarity with TOGAF (preferred), Zachman, or other EA frameworks.

  • Experience designing self-service data platforms, modern API-based architectures, and enterprise governance frameworks.

  • Ability to translate complex business problems into scalable, high-performing architectures.

  • Strong stakeholder management, presentation, and leadership skills, with the ability to align cross-functional teams on technical decisions.

Added Value:

  • Databricks Certifications

  • AWS Certifications

BENEFITS

It is our priority to provide competitive compensation and a benefit package that bridges your personal life with your professional career. Amongst our benefits are:

  • Competitive Salary + Performance Annual Bonus

  • Flexible work environment, including hybrid working

  • Comprehensive Healthcare Insurance Plans for self, spouse, and children

  • Group Term Life Insurance and Group Accident Insurance programs

  • Health & Wellness programs including annual health screening, weekly health sessions for employees.

  • Employee Assistance Program

  • 3 days of leave every year for Voluntary Service in additional to Humanitarian Leaves

  • Broad Variety of learning platforms

  • Diversity, Equity, and Inclusion Programs

  • Reimbursements – Home Internet & Mobile Phone

  • Employee Referral Program

  • Leaves – Paternity Leave (4 Weeks) , Maternity Leave (up to 26 weeks), Bereavement Leave (5 calendar days)

ABOUT ICC IN TAKEDA:

  • Takeda is leading a digital revolution. We’re not just transforming our company; we’re improving the lives of millions of patients who rely on our medicines every day.

  • As an organization, we are committed to our cloud-driven business transformation and believe the ICCs are the catalysts of change for our global organization.

Locations

IND - Bengaluru

Worker Type

Employee

Worker Sub-Type

Regular

Time Type

Full time
Share job
Similar Jobs
View All
1 Day ago
TrueFan - Senior Machine Learning Engineer
Information Technology
  • Thiruvananthapuram, Kerala, India
About UsTrueFan is at the forefront of AI-driven content generation, leveraging cutting-edge generative models to build next-generation products. Our mission is to redefine content generation space through advanced AI technologies, including deep ge...
decor
1 Day ago
Salesforce commerce cloud consultant
Information Technology
  • Thiruvananthapuram, Kerala, India
Salesforce Commerce Cloud consultant  5+ Years of Experience 6 to 12 months Mode - Remote 1.1LPM - 1.2LPM Max Key Responsibilities Translate business requirements into scalable Salesforce Service Cloud solutions, in collaboration with CAE's technic...
decor
1 Day ago
Cloud Infrastructure Engineer
Information Technology
  • Thiruvananthapuram, Kerala, India
DescriptionInvent the future with us. Recognized by Fast Company’s 2023 100 Best Workplaces for Innovators List, Ampere is a semiconductor design company for a new era, leading the future of computing with an innovative approach to CPU design focuse...
decor
1 Day ago
Devops Engineer- Intermetiate
Information Technology
  • Thiruvananthapuram, Kerala, India
BackJD: Dev ops Engineer:As a DevOps Specialist- should be able to take ownership of the entire DevOps process, including Automated CI/CD pipelines and deployment to production.They should also be comfortable with risk analysis and prioritization.Le...
decor
1 Day ago
Sr Data Scientist (London)
Information Technology
  • Thiruvananthapuram, Kerala, India
AryaXAI stands at the forefront of AI innovation, revolutionizing AI for mission-critical, highly regulated industries by building explainable, safe, and aligned systems that scale responsibly. Our mission is to create AI tools that empower research...
decor
1 Day ago
Software Test Engineer
Information Technology
  • Thiruvananthapuram, Kerala, India
By clicking the “Apply” button, I understand that my employment application process with Takeda will commence and that the information I provide in my application will be processed in line with Takeda’s Privacy Notice and Terms of Use. I further att...
decor
1 Day ago
Software Developer 5 (Java Fullstack)
Information Technology
  • Thiruvananthapuram, Kerala, India
Job DescriptionBuilding off our Cloud momentum, Oracle has formed a new organization - Oracle Health Applications & Infrastructure. This team focuses on product development and product strategy for Oracle Health, while building out a complete platfo...
decor
1 Day ago
Java Developer - Spring Frameworks
Information Technology
  • Thiruvananthapuram, Kerala, India
Java DescriptionWe are looking for a passionate and talented Java Developer with 2-3 years of hands-on experience to join our growing development team.The ideal candidate should have a strong foundation in Java technologies and the ability to develo...
decor

Talk to us

Feel free to call, email, or hit us up on our social media accounts.
Social media