Senior Data Architect - Databricks and AWS

Overview

Work Location - Remote

Lead Time - Immediate

Key Skills – Databricks exp is required and AWS data Architect exp

Job Overview – Senior Data Architect

We are on an exciting journey to build and scale our advanced analytics practice. We are looking for a Senior Data Architect that has experience in building data lake and data warehouse architectures using On-Prem and Cloud technologies.

We are seeking a highly skilled and experienced Data Architect with a strong background in Big Data technologies, Databricks solutioning, and SAP integration within the manufacturing industry. The ideal candidate will have a proven track record of leading data teams, architecting scalable data platforms, and optimizing cloud infrastructure costs. This role requires deep hands-on expertise in Apache Spark, Python, SQL, and cloud platforms (Azure/AWS/GCP).

Organization and is responsible for driving organic growth by leveraging big data and advanced analytics. The team reports to the VP and Chief Data Officer, works closely with the SVP of Corporate Strategy, and has regular interactions with the company’s C-Suite.

Key Responsibilities:

· Design and implement scalable, secure, and high-performance Big Data architectures using Databricks, Apache Spark, and cloud-native services.

· Lead the end-to-end data architecture lifecycle, from requirements gathering to deployment and optimization.

· Design repeatable and reusable data ingestion pipelines for bringing in data from ERP source systems like SAP, Salesforce, HR, Factory, Marketing systems etc.

· Collaborate with cross-functional teams to integrate SAP data sources into modern data platforms.

· Drive cloud cost optimization strategies and ensure efficient resource utilization.

· Provide technical leadership and mentorship to a team of data engineers and developers.

· Develop and enforce data governance, data quality, and security standards.

· Translate complex business requirements into technical solutions and data models.

· Stay current with emerging technologies and industry trends in data architecture and analytics.

Required Skills & Qualifications:

· 6+ years of experience in Big Data architecture, Data Engineering and AI assisted BI solutions within Databricks and AWS technologies.

· 3+ Years of Experience with AWS Data services like S3, Glue, Lake Formation, EMR, Kinesis, RDS, DMS and others

· 3+ Years of Experience in building Delta Lakes and open formats using technologies like Databricks and AWS Analytics Services.

· Bachelor’s degree in computer science, information technology, data science, data analytics or related field

· Proven expertise in Databricks, Apache Spark, Delta Lake, and MLflow.

· Strong programming skills in Python, SQL, and PySpark.

· Experience with SAP data extraction and integration (e.g., SAP BW, S/4HANA, BODS).

· Hands-on experience with cloud platforms (Azure, AWS, or GCP), especially in cost optimization and data lakehouse architectures.

· Solid understanding of data modeling, ETL/ELT pipelines, and data warehousing.

· Demonstrated team leadership and project management capabilities.

· Excellent communication, problem solving and stakeholder management skills.

Preferred Qualifications:

· Experience in the manufacturing domain, with knowledge of production, supply chain, and quality data.

· Certifications in Databricks, cloud platforms, or data architecture.

· Familiarity with CI/CD pipelines, DevOps practices, and infrastructure as code (e.g., Terraform).

Job Type: Full-time

Pay: ₹856,881.62 - ₹2,517,359.09 per year

Education:

• Bachelor's (Preferred)

Experience:

• Big Data architecture, AI assisted BI solutions ,AWS : 6 years (Preferred)
• AWS services like Glue, Lake Formation, EMR, Kinesis, DMS : 3 years (Preferred)
• Databricks and AWS Analytics Services: 3 years (Preferred)
• in Databricks, Apache Spark, Delta Lake, and MLflow.: 3 years (Preferred)
• Python, SQL, and PySpark: 3 years (Preferred)
• SAP data extraction and integration (SAP BW, S/4HANA, BODS): 3 years (Preferred)

Work Location: Remote