Sahibzada ajit singh nagar, Punjab, India
Information Technology
Other
Prodapt
Overview
Overview:
Work Location: Hyderabad, Bangalore and Chennai Requirements:
Preferred Skills (Nice to Have):
We are hiring a Senior Data Engineer (6 to 10 years) with deep expertise in Azure Data Bricks, Azure Data Lake, and Azure Synapse Analytics to join our high-performing team. The ideal candidate will have a proven track record in designing, building, and optimizing big data pipelines and architectures while leveraging their technical proficiency in cloud-based data engineering.
This role requires a strategic thinker who can bridge the gap between raw data and actionable insights, enabling data-driven decision-making for large-scale enterprise initiatives. A strong foundation in distributed computing, ETL frameworks, and advanced data modeling is crucial. The individual will work closely with data architects, analysts, and business teams to deliver scalable and efficient data solutions.
Work Location: Hyderabad, Bangalore and Chennai
Work Mode: Work from Office (5 days)
Notice Period: Immediate to 15 days
Responsibilities: Key Responsibilities:
- Data Engineering & Architecture:
- Design, develop, and maintain high-performance data pipelines for structured and unstructured data using Azure Data Bricks and Apache Spark.
- Build and manage scalable data ingestion frameworks for batch and real-time data processing.
- Implement and optimize data lake architecture in Azure Data Lake to support analytics and reporting workloads.
- Develop and optimize data models and queries in Azure Synapse Analytics to power BI and analytics use cases.
- Cloud-Based Data Solutions:
- Architect and implement modern data lakehouses combining the best of data lakes and data warehouses.
- Leverage Azure services like Data Factory, Event Hub, and Blob Storage for end-to-end data workflows.
- Ensure security, compliance, and governance of data through Azure Role-Based Access Control (RBAC) and Data Lake ACLs.
- ETL/ELT Development:
- Develop robust ETL/ELT pipelines using Azure Data Factory, Data Bricks notebooks, and PySpark.
- Perform data transformations, cleansing, and validation to prepare datasets for analysis.
- Manage and monitor job orchestration, ensuring pipelines run efficiently and reliably.
- Performance Optimization:
- Optimize Spark jobs and SQL queries for large-scale data processing.
- Implement partitioning, caching, and indexing strategies to improve performance and scalability of big data workloads.
- Conduct capacity planning and recommend infrastructure optimizations for cost-effectiveness.
- Collaboration & Stakeholder Management:
- Work closely with business analysts, data scientists, and product teams to understand data requirements and deliver solutions.
- Participate in cross-functional design sessions to translate business needs into technical specifications.
- Provide thought leadership on best practices in data engineering and cloud computing.
- Documentation & Knowledge Sharing:
- Create detailed documentation for data workflows, pipelines, and architectural decisions.
- Mentor junior team members and promote a culture of learning and innovation.
Required Qualifications:
- Experience:
- 7+ years of experience in data engineering, big data, or cloud-based data solutions.
- Proven expertise with Azure Data Bricks, Azure Data Lake, and Azure Synapse Analytics.
- Technical Skills:
- Strong hands-on experience with Apache Spark and distributed data processing frameworks.
- Advanced proficiency in Python and SQL for data manipulation and pipeline development.
- Deep understanding of data modeling for OLAP, OLTP, and dimensional data models.
- Experience with ETL/ELT tools like Azure Data Factory or Informatica.
- Familiarity with Azure DevOps for CI/CD pipelines and version control.
- Big Data Ecosystem:
- Familiarity with Delta Lake for managing big data in Azure.
- Experience with streaming data frameworks like Kafka, Event Hub, or Spark Streaming.
- Cloud Expertise:
- Strong understanding of Azure cloud architecture, including storage, compute, and networking.
- Knowledge of Azure security best practices, such as encryption and key management.
Preferred Skills (Nice to Have):
- Experience with machine learning pipelines and frameworks like MLFlow or Azure Machine Learning.
- Knowledge of data visualization tools such as Power BI for creating dashboards and reports.
- Familiarity with Terraform or ARM templates for infrastructure as code (IaC).
- Exposure to NoSQL databases like Cosmos DB or MongoDB.
- Experience with data governance tools like Azure Purview.
Similar Jobs
View All
29 Minutes ago
Senior Database Admin - SQL
Pharmaceuticals
- 7 - 12 Yrs
- Gurgaon / Gurugram
Job Summary
We are seeking an experienced Database Administrator to join our Shared Services IT team. In this role, you will design, implement, and maintain data management systems, ensuring security, compliance, and optimal performance. This posi...
43 Minutes ago
Oracle cloud Functional Lead
Information Technology
- 30,00,000 - 35,00,000 INR - Annual
- 8 - 15 Yrs
- Bangalore, Chennai
We’re Hiring | Oracle Functional Consultant - Finance
Location: Chennai/ Bangalore, India
Domain: Finance
Responsibilities: ...
4 Hours ago
Head of Product Engineering
Information Technology
- 8 - 12 Yrs
- Mumbai (All Areas)
Role Overview
We are seeking a visionary and hands-on Head of Product Engineering to lead and scale the product engineering function. This role involves full ownership of delivering a cutting-edge AI security product, ensuring it is scalable, secu...
5 Hours ago
Senior Vice President of Engineering
Information Technology
- 12 - 16 Yrs
- Mumbai (All Areas)
We are hiring a Senior Vice President (SVP) of Engineering for a high-growth, innovation-led cybersecurity company specializing in AI-powered security solutions. The company secures over 500 million+ devices, is backed by top-tier investors, and is t...
6 Hours ago
FP&A Analyst
Information Technology
- 12,00,000 - 15,00,000 INR - Annual
- 3 - 5 Yrs
- Bangalore
Key Responsibilities
Lead the planning and analysis cycle by building, updating, and refining quarterly and annual operating plans (AOP) across all geographies.
Conduct regular variance analysis (MIS) to explain actuals versus plan, identifyi...
7 Hours ago
AWS/Azure Software Engineer
Information Technology
- 9 - 13 Yrs
- Chennai
Software Engineer III
Good Communication is mandatory
Shift :2:00 PM -11:00 PM
Yrs Of Exp : 9-13
Location : Chennai (Work from office )
Budget : 1.9 Lac Per month
Notice period - Maximum 20 days
EDUCATION - Bachelor's Degree or e...
7 Hours ago
Tech Lead
Information Technology
- 6 - 10 Yrs
- Mumbai
Job Profile
Position: Tech Lead / Engineering Manager
Location: Mumbai
Qualification: Master’s in Computer Science, Engineering, or related technical discipline
Key Responsibilities
• Lead and manage a cross-functional team of Flutter,...
1 Day ago
Senior Software Engineer - React.js
Information Technology
- 20,00,000 - 27,00,000 INR - Annual
- 2 - 5 Yrs
- Bangalore
Your role: As a Senior Frontend Engineer, you will work on our React web applications to build usable, performant, and beautiful user interfaces. We are looking for someone who is experienced with React and loves to build high quality user experience...
Talk to us
Feel free to call, email, or hit us up on our social media accounts.
Email
info@antaltechjobs.in