Sirsa, Haryana, India
Information Technology
Full-Time
LeewayHertz
Overview
Job Description
This is a remote position.
Job Description
We are seeking a highly experienced and innovative Senior Data Engineer with a strong background in hybrid cloud data integration, pipeline orchestration, and AI-driven data modeling. This role is responsible for designing, building, and optimizing robust, scalable, and production-ready data pipelines across both AWS and Azure platforms, supporting
modern data architectures such as CEDM and Data Vault 2.0.
Responsibilities
Essential Skills:
Job
Job
check(event) ; career-website-detail-template-2 => apply(record.id,meta)">
This is a remote position.
Job Description
We are seeking a highly experienced and innovative Senior Data Engineer with a strong background in hybrid cloud data integration, pipeline orchestration, and AI-driven data modeling. This role is responsible for designing, building, and optimizing robust, scalable, and production-ready data pipelines across both AWS and Azure platforms, supporting
modern data architectures such as CEDM and Data Vault 2.0.
Responsibilities
- Design and develop hybrid ETL/ELT pipelines using AWS Glue and Azure Data Factory (ADF).
- Process files from AWS S3 and Azure Data Lake Gen2, including schema validation and data profiling.
- Implement event-based orchestration using AWS Step Functions and Apache Airflow (Astronomer).
- Develop and maintain bronze → silver → gold data layers using DBT or Coalesce.
- Create scalable ingestion workflows using Airbyte, AWS Transfer Family, and Rivery.
- Integrate with metadata and lineage tools like Unity Catalog and OpenMetadata.
- Build reusable components for schema enforcement, EDA, and alerting (e.g., MS Teams).
- Work closely with QA teams to integrate test automation and ensure data quality.
- Collaborate with cross-functional teams including data scientists and business stakeholders to align solutions with AI/ML use cases.
- Document architectures, pipelines, and workflows for internal stakeholders.
Essential Skills:
Job
- Experience with cloud platforms: AWS (Glue, Step Functions, Lambda, S3, CloudWatch, SNS, Transfer Family) and Azure (ADF, ADLS Gen2, Azure Functions,Event Grid).
- Skilled in transformation and ELT tools: Databricks (PySpark), DBT, Coalesce, and Python.
- Proficient in data ingestion using Airbyte, Rivery, SFTP/Excel files, and SQL Server extracts.
- Strong understanding of data modeling techniques including CEDM, Data Vault 2.0, and Dimensional Modeling.
- Hands-on experience with orchestration tools such as AWS Step Functions, Airflow (Astronomer), and ADF Triggers.
- Expertise in monitoring and logging with CloudWatch, AWS Glue Metrics, MS Teams Alerts, and Azure Data Explorer (ADX).
- Familiar with data governance and lineage tools: Unity Catalog, OpenMetadata, and schema drift detection.
- Proficient in version control and CI/CD using GitHub, Azure DevOps, CloudFormation, Terraform, and ARM templates.
- Experienced in data validation and exploratory data analysis with pandas profiling,
- AWS Glue Data Quality, and Great Expectations.
- Excellent communication and interpersonal skills, with the ability to engage with teams.
- Strong problem-solving, decision-making, and conflict-resolution abilities.
- Proven ability to work independently and lead cross-functional teams.
- Ability to work in a fast-paced, dynamic environment and handle sensitive issues with discretion and professionalism.
- Ability to maintain confidentiality and handle sensitive information with attention to detail with discretion.
- The candidate must have strong work ethics and trustworthiness
- Must be highly collaborative and team oriented with commitment to excellence.
Job
- Proficiency in SQL and at least one programming language (e.g., Python, Scala).
- Experience with cloud data platforms (e.g., AWS, Azure, GCP) and their data and AI services.
- Knowledge of ETL tools and frameworks (e.g., Apache NiFi, Talend, Informatica).
- Deep understanding of AI/Generative AI concepts and frameworks (e.g., TensorFlow, PyTorch, Hugging Face, OpenAI APIs).
- Experience with data modeling, data structures, and database design.
- Proficiency with data warehousing solutions (e.g., Redshift, BigQuery, Snowflake).
- Hands-on experience with big data technologies (e.g., Hadoop, Spark, Kafka).
- Demonstrate proactive thinking
- Should have strong interpersonal relations, expert business acumen and mentoring skills
- Have the ability to work under stringent deadlines and demanding client conditions
- Ability to work under pressure to achieve the multiple daily deadlines for client deliverables with a mature approach
- Bachelor’s in Engineering with specialization in Computer Science or Artificial Intelligence or Information Technology or a related field.
- 9+ years of experience in data engineering and data architecture.
check(event) ; career-website-detail-template-2 => apply(record.id,meta)">
Similar Jobs
View All
22 Hours ago
Data Analyst
Information Technology
- 300000 - 1200000 INR - Annual
- 1 - 7 Yrs
- Anywhere in India/Multiple Locations
Location: Remote – India
Experience: 1–7 years
Employment Type: Full-time & Part-time options available
About the Role
We’re seeking analytical minds who love turning data into actionable insights. As a Data Analyst, you’ll work closely w...
23 Hours ago
Sr. ServiceNow Developer | 8+ Years | Pune
Information Technology
- 30 - 40 INR - Annual
- 8 - 11 Yrs
- Pune
About the Role:
This is a senior-level ServiceNow Developer role where you’ll be responsible for driving the technical vision and execution of complex ServiceNow solutions across the organization.
What We’re Looking For:
Strong hands-...
23 Hours ago
Mobile Developer
Information Technology
- 350000 - 1000000 INR - Annual
- 1 - 6 Yrs
- Anywhere in India/Multiple Locations
About the Role
We're looking for talented Mobile Developers—whether you're a rising star with a year of experience or a seasoned engineer with eight—to build intuitive, high-performance mobile applications for Android and iOS. You’ll collaborate w...
2 Days ago
IT Support Executive
Information Technology
- Hyderabad, Telangana, India
hackajob is collaborating with British Airways to connect them with exceptional tech professionals for this role.A career without limitsAs the nation’s flag carrier, we take great pride in connecting Britain with the world and the world with Britain...
2 Days ago
Full Stack Developer (Java + AWS DevOps Focus)
Information Technology
- Hyderabad, Telangana, India
Company Description We are a Stealth GenAI B2B Startup focused on revolutionizing construction industry using Artificial General Intelligence. Our team is comprised of visionaries, researchers, and engineers dedicated to pushing the boundaries of AI...
2 Days ago
DevOps Engineer
Information Technology
- Surat, Gujarat, India
About QuantaniteQuantanite is a customer experience (CX) solutions company that helps fast-growing companies and leading global brands to transform and grow. We do this through a collaborative and consultative approach, rethinking business processes...
2 Days ago
Tech Carrot - Data Engineer - Python/ETL
Information Technology
- Hyderabad, Telangana, India
Skills Advanced working knowledge and experience with relational and non-relational databases. Experience building and optimizing Big Data pipelines, architectures, and datasets. Strong analytic skills related to working with structured and unstr...
2 Days ago
Software Engineer - Java8 & React Fullstack_C2E
Information Technology
- Surat, Gujarat, India
Reference 250008EIResponsibilitiesJob description:As a full stack developer, you’ll will work in agile. You will participate to all the agile ceremonies (sprint planning, refinement, retrospective, poker planning et sprint review)As part of the Pari...
Talk to us
Feel free to call, email, or hit us up on our social media accounts.
Email
info@antaltechjobs.in