Hyderabad, Telangana, India
Information Technology
Full-Time
LeewayHertz
Overview
Job Description
This is a remote position.
Job Summary
As a Senior Data Engineer, you will be responsible for designing, building, and optimizing data pipelines and lakehouse architectures on AWS. You will ensure data availability, quality, lineage, and governance across analytical and operational platforms. Your expertise will enable scalable, secure, and cost-effective data solutions that power advanced analytics and business intelligence.
Responsibilities
Essential Skills
Job
Job
check(event) ; career-website-detail-template-2 => apply(record.id,meta)">
This is a remote position.
Job Summary
As a Senior Data Engineer, you will be responsible for designing, building, and optimizing data pipelines and lakehouse architectures on AWS. You will ensure data availability, quality, lineage, and governance across analytical and operational platforms. Your expertise will enable scalable, secure, and cost-effective data solutions that power advanced analytics and business intelligence.
Responsibilities
- Implement and manage S3 (raw, staging, curated zones), Glue Catalog, Lake Formation, and Iceberg/Hudi/Delta Lake for schema evolution and versioning.
- Develop PySpark jobs on Glue/EMR, enforce schema validation, partitioning, and scalable transformations.
- Build workflows using Step Functions, EventBridge, or Airflow (MWAA), with CI/CD deployments via CodePipeline & CodeBuild.
- Apply schema contracts, validations (Glue Schema Registry, Deequ, Great Expectations), and maintain lineage/metadata using Glue Catalog or third-party tools (Atlan, OpenMetadata, Collibra).
- Enable Athena and Redshift Spectrum queries, manage operational stores (DynamoDB/Aurora), and integrate with OpenSearch for observability.
- Design efficient partitioning/bucketing strategies, adopt columnar formats (Parquet/ORC), and implement spot instance usage/bookmarking.
- Enforce IAM-based access policies, apply KMS encryption, private endpoints, and GDPR/PII data masking.
- Prepare Gold-layer KPIs for dashboards, forecasting, and customer insights with QuickSight, Superset, or Metabase.
- Partner with analysts, data scientists, and DevOps to enable seamless data consumption and delivery.
Essential Skills
Job
- Hands-on expertise with AWS data stack (S3, Glue, Lake Formation, Athena, Redshift, EMR, Lambda).
- Strong programming skills in PySpark & Python for ETL, scripting, and automation.
- Proficiency in SQL (CTEs, window functions, complex aggregations).
- Experience in data governance, quality frameworks (Deequ, Great Expectations).
- Knowledge of data modeling, partitioning strategies, and schema enforcement.
- Familiarity with BI integration (QuickSight, Superset, Metabase).
- Strong problem-solving ability in complex data environments.
- Ability to communicate technical insights to non-technical stakeholders.
- Commitment to best practices in data governance, compliance, and security.
- Collaborative mindset with cross-functional teams.
Job
- Real-time ingestion experience (Kinesis, MSK, Kafka on AWS).
- Exposure to ML feature store integration with SageMaker.
- Infrastructure as Code (Terraform, CloudFormation, or CDK).
- Experience with Data Mesh or domain-driven data architecture.
- Experience mentoring junior data engineers.
- Ability to lead data projects from design to production.
- Proactive in learning new AWS and data ecosystem technologies.
- Bachelor’s/Master’s degree in Computer Science, Information Technology, or related field.
- Minimum 4 years of proven experience in data engineering with AWS.
- This role offers the flexibility of working remotely in India.
check(event) ; career-website-detail-template-2 => apply(record.id,meta)">
Similar Jobs
View All
1 Day ago
Chief Technology Officer (CTO)
Information Technology
- 9 - 15 Yrs
- Mumbai (All Areas)
Role & responsibilities
* Develop and execute the company's technology roadmap aligned with business goals.
* Lead and mentor the engineering and IT teams to ensure high performance.
* Identity and implement emerging technologies to enhance ...
2 Days ago
Website Developer
Information Technology
- 700000 - 1200000 INR - Annual
- 5 - 10 Yrs
- Udaipur
Job Title: Website Developer
Location: Udaipur, Rajasthan (HO)
Job Type: Full-time
Experience Level: Senior (5 TO 10 Years)
About the Role:
We are looking for a skilled and detail-oriented Website Developer to join our team. The ideal candidat...
2 Days ago
Business Analyst
Information Technology
- Delhi, Delhi, India
Job DescriptionBusiness Analyst, Sales Operations:As Gartner Conferences continues its rapid expansion, we are seeking a highly motivated and analytical Sales Operations Analyst to join our dynamic team. This pivotal role supports the operational ne...
2 Days ago
ChicMic Studios - Python/Django Developer
Information Technology
- Delhi, Delhi, India
Profile : Python developer.Experience : 2-6 Years.Location : Mohali, Punjab (WFO).ChicMic Studios is on the lookout for a skilled Python Developer who loves turning complex problems into elegant solutions.If you have a passion for Django, Flask, AWS...
2 Days ago
Sr Data Engineer
Information Technology
- Delhi, Delhi, India
Job DescriptionAbout HighspotHighspot is a software product development company and a recognized global leader in the sales enablement category, leveraging cutting-edge AI and GenAI technologies at the core of its robust Software-as-a-Service (SaaS)...
2 Days ago
ChicMic Studios - Angular Developer - TypeScript/Javascript
Information Technology
- Delhi, Delhi, India
Key Responsibilities Develop and maintain responsive, high-performance web applications using Angular. Collaborate with UX/UI designers and back-end developers to implement user-friendly features. Optimize application for maximum speed and scalab...
2 Days ago
Oracle PL-SQL Developer
Information Technology
- Delhi, Delhi, India
Role SummaryWe are seeking a skilled Oracle PL/SQL Developer to design, develop, optimize, and maintain PL/SQL code for a regulatory database application which integrates with middleware and external APIs of MHA and DOT. The ideal candidate will hav...
2 Days ago
Software Engineer Senior
Information Technology
- Delhi, Delhi, India
The CompanyGentrack provides leading utilities across the world with innovative cleantech solutions. The global pace of change is accelerating, and utilities need to rebuild for a more sustainable future. Working with some of the world’s biggest ene...
Talk to us
Feel free to call, email, or hit us up on our social media accounts.
Email
info@antaltechjobs.in