Mumbai, Maharashtra, India
Information Technology
Full-Time
UST
Overview
Role Description
We are seeking an experienced Data Engineer with expertise in DBT (Data Build Tool) to join our dynamic and fast-growing team. In this role, you will be responsible for transforming critical data, specifically focusing on the creation and management of silver and gold data tiers within our data pipeline. Working closely with data architects and engineers, you will help design, develop, and optimize data transformation processes, ensuring that data is clean, reliable, and ready for business intelligence and analytics teams.
Key Responsibilities
Data build tool, PySpark
We are seeking an experienced Data Engineer with expertise in DBT (Data Build Tool) to join our dynamic and fast-growing team. In this role, you will be responsible for transforming critical data, specifically focusing on the creation and management of silver and gold data tiers within our data pipeline. Working closely with data architects and engineers, you will help design, develop, and optimize data transformation processes, ensuring that data is clean, reliable, and ready for business intelligence and analytics teams.
Key Responsibilities
- DBT-Based Data Transformations: Lead the design, development, and implementation of data transformations using DBT, with a focus on creating and managing silver and gold data tiers within the pipeline.
- Data Workflow Management: Oversee DBT workflows from data ingestion to transformation and final storage in optimized data models. Ensure seamless integration between DBT models and source systems.
- Integration with Data Adapters: Work with data adapters like AWS Glue, Amazon Athena, and Amazon Redshift to ensure smooth data flow and transformation across platforms.
- Data Quality & Optimization: Implement best practices to ensure data transformations are efficient, scalable, and maintainable. Optimize data models for query performance and reduced processing time.
- Cross-Functional Collaboration: Collaborate with data analysts, business intelligence teams, and data architects to understand data needs and deliver high-quality datasets for analytics and reporting.
- Documentation & Best Practices: Develop and maintain comprehensive documentation for DBT models, workflows, and configurations. Establish and enforce best practices in data engineering.
- Data Warehousing Concepts: Apply core data warehousing principles, including star schema, dimensional modeling, ETL processes, and data governance, to build efficient data pipelines and structures.
- DBT Expertise: Strong hands-on experience with DBT for data transformations and managing data models, including advanced DBT concepts like incremental models, snapshots, and macros.
- ETL and Cloud Integration: Proven experience with cloud data platforms, particularly AWS, and tools like AWS Glue, Amazon Athena, and Amazon Redshift for data extraction, transformation, and loading (ETL).
- Data Modeling Knowledge: Solid understanding of data warehousing principles, including dimensional modeling, star schemas, fact tables, and data governance.
- SQL Expertise: Proficient in writing and optimizing complex SQL queries for data manipulation, transformation, and reporting.
- Version Control: Experience with Git or similar version control systems for code management and collaboration.
- Data Orchestration: Familiarity with orchestration tools like Apache Airflow for managing ETL workflows.
- Data Pipeline Monitoring: Experience with monitoring and ing tools for data pipelines.
- Additional Tools: Familiarity with other data transformation tools or languages such as Apache Spark, Python, or Pandas is a plus.
Data build tool, PySpark
Similar Jobs
View All
1 Day ago
TrueFan - Senior Machine Learning Engineer
Information Technology
- Thiruvananthapuram, Kerala, India
About UsTrueFan is at the forefront of AI-driven content generation, leveraging cutting-edge generative models to build next-generation products. Our mission is to redefine content generation space through advanced AI technologies, including deep ge...
1 Day ago
Salesforce commerce cloud consultant
Information Technology
- Thiruvananthapuram, Kerala, India
Salesforce Commerce Cloud consultant 5+ Years of Experience 6 to 12 months Mode - Remote 1.1LPM - 1.2LPM Max Key Responsibilities Translate business requirements into scalable Salesforce Service Cloud solutions, in collaboration with CAE's technic...
1 Day ago
Cloud Infrastructure Engineer
Information Technology
- Thiruvananthapuram, Kerala, India
DescriptionInvent the future with us. Recognized by Fast Company’s 2023 100 Best Workplaces for Innovators List, Ampere is a semiconductor design company for a new era, leading the future of computing with an innovative approach to CPU design focuse...
1 Day ago
Devops Engineer- Intermetiate
Information Technology
- Thiruvananthapuram, Kerala, India
BackJD: Dev ops Engineer:As a DevOps Specialist- should be able to take ownership of the entire DevOps process, including Automated CI/CD pipelines and deployment to production.They should also be comfortable with risk analysis and prioritization.Le...
1 Day ago
Sr Data Scientist (London)
Information Technology
- Thiruvananthapuram, Kerala, India
AryaXAI stands at the forefront of AI innovation, revolutionizing AI for mission-critical, highly regulated industries by building explainable, safe, and aligned systems that scale responsibly. Our mission is to create AI tools that empower research...
1 Day ago
Software Test Engineer
Information Technology
- Thiruvananthapuram, Kerala, India
By clicking the “Apply” button, I understand that my employment application process with Takeda will commence and that the information I provide in my application will be processed in line with Takeda’s Privacy Notice and Terms of Use. I further att...
1 Day ago
Software Developer 5 (Java Fullstack)
Information Technology
- Thiruvananthapuram, Kerala, India
Job DescriptionBuilding off our Cloud momentum, Oracle has formed a new organization - Oracle Health Applications & Infrastructure. This team focuses on product development and product strategy for Oracle Health, while building out a complete platfo...
1 Day ago
Java Developer - Spring Frameworks
Information Technology
- Thiruvananthapuram, Kerala, India
Java DescriptionWe are looking for a passionate and talented Java Developer with 2-3 years of hands-on experience to join our growing development team.The ideal candidate should have a strong foundation in Java technologies and the ability to develo...
Talk to us
Feel free to call, email, or hit us up on our social media accounts.
Email
info@antaltechjobs.in