Bangalore, Karnataka, India
Information Technology
Full-Time
SolarSquare
Overview
Description
As a Data Engineer, you will own the end-to-end lifecycle of our data infrastructure.
You Will
Your impact will be measured by how effectively data is delivered, modeled, and leveraged to drive business outcomes.
Key Responsibilities
As a Data Engineer, you will own the end-to-end lifecycle of our data infrastructure.
You Will
- design and implement robust, scalable data pipelines and architect modern data solutions.
- using a best-in-class technology stack.
Your impact will be measured by how effectively data is delivered, modeled, and leveraged to drive business outcomes.
Key Responsibilities
- Architect & Build : Design, implement and manage cloud-based data platform using a modern ELT (Extract, Load, Transform) approach.
- Data Ingestion : Develop and maintain robust data ingestion pipelines from a variety of sources, including operational databases (MongoDB, RDS), real-time IoT streams, and third-party APIs using services like AWS Kinesis/Lambda or Azure Event Hubs/Functions.
- Data Lake Management : Build and manage a scalable and cost-effective data lake on AWS S3 or Azure Data Lake Storage (ADLS Gen2), using open table formats like Apache Iceberg or Delta Lake.
- Data Transformation : Develop, test, and maintain complex data transformation models using dbt.
- Champion a software engineering mindset by applying principles of version control (Git), CI/CD, and automated testing to all data logic.
- Orchestration : Implement and manage data pipeline orchestration using modern tools like Dagster, Apache Airflow, or Azure Data Factory.
- Data Quality & Governance : Establish and enforce data quality standards. Implement automated testing and monitoring to ensure the reliability and integrity of all data assets.
- Performance & Cost Optimization : Continuously monitor and optimize the performance and cost of the data platform, ensuring our serverless query engines and storage layers are used efficiently.
- Collaboration : Work closely with data analysts and business stakeholders to understand their needs, model data effectively, and deliver datasets that power our BI tools (Metabase, Power BI).
- 3+ years of professional experience in a data engineering role.
- Expert-level proficiency in SQL and the ability to write complex, highly-performant queries.
- Proficient in Python based data cleaning packages and tools.
- Experience in python is a must.
- Hands-on experience building data solutions on a major cloud provider (AWS or Azure), utilizing core services like AWS S3/Glue/Athena or Azure ADLS/Data Factory/Synapse.
- Proven experience building and maintaining data pipelines in Python.
- Experience with NoSQL databases like MongoDB, including an understanding of its data modeling, aggregation framework, and query patterns.
- Deep understanding of data warehousing concepts, including dimensional modeling, star/snowflake schemas, and data modeling best practices.
- Hands-on experience with modern data transformation tools, specifically dbt.
- Familiarity with data orchestration tools like Apache Airflow, Dagster, or Prefect.
- Proficiency with Git and experience working with CI/CD pipelines for data projects.
- Experience with real-time data streaming technologies, specifically AWS Kinesis or Azure Event Hubs.
- Experience with data cataloging and governance tools (i.e, OpenMetadata, DataHub, Microsoft Purview).
- Knowledge of infrastructure-as-code tools like Terraform or CloudFormation.
- Experience with containerization technologies (Docker, Kubernetes).
Similar Jobs
View All
1 Day ago
Java Developer
Information Technology
- Bangalore, Karnataka, India
Position: Java Developer Experience: 3+ yrsLocation: Baner, Pune Position Requirements BE/B.Tech or Similar (Technology/IT/Computer Science or related field). 3+ years relevant work experience designing and developing applications in high-paced env...
1 Day ago
Data Analyst - Senior Engineer
Information Technology
- Bangalore, Karnataka, India
Why Join Iris?Are you ready to do the best work of your career at one of India’s Top 25 Best Workplaces in IT industry? Do you want to grow in an award-winning culture that truly values your talent and ambitions?Join Iris Software — one of the faste...
1 Day ago
Software Developer- .Net C#
Information Technology
- Bangalore, Karnataka, India
Why Join Siemens?Siemens Energy is focused on helping customers navigate the worlds most pressing energy problems.As a world leader in developing and producing the most advanced engineering technologies, we improve lives and further human achievemen...
1 Day ago
Automation Tester
Information Technology
- Bangalore, Karnataka, India
We are seeking a skilled Automation Tester proficient in Java, Selenium WebDriver, and API testing. You will be responsible for developing and maintaining automation frameworks and scripts to improve testing efficiency and product quality. Key Respon...
1 Day ago
Power BI Technical Lead - Data Analytics
Information Technology
- Bangalore, Karnataka, India
DescriptionWe are seeking a highly skilled and experienced Power BI Tech Lead to drive the design, development, and deployment of business intelligence solutions. The ideal candidate will have deep expertise in Power BI, SQL, SSRS, and Report Builde...
1 Day ago
Field Network Engineer
Information Technology
- Bangalore, Karnataka, India
This job is provided by apna.co About Atria Convergence Technologies Ltd:ACT (Atria Convergence Technologies Ltd.) was born out of a vision to become the most admired in home entertainment, education, and interactive services provider in India. Toda...
1 Day ago
Data Scientist
Information Technology
- Bangalore, Karnataka, India
Job Description: We are seeking a highly motivated and enthusiastic Data Scientist withover 2-3 years of experience to join our dynamic team. The ideal candidate will have astrong background in AI/ML analytics and a passion for leveraging data to dr...
1 Day ago
Automation Test Engineer
Information Technology
- Bangalore, Karnataka, India
Key Responsibilities Design, develop, and maintain automated test cases using the Leapwork platform. Collaborate with QA, development, and product teams to understand testing requirements and develop comprehensive automation strategies. Analyze t...
Talk to us
Feel free to call, email, or hit us up on our social media accounts.
Email
info@antaltechjobs.in