Bangalore, Karnataka, India
Information Technology
Full-Time
Brillio
Overview
Lead Data Engineer
Primary Skills
Primary Skills
- IICS, Alation, Data Modelling Fundamentals, Data Warehousing, ETL Fundamentals, Modern Data Platform Fundamentals, PLSQL, T-SQL, Stored Procedures, Python, SQL, SQL (Basic + Advanced), Talend
- About the Role
- We are seeking a Senior Data Engineer with deep expertise in Google Cloud Platform (GCP) and BigQuery to lead cloud modernization initiatives, develop scalable data pipelines, and enable real-time data processing for enterprise-level systems. This is a high-impact role focused on driving the transformation of legacy infrastructure into a robust, cloud-native data ecosystem.
- Key Responsibilities
- 1. Data Migration & Cloud Modernization
- Analyze legacy on-premises and hybrid cloud data warehouse environments (e.g., SQL Server).
- Lead the migration of large-scale datasets to Google BigQuery.
- Design and implement data migration strategies ensuring data quality, integrity, and performance.
- 2. Data Integration & Streaming
- Integrate data from various structured and unstructured sources, including APIs, relational databases, and IoT devices.
- Build real-time streaming pipelines for large-scale ingestion and processing of IoT and telemetry data.
- 3. ETL / Data Pipeline Development
- Modernize and refactor legacy SSIS packages into cloud-native ETL pipelines.
- Develop scalable, reliable workflows using Apache Airflow, Python, Spark, and GCP-native tools.
- Ensure high-performance data transformation and loading into BigQuery for analytical use cases.
- 4. Programming & Query Optimization
- Write and optimize complex SQL queries, stored procedures, and scheduled jobs within BigQuery.
- Develop modular, reusable transformation scripts using Python, Java, Spark, and SQL.
- Continuously monitor and optimize query performance and cost efficiency in the cloud data environment.
- Required Skills & Experience
- 5+ years in Data Engineering with a strong focus on cloud and big data technologies.
- Minimum 2+ years of hands-on experience with GCP, specifically BigQuery.
- Proven experience migrating on-premise data systems to the cloud.
- Strong development experience with Apache Airflow, Python, and Apache Spark.
- Expertise in streaming data ingestion, particularly in IoT or sensor data environments.
- Strong SQL development skills; experience with BigQuery performance tuning.
- Solid understanding of cloud architecture, data modeling, and data warehouse design.
- Familiarity with Git and CI/CD practices for managing data pipelines.
- Preferred Qualifications
- GCP Professional Data Engineer certification.
- Experience with modern data stack tools like dbt, Kafka, or Terraform.
- Exposure to ML pipelines, analytics engineering, or DataOps/DevOps methodologies.
- Why Join Us?
- Work with cutting-edge technologies in a fast-paced, collaborative environment.
- Lead cloud transformation initiatives at scale.
Similar Jobs
View All
7 Hours ago
Engineering Manager
Information Technology
- 30 - 40 INR - Annual
- 10 - 12 Yrs
- Bangalore
We are hiring a hands-on Engineering Manager to lead our 100+ member engineering team across web, mobile, and backend (Shopify, Node.js, React.js, React Native). They will drive strategy, delivery, and team performance while working closely with the ...
12 Hours ago
Dot Net Developer
Finance & Banking
- 200000 - 300000 INR - Annual
- 2 - 6 Yrs
- Mumbai
Key Responsibilities:
Design, develop, and maintain WinForms applications.
Knowledge of .NET languages including C#, & Winforms is Mandatory. Collaborate with cross-functional teams to define, design, and implement new features. Write c...
12 Hours ago
Data Analyst
Information Technology
- 2 - 8 Yrs
- Delhi / NCR, Mumbai, Bangalore, Chennai, Jaipur, Lucknow, Hyderabad, Pune
Job description
Were hiring a Data Analyst to turn raw data into meaningful visualizations and reports.
Location - Delhi / Pune / Mumbai / Jaipur / Chennai / Lucknow / Hyderabad / Banglore ( Remote / Hybrid )
Key Responsibilities:
Analyze...
1 Day ago
Fullstack Developer .Net
Information Technology
- 1200000 - 1500000 INR - Annual
- 2 - 3 Yrs
- Chennai
Your Responsibilities:
Develop responsive, modular front-end components using React, TypeScript/JSX, and the MUI library.
Design and implement secure, scalable backend services with .NET 8 and C#.
Optimize SQL queries and wor...
1 Day ago
Business Analyst - Financial Services Domain
Information Technology
- Bangalore, Karnataka, India
Job Title : Business AnalystReporting StructureThe role is part of Project Management VerticalJob DescriptionWe are seeking an experienced Business Analyst with 1-3 years of experience with a background in financial services and in handling regulat...
1 Day ago
Data Engineer Job
Information Technology
- Pune, Maharashtra, India
We use cookies to offer you the best possible website experience. Your cookie preferences will be stored in your browser’s local storage. This includes cookies necessary for the website's operation. Additionally, you can freely decide and change any...
1 Day ago
Software Engineer, Cyber Security Embedded
Information Technology
- Pune, Maharashtra, India
The Impact You'll MakeDanfoss Power Solutions designs and manufactures a complete range of engineered components and systems. From hydraulics and electrification to fluid conveyance, electronic controls, and software, our solutions are engineered wi...
1 Day ago
Business Analyst
Information Technology
- Bangalore, Karnataka, India
Lead process discovery and analysis of the current-state RFI process across compliance, business, and operational teams. Design an optimized future-state RFI process, aligned with regulatory expectations, internal risk appetite, and digital bank o...
Talk to us
Feel free to call, email, or hit us up on our social media accounts.
Email
info@antaltechjobs.in