Bangalore, Karnataka, India
Information Technology
Full-Time
Srijan Now Material
Overview
Senior/Lead Data Engineer - Databricks
We are seeking a highly skilled and experienced Data Engineering Lead with a strong background in the retail domain and exceptional programming abilities. As a Lead, you will play a pivotal role in implementing, and optimizing data architecture to support our retail business operations and analytics initiatives. Your expertise in Spark programming, optimization techniques, and familiarity with Databricks and CI/CD practices will be instrumental in ensuring the efficient and effective management of our data ecosystem.
Imp Pointers as below skills should be in the CV
Data Platform (Data Engineers)
We are seeking a highly skilled and experienced Data Engineering Lead with a strong background in the retail domain and exceptional programming abilities. As a Lead, you will play a pivotal role in implementing, and optimizing data architecture to support our retail business operations and analytics initiatives. Your expertise in Spark programming, optimization techniques, and familiarity with Databricks and CI/CD practices will be instrumental in ensuring the efficient and effective management of our data ecosystem.
Imp Pointers as below skills should be in the CV
Data Platform (Data Engineers)
- Data Lake: AWS S3 / Azure / Dell Cloud Data Lake with Delta Lake format
- Data Warehouse: Databricks, IOMETE, Google BigQuery for analytical workloads
- Batch Processing /Stream Processing: Batch/real-time processing of data pipelines
- Database: PostgreSQL for transactional data, NoSQL (ex. MongoDB, Cassendra) for document storage
- Design and develop data models, data integration processes, and data pipelines to capture, transform, and load structured and unstructured data from various retail sources.
- Hands-on programming in Spark to develop and optimize data processing applications and analytics workflows.
- Apply optimization techniques to enhance the performance and efficiency of data processing and analytical tasks.
- Evaluate and implement appropriate tools and technologies, including Databricks, to streamline data operations and ensure scalability and reliability.
- Work closely with other team members to ensure data integrity, consistency, and accessibility across the organization.
- Define and enforce best practices for data governance and data management, including data quality, metadata management, and data security.
- Collaborate with DevOps teams to establish and maintain CI/CD pipelines for data engineering and analytics workflows.
- Peer Review of team members' deliverables
- Stay updated with the latest advancements and trends in the retail domain, data architecture, and programming languages to drive continuous improvement.
- At least 5+ years of experience in the data engineering domain.
- Proven experience as a senior Data Engineer /Lead, preferably within the retail industry.
- Strong programming skills with expertise in PySpark programming and optimization techniques.
- Hands-on experience with Databricks, Deltalake and its components for data processing and analytics.
- Hands-on experience in data modelling, data integration, and ETL/ELT processes.
- Experience in working with Gitlab pipelines and an in-depth understanding of CI/CD pipeline designs.
- Experience with data governance, data quality, and metadata management.
- Strong analytical and problem-solving abilities with a detail-oriented mindset.
- Excellent communication and collaboration skills to work effectively with cross-functional teams.
- Ability to adapt to a fast-paced and evolving environment while managing multiple priorities.
- Good to have experience in at least one of the Cloud Vendor (AWS / Azure / GCP)
- Good to have experience with streaming technologies as well
Similar Jobs
View All
5 Hours ago
SDE 2 Backend
Finance & Banking
- 1800000 - 2200000 INR - Annual
- 2 - 4 Yrs
- Bangalore
Key Responsibilities
Design, develop, and maintain robust microservices in GoLang, with clean abstractions, high reliability, and readiness for scale.
Mentor junior engineers, review code, and help maintain a high bar for technical excellence...
6 Hours ago
IOS Developer
Fintech
- 3 - 8 Yrs
- Mumbai
Job Description: iOS Developer
Industry: Insurance
Location: On-site
Role Overview
As an iOS Developer , you will be responsible for designing,
developing, and optimizing robust, high-quality iOS applications that drive our insurance
products...
7 Hours ago
Senior Automation Systems Engineer
Internet
- 2500000 - 2600000 INR - Annual
- 6 - 8 Yrs
- Hyderabad
Job Description:
Key Responsibilities:
• Design, develop, and maintain robust automation frameworks and scripts primarily using Python.
• Lead efforts to automate testing processes (functional, regression, performance) and deployment pi...
22 Hours ago
Data Analyst - Product
Fintech
- 1 - 3 Yrs
- Bangalore
Data Analyst (Product)
Key Responsibilities
Own analytics for the Product Team, from defining metrics to building dashboards to surfacing insights.
Work with PMs & Designers to measure feature adoption, funnel performance, and user behavio...
22 Hours ago
Software Engineer
Information Technology
- 800000 - 1600000 INR - Annual
- 3 - 5 Yrs
- Telangana, Tamil Nadu
Job Description:
We are looking for experienced product development engineers/experts who could join our cloud product engineering team to build the next gen applications for our global customers. If you are a technology enthusiast and have passi...
23 Hours ago
Software Tester (Manual + Automation)
Information Technology
- 3 - 7 Yrs
- Anywhere in India/Multiple Locations
Job Opening: Software Tester (Manual + Automation) Contract Role
Position: Software Tester (Manual + Automation)
Engagement: Contract (69 Months)
Location: Remote
Experience Required: 3+ Years
Key Responsibilities:
Develop and execute test ...
1 Day ago
Senior Principal Software Engineer (Analytics)
Digital Transformation & Innovation
- 15 - 25 Yrs
- Bangalore
About the Role -
As a member of the development group, you will become part of a team that develops and maintains one of companys software products developed using Ruby and React, built as a multi-tenant SaaS solution on all Cloud Platforms like AWS...
1 Day ago
Data Architect
Information Technology
- Bangalore, Karnataka, India
About UsAbout DATAECONOMY: We are a fast-growing data & analytics company headquartered in Dublin with offices inDublin, OH, Providence, RI, and an advanced technology center in Hyderabad,India. We are clearly differentiated in the data & analytics ...
Talk to us
Feel free to call, email, or hit us up on our social media accounts.
Email
info@antaltechjobs.in