Chennai, TN, India
Information Technology
Full-Time
Bridgenext Digital Engineering
Overview
Company Overview:
Don't just work, thrive. At Bridgenext, you have an opportunity to make a real difference - driving tangible business value for clients, while simultaneously propelling your own career growth. Our flexible and inclusive work culture provides you with the autonomy, resources, and opportunities to succeed. Position Description:
Key Responsibilities:
Preferred Skills:
Professional Skills:
Bridgenext is an Equal Opportunity Employer
Bridgenext is a digital consulting services leader that helps clients innovate with intention and realize their digital aspirations by creating digital products, experiences, and solutions around what real people need. Our global consulting and delivery teams facilitate highly strategic digital initiatives through digital product engineering, automation, data engineering, and infrastructure modernization services, while elevating brands through digital experience, creative content, and customer data analytics services.
Don't just work, thrive. At Bridgenext, you have an opportunity to make a real difference - driving tangible business value for clients, while simultaneously propelling your own career growth. Our flexible and inclusive work culture provides you with the autonomy, resources, and opportunities to succeed.
As a Lead Data Engineer specializing in Databricks, you will be a key player in designing, developing, and optimizing our enterprise transportation client's next-generation data platform. You will lead a team of data engineers, providing technical guidance, mentorship, and ensuring the scalable, and high-performance data solutions.
Key Responsibilities:
- Technical Leadership:
- Lead the design, development, and implementation of scalable and reliable data pipelines using Databricks, Spark, and other relevant technologies
- Define and enforce data engineering best practices, coding standards, and architectural patterns
- Provide technical guidance and mentorship to junior and mid-level data engineers
- Conduct code reviews and ensure the quality, performance, and maintainability of data solutions
- Databricks Expertise:
- Architect and implement data solutions on the Databricks platform, including Databricks Lakehouse, Delta Lake, and Unity Catalog
- Optimize Spark workloads for performance and cost efficiency on Databricks
- Develop and manage Databricks notebooks, jobs, and workflows
- Proficiently use Databricks features such as Delta Live Tables (DLT), Photon, and SQL Analytics
- Pipeline Development & Operations:
- Develop, test, and deploy robust ETL/ELT pipelines for data ingestion, transformation, and loading from various sources (e.g., relational databases, APIs, streaming data)
- Implement monitoring, alerting, and logging for data pipelines to ensure operational excellence
- Troubleshoot and resolve complex data-related issues
- Collaboration & Communication:
- Work closely with cross-functional teams including product managers, data scientists, and software engineers
- Communicate complex technical concepts clearly to both technical and non-technical stakeholders
- Stay updated with industry trends and emerging technologies in data engineering and Databricks
Must Have Skills:
- Experience:
- 8+ years of experience in data engineering, with at least 2-3 years in a lead or senior capacity
- Proven experience designing and building large-scale data platforms
- Primary Skills - Databricks:
- Extensive hands-on experience with Databricks platform, including Databricks Workspace, Spark on Databricks, Delta Lake, and Unity Catalog
- Strong proficiency in optimizing Spark jobs and understanding Spark architecture
- Experience with Databricks features like Delta Live Tables (DLT), Photon, and Databricks SQL Analytics
- Programming:
- Expertise in Python (PySpark) is essential
- Strong proficiency in SQL
- Data Warehousing/Lakes:
- Deep understanding of data warehousing concepts, dimensional modeling, and data lake architectures
- Experience with various data storage formats (Parquet, ORC, JSON, CSV)
- Cloud Platforms:
- Experience with at least one major cloud platform (AWS, Azure, or GCP) and their data-related services (e.g., S3, ADLS, GCS, EC2, Azure VMs, Google Compute Engine)
- Tools & Technologies:
- Experience with version control systems (Git)
- Familiarity with CI/CD pipelines for data solutions
- Knowledge of workflow orchestration tools (e.g., Apache Airflow, Databricks Workflows)
- Education:
- Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field
Preferred Skills:
- Experience with stream processing technologies (e.g., Kafka, Kinesis) is a plus
- Contributions to open-source data projectsExperience with Scala or Java is a plus
- Databricks certifications (e.g., Databricks Certified Data Engineer Associate/Professional)
- Experience with MLOps and integrating data pipelines with machine learning workflows
Professional Skills:
- Excellent problem-solving and analytical skills
- Strong leadership, communication, and interpersonal skills
- Ability to work independently and as part of a team in a fast-paced environment
Bridgenext is an Equal Opportunity Employer
Similar Jobs
View All
1 Day ago
Network Engineer (WLAN / Switching / Software)
Information Technology
- Chennai, TN, India
Network Engineer (WLAN / Switching / Software)This role has been designed as ‘Hybrid’ with an expectation that you will work on average 2 days per week from an HPE office. Who We Are: Hewlett Packard Enterprise is the global edge-to-cloud company adv...
3 Days ago
Senior Web UI Developer - HTML, JavaScript, React.js
Information Technology
- Chennai, TN, India
Chennai, Tamil Nadu Job Summary The Senior React.js Developer is responsible for developing, enhancing, and maintaining high-quality web applications that meet both client requirements and organizational standards. This role plays a crucial part in ...
3 Days ago
AWS Devops Engineer
Information Technology
- Chennai, TN, India
Choosing Capgemini means choosing a company where you will be empowered to shape your career in the way you’d like, where you’ll be supported and inspired by a collaborative community of colleagues around the world, and where you’ll be ab...
3 Days ago
Senior Automation Test Lead Embedded
Information Technology
- Chennai, TN, India
Lucknow, Uttar Pradesh Job Summary The Senior Test Lead will be responsible for overseeing the testing activities related to Testing Tools and Test Automation (EMB), selenium, java. The primary objective will be to ensure the quality and efficiency o...
3 Days ago
Technical Lead-Cloud & Infra Engg
Information Technology
- Chennai, TN, India
Country/Region: IN Requisition ID: 37003 Work Model: Position Type: Salary Range: Location: INDIA - CHENNAI - BIRLASOFT OFFICE Title: Technical Lead-Cloud & Infra Engg Description: Area(s) of responsibility Architecture & Solution Design Arch...
3 Days ago
IT System Administrator Technical Specialist
Information Technology
- Chennai, TN, India
About Us Ribbon Communications (Nasdaq: RBBN) delivers communications software, IP and optical networking solutions to service providers, enterprises and critical infrastructure sectors globally. We engage deeply with our customers, helping them mode...
3 Days ago
Senior MLOps Technical Lead
Information Technology
- Chennai, TN, India
Noida, Uttar Pradesh Job Summary Overview Senior‑level role on a five‑person engineering team building a production‑grade healthcare conversational GenAI platform. The stack centers on Python 3.12+ , FastAPI , and Google ADK 2.0 as the primary multi...
3 Days ago
Senior Selenium Automation Tester - Cucumber, Java
Information Technology
- Chennai, TN, India
Hyderabad, Telangana Job Summary The senior automation tester with expertise in cucumber, selenium, and Java will be responsible for developing, and executing automated tests to ensure the quality of software applications. This role will involve coll...
Talk to us
Feel free to call, email, or hit us up on our social media accounts.
Email
info@antaltechjobs.in