Free cookie consent management tool by TermsFeed Data Engineer - Senior | Antal Tech Jobs
Back to Jobs
1 Week ago

Data Engineer - Senior

decor
Thiruvananthapuram, Kerala, India
Information Technology
Full-Time
Cummins West Africa Limited

Overview

Description

Although the role category specified in the GPP is Remote, the requirement is for Hybrid.

Key Responsibilities

  • Design and Automation : Deploy distributed systems for ingesting and transforming data from various sources (relational, event-based, unstructured).
  • Data Quality and Integrity : Implement frameworks to monitor and troubleshoot data quality and integrity issues.
  • Data Governance : Establish processes for managing metadata, access, and retention for internal and external users.
  • Data Pipelines : Build reliable, efficient, scalable, and quality data pipelines with monitoring and alert mechanisms using ETL/ELT tools or scripting languages.
  • Database Structure : Design and implement physical data models to optimize database performance through efficient indexing and table relationships.
  • Optimization and Troubleshooting : Optimize, test, and troubleshoot data pipelines.
  • Large Scale Solutions : Develop and operate large-scale data storage and processing solutions using distributed and cloud-based platforms (e.g., Data Lakes, Hadoop, Hbase, Cassandra, MongoDB, Accumulo, DynamoDB).
  • Automation : Use modern tools and techniques to automate common, repeatable, and tedious data preparation and integration tasks.
  • Infrastructure Renovation : Renovate data management infrastructure to drive automation in data integration and management.
  • Agile Development : Ensure the success of critical analytics initiatives using agile development technologies such as DevOps, Scrum, Kanban.
  • Team Development : Coach and develop less experienced team members.

Responsibilities

Qualifications:

  • College, university, or equivalent degree in a relevant technical discipline, or equivalent experience required. Licensing may be required for compliance with export controls or sanctions regulations.

Competencies

  • System Requirements Engineering : Translate stakeholder needs into verifiable requirements; establish acceptance criteria; track requirements status; assess impact of changes.
  • Collaboration : Build partnerships and work collaboratively to meet shared objectives.
  • Communication : Develop and deliver communications that convey a clear understanding of the unique needs of different audiences.
  • Customer Focus : Build strong customer relationships and deliver customer-centric solutions.
  • Decision Quality : Make good and timely decisions to keep the organization moving forward.
  • Data Extraction : Perform ETL activities from various sources using appropriate tools and technologies.
  • Programming : Create, write, and test computer code, test scripts, and build scripts to meet business, technical, security, governance, and compliance requirements.
  • Quality Assurance Metrics : Apply measurement science to assess solution outcomes using ITOM, SDLC standards, tools, metrics, and KPIs.
  • Solution Documentation : Document information and solutions to enable improved productivity and effective knowledge transfer.
  • Solution Validation Testing : Validate configuration item changes or solutions using SDLC standards, tools, and metrics.
  • Data Quality : Identify, understand, and correct data flaws to support effective information governance.
  • Problem Solving : Solve problems using systematic analysis processes; implement robust, data-based solutions; prevent problem recurrence.
  • Values Differences : Recognize the value of different perspectives and cultures.

Qualifications

Skills:

  • ETL/Data Engineering Solution Design and Architecture : Expert level.
  • SQL and Data Modeling : Expert level (ER Modeling and Dimensional Modeling).
  • Team Leadership : Ability to lead a team of data engineers.
  • MSBI (SSIS, SSAS) : Experience required.
  • Databricks (Pyspark) and Python : Experience required.
  • Additional Skills : Snowflake, Power BI, Neo4j (good to have).
  • Communication : Good communication skills.

Preferred Experience

  • 8+ years of overall experience.
  • 5+ years of relevant experience in data engineering.
  • Knowledge of the latest technologies and trends in data engineering.
  • Technologies : Familiarity with analyzing complex business systems, industry requirements, and data regulations.
  • Big Data Platform : Design and development using open source and third-party tools.
  • Tools : SPARK, Scala/Java, Map-Reduce, Hive, Hbase, Kafka.
  • SQL : Proficiency in SQL query language.
  • Cloud-Based Implementation : Experience with clustered compute cloud-based implementations.
  • Large File Movement : Experience developing applications requiring large file movement for cloud environments.
  • Analytical Solutions : Experience in building analytical solutions.
  • IoT Technology : Intermediate experience preferred.
  • Agile Software Development : Intermediate experience preferred.
Share job
Similar Jobs
View All
1 Day ago
TrueFan - Senior Machine Learning Engineer
Information Technology
  • Thiruvananthapuram, Kerala, India
About UsTrueFan is at the forefront of AI-driven content generation, leveraging cutting-edge generative models to build next-generation products. Our mission is to redefine content generation space through advanced AI technologies, including deep ge...
decor
1 Day ago
Salesforce commerce cloud consultant
Information Technology
  • Thiruvananthapuram, Kerala, India
Salesforce Commerce Cloud consultant  5+ Years of Experience 6 to 12 months Mode - Remote 1.1LPM - 1.2LPM Max Key Responsibilities Translate business requirements into scalable Salesforce Service Cloud solutions, in collaboration with CAE's technic...
decor
1 Day ago
Cloud Infrastructure Engineer
Information Technology
  • Thiruvananthapuram, Kerala, India
DescriptionInvent the future with us. Recognized by Fast Company’s 2023 100 Best Workplaces for Innovators List, Ampere is a semiconductor design company for a new era, leading the future of computing with an innovative approach to CPU design focuse...
decor
1 Day ago
Devops Engineer- Intermetiate
Information Technology
  • Thiruvananthapuram, Kerala, India
BackJD: Dev ops Engineer:As a DevOps Specialist- should be able to take ownership of the entire DevOps process, including Automated CI/CD pipelines and deployment to production.They should also be comfortable with risk analysis and prioritization.Le...
decor
1 Day ago
Sr Data Scientist (London)
Information Technology
  • Thiruvananthapuram, Kerala, India
AryaXAI stands at the forefront of AI innovation, revolutionizing AI for mission-critical, highly regulated industries by building explainable, safe, and aligned systems that scale responsibly. Our mission is to create AI tools that empower research...
decor
1 Day ago
Software Test Engineer
Information Technology
  • Thiruvananthapuram, Kerala, India
By clicking the “Apply” button, I understand that my employment application process with Takeda will commence and that the information I provide in my application will be processed in line with Takeda’s Privacy Notice and Terms of Use. I further att...
decor
1 Day ago
Software Developer 5 (Java Fullstack)
Information Technology
  • Thiruvananthapuram, Kerala, India
Job DescriptionBuilding off our Cloud momentum, Oracle has formed a new organization - Oracle Health Applications & Infrastructure. This team focuses on product development and product strategy for Oracle Health, while building out a complete platfo...
decor
1 Day ago
Java Developer - Spring Frameworks
Information Technology
  • Thiruvananthapuram, Kerala, India
Java DescriptionWe are looking for a passionate and talented Java Developer with 2-3 years of hands-on experience to join our growing development team.The ideal candidate should have a strong foundation in Java technologies and the ability to develo...
decor

Talk to us

Feel free to call, email, or hit us up on our social media accounts.
Social media