Noida, Uttar Pradesh, India
Information Technology
Full-Time
Litmus7
Overview
- Design, develop, and maintain scalable and efficient big data processing pipelines distributed computing systems.
- Collaborate with cross-functional teams to understand data requirements and design appropriate data solutions.
- Implement data ingestion, processing, and transformation processes to support various analytical and machine learning use cases.
- Optimize and tune data pipelines for performance, scalability, and reliability.
- Monitor and troubleshoot pipeline performance issues, identifying and resolving bottlenecks.
- Ensure data quality and integrity throughout the pipeline, implementing data validation and error handling mechanisms.
- Stay updated on emerging technologies and best practices in big data processing and analytics, incorporating them into our data engineering practices.
- Document design decisions, technical specifications, and data workflows.
- Expertise in Data modelling, Data warehousing concepts, data governance best practices and ETL processes.
- Understanding of distributed computing principles and experience with distributed data processing frameworks like Apache Spark or Hadoop.
- Familiarity with containerization technologies like Docker and Orchestration tools like Kubernetes.
- Develop data models, schemas and structures to support business needs.
- Ensure data quality, integrity and security across all data solutions.
- Good understanding of a DW platform like Snowflake, RedShift etc.
Similar Jobs
View All
9 Hours ago
Senior Data Engineer
Information Technology
- 4 - 8 Yrs
- Bangalore
We are looking for a skilled and driven Data Engineer to join our dynamic technology team. As a Data Engineer, you will be responsible for building and maintaining robust, scalable, and high-performance data pipelines and platforms. This role involve...
1 Day ago
Senior Software Engineer - Python Developer
Information Technology
FactSet creates flexible, open data and software solutions for over 200,000 investment professionals worldwide, providing instant access to financial data and analytics that investors use to make crucial decisions.At FactSet, our values are the foun...
1 Day ago
Software Engineer - Angular/React
Information Technology
Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data a...
1 Day ago
IT Business Analyst (Foreign Exchange markets) – C11 (5-9 yrs) - Chennai
Information Technology
The IT Business Intermediate Analyst is an intermediate-level position responsible for liaising between business users and technologists to exchange information in a concise, logical and understandable way in coordination with the Technology team. T...
1 Day ago
Software Configuration Engineer
Information Technology
Project Role : Software Configuration EngineerProject Role Description : Implement the configuration management plan as directed by the Configuration Lead. Assist in the design of software configuration and customization to meet the business process...
1 Day ago
Software Developer 2
Information Technology
Job DescriptionDesign, develop, troubleshoot and debug software programs for databases, applications, tools, networks etc. As a member of the software engineering division, you will take an active role in the definition and evolution of standard pra...
1 Day ago
Data Engineer
Information Technology
DescriptionThe Data Engineer supports, develops, and maintains a data and analytics platform to efficiently process, store, and make data available to analysts and other consumers. This role collaborates with Business and IT teams to understand requ...
1 Day ago
Data Scientist II, Regulatory Intelligence, Safety, and Compliance (RISC)
Information Technology
DescriptionWe are seeking an exceptional Data Scientist to join a team of experts in the field of machine learning, and work together to tackle challenging problems across diverse compliance domains. We leverage risk models (including boosted trees ...
Talk to us
Feel free to call, email, or hit us up on our social media accounts.
Email
info@antaltechjobs.in