Big Data Engineer

Overview

Job Summary

We are looking for a Big Data Engineer with 3–4 years of experience to design, build, and maintain large-scale data processing systems. The ideal candidate will have hands-on experience with big data technologies, strong programming skills, and a deep understanding of data pipelines and distributed systems.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL processes
• Work with large, complex datasets in distributed environments
• Implement data processing solutions using tools like Hadoop, Spark, or Kafka
• Optimize data workflows for performance, scalability, and reliability
• Collaborate with data scientists and analysts to support data needs
• Ensure data quality, integrity, and governance across systems
• Integrate data from multiple sources including APIs and databases
• Monitor and troubleshoot data pipeline issues

Required Skills & Qualifications

• 3–4 years of experience in big data engineering or data engineering
• Strong proficiency in Python, Java, or Scala
• Hands-on experience with Apache Spark, Hadoop, or similar frameworks
• Experience with data streaming tools like Kafka or Flink
• Strong SQL skills and experience with relational and NoSQL databases
• Knowledge of ETL processes and data warehousing concepts
• Familiarity with cloud platforms (AWS, Azure, or GCP)
• Understanding of distributed systems and data architecture

Preferred Qualifications

• Experience with workflow orchestration tools like Airflow
• Knowledge of data lake architectures and modern data stack
• Experience with containerization (Docker, Kubernetes)
• Exposure to real-time data processing systems