Data Engineer

Overview

About the job:

Key Responsibilities:

A. Data pipeline development:
1. Build and maintain scalable ETL/ELT pipelines for structured and unstructured data.
2. Ingest data from diverse sources (APIs, streaming, batch systems).

B. Data modeling & warehousing:
1. Design efficient data models to support analytics and AI workloads.
2. Develop and optimize data warehouses/lakes using Redshift, BigQuery, Snowflake, or Delta Lake.

C. Big data & streaming:
1. Work with distributed systems like Apache Spark, Kafka, or Flink for real-time/large-scale data processing.
2. Manage feature stores for ML pipelines.

D. Collaboration & best practices:
1. Work closely with data scientists and ML engineers to ensure high-quality training data.
2. Implement data quality checks, observability, and governance frameworks.

Who can apply:

Only those candidates can apply who:

1. have minimum 1 years of experience
2. are from Hyderabad only

Salary:

₹ 4,00,000 - 5,00,000 /year

Experience:

1 year(s)

Deadline:

2025-10-19 23:59:59

Skills required:

Python, SQL, Microsoft Azure, Data Warehousing, Amazon Web Services (AWS), Apache Kafka , Google Cloud Platforms (GCP), Quality Assurance/Quality Control (QA/QC), Apache Spark, Machine Learning Operations (MLOps), Generative AI Tools and LLMOps

Other Requirements:

1. Education: Bachelor’s/Master’s in computer science, data engineering, or related field.

2. Experience: 0–1 years in data engineering with expertise in:

3. Programming: Python/Scala/Java (Python preferred).

4. Big Data & Processing: Apache Spark, Kafka, Hadoop.

5. Databases: SQL/NoSQL (Postgres, MongoDB, Cassandra).

6. Data Warehousing: Snowflake, Redshift, BigQuery, or similar.

7. Orchestration: Airflow, Luigi, or similar.

8. Cloud Platforms: AWS, Azure, or GCP (data services).

9. Version Control & CI/CD: Git, Jenkins, GitHub Actions.

10. MLOps/GenAI pipelines (feature engineering, embeddings, vector DBs).

About Company:

Soothsayer Analytics is a company that provides AI solutions for various industries. They have worked on projects such as vehicle scheduling and route optimization, vision-based quality inspection for a leading appliance manufacturer, and cognitive automation of resume assessment.