T&T | Cyber:D&R | AM | Cribl Data Engineer | Bengaluru

Key Responsibilities:

• Design, develop, and maintain ETL pipelines to process structured and unstructured data.
• Strong at Python coding.
• Work with large datasets to ensure efficient storage, retrieval, and processing.
• Develop and optimize SQL & NoSQL databases for performance and scalability.
• Collaborate with data analysts and business teams to understand reporting and analytical needs.
• Ensure data integrity, accuracy, and security across data platforms.
• Monitor data workflows and troubleshoot performance bottlenecks.
• Work with cloud-based data services (Azure, AWS/GCP a plus) for data processing and storage.
• Pyspark for Data extraction.
• Automate data pipelines using orchestration tools such as Azure Data Factory, Apache Airflow, or dbt.
• Implement data governance and best practices for data quality and compliance.

Required Skills & Qualifications:

• Strong expertise in ETL processes and data pipeline development
• Proficiency in SQL and NoSQL databases (PostgreSQL, MySQL, MongoDB, Cassandra, etc.)
• Experience with data modelling, indexing, and query optimization.
• Hands-on experience with big data processing frameworks (Apache Spark, Databricks, or Hadoop)
• Python (Must), Knowledge of Scala, or Java for data processing preferred.
• Experience with cloud-based data solutions (Azure Synapse, Azure Data Lake, or AWS/GCP equivalents)
• Familiarity with Power BI, Tableau, or other BI tools for reporting (Good to have)
• Experience with streaming technologies (Kafka, Kinesis, or Event Hubs) is a plus.
• Data engineering certifications, especially Microsoft Certified: Azure Data Engineer Associate (DP-203), AZ900 preferred.

Preferred Qualifications:

• Experience with Azure Data Factory, Databricks, or Synapse Analytics
• Knowledge of machine learning workflows (optional but a plus)
• Strong problem-solving and analytical skills
• Ability to communicate complex technical concepts clearly to non-technical stakeholders.