Senior Data Engineer - ETL/Python

Overview

• We are seeking a skilled Senior Data Engineer with strong experience in data architecture, pipeline development, and AI-driven engineering.
• The role focuses on building scalable data systems, integrating AI/LLM components, and collaborating across teams to deliver high-impact solutions.
  
  

• Design, develop, and optimize end-to-end data pipelines including ingestion, transformation, and orchestration.
• Build and maintain high-performance data models, ETL processes, and storage layers for large datasets.
• Integrate AI capabilities by implementing LLM workflows, embeddings, vector stores, and inference services.
• Collaborate with engineering, data, and product teams to ensure data quality, reliability, lineage, and security.
• Develop APIs, microservices, and interfaces that support analytics, reporting, and AI-driven features.
• Implement monitoring, logging, and automation frameworks for robust data operations.
• Conduct performance optimization of pipelines, queries, and data infrastructure.
• Evaluate and recommend modern tools and technologies in cloud, big data, and AI domains.
  
  

• 58+ years of hands-on experience as a Data Engineer or related role.
• Strong programming expertise in Python / TypeScript / SQL / Spark / Databricks.
• Experience with Jupyter Notebooks for data exploration, prototyping, and validation of AI/data workflows.
• Solid knowledge of cloud platforms (Azure / AWS / GCP), data lakes/lakehouses, and distributed computing.
• Practical experience integrating AI/LLM APIs, vector databases (Pinecone, Chroma, etc.), and RAG workflows.
• Strong understanding of data warehousing, schema design, metadata management, and large-scale processing.
• Proficiency in orchestration tools like Airflow / Prefect / Dagster.
• Familiarity with containerization and deployment using Docker / Kubernetes.
• Experience building CI/CD pipelines for data and AI systems.
• Strong analytical and problem-solving skills.
  
  

• Exposure to AI agent frameworks, LLM prompt automation, or retrieval pipelines.
• Understanding of enterprise security, RBAC, and compliance requirements.
• Prior experience working in large-scale enterprise data environments.
  
  

• Excellent communication and cross-functional collaboration skills.
• Ability to simplify complex technical concepts and convert them into actionable work.
• High ownership mindset and focus on delivering reliable, scalable solutions.
• Comfortable working independently as a high-impact individual contributor.
  
  

• You will contribute to the development of next-generation data and AI capabilities, enabling intelligence, automation, and insights across key business initiatives.
• This role offers significant opportunities for innovation and technical impact.
  
  

(ref:hirist.tech)