Senior AI & Data Engineer

We are seeking a Senior AI & Data Engineer who will be responsible for designing and maintaining robust data infrastructure, building machine learning pipelines (with a focus on Generative AI and LLMs), and creating insightful data visualizations. This role blends data engineering, data science, and ML engineering to support next-generation AI solutions and real-time analytics. The ideal candidate will have expertise in ETL processes, data warehousing, LLM-based models, and Power BI or similar reporting tools.

Key Responsibilities

Data Pipeline & Architecture

• Design, implement, and optimize ETL/ELT pipelines to handle large volumes of structured and unstructured data.
• Manage data ingestion, cleaning, and transformation for model training and business analytics.
• Maintain databases and data warehouses (e.g., Redshift, BigQuery, Snowflake) to ensure scalability and reliability.

Generative AI & LLM Development

• Train, fine-tune, and deploy Large Language Models (LLMs) for various use cases.
• Implement vector databases (e.g., FAISS, Pinecone, Milvus, Weaviate) to enable semantic search and retrieval-augmented generation.
• Collaborate with data scientists to experiment with cutting-edge techniques in generative AI (e.g., GPT, BERT, T5, etc.).

AI/ML Model Integration & MLOps

• Develop and maintain CI/CD pipelines for AI/ML model deployment and continuous monitoring.
• Automate model retraining, scaling, and version control (MLflow, Kubeflow, Airflow).
• Ensure high availability and performance of AI services in production environments.

Data Visualization & Reporting

• Create Power BI dashboards and reports to visualize complex data in a clear, actionable manner.
• Work with stakeholders to define KPIs and ensure data is accurately represented for business insights.
• Provide ad-hoc analyses and data storytelling to drive strategic decision-making.

Collaboration & Strategy

• Partner with cross-functional teams (data scientists, product managers, business analysts) to understand requirements and deliver end-to-end solutions.
• Define best practices for data governance, security, and compliance.
• Stay updated with emerging trends in AI, data engineering, and MLOps to drive innovation.

Qualifications (Must-Haves)

• 8-10 years of experience in data engineering, ML engineering, or a related field.
• Proficiency in SQL and at least one NoSQL database (e.g., MongoDB, Cassandra).
• Strong programming skills in Python (and/or Java, Scala) for data manipulation and ML model development.
• Demonstrated experience with ETL tools (Airflow, Talend, AWS Glue) and data warehousing solutions.
• Hands-on experience in ML model development, especially LLMs (Hugging Face Transformers, GPT-based architectures).
• Familiarity with vector databases for semantic search and generative AI applications.
• Knowledge of MLOps frameworks (MLflow, Kubeflow) and containerization (Docker, Kubernetes).
• Power BI (or similar visualization tools) proficiency for creating advanced dashboards and reports.
• Solid understanding of distributed computing (Spark, Hadoop) and cloud platforms (AWS, GCP, Azure).
• Excellent communication and collaboration skills to work in cross-functional, agile teams.

Nice-to-Have Skills

• Advanced knowledge of transformer architectures, prompt engineering, and fine-tuning strategies (e.g., RLHF).
• Experience with real-time streaming data pipelines (Kafka, Flink).
• Data governance and security best practices (GDPR, HIPAA, PCI DSS, etc.).
• Familiarity with data lake frameworks (Delta Lake, Apache Iceberg).
• Exposure to DevOps tools (Terraform, Ansible) for infrastructure automation.
• Open-source contributions or publications in AI/ML.

Why Join Us?

• Opportunity to pioneer next-generation AI solutions involving Generative AI and LLMs.
• Work in a fast-paced, innovative environment with a diverse and talented team.
• Access to cutting-edge tools and technologies for AI, data engineering, and analytics.
• Competitive compensation package, comprehensive benefits, and career growth opportunities.