Data Engineer: Big Data & Cloud Solutions (MA48T RM 3058)

Position: Data Engineer (MA48T RM 3058)

Key Responsibilities:

Data Engineering & Data Science:
o Preprocess structured and unstructured data to prepare for AI/ML model development.
o Apply strong skills in feature engineering, data augmentation, and normalization techniques.
o Manage and manipulate data using SQL, NoSQL, and cloud-based data storage solutions such as Azure Data Lake.
o Design and implement efficient ETL pipelines, data wrangling, and data transformation strategies.

Model Deployment and MLOps:
o Deploy ML models into production using Azure Machine Learning (Azure ML) and Kubernetes.
o Implement MLOps best practices, including CI/CD pipelines, model versioning, and monitoring frameworks.
o Design mechanisms for model performance monitoring, alerting, and retraining.
o Utilize containerization technologies (Docker/Kubernetes) to support deployment and scalability.

Business & Analytical Insights:
o Work closely with stakeholders to understand business KPIs and decision-making frameworks.
o Analyse large datasets to identify trends, patterns, and actionable insights that inform business strategies.
o Develop data visualizations using tools like Power BI, Tableau, and Matplotlib to communicate insights effectively.
o Conduct A/B testing and evaluate model performance using metrics such as precision, recall, F1-score, MSE, RMSE, and model validation techniques.

Desired Profile:

Experience and Expertise:
o Proven experience in data engineering, AI/ML data preprocessing, and model deployment.
o Strong expertise in working with both structured and unstructured datasets.
o Hands-on experience with SQL, NoSQL databases, and cloud data platforms (e.g., Azure Data Lake).
o Deep understanding of MLOps practices, containerization (Docker/Kubernetes), and production-level model deployment.

Technical Skills:
o Proficient in ETL pipeline creation, data wrangling, and transformation methods.
o Strong experience with Azure ML, Kubernetes, and other cloud-based deployment technologies.
o Excellent knowledge of data visualization tools (Power BI, Tableau, Matplotlib).
o Expertise in model evaluation and testing techniques, including A/B testing and performance metrics.

Soft Skills:
o Strong analytical mindset with the ability to solve complex data-related problems.
o Ability to collaborate with cross-functional teams to understand business needs and provide actionable insights.
o Clear communication skills to convey technical details to non-technical stakeholders.

*******************************************************************************************************************************************