Thiruvananthapuram, Kerala, India
Information Technology
Full-Time
UsefulBI Corporation
Overview
Job Description
- Design, develop, and maintain scalable and reliable data pipelines and ETL/ELT processes on both AWS and Azure cloud platforms.
- Build and optimize data warehouses and data lakes leveraging services such as Amazon Redshift, Amazon S3, Azure Synapse Analytics, and Azure Data Lake Storage.
- Implement data ingestion, processing, and transformation workflows for various data sources, both on-premises and in the cloud.
- Utilize cloud-native data services on AWS (e.g, Glue, Lambda, Kinesis, SQS, SNS) and Azure (e.g, Data Factory, Functions, Event Hub, Service Bus, Logic Apps) to build efficient and cost-effective data solutions.
- Ensure data quality, integrity, and security across all data pipelines and storage systems.
- Collaborate with data scientists, analysts, and other engineers to understand data requirements and deliver effective data solutions.
- Monitor and troubleshoot data pipelines and data infrastructure performance.
- Implement and manage data governance policies and procedures.
- Stay up-to-date with the latest advancements in cloud data technologies and best practices on both AWS and Azure.
- Contribute to the development of data engineering standards and best practices within the team.
- Potentially lead and mentor junior data engineers.
- Bachelor's or Master's degree in Computer Science, Engineering, Data Science, or a related field.
- 5+ years of proven experience in Data Engineering.
- Strong hands-on experience with both Amazon Web Services (AWS) and Microsoft Azure cloud platforms.
- Deep understanding of data warehousing concepts and data modeling techniques.
- Extensive experience in designing and implementing ETL/ELT processes.
- Proficiency in SQL and experience with various database technologies (both relational and NoSQL).
- Strong programming skills in Python or Scala, with experience in data processing libraries (e.g, PySpark, Pandas).
- Experience with cloud-native data services on both AWS and Azure (mention specific services used).
- Understanding of data governance, data quality, and data security principles.
- Excellent problem-solving and analytical skills.
- Strong communication and collaboration skills.
- Experience with version control systems (e.g, Git).
- Experience with data orchestration tools (e.g, Apache Airflow, AWS Step Functions, Azure Logic Apps).
- Familiarity with big data technologies (e.g, Hadoop, Spark).
- Experience with data visualization tools (e.g, Tableau, Power BI).
- Knowledge of DevOps principles and CI/CD pipelines for data infrastructure.
- Relevant certifications (e.g, AWS Certified Data Engineer - Associate, Azure Data Engineer Associate)
Similar Jobs
View All
1 Day ago
TrueFan - Senior Machine Learning Engineer
Information Technology
- Thiruvananthapuram, Kerala, India
About UsTrueFan is at the forefront of AI-driven content generation, leveraging cutting-edge generative models to build next-generation products. Our mission is to redefine content generation space through advanced AI technologies, including deep ge...
1 Day ago
Salesforce commerce cloud consultant
Information Technology
- Thiruvananthapuram, Kerala, India
Salesforce Commerce Cloud consultant 5+ Years of Experience 6 to 12 months Mode - Remote 1.1LPM - 1.2LPM Max Key Responsibilities Translate business requirements into scalable Salesforce Service Cloud solutions, in collaboration with CAE's technic...
1 Day ago
Cloud Infrastructure Engineer
Information Technology
- Thiruvananthapuram, Kerala, India
DescriptionInvent the future with us. Recognized by Fast Company’s 2023 100 Best Workplaces for Innovators List, Ampere is a semiconductor design company for a new era, leading the future of computing with an innovative approach to CPU design focuse...
1 Day ago
Devops Engineer- Intermetiate
Information Technology
- Thiruvananthapuram, Kerala, India
BackJD: Dev ops Engineer:As a DevOps Specialist- should be able to take ownership of the entire DevOps process, including Automated CI/CD pipelines and deployment to production.They should also be comfortable with risk analysis and prioritization.Le...
1 Day ago
Sr Data Scientist (London)
Information Technology
- Thiruvananthapuram, Kerala, India
AryaXAI stands at the forefront of AI innovation, revolutionizing AI for mission-critical, highly regulated industries by building explainable, safe, and aligned systems that scale responsibly. Our mission is to create AI tools that empower research...
1 Day ago
Software Test Engineer
Information Technology
- Thiruvananthapuram, Kerala, India
By clicking the “Apply” button, I understand that my employment application process with Takeda will commence and that the information I provide in my application will be processed in line with Takeda’s Privacy Notice and Terms of Use. I further att...
1 Day ago
Software Developer 5 (Java Fullstack)
Information Technology
- Thiruvananthapuram, Kerala, India
Job DescriptionBuilding off our Cloud momentum, Oracle has formed a new organization - Oracle Health Applications & Infrastructure. This team focuses on product development and product strategy for Oracle Health, while building out a complete platfo...
1 Day ago
Java Developer - Spring Frameworks
Information Technology
- Thiruvananthapuram, Kerala, India
Java DescriptionWe are looking for a passionate and talented Java Developer with 2-3 years of hands-on experience to join our growing development team.The ideal candidate should have a strong foundation in Java technologies and the ability to develo...
Talk to us
Feel free to call, email, or hit us up on our social media accounts.
Email
info@antaltechjobs.in