Free cookie consent management tool by TermsFeed Interesting Job Opportunity: AWS Data Engineer - Python/PySpark | Antal Tech Jobs
Back to Jobs
1 Day ago

Interesting Job Opportunity: AWS Data Engineer - Python/PySpark

decor
Information Technology
Full-Time
AKIRA Insights

Overview

Key Responsibilities

  • Hands on experience in data related activities such as data parsing, cleansing quality definition data pipelines, storage and ETL scripts.
  • Expert knowledge in AWS Data Lake implementation and support (S3, Glue, DMS Athena, Lambda, API Gateway, Redshift).
  • Experiences in programming language Python/PySpark/SQL.
  • Experience with data migration with hands-on experience.
  • Experiences in consuming rest API using various authentication options with in AWS Lambda architecture.
  • Orchestrate triggers, debug and schedule batch job using a AWS Glue, Lambda and step functions.
  • Hands-on experience with Redshift, including data models, storage, and writing effective queries.
  • Understanding of AWS security features such as IAM roles and policies.
  • Knowledge of the Devops tools and CI/CD process.
  • AWS certification in AWS is highly Skills & Expertise :

Cloud (AWS) Expertise

S3 (Simple Storage Service) :

  • Data storage and partitioning, lifecycle policies, data replication, encryption at rest and transit, and data versioning, Data archive and Data sharing :
  • Creation of Lambdas, configuring event-driven functions, monitoring, and integration with other AWS services of s3,Glue,API Gateway, Redshift etc.

Glue

  • Creating & Managing ETL pipelines, Glue crawlers, job scheduling, and integration with S3, Redshift, and :
  • Creating Tables, Views and Stored procedures, parameter tuning, workload management, configuring triggers, Redshift Spectrum usage for querying S3 data.

API Gateway

  • Designing Restful APIs, securing endpoints using IAM or Cognito, throttling and logging API usage.

VPC (Virtual Private Cloud)

  • Aware of existing VPC design, subnets, NAT gateways, peering, routing, and network ACLs for services creation.

ELB (Elastic Load Balancer)

  • Configuring ALB/NLB for load distribution, health checks, sticky sessions, and SSL termination. CloudTrail :
  • Enabling auditing and compliance logging, managing trails, integrating with CloudWatch and third-party SIEM :
  • Knowledge about Sagemaker ML model training and deployment workflows, managing notebooks, endpoints,

and model & DevOps Tools : / GitHub Actions :

  • Managing version control, branching strategies, and automating workflows for testing and deployment.

Jenkins / CloudBees

  • Building pipelines for build-test-deploy automation, plugin management, parallel execution, and integrations with SCM and artifact :
  • Static code analysis, security vulnerability checks, technical debt reporting, integration into CI/CD.

DevOps/MLOps & AI/ML Awareness

  • Understanding of ML model lifecycle : data preprocessing, training, evaluation, deployment, and monitoring.
  • Experience supporting Data Scientists and ML Engineers in deploying models to production.
  • Familiarity with tools and workflows like SageMaker, MLflow, Airflow (optional), and pipelines integration

(ref:hirist.tech)
Share job
Similar Jobs
View All
1 Day ago
SDE III - Frontend Heavy
Internet
  • 3 - 5 Yrs
  • Anywhere in India/Multiple Locations
Candidates from Saas, Software Development, Product based companies only. About the Role: We are looking for an experienced software engineer with strong technical and communication skills who has developed full stack web apps, created solid AP...
decor
1 Day ago
VAYUZ Technologies - Python Developer - Django/Flask
Information Technology
Job DescriptionRole Expectations : Provide technical leadership and guidance to a team of Python developers. Mentor junior developers in best practices, coding standards, and problem-solving techniques. Participate in code reviews to ensure code ...
decor
1 Day ago
Go (Golang) Software Engineer, Developer Tooling and Containers
Information Technology
Canonical is a leading provider of open source software and operating systems to the global enterprise and technology markets. Our platform, Ubuntu, is very widely used in breakthrough enterprise initiatives such as public cloud, data science, AI, e...
decor
1 Day ago
Go (Golang) Software Engineer, Developer Tooling and Containers
Information Technology
Canonical is a leading provider of open source software and operating systems to the global enterprise and technology markets. Our platform, Ubuntu, is very widely used in breakthrough enterprise initiatives such as public cloud, data science, AI, e...
decor
1 Day ago
Security Consultant (Operational Technology Security)
Information Technology
Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diver...
decor
1 Day ago
Go (Golang) Software Engineer, Developer Tooling and Containers
Information Technology
Canonical is a leading provider of open source software and operating systems to the global enterprise and technology markets. Our platform, Ubuntu, is very widely used in breakthrough enterprise initiatives such as public cloud, data science, AI, e...
decor
1 Day ago
Senior Software Engineer - Digital Workplace
Information Technology
Canonical is a leading provider of open source software and operating systems to the global enterprise and technology markets. Our platform, Ubuntu, is very widely used in breakthrough enterprise initiatives such as public cloud, data science, AI, e...
decor
1 Day ago
Software Engineer - Solutions Engineering
Information Technology
Canonical is a leading provider of open source software and operating systems to the global enterprise and technology markets. Our platform, Ubuntu, is very widely used in breakthrough enterprise initiatives such as public cloud, data science, AI, e...
decor

Talk to us

Feel free to call, email, or hit us up on our social media accounts.
Social media