Senior AWS Data Pipeline Engineer

Who We Are

At Kyndryl, we design, build, manage and modernize the mission-critical technology systems that the world depends on every day. So why work at Kyndryl? We are always moving forward – always pushing ourselves to go further in our efforts to build a more equitable, inclusive world for our employees, our customers and our communities.

The Role

Are you ready to dive headfirst into the captivating world of data engineering at Kyndryl? As an AWS Data Engineer, you'll be the visionary behind our data platforms, crafting them into powerful tools for decision-makers. Your role? Ensuring a treasure trove of pristine, harmonized data is at everyone's fingertips.
In this role, you'll be engineering the backbone of our data infrastructure, ensuring the availability of pristine, refined data sets. With a well-defined methodology, critical thinking, and a rich blend of domain expertise, consulting finesse, and software engineering prowess, you'll be the mastermind of data transformation.

Key Responsibilities:

1. Data Pipeline Design & Development

• Design and develop scalable, resilient, and secure ETL/ELT data pipelines using AWS services.

• Build and optimize data workflows leveraging AWS Glue, EMR, Lambda, and Step Functions.

• Implement batch and real-time data ingestion using Kafka, Kinesis, or AWS Data Streams.

• Ensure efficient data movement across S3, Redshift, DynamoDB, RDS, and Snowflake.

2. Cloud Data Engineering & Storage

• Architect and manage data lakes and data warehouses using Amazon S3, Redshift, and Athena.

• Optimize data storage and retrieval using Parquet, ORC, Avro, and columnar storage formats.

• Implement data partitioning, indexing, and query performance tuning.

• Work with NoSQL databases (DynamoDB, MongoDB) and relational databases (PostgreSQL, MySQL, Aurora).

3. Infrastructure as Code (IaC) & Automation

• Deploy and manage AWS data infrastructure using Terraform, AWS CloudFormation, or AWS CDK.

• Implement CI/CD pipelines for automated data pipeline deployments using GitHub Actions, Jenkins, or AWS CodePipeline.

• Automate data workflows and job orchestration using Apache Airflow, AWS Step Functions, or MWAA.

4. Performance Optimization & Monitoring

• Optimize Spark, Hive, and Presto queries for performance and cost efficiency.

• Implement auto-scaling strategies for AWS EMR clusters.

• Set up monitoring, logging, and alerting with AWS CloudWatch, CloudTrail, and Prometheus/Grafana.

5. Security, Compliance & Governance

• Implement IAM policies, encryption (AWS KMS), and role-based access controls.

• Ensure compliance with GDPR, HIPAA, and industry data governance standards.

• Monitor data pipelines for security vulnerabilities and unauthorized access.

6. Collaboration & Stakeholder Engagement

• Work closely with data analysts, data scientists, and business teams to understand data needs.

• Document data pipeline designs, architecture decisions, and best practices.

• Mentor and guide junior data engineers on AWS best practices and optimization techniques.

Your journey begins by understanding project objectives and requirements from a business perspective, converting this knowledge into a data puzzle. You'll be delving into the depths of information to uncover quality issues and initial insights, setting the stage for data excellence. But it doesn't stop there. You'll be the architect of data pipelines, using your expertise to cleanse, normalize, and transform raw data into the final dataset—a true data alchemist.

So, if you're a technical enthusiast with a passion for data, we invite you to join us in the exhilarating world of data engineering at Kyndryl. Let's transform data into a compelling story of innovation and growth.

Your Future at Kyndryl
Every position at Kyndryl offers a way forward to grow your career. We have opportunities that you won’t find anywhere else, including hands-on experience, learning opportunities, and the chance to certify in all four major platforms. Whether you want to broaden your knowledge base or narrow your scope and specialize in a specific sector, you can find your opportunity here.

Who You Are

You’re good at what you do and possess the required experience to prove it. However, equally as important – you have a growth mindset; keen to drive your own personal and professional development. You are customer-focused – someone who prioritizes customer success in their work. And finally, you’re open and borderless – naturally inclusive in how you work with others.

Required Skills and Experience

• 7+ years of experience in data engineering with a focus on AWS cloud technologies.

• Expertise in AWS Glue, Lambda, EMR, Redshift, Kinesis, and Step Functions.

• Proficiency in SQL, Python, and PySpark for data transformations.

• Strong understanding of ETL/ELT best practices and data warehousing concepts.

• Experience with Apache Airflow or Step Functions for orchestration.

• Familiarity with Kafka, Kinesis, or other streaming platforms.

• Knowledge of Terraform, CloudFormation, and DevOps for AWS.

• Expertise in data mining, data storage, and Extract-Transform-Load (ETL) processes.

• Experience in data pipelines development and tooling, such as Glue, Databricks, Synapse, or Dataproc.

• Experience with both relational and NoSQL databases, including PostgreSQL, DB2, and MongoDB.

• Excellent problem-solving, analytical, and critical thinking skills.

• Ability to manage multiple projects simultaneously while maintaining attention to detail.

• Communication skills: Ability to communicate with both technical and non-technical colleagues to derive technical requirements from business needs and problems.

Preferred Skills and Experience

• Experience working as a Data Engineer and/or in cloud modernization.

• Experience with AWS Lake Formation and Data Catalog for metadata management.

• Knowledge of Databricks, Snowflake, or BigQuery for data analytics.
  

Being You

Diversity is a whole lot more than what we look like or where we come from, it’s how we think and who we are. We welcome people of all cultures, backgrounds, and experiences. But we’re not doing it single-handily: Our Kyndryl Inclusion Networks are only one of many ways we create a workplace where all Kyndryls can find and provide support and advice. This dedication to welcoming everyone into our company means that Kyndryl gives you – and everyone next to you – the ability to bring your whole self to work, individually and collectively, and support the activation of our equitable culture. That’s the Kyndryl Way.

What You Can Expect

With state-of-the-art resources and Fortune 100 clients, every day is an opportunity to innovate, build new capabilities, new relationships, new processes, and new value. Kyndryl cares about your well-being and prides itself on offering benefits that give you choice, reflect the diversity of our employees and support you and your family through the moments that matter – wherever you are in your life journey. Our employee learning programs give you access to the best learning in the industry to receive certifications, including Microsoft, Google, Amazon, Skillsoft, and many more. Through our company-wide volunteering and giving platform, you can donate, start fundraisers, volunteer, and search over 2 million non-profit organizations. At Kyndryl, we invest heavily in you, we want you to succeed so that together, we will all succeed.

Get Referred!

If you know someone that works at Kyndryl, when asked ‘How Did You Hear About Us’ during the application process, select ‘Employee Referral’ and enter your contact's Kyndryl email address.