Bangalore, Karnataka, India
Information Technology
Full-Time
SecureKloud Technologies
Overview
About The Role
We are seeking a highly skilled Senior Data Engineer to join our dynamic team. In this role, you will be instrumental in designing, building, and maintaining scalable, efficient, and reliable data pipelines and solutions. You will work closely with cross-functional teams to enable data-driven decision-making by ensuring the availability, accuracy, and performance of data infrastructure.
Key Responsibilities
Cloud Platforms & Services :
We are seeking a highly skilled Senior Data Engineer to join our dynamic team. In this role, you will be instrumental in designing, building, and maintaining scalable, efficient, and reliable data pipelines and solutions. You will work closely with cross-functional teams to enable data-driven decision-making by ensuring the availability, accuracy, and performance of data infrastructure.
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT data pipelines for batch and near real-time processing using tools such as Talend, AWS Glue, Lambda, and SQS.
- Develop and optimize SQL queries and perform data modeling, cleansing, transformation, and validation to ensure data quality and accuracy.
- Write efficient, reusable, and scalable ETL scripts primarily using Python, PySpark, and SQL. Utilize AWS SDK for Python (Boto3) for automation and integration with AWS services.
- Collaborate with data analysts, data scientists, and business stakeholders to understand data requirements and deliver actionable insights.
- Build and orchestrate workflows using Talend and AWS Step Functions to automate complex data processes.
- Monitor and troubleshoot data pipelines and infrastructure using AWS CloudWatch and other monitoring tools to ensure optimal performance and availability.
- Implement best practices for data security, access control (using AWS IAM), and compliance in cloud environments.
- Participate in the design and architecture of data storage solutions using AWS services such as S3, Redshift, DynamoDB, and EC2.
- Stay current with emerging technologies and industry trends, recommending and implementing improvements to our data engineering practices.
- Mentor junior data engineers and contribute to team knowledge sharing.
Cloud Platforms & Services :
- AWS (Amazon Web Services) expertise with hands-on experience in :
- Data processing and querying : Athena, AWS Glue, Lambda
- Storage and databases : S3, Redshift, DynamoDB, EBS
- EC2, VPC
- SQS, SNS, Step Functions, API Gateway
- Security and monitoring : IAM, :
- QuickSight
- Python (including AWS SDK - Boto3)
- PySpark
- Java (preferred but not mandatory)
- SQL (advanced query optimization and tuning)
- Talend or similar ETL tools
- Apache Airflow or other orchestration tools for workflow :
- Bachelors or Masters degree in Computer Science, Engineering, Information Technology, or a related field.
- Minimum 7+ years of professional experience in data engineering or related roles.
- Proven experience designing and building complex data pipelines and data warehousing solutions on AWS.
- Strong proficiency in SQL and database technologies.
- Experience with big data processing frameworks and distributed systems.
- Excellent problem-solving skills and ability to work with large datasets.
- Strong communication skills and ability to work collaboratively in cross-functional teams
Similar Jobs
View All
18 Hours ago
Principal Engineer - UI/UX
Information Technology
- 6 - 11 Yrs
- Mumbai (All Areas)
looking for a technically savvy and experienced Principal Engineer to take up front-end development efforts. You will design and develop elegant interfaces that exceed client expectations in terms of value and benefit. You will collaborate on scalabi...
18 Hours ago
Principal Engineer - Beckend
Information Technology
- 7 - 11 Yrs
- Mumbai (All Areas)
Principal Engineer - Backend
Mumbai, Maharashtra, India | Product Engineering | Full-time
We are looking for a technically savvy and experienced senior developer to lead development efforts. You will help the team grow in size and skills, optim...
18 Hours ago
Principal Engineer - DevOps/DBA
Information Technology
- 7 - 11 Yrs
- Mumbai (All Areas)
we are looking for a technically savvy and passionate Principal DevOps Engineer or Senior Database Administrator to cater to the development and operations efforts in product. You will choose and deploy tools and technologies to build and support a r...
20 Hours ago
C++ Developer | 5 - 8 Years
Information Technology
- 30 - 45 INR - Yearly
- 5 - 12 Yrs
- Pune
About the Client:
The company has been a global leader in delivering cutting-edge in-flight entertainment and connectivity (IFEC) solutions for over 40 years.
About the Role:
Title:
SDE 3 C++ - 5 to 8 Years
SDE 4 C++ - 8 to 13 Years
...
1 Day ago
Software Engineer II - Java, Python, AWS, Pyspark, DPL
Finance & Banking
- Bangalore, Karnataka, India
hackajob is collaborating with J.P. Morgan to connect them with exceptional tech professionals for this role.Organization DescriptionOur Consumer & Community Banking division serves our Chase customers through a range of financial services, includin...
1 Day ago
Lead Software Engineer- Devportal Node fullstack
Finance & Banking
- Bangalore, Karnataka, India
Reference 25000GPGResponsibilities In-depth knowledge of UI development with React JS, Java script, Type script including its core principles and workflows. Expertise on backend development with Node Js and framework nest JS Familiarity with version...
1 Day ago
QA Engineer (Manual & Automation) in Delhi, Faridabad, Gurgaon, Noida (Hybrid)
Finance & Banking
- Bangalore, Karnataka, India
We are looking for a talented 'QA engineer (automation + manual)' to join our team at AppQuarterz Technologies! As a 'QA engineer', you will play a crucial role in ensuring the quality and performance of our software products.Key Responsibilities D...
1 Day ago
Software Engineer III - Infrastructure
Finance & Banking
- Bangalore, Karnataka, India
hackajob is collaborating with J.P. Morgan to connect them with exceptional tech professionals for this role.Job DescriptionEmpowering projects with innovative technical solutions, our team excels in optimizing data and systems at scale while ensuri...
Talk to us
Feel free to call, email, or hit us up on our social media accounts.
Email
info@antaltechjobs.in