Data Engineer

You will join the team as a Data Engineer, responsible for building and maintaining data pipelines, as well as developing data visualizations that drive company-wide analytics.

A strong understanding of good API design, data pipelining, and a commitment to data security is essential for this role. Proficiency in SQL is a must, as you'll regularly work with complex queries, data transformations, and optimizations to support scalable, high-performance data workflows.

Responsibilities:

Data Pipeline Development

• Design and implement scalable, production-grade data pipelines using PySpark and Python
• Develop ETL/ELT workflows within the AWS ecosystem, leveraging services like AWS Glue, Lambda, and Step Functions.
• Ingest and transform data from a variety of sources, including files, APIs, SQL databases (e.g., PostgreSQL, MySQL), NoSQL databases (e.g., DynamoDB, Cassandra) , events and streaming data
• Write robust unit and integration tests to validate pipeline logic and ensure data quality.
• Monitor, optimize, and troubleshoot pipeline performance, with logging and alerting for failures and delays

Data Lake Table Management (Hudi & Iceberg)

• Work with modern table formats like Apache Hudi and Apache Iceberg to enable incremental processing, upserts, and time-travel queries.
• Implement efficient data modelling strategies using these formats to support both batch and streaming data needs
• Optimize table partitioning, compaction, and schema evolution in large-scale data lake environments

Data Visualization & Reporting

• Create impactful dashboards and data visualizations using tools like QuickSight,Power BI, Tableau or similar tools
• Translate complex data into actionable insights for business stakeholders
• Provide support and training to stakeholders on accessing and using analytics tools and data assets

Collaboration & Stakeholder Engagement

• Partner with product, data scientists, and business teams to gather data requirements and deliver integrated solutions
• Translate business logic into efficient data transformations and visual outputs
• Optimize application performance for speed and responsiveness

Data Governance & Infrastructure

• Manage cloud-based data infrastructure (e.g., AWS Glue, Redshift, S3, EMR) ensuring security, reliability, and scalability
• Ensure compliance with data governance policies, privacy regulations, and access control standards
• Maintain proper data documentation, versioning, and lineage tracking.

Deployment & Release Management

• Design, implement, and maintain the build, deployment, and release process for data pipelines using AWS CloudFormation (CFN) or Terraform.
• Collaborate with team members to integrate code changes into CI/CD pipelines and ensure smooth deployment across multiple environments

Debugging, Collaboration & Growth

• Mentor and support junior developers by guiding them on creating and maintaining pipelines.
• Debug and troubleshoot cross-platform issues efficiently
• Collaborate with developers, DevOps, and stakeholders to deliver end-to-end features
• Stay updated on industry trends in the data and generative BI world

Knowledge, Education & Experience:

• Bachelor's or Master’s degree in Engineering in computer science or a related field (preferred)
• 3–5 years of experience Data engineering and pipeline development using Pyspark and python, SQL, No-SQL and understanding of modern development patterns
• Understanding and familiarity of modern concepts like data lake, lake house, open table formats
• Strong knowledge and experience to work with complex SQL queries
• Hands-on with AWS or similar cloud platforms; knowledge of cloud architecture (serverless)
• Experience with Git, CI/CD tools
• Basic experience in Microservice architecture, web services (Restful API’s).
• Experience with Redshift will be an added advantage.
• Strong problem-solving, analytical, communication, and collaboration skills
• Familiarity with Agile software development methodologies.

Why Join bswift?

At bswift, we empower our employees to make a meaningful impact, innovate, and grow. Joining our team means stepping into a collaborative and dynamic environment that values creativity, initiative, and a passion for client success. We are dedicated to fostering an inclusive workplace that celebrates diversity and values each team member’s unique contributions.

Benefits of Working at bswift:

• Comprehensive Health Benefits: Medical, Accidental and Term Life Insurance coverage to support your wellness and that of your family.
• Competitive Compensation: A compensation package that recognizes your skills, experience, and contributions, including performance-based incentives for most roles.
• Hybrid work-model: With flexible working hours
• Retirement Savings Plans: Options like Provident Fund and Gratuity to help you plan for a secure financial future with employer contribution
• Professional Development: Opportunities for career growth, including training and access to resources to support your career progression.
• Supportive Culture: A work environment that encourages collaboration, open communication, and creative problem-solving, where your voice and ideas are valued.
• Employee Wellbeing Initiatives: Programs focused on mental health, financial planning, and wellness resources to help you thrive inside and outside of work.

Make an Impact: At bswift, your work directly contributes to transforming how organizations approach benefits administration and client engagement. Join us to be part of an organization that is making a meaningful difference in the lives of our clients and their employees.