Associate Consultant / Manager Consulting Expert- Data Streaming Architect – Real-Time Replication

Overview

Position Description

Company Profile:

Founded in 1976, CGI is among the largest independent IT and business consulting services firms in the world. With 94,000 consultants and professionals across the globe, CGI delivers an end-to-end portfolio of capabilities, from strategic IT and business consulting to systems integration, managed IT and business process services and intellectual property solutions. CGI works with clients through a local relationship model complemented by a global delivery network that helps clients digitally transform their organizations and accelerate results. CGI Fiscal 2024 reported revenue is CA$14.68 billion and CGI shares are listed on the TSX (GIB.A) and the NYSE (GIB). Learn more at cgi.com.

Position: Associate Consultant / Manager Consulting Expert- Data Streaming Architect – Real-Time Replication

Experience: 10-16 years

Category: Software Development/ Engineering

Location: Chennai/Mumbai

Shift Timing-General Shift

Position ID: J1025-0514

Employment Type: Full Time

Education Qualification: Bachelor's degree in computer science or related field or higher with minimum 10 years of relevant experience.

We are seeking a Data Streaming Architect to lead the design and stabilization of large-scale replication pipelines into Snowflake. The source systems generate heavy transactional volumes that must be captured and replicated with strict SLAs.

You will be responsible for building and optimizing real-time CDC pipelines (Oracle LogMiner / SQL Server CDC / PostgreSQL CDC -> Kafka -> Snowflake) that deliver data with near real time, even under heavy workloads. This is a hands-on architecture role requiring deep technical expertise, operational maturity, and the ability to mentor engineers while engaging with cross-functional stakeholders.

Your future duties and responsibilities

• Architect, stabilize, and scale real-time CDC pipelines from Oracle, SQL Server, and PostgreSQL into Snowflake.
• Ensure replication consistently meets the near real time, even during high-volume transactional loads.
• Optimize Oracle LogMiner, SQL Server CDC, and PostgreSQL logical decoding / WAL-based replication for performance and integrity.
• Design and operate robust Kafka infrastructure (topics, partitions, replication, retention, lag management, schema registry).
• Configure and tune Kafka Connect / Debezium connectors and evaluate GoldenGate or other CDC tools where appropriate.
• Implement strong error handling, checkpointing, replay & backfill strategies to ensure fault-tolerance.
• Define and enforce schema evolution strategies with Avro, Protobuf, or JSON serialization.
• Collaborate with DBAs to manage redo logs, WAL files, and CDC configurations without impacting source systems.
• Design and optimize Snowflake ingestion (Snowpipe, streaming ingestion, staging) for high-throughput and low-latency delivery.
• Set up monitoring and observability: lag tracking, throughput, retries, failure alerts, anomaly detection.
• Lead capacity planning, scaling, and performance tuning to handle spikes gracefully.
• Establish best practices, governance, and technical standards for streaming data pipelines.
• Mentor engineers, review designs, and guide the team on modern streaming practices.
  
  

Must to have Skills-

• Experience: 10+ years in data engineering / architecture, with at least 3–5 years in real-time streaming and CDC replication.
• Databases: Deep expertise with Oracle redo logs / LogMiner / GoldenGate, SQL Server CDC, and PostgreSQL logical decoding / WAL.
• Streaming Platforms: Advanced hands-on with Apache Kafka, Kafka Connect, and Debezium.
• Snowflake Integration: Strong knowledge of Snowpipe, streaming ingestion, staging, clustering.
• Performance Engineering: Proven ability to tune pipelines to achieve 15 minutes replication lag consistently.
• Data Consistency: Strong in exactly-once semantics, idempotent writes, deduplication, ordering guarantees.
• Monitoring & Ops: Experience with Prometheus, Grafana, Confluent Control Center, JMX metrics, ELK/Splunk.
• Programming: Proficiency in Java/Scala or Python for custom connectors, orchestration, or automation.
• Cloud/Infra: Experience in AWS / Azure / GCP, containerization (Docker, Kubernetes), and infrastructure-as-code.
• Soft Skills: Problem-solving under SLA pressure, stakeholder communication, and team mentorship.
  
  

Good to have Skills-

• Stream processing engines (Flink, Kafka Streams, Spark Structured Streaming, ksqlDB).
• Confluent Enterprise features (Tiered Storage, Schema Registry, RBAC).
• Metadata/lineage tools and data governance/security exposure.
  
  

Life at CGI is rooted in ownership, teamwork, respect and belonging. Here, you’ll reach your full potential because…

You are invited to be an owner from day 1 as we work together to bring our Dream to life. That’s why we call ourselves CGI Partners rather than employees. We benefit from our collective success and actively shape our company’s strategy and direction.

Your work creates value. You’ll develop innovative solutions and build relationships with teammates and clients while accessing global capabilities to scale your ideas, embrace new opportunities, and benefit from expansive industry and technology expertise.

You’ll shape your career by joining a company built to grow and last. You’ll be supported by leaders who care about your health and well-being and provide you with opportunities to deepen your skills and broaden your horizons.

Come join our team—one of the largest IT and business consulting services firms in the world.