Free cookie consent management tool by TermsFeed Senior Data Engineer | Antal Tech Jobs
Back to Jobs
1 Day ago

Senior Data Engineer

decor
Karve, MH, India
Information Technology
Full-Time
Darkroom

Overview

What we're building

We're empowering small teams with technology that makes it easier to market and grow businesses. Our current focus is to help consumer brands shift from "workflow automation" to "agent management" within their marketing operations. Shadow is the AI coordination layer — providing shared AI memory, centralized agent control, and model orchestration for marketing teams.

Why join Shadow?

  • Product Ownership You'll ship production code daily and help steer key product and technical decisions.

  • Shape the Engineering Culture You'll influence how we work—tools, processes, standards, and hiring.

  • Work with Challenger Consumer Brands Talk directly to customers (CEOs, CMOs, VP's) of fast-growing consumer brands—some doing $80M–$500M in revenue.

The agency behind the product

Shadow is built alongside Darkroom — a performance marketing agency that's been operating for 10 years, employs 100+ people, runs 100+ clients at a time, and has worked with over 1,000 consumer brands. The agency is both our proving ground and our first user, which means the data you build with is real marketing data at real volume from day one — not a synthetic demo.

The role

You own the pipelines that bring the world's marketing data into Shadow — and keep them fast, accurate, and reliable as we scale to thousands of users. Every brand connects its full stack (ad platforms, ecommerce, analytics, email/SMS), and you make that data land cleanly, normalize into shared schemas, and stay in sync. The agent is only as good as the data underneath it; that layer is yours.

This is a hands-on, build-heavy engineering role for someone who has run large data systems before and wants to do it again in a smaller, faster environment.

What you'll own

  • Build and scale the ingestion layer across third-party marketing APIs (Meta, Google, TikTok, GA4, Shopify, Klaviyo, and more) — auth, extraction, rate-limit handling, backfill, and incremental sync.

  • Design normalization and transformation pipelines that map messy, platform-specific data into shared, queryable schemas (e.g. a unified creative/campaign/order model).

  • Own data reliability at scale — sync accuracy, freshness, coverage, and observability. Build the systems that detect when a connection breaks or a number looks wrong before a user does.

  • Engineer for multi-tenant scale and security: pipelines and storage that stay performant and cost-efficient across 1,000+ users and hundreds of connected brands — with strict data isolation, privacy, and compliance built in, not bolted on.

  • Partner with the AI and data-science teams to expose clean, well-modeled data the agent can retrieve and reason over.

Must haves

  • Experience building and operating large enterprise data pipelines engineered for scale — systems serving 1,000+ users (or equivalent data volume / tenancy), where reliability, isolation, and cost at scale were real constraints you solved.

  • Strong SQL and Python, with production experience in a modern data warehouse (BigQuery, Snowflake, Redshift, or similar).

  • Deep familiarity with ETL/ELT patterns, incremental sync, schema design, and data modeling for analytics.

  • Built and maintained integrations against third-party APIs — OAuth flows, pagination, rate limits, schema drift, and the operational reality of connectors that break.

  • A bias toward observability and data quality: you instrument your pipelines and you don't ship data you can't trust.

  • Experience building or operating within SOC 2-compliant systems with enterprise-grade security and privacy — you've handled sensitive customer data under real compliance constraints (access controls, encryption, data isolation, auditability) and treat it as a first-class engineering requirement.

Nice to haves

  • Experience in martech, adtech, or an adjacent data-heavy marketing domain — you've worked with ad platform or ecommerce data before and know where the bodies are buried (attribution windows, currency/timezone messes, deduping across platforms).

  • Familiarity with our stack: GCP (BigQuery, Cloud Run), PostgreSQL + pgvector, and orchestration/transformation tooling (dbt, Airflow, Dagster, or similar).

  • Experience with pipeline observability and tracing in an AI/LLM context (e.g. Langfuse).

  • Comfort supporting data that feeds AI agents and retrieval systems, not just dashboards.

Culture fit

  • Obsessive about data organization at scale. We're hiring for someone who lives in the data layer and wants to own it end to end.

  • You’re a power AI user. You've embedded AI into every workflow you touch and you think in systems — not one-off prompts, but repeatable structures that compound.

  • Entrepreneurial. You don't need much direction to move fast, you pivot when the situation demands it, and what you ship is production-grade, not a prototype you hand off for someone else to finish.

Share job
Similar Jobs
View All
1 Day ago
Senior Software Engineer - AI/ML Engineer AWS
Information Technology
  • Karve, MH, India
Founded in 1976, CGI is among the largest independent IT and business consulting services firms in the world. With 94,000 consultants and professionals across the globe, CGI delivers an end-to-end portfolio of capabilities, from strategic IT and busi...
decor
1 Day ago
Java Developer (Future Platform Architect)
Information Technology
  • 25000 - 40000 INR - Monthly
  • Karve, MH, India
About UsWe work on enterprise software for healthcare, laboratories, manufacturing, and digital transformation projects.Our current projects involve Java-based Laboratory Information Management Systems (LIMS), MIS platforms, integrations, and futur...
decor
1 Day ago
SSIS and Azure Data Engineer
Information Technology
  • 1000000 - 1800000 INR - Yearly
  • Karve, MH, India
This role is for one of our clients Industry: IT Services and IT Consulting Seniority level: Mid-Senior level Min Experience: 5+ years Location: Bengaluru, Pune, Hyderabad JobType: full-time Primary Duties and Responsibilities: Over 5 years o...
decor
1 Day ago
Cloud Architect AWS - Migration
Information Technology
  • Karve, MH, India
Presidio, Where Teamwork and Innovation Shape the Future At Presidio, we’re at the forefront of a global technology revolution, transforming industries through cutting-edge digital solutions and next-generation AI. We empower businesses—and their cus...
decor
1 Day ago
Mobile Software Developer
Information Technology
  • Karve, MH, India
L40 - Mobile Software Engineer II Role Overview: As a Lead Mobile Software Engineer, you will actively engage in your engineering craft, taking a hands-on approach to multiple high-visibility projects. Your expertise will be pivotal in delivering sol...
decor
1 Day ago
Python Developer Intern
Information Technology
  • 5000 - 8000 INR - Monthly
  • Karve, MH, India
Job Description: Python Developer InternAbout Us:Techquadra Software Solutions is a software development company that provides innovative and reliable IT solutions. We specialize in ERP systems, customized software, and mobile application developme...
decor
1 Day ago
Data Engineer
Information Technology
  • Karve, MH, India
Job Description: Data Engineers design and build data systems and pipelines. Responsibilities include developing data processing workflows, optimizing data storage, and ensuring data accuracy. You will collaborate with data scientists and analysts ...
decor
1 Day ago
Software Engineer - Android Application Development
Information Technology
  • Karve, MH, India
Location: IN_Bangalore_Sattva Knowledge Court Bdg_HII Job Family: Engineering Worker Type Reference: Regular - Permanent Pay Rate Type: Salary Career Level: T3(A) Job ID: R-53197-2026Description & Requirements Introduction: A Career at HARMAN Lifes...
decor

Talk to us

Feel free to call, email, or hit us up on our social media accounts.
Social media