Mumbai, Maharashtra, India
Information Technology
Full-Time
OA Compserve Pvt. Ltd
Overview
We are seeking a highly skilled and results-driven Data Engineer with proven expertise in data engineering, reporting, and analytics, specializing in Palantir Foundry. The ideal candidate will have experience designing scalable data architectures, building robust pipelines, and enabling actionable insights through optimized data solutions.
This role requires hands-on technical excellence, a strong understanding of big data ecosystems, and the ability to work closely with cross-functional teams to translate business requirements into efficient technical implementations.
Key Responsibilities
This role requires hands-on technical excellence, a strong understanding of big data ecosystems, and the ability to work closely with cross-functional teams to translate business requirements into efficient technical implementations.
Key Responsibilities
- Design, develop, and maintain end-to-end data pipelines using Palantir Foundry, including the code repository, Contour for data analysis, health checks, and data expectations.
- Build and optimize PySpark-based ETL pipelines for large-scale structured and unstructured data processing.
- Manage data ingestion, transformation, and quality monitoring for high volume, high-velocity datasets.
- Design scalable and efficient data models aligning with business requirements and analytics needs.
- Implement data warehousing solutions and manage data lake / lakehouse architecture for analytics.
- Optimize Spark and PySpark pipelines for cost efficiency, scalability, and low latency.
- Leverage Hadoop/Spark ecosystems to handle distributed data processing effectively.
- Collaborate with analytics and BI teams to enable self-service analytics and robust reporting capabilities.
- Support financial reporting and valuations (e.g., IFRS17) where applicable.
- Define and implement data validation, profiling, and lineage tracking within Palantir Foundry.
- Establish best practices for data security, privacy, and compliance.
- Work closely with data scientists, analysts, and business teams to deliver actionable insights.
- Translate complex business problems into scalable, technical data solutions.
- 3+ years of hands-on experience in data engineering and big data ecosystems.
- Expertise in Palantir Foundry (pipelines, Contour, health checks, expectations).
- Strong programming skills in PySpark, Python, and SQL for data manipulation and analysis.
- Solid understanding of Hadoop, Spark, Kafka, and distributed data systems.
- Experience in data warehousing concepts (Snowflake, Redshift, BigQuery, or similar) is a plus.
- Exposure to cloud platforms (AWS, GCP, or Azure) and their big data services (Glue, EMR, Dataproc, Databricks).
- Experience in CI/CD for data pipelines (Git, Jenkins, or equivalent).
- Understanding of containerization & orchestration (Docker, Kubernetes, Airflow, or Dagster).
- Knowledge of real-time data streaming frameworks (Kafka, Flink, or Kinesis).
- Familiarity with data cataloging & governance tools (Alation, Collibra).
- Familiarity with TypeScript, HTML, CSS, JavaScript, and frameworks like React, Angular, D3.js, or Chart.js.
- Experience with modern visualization tools (Tableau, Power BI, Looker, or Superset).
- Exposure to financial services, risk modeling, or IFRS17 reporting is a strong plus.
- Bachelors or Masters degree in Mathematics, Statistics, Operations Research, Industrial Engineering, Computer Science, Econometrics, or Information Technology.
- Strong analytical and problem-solving skills.
- Excellent communication and stakeholder management abilities.
- Ability to work in agile, cross-functional teams with a high degree of ownership.
Similar Jobs
View All
1 Day ago
SAP S/4 HANA Platforms – Industrial (Analyst / Consultant)
Information Technology
- 1800000 - 2000000 INR - Annual
- 2 - 7 Yrs
- Mumbai, Bangalore, Chennai, Gurgaon / Gurugram, Delhi, Hyderabad, Pune
About the Job
Locations: Delhi, Gurgaon, Mumbai, Bangalore, Hyderabad, Pune, Chennai
Position: SAP S/4 HANA Platforms – Industrial (Analyst / Consultant)
Job Type: Full-time | Permanent
Salary: Up to ₹20 LPA + 23% Variable Pay
Experience: 2 – ...
1 Day ago
SAP Platforms Analyst / Consultant
Information Technology
- 1800000 - 2200000 INR - Annual
- 3 - 7 Yrs
- Mumbai, Bangalore, Gurgaon / Gurugram, Hyderabad
About the Job
Locations: Bangalore | Gurugram | Mumbai | Hyderabad
Position: SAP Platforms Analyst / Consultant
Job Type: Full-time | Permanent
Salary: Up to ₹20 LPA + 23% Variable Pay
About the Role
Our client, a major global consulti...
1 Day ago
Manager / Sr. Manager – SAP S/4 HANA Platforms (Industrial)
Information Technology
- 3500000 - 4000000 INR - Annual
- 7 - 15 Yrs
- Mumbai, Bangalore, Chennai, Delhi, Hyderabad, Pune, Gurugram
About the Job
Locations: Delhi | Gurugram | Mumbai | Bangalore | Hyderabad | Pune | Chennai
Position: SAP S/4 HANA Platforms – Industrial (Manager / Sr. Manager)
Job Type: Full-time | Permanent
Salary: Up to ₹40 LPA + 30% Variable Pay
Abou...
1 Day ago
Product Designer
Fintech
- 1800000 - 2500000 INR - Annual
- 3 - 8 Yrs
- Mumbai, Navi Mumbai, Thane
Job Description :-
Looking for potential candidates only from Product Based Organization who have implemented B2C projects.
● A Compelling Portfolio:
A link to your online portfolio is required for consideration. Your portfolio must showc...
1 Day ago
Technical Lead
Aerospace & Defense
- 12 - 20 Yrs
- Bangalore
We are hiring for Technical Lead, client based in Bangalore
Techanical Skills: Deep knowledge of Application Architecture, System Integration, Cloud computing, IoT architecture and business requirement standards.
· Experience with core systems ...
1 Day ago
Fullstack and Senior Full stack Developer
Information Technology
- 6 - 12 Yrs
- Bangalore
We are Hiring for NGO, who is based at Bangalore location " Full stack and Senior Full stack Developer"
Required Experience:
• Professional Experience: Minimum of 5 years in full-stack web and mobile application development roles.
• AI Appli...
1 Day ago
Junior AI Data Scientist
AI & Machine Learning Advancement
- 2 - 4 Yrs
- Pune
We are Hiring Junior Data Scientist for Pune Location
Required Skills & Qualifications
● Education: Bachelor’s or Master’s degree in Computer Science, Data Science, Artificial
Intelligence, or a related quantitative fieldanalyticsvidhya.com...
1 Day ago
TA head
Manufacturing & Industrial
- 12 - 18 Yrs
- Chennai
We are hiring for TA head position for manufacturing company based in Chennai:
Key Skills & Competencies
• Deep TA expertise across sales, technical, and leadership hiring.
• Strong experience in ATS implementation and lifecycle management.
...
Talk to us
Feel free to call, email, or hit us up on our social media accounts.
Email
info@antaltechjobs.in