Pune, Maharashtra, India
Information Technology
Full-Time
Go Digital Technology Consulting LLP
Overview
Location: Mumbai
Experience: 0-6months
Technologies / Skills: Advanced SQL, Python and associated libraries like Pandas, Numpy etc., Pyspark , Shell scripting, Data Modelling, Big data, Hadoop, Hive, ETL pipelines.
Responsibilities
Experience: 0-6months
Technologies / Skills: Advanced SQL, Python and associated libraries like Pandas, Numpy etc., Pyspark , Shell scripting, Data Modelling, Big data, Hadoop, Hive, ETL pipelines.
Responsibilities
- Proven success in communicating with users, other technical teams, and senior management to collect requirements, describe data modeling decisions and develop data engineering strategy.
- Ability to work with business owners to define key business requirements and convert to user stories with required technical specifications.
- Communicate results and business impacts of insight initiatives to key stakeholders to collaboratively solve business problems.
- Working closely with the overall Enterprise Data & Analytics Architect and Engineering practice leads to ensure adherence with the best practices and design principles.
- Assures quality, security and compliance requirements are met for supported area.
- Design and create fault-tolerance data pipelines running on cluster
- Excellent communication skills with the ability to influence client business and IT teams
- Should have design data engineering solutions end to end. Ability to come up with scalable and modular solutions
- 0-6months of hands-on experience Designing and developing Data Pipelines for Data Ingestion or Transformation using Python (PySpark)/Spark SQL in AWS cloud
- Experience in design and development of data pipelines and processing of data at scale.
- Advanced experience in writing and optimizing efficient SQL queries with Python and Hive handling Large Data Sets in Big-Data Environments
- Experience in debugging, tunning and optimizing PySpark data pipelines
- Should have implemented concepts and have good knowledge of Pyspark data frames, joins, caching, memory management, partitioning, parallelism etc.
- Understanding of Spark UI, Event Timelines, DAG, Spark config parameters, in order to tune the long running data pipelines.
- Experience working in Agile implementations
- Experience with building data pipelinesin streaming and batch mode.
- Experience with Git and CI/CD pipelines to deploy cloud applications
- Good knowledge of designing Hive tables with partitioning for performance.
- Experience in data modelling
- Hands on creating workflows on any Scheduling Tool like Autosys, CA Workload Automation
- Proficiency in using SDKsfor interacting with native AWS services
- Strong understanding of concepts of ETL, ELT and data modeling.
Similar Jobs
View All
2 Hours ago
Data Analyst (Kannada Speakers)
AI & Machine Learning Advancement
- 1 - 20 Yrs
- Karnataka, India
For thousands of years, maps have provided humans with the knowledge they need to make decisions. As a Maps Evaluator, you will have the opportunity to provide ground truth for your town, city or country.
At Peroptyx, we are looking for Data Ana...
2 Hours ago
Data Analyst (Marathi Speakers)
AI & Machine Learning Advancement
- 1 - 20 Yrs
- India, Pune
For thousands of years, maps have provided humans with the knowledge they need to make decisions. As a Maps Evaluator, you will have the opportunity to provide ground truth for your town, city or country.
At Peroptyx, we are looking for Data...
3 Hours ago
Data Analyst (Telugu Speakers)
AI & Machine Learning Advancement
- 1 - 20 Yrs
- Andhra Pradesh, Telangana, India
For thousands of years, maps have provided humans with the knowledge they need to make decisions. As a Maps Evaluator, you will have the opportunity to provide ground truth for your town, city or country.
At Peroptyx, we are looking for Data Ana...
1 Day ago
Java Developer – Payments Domain
Information Technology
- 4 - 7 Yrs
- Mumbai (All Areas)
We are hiring Java Developers with 4–6 years of hands-on experience in backend development, particularly within the Payments or FinTech domain. The ideal candidate should possess a strong foundation in Java technologies and be capable of working in a...
1 Day ago
SAP Functional Architect
Information Technology
- 40,00,000 - 45,00,000 INR - Annual
- 12 - 15 Yrs
- Bangalore, Chennai
We are seeking an experienced SAP Pre-Sales Architect with a strong functional background and deep expertise in at least one SAP functional area. The ideal candidate will have extensive knowledge of cross-module integrations and a proven track record...
1 Day ago
Senior React Native Developer
Information Technology
- 7 - 12 Yrs
- Jaipur
The NineHertz is on the lookout for a Senior React Native Developer who is passionate about mobile app development and thrives in a fast-paced environment. This is a fantastic opportunity to work with a dynamic team, drive innovation, and help delive...
1 Day ago
Senior Data & AI Analytics Engineer (Remote)
AI & Machine Learning Advancement
- 18,00,000 - 24,00,000 INR - Annual
- 5 - 8 Yrs
- Pune
Job Ref: NT-DAAI-003
Experience: 5–8 years
Client: A prestigious AI-first tech company
Notice: Early joiners preferred (Immediate- 30 days)
We are hiring on behalf of a prestigious AI-first technology client for a Senior Data & AI Analytics En...
1 Day ago
AI Engineering Manager (Remote)
Information Technology
- 40,00,000 - 50,00,000 INR - Annual
- 10 - 15 Yrs
- Pune
Experience: 10 to 15 years
Location: Remote
Notice Period: Immediate to 30 days preferred
Client: Leading mid-sized firm specializing in AI-driven solutions
Overview:
We are looking for an AI Engineering Manager to lead a dynamic team of ...
Talk to us
Feel free to call, email, or hit us up on our social media accounts.
Email
info@antaltechjobs.in