Gurugram, Haryana, India
Information Technology
Full-Time
TekWissen India
Overview
Overview
TekWissen Group is a workforce management provider operating throughout India and several other countries worldwide. The client below is a leading technology company offering a range of IT solutions to businesses and organizations, enabling them to transform their digital futures
Position: Senior Software Engineer
Location: Hyderabad
Duration: 24 Months
Job Type: Contract
Work Type: Hybrid
Shift Timings: 9:00 AM-6:00 PM
Job Description
Expertise in Data Ingestion and Transformation tools and technologies
TekWissen Group is a workforce management provider operating throughout India and several other countries worldwide. The client below is a leading technology company offering a range of IT solutions to businesses and organizations, enabling them to transform their digital futures
Position: Senior Software Engineer
Location: Hyderabad
Duration: 24 Months
Job Type: Contract
Work Type: Hybrid
Shift Timings: 9:00 AM-6:00 PM
Job Description
- Install, configure, and administer data ingestion and transformation platforms including StreamSets, Apache Spark, Informatica, and Apache NiFi for maximum utilization and throughput.
- Perform StreamSets Data Collector and Control Hub administration including pipeline deployment, monitoring, and optimization across multiple environments.
- Experience in designing and implementing real-time and batch data pipelines using Apache Spark with Python (PySpark), with Scala knowledge as an added advantage.
- Hands-on experience with Apache NiFi for data flow automation, including processor configuration, flow management, and cluster coordination.
- Experience in upgrading and migrating data ingestion tools and pipelines to higher versions while maintaining data integrity and minimal downtime.
- Highly proficient in Python programming for data processing, pipeline development, and automation scripting.
- Expert-level SQL skills for complex data transformations, performance optimization, and database interactions across various RDBMS platforms.
- Experience with diverse database technologies and data sources:
- MPP databases: Teradata, Greenplum for large-scale analytical workloads
- OLTP databases: Oracle, SQL Server, PostgreSQL for transactional processing
- In-memory databases: SAP HANA, MemSQL for high-performance analytics
- Big Data platforms: Hive, HBase, and Kudu for distributed data storage and processing
- Additional data sources including SFDC and modern cloud data platforms
- Ability to write efficient, scalable code following best practices and coding standards for data engineering projects.
- Extensive experience with automated testing integration using CI/CD tools, preferably GitLab (Jenkins experience is beneficial).
- Implement and maintain automated deployment pipelines for data ingestion and transformation workflows.
- Experience in version control, branching strategies, and collaborative development practices for data engineering projects.
- Knowledge of infrastructure as code and containerization technologies for deployment automation.
- Perform data pipeline deployments across versioned repositories from development to production environments.
- Experience with TLS hardening and security best practices for data platforms and APIs.
- Proficient in using command-line tools and APIs for platform administration and monitoring.
- Create, maintain, and restore backup strategies for data pipelines, configurations, and metadata repositories.
- Expertise in troubleshooting performance bottlenecks in data processing workflows and implementing optimization strategies.
- Experience in cluster maintenance and distributed system administration for Spark and other big data technologies.
- Production environment health monitoring, alerting, and incident response for data pipelines.
- Root cause analysis of pipeline failures with comprehensive documentation of issues and resolutions.
- Develop and enforce data engineering coding standards and best practices across development teams.
- Experience with data quality validation, testing frameworks, and automated data validation processes.
- Knowledge of data governance principles and implementation of data lineage tracking.
- Scheduling and orchestration of data workflows with proper error handling and retry mechanisms.
Expertise in Data Ingestion and Transformation tools and technologies
- StreamSets
- Spark (python preferable, scala good to have)
- Informatica
- Apache Ni-fi
- Highly proficient in Python and SQL
- Has working experience with automated test integration using CI/CD tools
- Gitlab preferable (Jenkins good to have)
- OLTP DB's like Oracle, SQL Server and Postgres
- In-memory DB's like HANA, MemSQL
- Hive, HBase and Kudu
- Total Exp: 5- 8 years
- Rel Exp: 5+ years
Similar Jobs
View All
1 Day ago
Technical Lead Manager, Megamind, Contributor Trust Modeling
Information Technology
- Gurugram, Haryana, India
Google's software engineers develop the next-generation technologies that change how billions of users connect, explore, and interact with information and one another. Our products need to handle information at massive scale, and extend well beyond w...
1 Day ago
Cloud Architect
Information Technology
- 1200000 - 1320000 INR - Yearly
- Gurugram, Haryana, India
Overview:We are seeking an experienced Cloud Architect with over 2 years of experience to lead the design and implementation of our cloud infrastructure, specifically tailored for our advanced AI and machine learning initiatives. You will be the ar...
1 Day ago
Data Scientist Manager, Product, Cloud
Information Technology
- Gurugram, Haryana, India
Help serve Google's worldwide user base of more than a billion people. Data Scientists provide quantitative support, market understanding and a strategic perspective to our partners throughout the organization. As a data-loving member of the team, yo...
1 Day ago
IT Executive
Information Technology
- 15000 - 20000 INR - Monthly
- Gurugram, Haryana, India
Skills Preferred End-user technical support Experience supporting Windows 7 and Windows 10 OS, switch, and routers. Installing and maintaining hardware and computer peripherals. Troubleshooting hardware and software issues.Experience in digital marke...
1 Day ago
Manual Tester
Information Technology
- 180000 - 650000 INR - Yearly
- Gurugram, Haryana, India
eGrove Systems is hiring Manual testerLocation: Chennai/MaduraiExperience: 3+ YearsOffice Timing - 1:00 PM to 10:00 PMNotice Period - Immediate to 15daysTesting & Quality Assurance:Design, develop, and execute manual test cases based on business and ...
1 Day ago
Senior Web Developer in Noida
Information Technology
- Gurugram, Haryana, India
Key Responsibilities Develop, test, and maintain responsive websites. Collaborate with designers and marketing teams to implement new features. Optimize website performance, speed, and SEO. Troubleshoot and resolve bugs/issues. Stay updated wit...
1 Day ago
Data Scientist - GenAI
Information Technology
- Gurugram, Haryana, India
Purpose of the Role:We are seeking a highly skilled Data Scientist to join our Advanced Analytics team. This is a technical role focused on developing statistical and machine learning models to drive insights, innovation, and decision-making. Domain...
1 Day ago
Data Analytics Architect
Information Technology
- Gurugram, Haryana, India
Company Summary DISH Network Technologies India Pvt. Ltd is a technology subsidiary of EchoStar Corporation. Our organization is at the forefront of technology, serving as a disruptive force and driving innovation and value on behalf of our custome...
Talk to us
Feel free to call, email, or hit us up on our social media accounts.
Email
info@antaltechjobs.in