Data Processing Engineer

As a Data Processing Engineer, you will play a crucial role in the data lifecycle, from ingestion to processing and delivery. Your responsibilities will include extracting data from various sources, integrating disparate datasets, ensuring data quality, migrating data to destination storage, testing processed data for suitability, and managing the data processing environment.

Key Skills:

• Data Ingestion: Extracting data from diverse sources using appropriate methods and tools to ensure efficient and reliable data transfer.

• Integration: Bringing together data from multiple sources, including structured and unstructured data, and integrating them seamlessly for unified processing.

• Data Quality: Ensuring the reliability and accuracy of data through validation, cleansing, and enrichment processes to maintain high data quality standards.

• Migration: Moving and processing data to designated storage environments efficiently while minimizing data loss and ensuring data integrity.

• Testing: Performing thorough testing of processed data to ensure it meets specified requirements and is fit for the intended purpose.

• Environment Management: Administering platforms and tools required for data processing, including but not limited to hosting environments, cloud/on-prem infrastructure, and associated services.

Preferred Skills:

• Hosting, Cloud/OnPrem: Experience with Google Cloud Platform or similar cloud services, as well as on-premises infrastructure management.

• Web Server: Proficiency in managing Tomcat web servers for hosting and serving web applications.

• Frameworks: Familiarity with SpringBoot and Spring MVC frameworks for building and deploying Java-based applications.

• API Management: Knowledge of Swagger and Postman for designing, documenting, and testing APIs.

• Database: Experience with SQL Server Integration Services (SSIS) for data integration and ETL processes.

• DevOps CI/CD: Proficiency in using Bitbucket and Jenkins for continuous integration and continuous deployment (CI/CD) pipelines.

• Test Automation: Familiarity with QTest or similar test automation tools for automating testing processes.

• ETL: Experience with Altova Mapforce or similar ETL (Extract, Transform, Load) tools for data transformation and manipulation.

Additional Requirements:

• Strong problem-solving skills and attention to detail.
• Excellent communication and collaboration abilities.
• Ability to work effectively in a fast-paced, dynamic environment.
• Continuous learning mindset to stay updated with emerging technologies and best practices in data processing and engineering

#LI-AU1

#LI-Remote