Lead Ab Initio Data Engineer – 1542658

Working with a manager who needs to hire a full time Lead Ab Initio ETL Engineer to their team full time (this is not a contract or C2C) and play a vital role in orchestrating, deploying, and maintaining big data infrastructure both on-premise and in the cloud. The primary focus will be on leading the migration and setup of the enterprise data platform on the cloud using Cloudera CDP public cloud and other AWS services.

Responsibilities:

• Orchestrate, deploy, and maintain on-premise and cloud-based big data infrastructure.
• Prepare and manipulate data using Hadoop or equivalent MapReduce platform.
• Administer and engineer relational databases (e.g., MySQL, PostgreSQL) and Big Data systems (e.g., Cloudera Data Platform Private Cloud and Public Cloud).
• Act as a subject matter expert (SME) in Apache Solr, ETL (e.g., Ab Initio), BI (e.g., MicroStrategy), and automation tools (e.g., Ansible, Terraform, Bit Bucket).
• Experience with cloud solutions, specifically data products on AWS.
• Manage metadata hub-MDH, Operational Console, and troubleshoot environmental issues affecting these components.
• Prior experience with migration from on-premise to AWS Cloud.
• Represent the team in architectural and design discussions, providing credible feedback and input in all impacted areas.
• Track and monitor projects and tasks as the lead.

Essential Functions:

• Lead the team in designing, configuring, implementing, monitoring, and managing the Data Integration Framework.
• Develop and maintain infrastructure systems, including data warehouses and data lakes, and data access APIs.
• Troubleshoot incidents to recover services and support root cause analysis.
• Implement best practices for data auditing, scalability, reliability, high availability, and application performance.
• Mentor junior and senior team members.
• Perform operating system administration, user account management, disaster recovery strategy, and networking configuration.

Qualifications:

• Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related field. (Or an additional 4 years of relevant work experience in addition to the required work experience if lacking a Bachelor’s degree).
• 8 years of experience in leading data engineering and cross-functional teams to implement scalable ETL/ELT solutions.
• Hands-on experience with application development, relational database layout, development, and data modeling.
• Experience with Cloudera CDP on-prem and public cloud; Solr SME.
• Strong technical, analytical, and problem-solving skills.
• Excellent organizational and communication skills.
• Knowledge of programming languages and web-based technologies (e.g., SQL, NoSQL, Python).