Data Engineer

Job Description:

• Development of data pipelines end-to-end, in a Data Lake environment.
• Development of data movement processes in relational environments.
• Python scripting development (with an emphasis on PySpark).
• Data development and querying using SQL.
• Performing tests (unit/integration).
• CI/CD processes.

Requirements:

• Over 2 years of experience in Python - Must
• Over 2 years of experience in Big Data, with an emphasis on Spark and Hive.
• Ability to read and write complex SQL queries.
• At least 3 years of experience in Microsoft Data environments (SSIS, SQL Server).
• Familiarity with DWH, DL, and ETL/ELT processes.
• Experience working with CI/CD processes.
• Experience working with CTRLM.
• Experience with TERADATA databases – an advantage.
• Familiarity with Qlik Sense and QlikView – an advantage.
• Familiarity with the banking world – an advantage.
• Responsibility, independence, organizational skills, "big picture" thinking, initiative, diligence.
• Eager to learn and grow both technically and personally.
• Ability to work in a team and excellent interpersonal skills.