Data Engineer

We are looking for an experienced Data Engineer to join a leading and major financial institution in Israel. In this role, you will play a key part in developing and implementing advanced data-driven solutions for regulatory compliance and strategic risk management in the financial sector.

Role Overview:

As a Data Engineer, you will be responsible for designing and building scalable data pipelines, optimizing data migration processes, and contributing to the development of high-performance data solutions in a Data Lake environment. Your work will support critical business areas such as compliance, anti-money laundering, capital adequacy, and overall risk management initiatives.

Key Responsibilities:

• Develop and maintain data pipelines in a Data Lake environment, ensuring seamless data processing from source to destination.
• Design and implement data migration processes in relational databases and distributed systems.
• Develop and optimize Python scripts, with a strong focus on PySpark for big data processing.
• Write and optimize complex SQL queries for efficient data retrieval and manipulation.
• Conduct unit and integration testing to ensure the accuracy and reliability of data systems.
• Work with CI/CD pipelines to automate data solution deployments and ensure continuous delivery.

Required Skills and Experience:

• 2+ years of hands-on experience with Python, including work with PySpark.
• 2+ years of experience with Big Data technologies, specifically Spark and Hive.
• Proficiency in writing and optimizing complex SQL queries.
• 3+ years of experience with Microsoft Data technologies (SSIS, SQL Server).
• Strong understanding of DWH, DL, and ETL/ELT processes.
• Experience with CI/CD processes and automation of deployments.
• Familiarity with CTRLM or similar risk and compliance management tools.
• Experience with TERADATA databases – advantage.
• Familiarity with Qlik Sense and Qlikup – advantage.
• Understanding of the banking and financial services sector – advantage.

Key Competencies:

• Strong problem-solving and analytical skills, with a focus on handling complex data workflows.
• Proactive, independent, and organized, with excellent time management abilities.
• Eagerness to learn, adapt, and stay updated with the latest technologies.
• Excellent communication and collaboration skills, with the ability to work in cross-functional teams.