Data Engineer

We are looking for an experienced Data Engineer to join a leading financial institution in Israel.

As a Data Engineer, you will be responsible for developing and maintaining data pipelines in a Data Lake environment, designing data migration processes, and optimizing workflows for big data processing. Your work will directly support the business areas involved in data analytics, financial risk management, and regulatory compliance. You will work closely with cross-functional teams to drive the development of high-quality data solutions.

Key Responsibilities:

• Develop and maintain end-to-end data pipelines in a Data Lake environment, ensuring seamless data flow from source to destination.
• Design and implement data migration processes for relational databases and distributed systems.
• Develop and optimize Python scripts, with a strong focus on PySpark for big data processing.
• Write and optimize complex SQL queries for data analysis, retrieval, and manipulation.
• Collaborate with Data Analysts to develop models and build dashboards.
• Ensure the accuracy and reliability of data by conducting thorough testing and implementing controls for ETL/ELT processes.

Required Skills and Experience:

• 2+ years of hands-on experience with Big Data technologies, specifically Spark and Hive - Must !!!
• 3+ years of experience with SQL, including writing and optimizing complex queries.
• Experience with Data Lake (DL), Data Warehousing (DWH), and ETL/ELT processes.
• 2+ years of experience with Python (familiarity with PySpark is a big advantage).
• Familiarity with Quilliup – advantage.
• Experience with Predictive Modeling – advantage.
• Agile methodology experience – advantage.

Key Competencies:

• Strong problem-solving and analytical skills, with a focus on handling complex data workflows.
• Proactive, independent, and organized, with excellent time management abilities.
• Ability to work under pressure and deliver high-quality results.
• Excellent interpersonal skills, with a collaborative approach to working with teams and stakeholders.
• A passion for innovation, learning, and adapting to new technologies in the ever-evolving data space.