Big Data Developer

An innovative Fintech start-up specializing in Algo-Trading, focusing on electronic trading based on algorithms developed by the company's software engineers.

The company is at the technological forefront, offering research freedom, a unique and innovative work environment, and many opportunities for personal and professional development.

The company has approximately 250 employees, is located in Herzliya (near a train line) and operates with two work-from-home days.

About the role

• Lead the architecture and development of mission-critical, diverse and large-scale data pipelines over both public and on-prem cloud solutions.

• Derive policies for data storage and versioning over peta-bytes scale complex storage solutions of multiple data lakes.

• Design and develop smart ML infrastructure in order to build Agile ML processes (MLOps).

• Work with Data Scientists to understand needs and design ML infrastructure solutions

• Be a part of a collaborative heterogeneous data team which includes developers, data scientists, data engineers, MLOps and data scouts.

Requirements:

• BSc / MSc degree in Computer Science/ Engineering / Mathematics or Statistics

• 5 years of experience working as a Data Engineer or Data Architect or ML Engineer

• 3 years of experience working in high level software development, preferably in Python

• Proven hands-on experience in designing, developing and optimizing complex solutions that move and/or manipulate large volumes of data

• Sound understanding of different big data file formats such as Parquet, Arrow, HDF5.

• Experience with Docker, Linux, CI/CD tools and concepts, Kubernetes.

• Experience with data pipelining tools such as Airflow, Kubeflow, MLFlow and similar technologies.

• Hands-on experience with large-scale distributed computing systems and related technologies

• Understanding of ML concepts and processes.

• Result oriented, self-motivated, fast learner, can do attitude and a great team player