Big Data Developer

n innovative, established, and successful Fintech startup that is a world leader in trading algorithms and trade execution technologies development. The company specializes in Algo-Trading and e-commerce based on algorithms developed by its software engineers. It is at the technological forefront, offering a research-driven environment, a unique and innovative workspace, and many opportunities for personal and professional development.

This values-driven company is committed to making a positive impact on the world in its day-to-day operations. With around 250 employees, the company is located in the Sharon area near a train line, with the option to work from home two days a week.

About the role-

• Lead the architecture and development of mission-critical, diverse and large-scale data pipelines over both public and on-prem cloud solutions.
• Derive policies for data storage and versioning over peta-bytes scale complex storage solutions of multiple data lakes.
• Design and develop smart ML infrastructure in order to build Agile ML processes (MLOps).
• Work with Data Scientists to understand needs and design ML infrastructure solutions
• Be a part of a collaborative heterogeneous data team which includes developers, data scientists, data engineers, MLOps and data scouts.

Requirements-

• BSc / MSc degree in Computer Science/ Engineering / Mathematics or Statistics
• 5 years of experience working as a Data Engineer or Data Architect or ML Engineer
• 3 years of experience working in high level software development, preferably in Python
• Proven hands-on experience in designing, developing and optimizing complex solutions that move and/or manipulate large volumes of data
• Sound understanding of different big data file formats such as Parquet, Arrow, HDF5, etc.
• Experience with Docker, Linux, CI/CD tools and concepts, Kubernetes.
• Experience with data pipelining tools such as Airflow, Kubeflow, MLFlow and similar technologies.
• Hands-on experience with large-scale distributed computing systems and related technologies (e.g., Spark, Hive)
• Understanding of ML concepts and processes.
• Result oriented, self-motivated, fast learner, can do attitude and a great team player