Senior Big Data Engineer

Final is a world leader in trading algorithms and trade execution technologies development. Our multi-disciplinary teams have developed a unique and highly successful machine learning algorithmic based HFT platform that delivers excellent results. In a world increasingly dominated by learning machines and artificial intelligence, we at Final are especially proud of our humans. Our elite team of exceptional people are the soul of our company, and it is our top priority to provide them with a professionally fulfilling environment that supports healthy work-life balance. Our employees are encouraged to pursue their passions outside of work and we are proud to offer them a variety of opportunities, multiple resources and an agile work environment which promotes their well-being.

We are searching for an innovative and experienced Senior Big Data Engineer that will join us and be part of our data group.

As a Senior Big Data Engineer, you will:

· Be a part of a cross functional team of data, backend and DevOps engineers.

· Work closely with Data Scientists to understand needs and design for data layout and infrastructure used in data ingestion for ML and analytics use-cases.

· Lead the architecture, planning, design and development of mission-critical, diverse and large-scale data pipelines and data lakes over both public and on-prem cloud solutions.

· Establish policies for data storage and versioning over peta-bytes scale complex storage solutions of multiple data lakes.

· Drive for continuous innovation and improvement in Final’s data stack.

Requirements:

· At least 5 years of experience working as a Big Data Engineer or Data Architect

· At least 3 years of experience working in python development with emphasis on data analysis tools such as numpy, pandas, polars, Jupyter notebook.

· Proven understanding in designing, developing and optimizing complex solutions that move and/or manipulate large volumes of data.

· Sound understanding of different big data file formats such as Parquet, DeltaLake, AVRO, HDF5.

· Hands-on experience with distributed query engines such as Spark and Hadoop.

· Experience with implementing open-source solutions on existing large-scale systems.

· Experience working on large scale and complex on-premises systems.

· Experience with Docker, Linux, CI/CD tools and concepts, Kubernetes.

· Experience with data pipelining tools such as Airflow, Kubeflow or similar.

· Understanding of ML concepts and processes.

· BSc / MSc degree in Computer Science/ Engineering / Mathematics or Statistics.

Advantages:

· Experience and understanding of various storage protocols (NFS, S3, CIFS)

· Experience working with data versioning tools such as DVC, LakeFS, GitLFS or similar technologies.

· Experience working with AWS data processing tools and concepts.

· Hands-on experience with DataBricks platform.

· Hands-on experience in ML frameworks and models training and implementation.

· Experience with various databases technologies, including Vertica, MongoDB, Neo4j, Postgres, influxDB

· Hands-on experience in lower-level programming languages such as C++ or RUST