Director of Data Engineering

Description

We seek an experienced and dynamic Data Engineering Manager to join our innovative technology team.

The Data platform of Datarails serves as the multi-tenant infrastructure for the Datarails application, running tens of thousands custom Data pipelines each day, serving thousands of different organizations across the globe.

You will have the opportunity to design, build, and manage the next generation of the Datarails Data Platform, build a data engineering team from the ground up, and manage all data-related aspects of the Datarails application.

What You'll Do

• Take a major role in designing, architecting, and selecting technology for the next generation of the Data platform.
• You will recruit, establish, and manage a data engineering team responsible for all Datarails data (Customer and business data), including Data Integrity, Data integrations, Pipelines, Data transformation, Data aggregation, Reports, Insights, Events, Monitoring, etc.
• You will define working methodology, KPIs for measuring success, and quality control measures.
• Design data profiling and monitoring to identify anomalies and maintain data integrity.
• Work closely with our AI and data scientists team, responsible for developing algorithms, analytical rules, and LLM/ML data models.
  
  

• We offer a supportive, agile, and creative work environment with brilliant colleagues!
• Competitive compensation, ongoing recognition, and most of all, a chance to learn, contribute, and develop your skills.
• Opportunity to join an explosive industry, a successful and growing organization, and an unmatched leadership team that is passionate about their employees!
• Most of all, we will give you fun, enthusiasm, support, a passion for healthy living, and never a dull moment!
  
  

• BSc in Computer Science or equivalent.
• 7+ years of experience as a Data Engineer
• 3+ years experience as a Data Engineering Manager
• Experience in building Data infrastructure.
• Experience and familiarity with the following tools:
• Big-data query engines (Spark, Databricks, Presto)
• Data lakes and data Lakehouse architecture
• Familiarity with Azure data stack - advantage.
• Experience creating and optimizing big data processes, pipelines, and architectures on cloud-native environments (Spark/Databricks).
• Proficiency with SQL, Python, and one of Java/Scala/CPP.
• Understanding of how to build robust and reliable solutions
• Familiarity with cloud Databases (Snowflake-Advantage)
• Have a strong analytical mind with proven problem-solving abilities