Data Engineer

A startup in the logging domain is developing an observability product aimed at providing clients with tools for their production environment. The product offers visibility through various tools, data accessibility, investigative management, and ML algorithms, as well as complex alert mechanisms for critical events.

The company is building a product platform for developers and DevOps professionals, which currently includes 10 products and is set to expand to 14, including a new security product designed to help monitor cybersecurity events.

The company handles high-scale operations with over 3 million events per second being monitored. It has over 2,000 paying clients monthly, including well-known global companies. The company has raised $240 million in Series D funding and has 5 sites worldwide: London, the US, India, and Israel.

Located in central Israel near a train station, the company operates with a hybrid work model of two days from home and has 330 employees, with 140 based in Israel.

A promising startup offering the chance to be at the heart of business development in a positive environment with growth opportunities.

Role Description:

• Setting up infrastructure from scratch and building a Data Lake.
• Developing server-side components in Python.
• Working with ETL tools in a Big Data environment.
• Partnering with business stakeholders and upstream infrastructure platform engineering teams to understand data and translate business requirements into scalable data pipeline designs.
• Developing and taking ownership of BI data pipelines and centralized data warehouses, ensuring trustworthy curated datasets and standardized metrics.
• Providing direction on data engineering and architecture, and determining the appropriate tools for various tasks.
• Implementing data engineering best practices such as dimensional modeling, ETL/ELT pipeline development, and large-scale distributed ETL/ELT pipelines to support large-scale machine learning.
• Creating and maintaining analytics data pipelines to generate data and insights that drive business decision-making.
• Providing feedback to Product and Engineering teams on new and upcoming features.
• Tech stack: Scala, NodeJS, AWS, Kubernetes, ETL, Elasticsearch, Angular 9, Kafka, Big Data, SQL, Looker.

Retirements

• 4+ years as a software engineer or data engineer
• Extensive knowledge of BI concepts (i.e. ETL, dimensional modeling, data warehouse design, data quality monitoring, dashboarding)
• Extensive knowledge of database query languages (i.e. SQL or its MPP equivalent), database design, optimizing queries, internals knowledge of query planning
• Extensive experience with distributed computing/MapReduce
• Extensive experience with data streaming solutions