Senior Big Data Engineer

Tired of big data performance bottlenecks? Our Tel-Aviv based client is an end-to-end performance platform that is focused on mitigating these. They help businesses become more data-driven, looking to make impactful decisions by identifying and alerting on performance issues in real-time. You’ll be at the forefront of building intelligent solutions that help companies understand and optimize their Spark applications performance, turning complex metrics into actionable insights.

What will you be doing?

Joining a 4 person team comprised entirely of seniors they’re looking for you to really come in and take ownership of what you’re building. Initially you’re going to be focusing on Spark applications; building and enhancing their real-time performance monitoring and alerting system and developing features that translate Spark metrics into actionable insights. There’s also a few other things to get stuck into:

• Designing and implementing algorithms to detect performance bottlenecks and anomalies.
• Creating intelligent alerting mechanisms for proactive performance issue detection.
• Working closely with customers to understand their performance optimization needs.
• Collaborating with the wider team to improve their bottleneck detection and analysis capabilities.

What experience do you need?

They’re looking for someone who comes from a software engineering background and that has prior experience working with Scala, with it being a huge bonus if you’ve worked on enterprise systems before. As they are very early stage, it’s really important that you possess that start-up mentality and ideally at least 5 or 6 years’ experience. You’re going to need to have experience with Apache Spark, as well as with cloud infrastructure (ideally AWS and alongside experience with Kubernetes and Docker). Any of the following would be a huge bonus:

• Python experience.
• A sound knowledge of the Databricks platform and ecosystem.
• Knowledge of performance optimization in JVM environments.
• Production experience in a SaaS environment.

What’s in it for me?

Offering up to 45,000 Shekels a month plus significant equity, these guys have managed to build a really strong, welcoming environment and culture. There’s a set limit for food each day and the co-founders are all big music fans and like to have fun, which they’re also trying to bring to the interview process. This is an initial half hour phone call, followed by an hour meet over coffee where you’d get to know each other a little more on a personal level. Third and finally, there is a technical task that is open sourced.

If this is of interest, then please get in touch with me on [email protected] and we can discuss further.