Big Data Developer

A successful startup company with disruptive technology in the underground infrastructure mapping market, often dubbed as the "Google Earth of underground infrastructure."

The company has developed an advanced system based on Big Data, algorithms, and AI for mapping underground infrastructure. Utilizing NLP capabilities for scanning data sources such as satellite images, employing Signal Processing, Computer Vision, and more, they provide a clear picture enabling navigation and understanding of what exists and where it's located, including knowledge about water lines, electricity, oil, gas, with potential savings estimated in the billions of dollars annually for construction companies.

The company has numerous paying customers, and its technology is already disrupting a trillion-dollar market in the US.

The company employs approximately 100 workers, located in Tel Aviv near the train line, and operates on a hybrid work model.

Role Description: Part of the company's Data infrastructure development team considered a core team member. Working with hundreds of thousands of different data sources (satellite imagery, sensors, data partners, etc.). Reporting to the team lead, collaborating with all development teams in the company, including developing data infrastructures from scratch, performing validation and standardization of data, and more.

Stack: Python, Kafka, Airflow, Kubernetes, AWS, Spark, Datalake, NoSQL, GIS Technologies.

Requirements:

• 5 years experience in Scala/Java or equivalent language.

• Experience with Python.

• Proven experience in building, optimizing, and maintaining large-scale big data pipelines using popular open-source frameworks (Spark, Kafka, Hive Metastore, Airflow, Iceberg, etc).

• Strong SQL abilities and hands-on experience with SQL and NoSQL DBs.

• Experience with AWS cloud services such as