Data Engineer - Billing Platform

We are looking for a Data Engineer to join our usage metering & billing platform team! with a strong background in writing ETLs and complicated data processing pipelines. Youre deeply familiar with modern data processing stack (especially Spark, Hadoop and Airflow), have a strong problem-solving attitude and, most importantly, youre passionate about understanding the business side of the platform. Our platform engineers have an end-to-end understanding of the data our products generate and the commercial models and data structures in which they fit.
Working with this enormous volume of data begins with massive ETL pipelines built using cutting-edge technologies such as Kafka, BigQuery, Airflow, S3, PostgreSQL, Athena, SQS, Redis, and especially Spark. We are big believers in working with the right tool for the right job, including programming languages. We run a truly polyglot shot with Scala, Python, Go and Node being especially common here . The extreme growth rate of our data and client base means that solutions need to be designed to handle 10X growth or more.
What youll do:
Develop end-to-end ETLs and data infrastructure features from data processing logic to database choice and implementation through to API development
Improve the scalability and performance of existing data pipelines
Delve into existing pipelines and business logic, target weaknesses and add functionality
Mentor, advise, and work collaboratively within the Biz Data & Dev group, with other development teams, and with all other business teams of the company
Continuously learn and follow new technologies both for personal growth and to ensure we are making the most informed technology choices possible