Big Data Developer

Join our dynamic team as a key player in our Data Group, leveraging your expertise to innovate and enhance our advanced big data platform. Take the lead in designing and implementing extensive data pipelines using the latest frameworks and technologies like Spark, Kafka, Iceberg, Flink, Imply, and so on. Collaborate closely with both data science and data analytics teams, guiding projects from inception to completion. Your role is crucial in driving technological excellence and establishing best practices in data operations and pipeline optimization. We’re looking for someone passionate and experienced in developing large-scale data platforms and applications, supporting everything from analytics &

Requirements:

• At least 3 years experience in Scala/Java or equivalent language.
• Experience with Python.
• Proven experience in building, optimizing, and maintaining large-scale big data pipelines using popular open-source frameworks (Spark, Kafka, Hive Metastore, Airflow, Iceberg, etc).
• Strong SQL abilities and hands-on experience with SQL and NoSQL DBs.
• Experience with AWS cloud services such as EMR, Redshift/Snowflake, S3, Athena, and Glue - advantage.
• Experience in stream processing frameworks such as Flink - advantage.
• Exceptional Team Collaboration: Demonstrated ability to work effectively in a team environment, contributing to a collaborative work culture that fosters innovation.
• Ownership and Initiative: Strong ability to own and drive solutions from concept to completion, showing initiative and a proactive approach to problem-solving.