Data Engineer Team Lead

Pipl is the identity trust company. Our solutions allow organizations to provide frictionless customer experiences and approve more transactions with greater confidence and speed. We use multivariate linking to establish deep connections among more than 330 billion trust signals—email, mobile phone and social media data that spans the globe—and then look at the big picture to derive identity trust.

We’re looking for a Senior Data Engineer to be a part of a growing Data team. Our Data-Team is developing data-centric algorithms that extract accurate and detailed information from vast amounts of data, which is then normalized and indexed for the use of our search engine.

Responsibilities:

• Design, build, expand and maintain our data platform using efficient and reliable data pipelines
• Collaborate with various stakeholders across pipl, including the engineering team, data analysts, and data scientists
• Understand their work, collect requirements and create the relevant infrastructure to build easy-to-use and highly productive data systems
• Develop in-house applications for simplifying data transformation, analysis, and accessibility
• Data Ownership. Design, build and launch our data models, how and where each data point will be stored in the most convenient format for data consumers (data analysis, data QA, data scientist etc.)
• Use various technologies and techniques to solve data problems like -Spark, BigQuery, Airflow, Python, Dataproc, DBT, etc.

Requirements:

• 4+ years experience as a data engineer
• At least 2 years of proven experience with writing complex SQL statements, Python, PySpark, Spark, Airflow, or other similar tools
• Experience with columnar and relational DBs (Bigquery, Casandra, Mysql, snowflake)
• Experience with data pipelines design, implementation, and maintenance
• Experience with data models design
• Experience working with cloud infrastructure (GCP, AWS )- Advantage
• Experience with Java- Advantage