Senior Data Engineer, Research Engineering

a Senior Data Engineer to join its Research Engineering team. The Research Engineering teams mission is to enable research needs. We succeed when our research analysts, data scientists, and engineers have access to the data, tools, knowledge, and support to do their jobs painlessly and efficiently.

We build the foundational infrastructure and key applications that allow internal users to derive actionable insights from vast data. Both enablingand runningBig Data processing at the scale of hundreds of TB is a core part of what our team does, but were not just a Big Data team. We work across the tech stack in support of our mission to enable data-driven research and investigation, from back-end to front-end to data stores.

What you'll be doing:

Design, build, and maintain foundational infrastructure and key applications that allow people to effectively leverage proprietary and public data
Use modern technologies daily, such as: Snowflake, Databricks, Apache Spark, Elasticsearch, AWS, Kafka, and more
Expand and optimize data warehouse and lakehouseweighing tradeoffs between performance, cost, and user experienceacross continually larger and more complex datasets
Model data and define storage strategy, to ensure that data is intuitive for researchers to find, and easy and fast for them to access
Enhance analysts most-used web application, which gives them in-depth insight into every decision we make
Collaborate directly with colleagues from various disciplines: Analysts, Data Scientists, and other Engineers (all of whom write production code!)
Self-manage project planning, milestones, designs, and estimations
Hold yourself and others to a high standard when working with production systems. We take pride in our tests, monitoring & alerting abilities just as we do with our systems.
Debug complex problems across the whole stack