Senior Data Engineer

Were looking for a Senior Data Engineer to join our Research Engineering Team and play a pivotal role in shaping the data infrastructure that powers innovative solutions. In this role, youll design, build, and scale data pipelines and data warehouses from the ground up, creating the backbone for the worlds leading data classification engine. Youll collaborate closely with researchers and machine learning experts to enable the development of advanced models, ensuring a seamless flow of high-quality, reliable, and well-structured data. Your contributions will directly impact the scalability, performance, and precision of our platform, advancing our mission to protect critical data.

What Youll Do

Design and build scalable batch and streaming data pipelines that process complex datasets from diverse sources, enabling reliable and high-performance model training and inference.

Collaborate closely with researchers and data scientists to deliver high-quality, structured datasets that accelerate experimentation and model iteration.

Lead large-scale historical backfills and migration initiatives to ensure data consistency and integrity across evolving storage and compute platforms.

Optimize data workflows through advanced query tuning, indexing, partitioning, and cost optimization strategies to support efficient large-scale analytics.

Architect and maintain high-performance cloud-based data platforms using modern data stack components across AWS, GCP, or Azure.

Operate distributed data processing engines to handle massive volumes of structured and unstructured data.

Design and implement event-driven architectures using large-scale queue systems such as Kafka or SQS to ensure reliable and efficient data movement.

Develop automated monitoring and validation systems that guarantee uptime, schema compatibility, and pipeline reliability.

Deploy and manage data infrastructure in containerized, Kubernetes-based environments to support scalable and resilient services.