Senior Data Engineer, NSV

We are looking for a Senior Data Engineer to join the NSV (Network Solutions Validation) group. NSV builds high-performing software automation for our Data Center environments and helps drive the data growth of the world's biggest companies. In this role, you will design, build, and maintain scalable, high-performance data pipelines that handle massive volumes of data from hardware, communication modules, firmware, and large-scale AI and HPC clusters. You'll also contribute to our growing Agentic AI initiatives, helping develop AI Agents that bring our data capabilities to the next level.
What you'll be doing:
Define and execute the group's data technical roadmap, aligning with Infra, DevOps, and Performance teams
Design and maintain flexible ETL/ELT frameworks for ingesting, transforming, and classifying cluster verification and telemetry data
Build and optimize streaming pipelines using Apache Spark, Kafka, and Databricks, ensuring high throughput, reliability, and adaptability to evolving data schemas
Ensure data quality and pipeline health through observability standards, schema validation, lineage tracking, monitoring, and alerting
Deliver reliable insights for cluster performance analysis, telemetry visibility, and end-to-end test coverage
Support self-service analytics for engineers and researchers via Databricks notebooks, APIs, and datasets
Drive best practices in data modeling, code quality, and operational excellence; collaborate with cross-functional teams to support data-driven decision-making
Contribute to the development of AI Agents that enhance the visibility and accessibility of insights and data for our users.