Senior Software Engineer — Kafka / Event Streaming

Description:

VAST Data is looking for a Senior Software Engineer to join our growing team!

This is a great opportunity to be part of one of the fastest-growing infrastructure companies in history, an organization that is in the center of the hurricane being created by the revolution in artificial intelligence.

"VAST's data management vision is the future of the market."- Forbes

VAST Data is the data platform company for the AI era. We are building the enterprise software infrastructure to capture, catalog, refine, enrich, and protect massive datasets and make them available for real-time data analysis and AI training and inference. Designed from the ground up to make AI simple to deploy and manage, VAST takes the cost and complexity out of deploying enterprise and AI infrastructure across data center, edge, and cloud.

Our success has been built through intense innovation, a customer-first mentality and a team of fearless VASTronauts who leverage their skills & experiences to make real market impact. This is an opportunity to be a key contributor at a pivotal time in our company’s growth and at a pivotal point in computing history.

What You'll Do:

• Design and develop core components of a next-generation distributed event-streaming and messaging platform.
• Implement and maintain a high-scale, high-performance Kafka-protocol-compatible broker.
• Build scalable and resilient distributed services handling petabytes of data across large clusters.
• Design solutions around correctness semantics (ordering guarantees, transactions, exactly-once / idempotence, repartitioning), load balancing, replication, fault tolerance, and scalability
• Investigate and resolve complex production issues across the full produce-to-consume path - from the Kafka client that produced it, through the broker and all the layers in between, all the way to the consumer.
• Collaborate closely with architects, researchers, and engineering teams to deliver high-performance data infrastructure powering AI, analytics, and enterprise workloads.

• Strong hands-on experience in C/C++ software development, including system design, implementation, debugging, performance optimization, and integration of large-scale production systems.
• B.Sc. or M.Sc. in Computer Science, Software Engineering, Electrical Engineering, or equivalent practical experience.
• Deep understanding of Linux internals, including processes, threads, memory management, synchronization primitives, I/O subsystems, and performance troubleshooting.
• Experience developing distributed systems with a focus on scalability, reliability, fault tolerance, concurrency, and high availability.
• Solid understanding of networking fundamentals, including TCP/IP, routing, transport protocols, distributed communication patterns, and OSI networking layers.
• Experience designing and implementing high-performance backend services operating in multi-node environments.
• Strong debugging and root-cause analysis skills across complex production environments.
• Experience with message brokers or event-streaming systems - advantage.
• Apache Kafka internals or Kafka wire protocol experience - advantage.