Data Engineering Team Leader

Join a Company That Invests in You

Seeking Alpha is the world’s leading community of engaged investors. We’re the go-to destination for investors looking for actionable stock market opinions, real-time market analysis, and unique financial insights. At the same time, we’re also dedicated to creating a workplace where our team thrives. We’re passionate about fostering a flexible, balanced environment with remote work options and an array of perks that make a real difference.

Here, your growth matters. We prioritize your development through ongoing learning and career advancement opportunities, helping you reach new milestones. Join Seeking Alpha to be part of a company that values your unique journey, supports your success, and champions both your personal well-being and professional goals.

What We're Looking For

We are seeking a hands-on Data Engineering Team Leader to oversee our real-time data domain and lead the evolution of our data platform as it transitions from a heavy build-out to stability, quality, and scale.

This role is responsible for maintaining, hardening, and extending our event processing infrastructure, focusing on adding targeted real-time processing capabilities to support analytics and informed decision-making. At Seeking Alpha, our decisions are deeply data-driven. This critical role is centered on maintaining, hardening, and extending our event processing infrastructure, with a primary focus on adding targeted real-time processing capabilities. Your work will directly support our analytics and enable informed, data-driven business decision-making based on production data. Specifically, you will be responsible for the core data component of capturing the clickstream events from the website, which is the foundational functionality for all our analytics.

Looking forward, this role will help lead the team into its next phase: improving data quality and developer experience, investing in modern table formats (Iceberg), empowering analysts and data scientists, and continuously reducing cost and operational friction.

This is a player–coach role, combining hands-on technical leadership with ownership, prioritization, and mentorship.

Requirements:

7+ years of experience in Data Engineering, including ownership of production-grade data platforms.
Proven experience in a technical leadership or lead IC role.
Strong hands-on experience with Scala and Python.
Solid experience with AWS, including S3, EC2, EMR, Kinesis, Firehose, Spark
Strong understanding of event-driven and real-time architectures, even in maintenance-heavy environments.
Strong experience with Airflow for orchestration of batch and near-real-time data pipelines, including production operations and troubleshooting.

Deep knowledge of data warehousing and analytics workflows, including SQL.
Ability to work across teams ( DevOps, IT, Dev, Product, and Analytics ) and translate business needs into technical solutions.

Nice-to-Have Qualifications

Experience with Apache Iceberg or other modern table formats (Delta, Hudi).
Hands-on expertise with Redshift and/or Vertica, including performance tuning.
Experience enabling analytics engineering workflows (dbt or similar).
Exposure to supporting data science workloads in production.
Strong opinions (and pragmatism) around observability, documentation, and cost management.

What You'll Do:

Own the end-to-end real-time and event data domain, including ingestion, processing, and downstream consumption.

Ensure stability, correctness, and observability of existing streaming and near-real-time pipelines.
Lead the design and implementation of select real-time processing components to support analytics and decision-making use cases.
Define clear ownership, SLAs, and best practices for event data usage across the company.
Lead the team’s investment in modern data infrastructure
Drive architectural decisions with a long-term view on maintainability, cost, and scalability.

Team Leadership & Execution

Lead, mentor, and support a team of data engineers; set technical standards and review designs and code.
Act as a hands-on contributor in critical areas of the system.
Own planning and prioritization, balancing new investments with platform stability and technical debt.
Promote a culture of quality, documentation, and operational excellence.
Partner with Data Analytics and Data Science teams to improve data accessibility, trust, and self-service capabilities.
Improve observability across the data platform (data quality, freshness, lineage, failures).
Explore and adopt AI-assisted development tools to improve engineering velocity and code quality.
Stay current with emerging technologies in data engineering and analytics, and evaluate their relevance to the platform.