Online Pipelines Team Lead

Come and shape the modern era of data and AI!!

Nimble is the first business external data platform, making data decision-making easier than ever, with our award-winning AI-powered data structuring technology Nimble connects business users with the public web knowledge.

We empower businesses with mission-critical real-time external data to unlock advanced competitors’ intelligence, price comparison, and companies’ data for sales and marketing. We translate data into immediate business value.

With Nimble, data teams build advanced AI applications that connect enterprise-ready real-time web data to their custom GPTs. Modern BI teams use Nimble to build business dashboards and unlock new opportunities with market visibility. Data is the king.

We will disrupt the traditional market intelligence space. The reward is great – we can disrupt the legacy data industry worth $ 100B and build a data connected world.

We are looking for an experienced Software Engineering Team Lead to join us in our TLV offices.

We’re on a search for a Team Lead who has a lot of passion for scaling software up. In this role, you will be Leading Nimble’s Online Pipelines team, who are responsible for the newest, most innovative product at Nimble, offering an end-to-end platform for data teams and business leaders that solve problems using web data in a way deemed impossible only a year ago.

You will be developing highly scalable web solutions, leading an innovative team designing and implementing the backbone that connects all of Nimble’s Data products.

What it entails:

• A unique opportunity to lead an innovative team designing the future of Nimble’s products
• Working closely with our AI team and other Software teams to collaborate, as the owners of the main new platform that leverages internal APIs to create an ecosystem.
• We work with Python, using FastAPI, using Postgres and S3 for persistence,Redis for Caching, Flyte for Orchestration, Airbyte to power data movement, OpenSearch for log search, Deployed on Kubernetes in a multi-cloud environment (AWS, OCI, DO) using Github and Github Actions for CI/CD.
• You will lead a team of mostly senior backend engineers

Responsibilities:

• Lead the design, implementation, and growth of the Nimble Online Pipelines Platform, bringing it from an early stage product in production to hyper scale and maturity.
• Grow a healthy, collaborative engineering culture in line with the Nimble values
• Develop your team members professionally and personally, encouraging a fun and collaborative working environment
• Shape product and architecture decisions together with the Product Manager, UX Designer, and other engineering teams
• Lead engineering best practices such as CRs, Scrum sprint management, CI/CD, Knowledge sharing, Brainstorms, Retros and Postmortems
• Collaborate closely with other Team Leaders
• Lead processes to analyze and improve the efficiency, scalability, and stability of various system components.

Technical Aspects of the team’s responsibilites:

• Customer-Facing APIs of the Online Pipelines Product, and the whole Backend Logic that defines and connects what Online Pipelines are and how they connect with 3+ other products at Nimble
• Large Scale Orchestration Engine for real-time or long running pipelines that can run anything from code to data collection tasks to using external apis and running spark jobs or use model serving APIs
• Developing the Pipelines Platform as a developer platform for community developers to build and add pipelines as code to our platform, leveraging our ecosystem
• Development of a web scale index of web data that can be used to make graphs, as a cache, and for advanced persistence of state to power complex pipeline scenarios

Professional Requirements:

• Proficiency with Python
• We believe strong candidates will have 6+ years of experience as Back-End engineers and 3+ years of experience as Team Leaders
• Experience designing and implementing large-scale apps in production
• Excellent emotional intelligence and a desire to understand, develop and motivate people
• Excellent understanding of modern web apps’ lifecycle and architecture
• A demonstrated ability to design and architect back-end solutions independently from scratch
• An understanding of modern data engineering concepts in production at scale, is a plus.
• A demonstrated ability to collaborate with other teams effectively
• Significant experience with API design and implementation.
• Fluent in written and conversational English.