Senior Data Engineer

Prisma Photonics, a fast-growing startup, transforms infrastructure monitoring with optical fibers. We eliminate the need for extra sensors by offering sensor-free solutions to monitor electrical power grids and oil & gas pipelines across thousands of kilometers.

Our fiber-sensing technology, integrated with AI and machine learning, enables our customers to achieve environmental and renewable energy targets, ensuring smooth utility operations on their path to net-zero emissions.

We are looking for the best minds and spirits to join us in our journey. We know our product is only as great as the individuals building the hardware and software and harnessing data for good causes. Being a great team member means being eager to learn and grow, able to challenge while accepting being challenged, and working for the team and the product with enthusiasm and passion.

We are now hiring a talented, self-driven and passionate Senior Data Engineer to build and maintain optimized and highly available data pipelines that facilitate deeper analysis and reporting.

What you'll do

• Design, develop, and maintain scalable data pipelines that integrate data from multiple sources, including APIs, databases, streaming platforms, edge computing devices, cloud services, and on-premise systems.
• Design reliable and business-critical data processing workflows for batch and real-time use cases.
• Develop data access services and APIs that enable efficient communication between edge devices, cloud infrastructure, and on-premise environments.
• Design and optimize storage solutions for structured, semi-structured, and high-dimensional sensor data to support AI model training, inference, and analytics.
• Strong understanding of distributed systems and scalable data processing architectures.
• Build and maintain scalable streaming data pipelines for low-latency processing and event-driven architectures.
• Analyze existing data architecture, storage models, and processing workflows, and continuously improve performance, scalability, reliability, and maintainability.
• Optimize cloud infrastructure and data storage costs while maintaining high availability and low-latency access.
• Collaborate closely with Software, AI, Algorithms, DevOps, and Product teams to translate business requirements into scalable technical solutions.
• Design monitoring, observability, and operational processes for data platforms.

What you bring

• Bachelor's degree in Computer Science, Engineering, Mathematics, or a related quantitative field.
• 5+ years of professional experience in Data Engineering or a related role.
• Strong experience designing and implementing large-scale data pipelines using orchestration frameworks such as Apache Airflow, Prefect, or similar.
• 5+ years of software development experience, including at least 2 years of Python development.
• Strong knowledge of relational and NoSQL databases such as PostgreSQL, MySQL, MongoDB, Elasticsearch/OpenSearch, ClickHouse, or similar technologies.
• Experience designing and implementing streaming and event-driven data architectures using technologies such as AWS Kinesis, Amazon SQS, RabbitMQ, Kafka, or similar messaging systems.
• Experience designing REST APIs and backend services (FastAPI or similar frameworks).
• Experience working with AWS cloud services (S3, EC2, Lambda, CloudWatch, IAM, etc.).
• Experience with Git, Docker, CI/CD pipelines, and modern software engineering practices.
• Excellent communication and collaboration skills with engineering, AI, and Product teams.
• Self-driven, innovative, and continuously looking for ways to improve systems and processes.

Nice to have

• Experience with Kubernetes and container orchestration.
• Experience with distributed computing platforms and distributed data processing systems.
• Experience building ML data pipelines supporting training and inference workloads.
• Experience working with large-scale sensor, IoT, or time-series data.
• Experience with monitoring and observability tools such as Grafana, Prometheus, ELK, Kibana, or OpenSearch.
• Experience working in edge computing or hybrid cloud environments.