Senior Data Engineer

A financial services company is seeking a talented and experienced Senior Data Engineer to join its team. This role involves designing, building, and maintaining robust data infrastructure to support the analysis of large data sets. The ideal candidate will have expertise in data management, ETL processes, data warehousing, and big data technologies such as Spark, Hadoop, and NoSQL databases.

The Senior Data Engineer will collaborate with data scientists, analytics teams, and other stakeholders to align data infrastructure with business goals. Strong communication and problem-solving skills are essential, as the role requires tackling complex challenges in a dynamic and fast-paced environment.

Responsibilities

• Collaborate with data scientists and stakeholders to prioritize data engineering projects and ensure alignment with business objectives.
• Design, build, and maintain data pipeline architectures for extracting, transforming, and loading data from diverse sources, including APIs, data streams, and data stores.
• Monitor and optimize data infrastructure performance, scalability, and security.
• Stay informed about the latest trends in data engineering to identify opportunities for innovation.
• Implement data privacy and security measures to comply with industry standards and frameworks.
• Enhance team dev-ops capabilities and contribute to process improvements.
  
  

• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
• At least 5 years of professional experience in data engineering, including building and maintaining data pipelines and data warehouses.
• Strong experience with Spark, Scala, and Python for developing high-performance, maintainable code.
• Proven knowledge of AWS services (EC2, S3, Athena, Lambda, EMR).
• Familiarity with data warehousing concepts and technologies, including columnar storage, data lakes, and SQL.
• Experience with data pipeline orchestration tools such as Airflow.
• Strong problem-solving skills and the ability to work independently or in a team.
• High proficiency in English (written and spoken).
• A collaborative team player with excellent communication skills.
  
  

• Expertise in Vertica or Redshift, including query optimization and performance tuning.
• Experience with machine learning or data science projects.
• Knowledge of data governance and security practices, including compliance with regulations such as GDPR and CCPA.
• Advanced understanding of Spark internals, including tuning and query optimization.