Data Developer

Responsibilities:

Data Feed Development:

Design and implement data feeds from external sources, ensuring accuracy, reliability, and efficiency.

Develop automated processes for data collection and ingestion, utilizing appropriate tools and technologies.

Data Scraping, Analysis and Modeling:

Conduct data scraping from diverse external sources to gather relevant information.

Perform heuristic analysis to find trends and patterns in complex data sets.

Build, maintain and refine LLM-based models to predict the escalation of toxicity within textual data over time.

Optimization and Feedback Integration:

Continuously optimize data feeds based on feedback from researchers and analysts, improving data quality and relevance.

Collaborate closely with stakeholders to understand requirements and implement necessary adjustments.

Project Management:

Gather requirements from stakeholders and translate them into detailed Product Requirement Documents (PRDs).

Develop and execute project plans, breaking tasks into smaller manageable components.

Collaboration and Communication:

Work effectively with a diverse team of multicultural freelancers, fostering a collaborative and inclusive work environment.

Maintain clear and open communication channels to facilitate seamless coordination and feedback exchange.

Strong working knowledge of AWS services (S3, Bedrock, Lambda) and cloud architecture.

Proficiency with Git and collaborative development workflows.

Requirements:

Technical Skills:

• Proficient in Python programming: The candidate must have at least 3 years of extensive hands-on experience in Python, capable of writing efficient, clean, and well-documented code.
• Hands-on experience with LLMs, prompt engineering and fine-tuning methodologies.
• Experience with data processing libraries: Candidates should have practical experience with PySpark and/or Pandas for data processing and analysis. Proficiency in handling large datasets and performing complex data transformations is essential.
• Familiarity with AWS Services: Knowledge of AWS cloud services is required, including but not limited to EC2, S3, Lambda.
• Project management skills: The role demands excellent project management capabilities, including planning, execution, and tracking project progress. The ability to manage timelines, resources, and stakeholder expectations is crucial.
  
  

• Independent: The ideal candidate should be able to work independently, with minimal supervision, efficiently managing their workload and making informed decisions.
• Self-Learner: The ability to learn new technologies and methodologies quickly and effectively is essential. Candidates should demonstrate a strong capacity for self-directed learning and staying current with industry trends.
• Proactive: We are looking for individuals who are proactive in nature, always looking for ways to improve processes, solve problems before they escalate, and take initiative in their work.
  
  

• Candidates with experience in using Databricks for data engineering and analysis will have an advantage.
• Proficiency in Arabic