Senior Data Engineer

Presight


Date: 20 hours ago
City: Abu Dhabi
Contract type: Full time
Overview

Job Description: Senior Data Engineer (12 months contract)

Location: Abu Dhabi

About Presight

Presight is an ADX-listed public company with Abu Dhabi based G42 as its majority shareholder and is the region’s leading big data analytics company powered by GenAI. It combines big data, analytics, and AI expertise to serve every sector, of every scale, to create business and positive societal impact. Presight excels at all-source data interpretation to support insight-driven decision-making that shapes policy and creates safer, healthier, happier, and more sustainable societies. Today, through its range of GenAI-driven products and solutions, Presight is bringing Applied AI to the private and public sector, enabling them to realize their AI strategy and ambitions faster.

The Opportunity

Presight is looking for an astute, proficient and qualified Senior Data Engineer to assess, analyze and work with data concepts, use-cases & complex new data sources to provide business insights to customers and support the implementation & integration of the data sources into the Presight AI platform.

Responsibilities

Key Responsibilities:

Functional

  • Solve challenging problems, using python coding skills.
  • Design, build and launch new data extraction, transformation & loading processes in production.
  • Web crawling, data cleaning, data annotation, data ingestion and data processing.
  • Reading and collating complex data sets.
  • Creating and maintaining data pipelines.
  • Continual focus on process improvement to drive efficiency and productivity within the team.
  • Use of Python, SQL, ES, Shell etc. to build the infrastructure required for optimal extraction, transformation, and loading of data.
  • Provide insights into key business performance metrics by building analytical tools that utilize the data pipeline.
  • Support the wider business with their data needs on an ad hoc basis.
  • Comply with QHSE (Quality Health Safety and Environment), Business Continuity, Information Security, Privacy, Risk, Compliance Management and Governance of Organizations policies, procedures, plans and related risk assessments.

Qualifications

Requirements:

  • Bachelor's degree in computer engineering, Computer Science, or Electrical Engineering and Computer Sciences.
  • 6+ years of programming experience, solid coding skills in Python, Shell, and Java
  • Experience with Web crawling, cleaning.
  • In-depth knowledge in the design and implementation of Spark jobs to execute, schedule, monitor, and control processes.
  • Experienced in Spark SQL and Postgres query languages .
  • Skilled in writing complex queries with joins for processing large datasets.
  • Proficient in using containerization technologies such as Docker.
  • Experienced with orchestration tools like Kubernetes.
  • Adept at implementing testing and monitoring systems for data pipelines to ensure high availability and reliability.
  • Experience with tools like Apache Kafka and Apache Flink for real-time data processing .
  • Skilled in using data orchestration tools like Apache Airflow and Apache NiFi.
  • Strong understanding of Elasticsearch architecture, queries, and ingestion techniques.
  • Experience with solution architecture, data ingestion, query optimization, data segregation, ETL, ELT, AWS, EC2, S3, SQS, lambda, Elastic Search, Redshift, CI/CD frameworks and workflows.
  • Working knowledge of data platform concepts - data lake, data warehouse, ETL, big data processing (designing and supporting variety/velocity/volume), real time processing architecture for data platforms, scheduling and monitoring of ETL/ELT jobs.
  • PostgreSQL and programming (preferably Java, Python), proficiency in understanding data, entity relationships, structured & unstructured data, SQL and NoSQL databases.
  • Knowledge of best practice in optimizing columnar and distributed data processing system and infrastructure.
  • Experienced in designing and implementing dimensional modelling.
  • Knowledge of machine learning and data mining techniques in one or more areas of statistical modelling, text mining and information retrieval.
  • Strong Analytical Skills & Problem-solving skills.

Ideally, you’ll also need

  • In-depth market and domain knowledge
  • A passion for constant improvement
  • An innovative and creative approach to problem-solving
  • Excellent communication skills

What We Look For

If you are a performance-driven, inquisitive mind with the agility to adapt to ambiguity, you will fit right in. You should be eager to explore opportunities to build meaningful collaborations with stakeholders and aspire to create unique customer-centric solutions. Bias for action and a passion to conquer new frontiers in the AI space is at the heart of the Presight community.

What Working At Presight Offers

Culture: An open, diverse and inclusive environment with a global vision that encourages personal growth and focuses on ground-breaking, industry-first innovations.

Career: Outstanding learning, development & growth opportunities via structured training programs and innovative, high-tech projects.

Rewards: A competitive remuneration package with a host of perks including healthcare, leave benefits and more.

How to apply

To apply for this job you need to authorize on our website. If you don't have an account yet, please register.

Post a resume

Similar jobs

Senior Pipeline Engineer

KBR, Inc., Abu Dhabi
18 hours ago
TitleSenior Pipeline Engineer"Belong, Connect, Grow, with KBR! The KBR team of teams delivers future-forward science, technology and engineering solutions and mission-critical services that help governments and companies around the world accomplish their most important objectives, while also helping achieve their sustainability goals.KBR Sustainable Technology Solutions provides holistic and value-added solutions across the entire asset life cycle. These include world-class licensed...

Interior Fitout Draughtsman | Royal Joinery

United Al Saqer Heavy Equipment LLC, Abu Dhabi
22 hours ago
Job DescriptionRoyal Joinery has its own dedicated workshop to assist clients on high-profile projects that require international expertise and a degree of workmanship that eliminates the need to source subcontractors and third-party furniture suppliers. Our primary focus is on carpentry and woodwork specialities across various projects and sectors through the delivery of sophisticated, qualified experts in their field and innovative...

Planning Manager

AtkinsRéalis, Abu Dhabi
23 hours ago
AtkinsRéalis, a world-leading design, engineering, and project management organization. We connect people, data and technology to transform the world's infrastructure and energy systems. We are active in the Middle East and India for over 10 years with our regional headquarter in Dubai, and offices in Abu Dhabi, Kuwait, Qatar, Oman, and Saudi Arabia. Our Middle East and India presence is...