Manager - Data Engineering

About the Department of Finance – Abu Dhabi

Established in 1966 with the objective of strengthening the financial sustainability of the Emirate, DoF is responsible for the development and execution of the Abu Dhabi government’s annual budget and for overseeing government returns, expenditures,

Objective: Take ownership of the strategy, architecture, and delivery of a secure, AI-optimized data platform. This platform will modernize the legacy state, institute data-as-a-product governance, and enable high-scale operations. It will align with the Department of Finance's Data Strategy and AI vision, positioning the department as the regional benchmark for data-driven public finance excellence.

Essential Duties

• Architect and oversee end to end Azure data platforms Data Factory, Databricks or similar ensuring scalability, security, and cost efficiency.
• Define and execute the enterprise data strategy and maturity roadmap, moving the organization from ad hoc reporting to fully productized, governed data services.
• Design, build, and manage high throughput data pipelines and ELT workflows (ADF pipelines/data flows, Spark, Delta Live Tables) with automated CI/CD.
• Lead migration of legacy EDWs and SSIS/ETL workloads (SQL Server, Teradata, Hadoop) to modern Azure architectures, balancing lift and shift and refactor approaches.
• Develop domain aligned data marts and semantic layers powering BI, AI, and real time decisioning at >100 TB scale.
• Champion a “data as a product” operating model defining dataset ownership, SLAs, observability, lineage (Purview), and version-controlled contracts.
• Implement robust data quality frameworks, test automation, and performance tuning in SQL, Python, and Scala to guarantee reliable, audit ready data.
• Establish secure architectures with RBAC, private endpoints, encryption, and enforce FinOps practices for optimal spend management.
• Integrate streaming and event driven data (Kafka/Event Hubs, Azure Functions) into real time analytics and AI workloads.
• Mentor and lead cross functional squads of data engineers, analytics engineers, and SREs; align delivery to business OKRs and foster a culture of continuous improvement.
• Engage finance and analytics stakeholders to translate business requirements into governed, reusable data products and services.
• Perform other duties and responsibilities as requested by the Executive Director or management.
• Provide the necessary support to the direct manager, and perform any other related tasks as requested.

Qualifications and Experience Required

• Bachelor's degree in computer science, business analytics or a similar field relevant to the job duties.
• A specialized certificate in the field of the job is preferred.
• Preferable to have 10-14 years of experience in a similar field/position.
• At least 5 years’ experience in end-to-end data-engineering, and architecting cloud data platforms on Azure
• Preferable experience with financial domain.
• Expert in Azure Data Factory (pipelines, dataflows, CI/CD), data bricks or similar, Analytics, Data Lake Gen2, Delta & parquet formats.
• Designs and executes enterprise data strategy & maturity roadmaps from “ad-hoc reporting” to fully productised, governed data services.
• Proven track record building domain-aligned data marts / semantic layers that power BI, AI and real-time decisioning at scale (> 100 TB).
• Led migration of legacy EDWs & SSIS/ETL workloads (SQL Server, Teradata, Hadoop) to Azure; balances lift-and-shift with refactor to ELT/Spark.
• Champions a “data-as-a-product” mindset—defines ownership, SLAs, observability, lineage (Purview) and versioned contract-driven datasets.
• Hands-on with modern stack: Databricks / Spark, Delta Live Tables, dbt, Kafka/Event Hubs, Azure Functions, Terraform, Ansible etc.
• Strong SQL performance tuning plus Python/Scala for orchestration and data-quality frameworks; automates tests & monitoring.
• Builds secure, cost-optimized architectures (RBAC, private endpoints, encryption, cost tagging) and drives FinOps discipline.
• Seasoned people leader, mentors cross-functional squads (data engineers, analytics engineers, platform SREs) and aligns delivery to business OKRs.