Data Engineer (Azure Data Factory & Power BI)

Overview:

This role will be responsible for creating and managing data orchestration with Azure Data Factory (ADF) pipelines and dataflows. The ideal candidate must have extensive enterprise-level data engineering experience, particularly in handling large and complex systems. Strong technical skills, clear communication, and the ability to adapt to changing project requirements are crucial for success in this role.

Key Responsibilities:

• Understand business requirements and provide valuable inputs from a data engineering perspective.
• Analyze and understand complex, large-scale data systems and workflows.
• Design, build, and manage simple to complex pipelines and dataflows using Azure Data Factory (ADF).
• Implement secure data modules with appropriate authorization and governance mechanisms (e.g., Role-Level Security, Data Governance).
• Stay responsive to changes in project size, scope, and function, adjusting processes as needed.
• Create and implement data visualizations and traditional reporting solutions using Power BI, ensuring the security of sensitive data.
  
  
  

Required Knowledge, Skills & Abilities:

• Expert-level experience in Azure Data Factory (ADF) and Power BI, particularly in large, enterprise environments.
• Strong background in data engineering within the Real Estate and Finance domains.
• Advanced knowledge of Azure SQL Database, Synapse Analytics, SSIS, SSAS, Logic Apps, and SQL Server best practices.
• Expertise in Azure Data Lake, SQL Databases, and Stream Analytics.
• Proficiency in Power BI, including DAX, and experience in implementing security measures (e.g., Row-Level Security).
• Familiarity with Azure DevOps, CI/CD pipelines, and modern data engineering practices.
• Practical experience with data modeling, database design, and Azure ETL processes.
• Clear understanding of the Data Warehouse lifecycle, contributing to design documents, unit tests, and code reviews.
• Knowledge of big data technologies like Databricks, Spark, Hive, and Azure Data Bricks is a strong plus.
• Experience working in Agile environments (Scrum, Lean, Kanban) is preferred.
• Excellent interpersonal and communication skills, with the ability to explain complex technical concepts clearly.
  
  
  

Qualifications & Experience:

• Bachelors or Masters degree in Computer Science or a related field.
• 6-10 years of data engineering experience, with a proven track record in enterprise-level environments managing large-scale, complex systems.
• Demonstrated expertise in managing big data and complex data flows, ideally within high-traffic environments.
  
  
  

Additional Desired Skills:

• Experience with the full Azure Synapse Analytics stack.
• Knowledge of big data technologies and frameworks such as Spark, NoSQL, and Hadoop ecosystem (Hive, Impala, HDFS, YARN, Oozie).
• Familiarity with Azure Active Directory (AD) for secure data access.