Main characteristics
- Location
- Milton Keynes, Buckinghamshire
- Company
- Agile Recruit
- Salary
- £70000 - £75000/annum
- Employment type
- Permanent
- Category
- IT
Job summary
Azure Data EngineerPermanent
Remote
70k
Responsibilities:
Design and implement scalable and secure Azure-based data platforms to manage large and intricate datasets
Craft Data Lake and Data Warehouse solutions to effectively store and organize data for analytical purposes
Utilize Data Vault and Dimensional modeling methodologies to design robust and adaptable data models
In-depth experience of Apache Kafka or other streaming technologies (Synapse streaming or Event hubs) for real-time data ingestion and processing
Develop automated, reusable, and efficient batch and streaming data pipelines to ensure seamless data ingestion and transformation
Collaborate closely with Data Science teams to provide data support and fulfill their data requirements
Partner with Reporting and Visualization teams to deliver insightful data visualizations that drive business decisions
Design and implement solutions to handle Personal and Sensitive data in compliance with data privacy regulations
Adhere to DevOps practices and work within an Agile environment to promote continuous delivery and collaboration
Experience:
In-depth experience of Apache Kafka or other streaming technologies (Synapse streaming or Event hubs) for real-time data ingestion and processing
Proven expertise in Azure Data Lake (ADLS Gen2) and its capabilities for storing and managing large datasets
Proficiency in Synapse Dedicated SQL Pool and SQL Data Warehouse, including the ability to create tables, stored procedures, and other database objects
Comprehensive understanding of Azure Data Factory and Mapping Data Flows, enabling the creation of metadata-driven pipelines
Familiarity with Azure Databricks for batch and streaming data processing, with the ability to develop metadata-driven pipelines
Knowledge of Data Lake storage formats such as Parquet and Delta Lake to efficiently store and manage data
Experience in handling complex data types such as JSON and XML to ensure accurate data processing
Familiarity with Azure Event Hubs and Event Grids for event streaming and triggering data pipelines
Understanding of Azure Functions and Cognitive Search for building serverless applications and enhancing search capabilities
Proficiency in a structured programming language like Python or C# for data manipulation and pipeline development
Inform me about similar jobs:
