Cloud Data Migration Engineer - Exeter or Reading - 12 months fixed term contract - £70,000 - £75,000 + benefits
The Cloud Migration Data Platform Engineer will work with the existing Data Management team to migrate existing on prem Data Lifecycle Management processes into Azure.
- Assess existing data lifecycle pipeline processes for cloud migration suitability.
- Work with the Data Architect to define target architecture for the cloud.
- Define data consumption paths for new cloud architecture with key stakeholders, ensuring smooth transition from on prem to cloud solution and allowing a wide variety of consumption end points to facilitate downstream use cases.
- Create automated data pipelines.
- Put in place DevOps processes and performance tuning to optimise solution performance.
- Migrate file-based systems to Azure-based storage solutions, including access and security requirements.
- Work with Data Asset program to capture processing and source meta data within the enterprise data catalogue.
- Put in place automated testing to enable ability to evolve data processing overtime without impacting existing processes.
- Document new data pipeline processes including runbooks for pass back into Data Management team ensuring a smooth handover.
- Upskill existing team members by demonstrating defined operational processes and reduce reliance of team members to run processes by automating.
- Help define change management process for managing cloud based DLCM pipeline post migration.
- Build in Azure cost reporting, including the tagging of infrastructure to enable understanding of cost disbursement across business units.
- Design and build of metrics to understand pipeline performance and resource consumption for operation of the migrated DLCM processes for predominantly geospatial data.
- Previous experience of ETL migration from on-prem to Microsoft Cloud repositories, and the technology stack involved.
- Previous experience in Spatial Data with Azure Cloud technologies, such as Data Factory, Synapse, Data Bricks and Azure-based DBS.
- Previous experience of FME Desktop/Server, with examples of authoring ETL/ELT processes following correct Best Practises, problem-solving, and finding efficiencies in already existing workspaces.
- Be able to effectively communicate new methods and ideas to a well-established high performing team.
- Project team management experience.
- SQL/PostgreSQL/SQL Server/SQLite experience desirable.
- Experience using JIRA reporting for project management desirable.
"Southern Lights Ltd act as an employment agency for the purposes of this advert"
For more information on the company and a detailed job specification please apply now.