Listings

Cloud Data Migration Engineer

Permanent

£65,000–£75,000

Exeter, Devon

Cloud Data Migration Engineer - Exeter or Reading - 12 months fixed term contract - £70,000 - £75,000 + benefits

The Opportunity

The Cloud Migration Data Platform Engineer will work with the existing Data Management team to migrate existing on prem Data Lifecycle Management processes into Azure.

Key Duties

  • Assess existing data lifecycle pipeline processes for cloud migration suitability.
  • Work with the Data Architect to define target architecture for the cloud.
  • Define data consumption paths for new cloud architecture with key stakeholders, ensuring smooth transition from on prem to cloud solution and allowing a wide variety of consumption end points to facilitate downstream use cases.
  • Create automated data pipelines.
  • Put in place DevOps processes and performance tuning to optimise solution performance.
  • Migrate file-based systems to Azure-based storage solutions, including access and security requirements.
  • Work with Data Asset program to capture processing and source meta data within the enterprise data catalogue.
  • Put in place automated testing to enable ability to evolve data processing overtime without impacting existing processes.
  • Document new data pipeline processes including runbooks for pass back into Data Management team ensuring a smooth handover.
  • Upskill existing team members by demonstrating defined operational processes and reduce reliance of team members to run processes by automating.
  • Help define change management process for managing cloud based DLCM pipeline post migration.
  • Build in Azure cost reporting, including the tagging of infrastructure to enable understanding of cost disbursement across business units.
  • Design and build of metrics to understand pipeline performance and resource consumption for operation of the migrated DLCM processes for predominantly geospatial data.

Key Skills

  • Previous experience of ETL migration from on-prem to Microsoft Cloud repositories, and the technology stack involved.
  • Previous experience in Spatial Data with Azure Cloud technologies, such as Data Factory, Synapse, Data Bricks and Azure-based DBS.
  • Previous experience of FME Desktop/Server, with examples of authoring ETL/ELT processes following correct Best Practises, problem-solving, and finding efficiencies in already existing workspaces.
  • Be able to effectively communicate new methods and ideas to a well-established high performing team.
  • Project team management experience.
  • SQL/PostgreSQL/SQL Server/SQLite experience desirable.
  • Experience using JIRA reporting for project management desirable.

"Southern Lights Ltd act as an employment agency for the purposes of this advert"

For more information on the company and a detailed job specification please apply now.

Send to a friend
Cloud Data Migration Engineer
Direct link: https://www.southern-lights.co.uk/jobs/cloud-data-migration-engineer
View Privacy Policy
Register your profile
View Privacy Policy