Best Option

Find the right accountancy solution for your requirements

Share
  • Location: London, London
  • Rate: Up to 570.00
  • Start Date: 04/09/2019
  • Duration: 6 months

Applicants must be eligible to work in the specified location


 


Data Platform Engineer (Python/GCP/Spark/Terraform/Kubernetes) required by my client, an eCommerce brand based in Central London 



This is a 6 month contract paying up to £570/day and is highly likely to extend



As this global eCommerce brand look to move from AWS over to GCP they need a Data Platform Engineer with strong python coding skills and experience in both AWS and GCP

Day to day jobs and experience expected 



– Develop the Back End data systems on AWS. Designed to support seamless integration, deployed across 20 countries, integrating thousands of partner brands using Spark and python for processing and parquet with Athena/Presto as a query engine

– Developed the architecture for a distributed crawler operating using server-less AWS lambda functions along with SQS and SNS that execute concurrently across all countries and retailers.

– Work extensively with the search infrastructure based on Elastic Search to improve its indexing and query performance, scalability and relevance of results

– Assisted with the migration from on-premise infrastructure to GCP using Big Query, Dataflow and Dataproc 

– Introduce a Data Warehouse for a product database to lay the data in dimension and fact tables that allowed the development of traditional BI analysis on the pricing data 



Tech stack

Python

AWS, GCP cloud technologies

Terraform 

Hadoop

Spark



This is an opportunity to join one o the fastest growing brands in the country, work with some seriously talented people on vastly interesting projects. 



If you would to understand more please get in touch



Apply