Lead Data Engineer - Python, Remote working with 1-2 days per month in Central London. Up to £90K
Via analysis of ethically derived datasets, this digital agency deliver powerful trend reporting to its clients that accurately reflect the thoughts and behaviours of people across the social web.
They’re currently looking for a Lead Data Engineer interested in leading projects at the forefront of the intersections between cyber security, current affairs, civil society and the digital space.
This Lead Data Engineer, will lead a small technical team including data engineers and infrastructure lead to consolidate and scale the technical product offering. Within their secure-by-design AWS infrastructure, the Lead Data Engineer will be comfortable running daily stand-ups and working closely with the Client Servicing team to ensure best-in-class national security product continues to evolve and maintain the precision and quality. This is an opportunity to be part of a small but ambitious team looking to scale rapidly throughout 2021 and 2022, with excellent career progression for the right candidate.
This Lead Data Engineer will:
- Have experience working in the private sector, building technical products for government customers in the field of open-source intelligence data solutions
- Have an in-depth understanding of the open-source information environment in particular the
opportunities and challenges presented by such an environment
- Be excited, motivated and have knowledge of analytics and OSINT products in the national security space
- Be comfortable working for Defence and Government customers in areas such as Disinformation, Terrorism and Extremism
This Lead Data Engineer’s responsibilities would be:
- Setup and maintain secure environments for data management and access on AWS
- Automate monitoring and evaluation of data quality
- Work with the Client Servicing team and PM to deliver data products to senior government clients
- Work closely with senior stakeholders to outline project scope and requirements
- With the technical PM, be able to rapidly and professionally advise on feasibility of technical requirements in terms of data availability and appropriateness.
- Work on pre-sales to respond to immediate business opportunities and longer-term development
projects
- Overall responsibility for the setup of API and third-party data pipelines for batch and near real-time data collection across platforms such as Twitter and Youtube
- Scale projects to ensure data is available as required
- Scope out new data collection opportunities and manage access
- Scope requirements and target architecture for project propositions
- Manage and support the rest of the tech team in debugging, blockers etc
- Oversee and run daily standups to distribute and prioritise tasks
- Architect data solutions for new projects and new technology
- Liaise with CEO and Business development on pitching new work, including costing technology, resource and timeframe
- Comfortable with being ‘hands-on’ as well as manage the more strategic / technical aspects of code
- Have experience in working with data scientists to productionise data science and AI models
- Professional attitude at all times and confident speaking directly to senior clients if required
Essential:
Security Cleared to SC
Technical:
- SQL
- Python
- AWS
- Data warehousing
- Serverless functions (AWS Lambda)
- Docker
- AWS Glue (PySpark)
- Desirable
- Terraform
- CI/CD pipelining
Are you the Lead Data Engineer that this high-flying business need right now? Please get in touch quoting job reference AP1536.