Big Data Engineer with Python/Scala BPDTS Ltd
| Reference | BPDTS 63/19 |
| Number of vacancies | 6 |
| Location | London, Leeds or Newcastle |
| Salary | Up to £49,337 |
| Work hours | 37.5 |
| Employment type | Permanent |
| Job benefits | Our flexible benefits package includes employer pension contributions of up to 10%, a performance related annual bonus of up to 10%, and lots more.
|
| Closing date | 29 March 2019 |
Job description
Do you want to be part of something really big?
Do you have expertise in Big Data pipelining in the Cloud?
Are you Proficient in Hive/Hadoop Stack, Python or Scala?
Are you ready to be part of an epic digital transformation?
Then click APPLY now, we’d love to hear from you…
We currently have a Data & Analytics community of 80 Engineers and we are looking for additional Big Data Engineers with expertise in data pipelining in Cloud/Cloudera.
You’ll help us to take raw data and land it so that DWP can shape real change to products and services used by millions. This includes one of our flagship projects the “Child Analytic Platform” ( CAP) where we will look at data analysis to help protect children.
Watch Tim talk about his role as a Data Engineer…
About you and the role
BPDTS Ltd is a new organisation that provides dedicated services to the Department for Work and Pensions (DWP).
Our digital, data and technology specialists work closely with experts in DWP Digital’s community to make digital services quicker, simpler and more efficient for the millions of people that use them every day.
As public servants, we help make a difference to people at critical times in their lives – as they look for work, deal with illness or disability, support children after separation, or plan for retirement.
Operating on a scale unrivalled almost anywhere, our people work with experts in DWP Digital’s community to create effortless, secure and intuitive digital services that are used by millions of people.
What are we looking for?
Technical Skills
- Cloud/ Cloudera
- Spark SQL
- Hive SQL
- Spark Streaming
- Python
- Scala
- HDFS
- Unix
The key skills and experience we value for this role are…
- Hands-on experience on Data Engineering projects where you have delivered data processing pipelines in a cloud or on premise environment (distributed ETL and machine learning) using Hadoop/Hive to support different analytics use cases, relating to high-volume, complex and distributed datasets.
- Experience of having created and maintained data pipelines, data streaming, working with large sets of unstructured and structured data and creating curated datasets for end users.
- A good understanding of big data engineering techniques and concepts using Hadoop; data streaming, data transformation, data loading strategy, data archiving, data reconciliation, data manipulation, data integration, error handling, error logging mechanism, standards and best practices with hands-on experience in developing data solutions.
- Experience of developing data solutions on Oracle, Cloudera or HortonWorks platforms.
- An ability to quickly comprehend and convert business requirements into technical solutions.
- Significant and demonstrable experience implementing Data Engineering best practices
Desirable Skills/Experience
- Core Java Spark
- Kafka
- NoSQL eg Hbase, MongoDB, Cassandra
- Sqoop
- Flume
- Java
- OOzie
- GitHub
- Java Programming
- Background in DevOps / Data Operations
Where will you work?
You’ll join us in one of our easy-to-reach digital hubs in either Newcastle, Leeds or London.
Each of our nationwide hubs has been designed for and by members of our digital community. We provide the latest tech and tooling and foster a collaborative culture to help our agile, multi-disciplinary teams think big and try new things.
Our central hubs have the best of each location on our doorstep, and we have a range of facilities inside from coffee shops and restaurants, to table tennis, a swimming pool, gym facilities and loads more.
We are an equal opportunity employer and we welcome applications regardless of age, gender, race or sexuality.
What is our offer?
- We’ll reward you with a very competitive salary of up to £49,337 dependant on level of expertise
- You will be eligible for a pension with employer contributions of up to 10%
- We can offer you a performance-related annual bonus of up to 10%
- We’ll help you balance your life and work
- We’ll give you a generous leave allowance, with the option to buy and sell extra days
We also offer much more as part of a flexible benefits package – including a cycle to work scheme and lots of family friendly perks – which can be tailored to your lifestyle.
Building our future
We believe the work we do really matters, so we’re serious about investing in you.
You’ll have access to a huge suite of training and learning opportunities to complement the new skills you’ll acquire on the job. You’ll get the chance to share knowledge with colleagues from across our digital communities, and to grow your skills through our coaching and mentoring programmes.
For further information and to start your application click Apply. You will need to register your CV and add a supporting statement on our online application portal.
Please contact tia.nicholas@dwp.gsi.gov.uk if you require any additional details.