Big Data jobs and careers
As the use of data in business gets more and more prevalent the ability to quickly and effectively produce data algorithms that can analyse large complex datasets is crucial. In order to do this, the right data infrastructure needs to be in place.
Why consider a career in Big Data?
Data Engineers build databases, data warehouses and data lakes to service the needs of the Data Analysts and Data Scientists in their businesses. This area is fast growing and Data Engineers are in massive demand as so many companies race to unlock the value in their data.
Typical role titles include
Big Data Engineer
Business Intelligence Developer
Business Intelligence Engineer
Big Data Developer
Data factory Developer
Data Availability Engineer
Data Science Engineer
Big Data Software Engineer
Data Warehouse Engineer
SQL Server Engineer
Data Infrastructure Engineer
Data Infrastructure Programmer
Data Infrastructure Developer
Data Analytics Engineer
Big Data Architect
Big Data Specialist
Big Data Lead
Head of Big Data
Data Architecture specialist
Head of Data Architecture
Career progression example
Junior Data Engineer
Senior Data Engineer
Lead Data Engineer
Head of Data Engineering
At an entry level, junior/graduate Data Engineering professionals on average will be earning between £30k-£35k. At the mid-level you’d be looking to earn £50k-£60k and seniors would be looking to earn £70k+.
Is it right for me? The skills it takes...
The skills needed can be split into 3 categories; programming skills, data science understanding and problem-solving.
Programming skills: Java | Python | Golang | Cython | Software Design - A background in APIs e.g. web APIs and static/dynamic libraries | ML Libraries - e.g. TensorFlow, CNTK, MLib, Spark, Kafka, Kinesis, Redshift.
What qualifications does it take?
A BSc, Masters (MSc) or Doctorate (PhD) in a maths, physics, statistics, artificial intelligence (AI), software engineering, engineering or computer science field with technical skills in programming. Demonstrable experience working with large data sets or alternatively candidates will work in Business Intelligence and then do online courses /self-learning/ get trained up on Big Data Engineering and Architecture and can move into these roles through this route.
Our Data recruitment team
Our latest Data jobsSearch all jobs
— Scale-up Tech FirmLocation:Central LondonSalary:Up to £75kGreenfield design and build. Architect data infrastructure for a cloud-native company. Great engineering culture, tech stack and family atmosphere. This scale-up technology firm is growing rapidly through both organic growth and acquisition. They require a Data Engineer that is up to the challenge of owning the design, build and management of their cloud data infrastructure. What you’ll be doing As the Data Engineer, you will evaluate serverless database technologies that will effectively host, integrate and migrate current and future customer data. You will design and architect the data pipelines, suggest different tools and continually improve the set-up. This role requires a variety of engineering activities including; prototype development, architecture, ETL, process automation and more. What you’ll need to apply Experience with Python/Java/Scala etc. Experience with Cloud e.g. AWS, GCP, Azure Ability to design, build and manage data infrastructure Experience of working on greenfield projects desired Ability to communicate problems and solutions to stakeholders What you’ll get in return for your experience A base salary of up to £75,000 plus benefits including work from home, training and investment in your skills and much more. What’s next Please get in touch with Scott with an up-to-date-cv today. Please don’t hesitate to email/call to discuss the finer details.
— Market-Leading E-Commerce BusinessLocation:MansfieldSalary:Up to £36kGain experience with advanced BI tools. Varied workload in a growing business. Contribute to complex business and technology projects. This leading E-commerce Supplier requires a BI Analyst to provide the information, insights and processes needed to drive the company forward. The role will support senior management with processing the vast amount of data that is created and consumed on a daily basis. What you'll be doing You'll collaborate with a wide range of stakeholders to develop and apply your BI analytics. This role covers various sources of information including customer, transactional, financial and website data. As a result, you'll capture and deliver reporting and analysis requirements to management teams across the business, visualising performance, identifying insights and trends and comparing with appropriate benchmark data. You'll capture and deliver data management requirements for a range of ongoing and new projects. This is a great opportunity for someone who is interested in developing data science skills and to work on customer behavioural modelling. What experience you'll need to apply Minimum of 1 years' experience in BI Solid SQL Experience with Tableau, Power BI, Qlik or similar Exposure to Python or R desirable Ability to communicate to senior stakeholders What you'll get in return for your experience A competitive salary of up to £36,000 plus bonus, benefits and the chance to develop data science skills in a commercial setting. What's next Please get in touch with Scott with an up-to-date CV today, don't hesitate to call/email to discuss the role in more detail.
— Leading Digital Transformation AgencyLocation:LondonSalary:£90K - £120KExcellent opportunity to work for a leading Global Digital Transformation Agency. Work on interesting and exciting project with cutting edge tech. 90% clients based in London. This leading Global Digital Transformation Agency are looking for experienced Data Architects who want to work on client-facing projects. They have a mixture of private and public sector clients and work on cutting edge digital transformation projects. What you’ll be doing You will be leading in the design and build of data architecture and engineering solutions for end clients. The work will be a combination of office based, remote and on site work. The client projects might be around dynamic pricing applications, experiential marketing solutions or migrations / upgrades of technology. What experience you’ll need to apply Data Architecture experience. Strong background in Python, Scala or Java. Experience of designing and developing cloud based data solutions. Experience of modern data solutions such as NoSQL, graph or real-time systems. Strong stakeholder management and communication skills. Excellent understanding of business use cases for analytics and how to provide the data. Excellent stakeholder management and influencing skills. What you’ll get in return for your experience A competitive salary of up to £120K + benefits + bonus. What’s next? Please get in touch with Alex with an up to date CV today. Don’t hesitate to call / email to discuss the finer details.
— Join one of the largest academic publishers in the World.Location:OxfordSalary:Up to £40,000Be a key figure in deriving insight from a global academic data stream Develop genuinely interesting customer segmentation insight Work on new and engaging projects from start to completion What you’ll be doing: You will be a key figure in deriving insight from the largest data stream flowing from global academic sales. This a massive reservoir of incoming data so you will be comfortable working with this amount of volume. The dashboards you create will be shared with stakeholders globally and will help drive forward their data-focused strategy. This will be a hands-on analytics role with the opportunity to have a real impact on the business. What experience you’ll need to apply: Be very proficient with Alteryx (essential) A very good working knowledge of SQL Salesforce familiarity Experience using at least one of the following data visualization platforms: Tableau, Power Bi, QlikView. An ability to be creative and derive meaningful insights from available data What you’ll get in return for your experience: Salary up to £40,000 but could pay higher for the right candidate. A generous benefit package Join a very good data team that helps train people up Opportunity to support personal development, they have helped people through their Alteryx courses A fantastic company culture What Next: Please get in touch with Kye Williams with an up to date CV today. Don’t hesitate to call / email to discuss in more detail.
— Analytics ProviderLocation:BathSalary:Up to £70KOn-prem and Cloud data development. Define analytic standards for the industry. Collaborate with software engineering and DevOps teams. This client is migrating to the cloud to support analytical services offered to customers. This project is set to define industry standards so the incoming hire will have a huge impact on this. What you'll be doing As a Senior Data Engineer, you will model, pipeline and integrate the current set-up to the cloud to support the analytics platform. This will require collaboration with the development teams to ensure the data is accessible across the platform. What you’ll need to apply Proven experience as a Data/Software Engineer Experience with AWS, Azure, GCP or similar SQL Server experience Sourcecode experience e.g Github or Bitbucket Data visualisation experience e.g Power BI, Tableau preferred Object-orientated or functional programming experience preferred What you’ll get in return for your experience A base salary of up to £70,000 DOE plus benefits including WFH 2 days a week. What’s next Please get in touch with Scott with an up-to-date CV, don’t hesitate to email/call to discuss the role in more detail.
— Data Science ConsultancyLocation:Central LondonSalary:Up to £90kDevelop Cloud, Big Data and on-prem solutions. Enable data science activities for clients. Cutting-edge data engineering projects. This global consultancy pride themselves on improving client data science and engineering activities. They are seeking data engineers that are capable of developing a wide range of solutions for their customers. What you’ll be doing You will work on-site with clients to provide them a variety of data solutions. This will require working with serverless and on-prem technologies to effectively host, integrate and migrate valuable client data. This role requires a variety of engineering activities including; prototype development, ETL, architecture, process automation and more. What you’ll need to apply Experience with Java/Scala/Python etc. Experience with Cloud services Experience with Big Data Tools e.g Spark, Hadoop Ability to communicate problems and solutions to stakeholders Degree in Computer Science, Physics or similar What you’ll get in return for your experience A base salary of up to £90,000 plus bonus & benefits including work from home, training and investment in your skills and much more. What’s next Please get in touch with Scott with an up-to-date-cv today. Please don’t hesitate to email/call to discuss the finer details.
— FinTech FirmLocation:CardiffSalary:Up to £65kOn-prem and cloud data development. Work in a team of data engineers and architects. Develop BI solutions, databases and scalable infrastructure. This client is migrating to the cloud to utilise new data services. They are a data-driven firm, it guides their strategic decision-making and service/product development. As a result, they require a Senior Data Engineer to join the team of data engineers to support this migration. What you'll be doing As a Senior Data Engineer, you will be influential in supporting modern, scalable, cloud-based data infrastructure. You will support the design, roll-out and execution of the data strategy across multiple products and collaborate with other data teams to ensure that the data is integrated and accessible. This role requires a range of data engineering activities including; prototyping, pipelining, architecture, process automation and more. What you’ll need to apply Proven experience as a Data/Software Engineer Experience with AWS, Azure, GCP or similar SQL Server experience CI/CD experience preferred e,g Docker, Azure DevOps Object-orientated or functional programming experience preferred Agile experience preferred What you’ll get in return for your experience A base salary of up to £65,000 plus bonus and benefits. What’s next Please get in touch with Scott with an up-to-date CV, don’t hesitate to email/call to discuss the role in more detail.
— ConsultancyLocation:BristolType:Salary:Up to £450 a dayExcellent opportunity to work for a growing consultancy firm Work on interesting projects Remote working and opportunities to travel abroad for projects This fast-growing Data Consultancy are looking for some interim data engineering & architecture support across a few key projects. What you’ll be doing You will be leading in the design and build of data architecture and engineering solutions for end clients. The work will be a combination of office-based, remote and on-site work. What experience you’ll need to apply - Data Architecture experience- Strong Python and SQL skills- Experience of developing APIs- Strong stakeholder management and communication skills- Excellent understanding of business use cases for analytics and how to provide the data- Excellent stakeholder management and influencing skills What you’ll get in return for your experience A competitive day rate of up to £450 per day What’s next?Please get in touch with Savannah with an up to date CV today. Don’t hesitate to call/email to discuss the finer details.
Data case studies
Alistair Rogers, Data Scientist
- Alistair is an experienced Data Scientist with working with clients across government, healthcare and banking sectors.
- While successfully working in London as a Data Scientist for one of the Big Four professional service companies, he was planning to move to Bristol and approached us to get an understanding of the local market, especially opportunities and skills in demand within Data in the South West well ahead of the move.
- We advised on opportunities and businesses in and around Bristol as well as in-demand skill sets so he could gather relevant exposure in advance.
- Once ready to make the move, we were able to match Alistair perfectly with an open opportunity. Equipping him with tips and advice, we guided him through the interview process and negotiated the salary that was in line with his expectations. Alistair started as a Data Scientist with OnCorps, a start-up FinTech firm.
"Alex and ADLIB were fantastic to work with and absolutely made the daunting experience of finding a job in a new city a lot easier! They take the time to understand the fields that they are recruiting for and even go to developer meetups to learn more about the areas that they recruit in. Would definitely recommend working with Alex and ADLIB."
Alistair Rogers, Data Scientist
Two niche hires in record time
- SunLife are a financial services company and one of the largest life insurance companies in the world.
- As part of a small restructure, they were looking to expand their Performance Optimisation team, by adding two analysts.
- Low quality of applications and scarcity of top analytics talent can make finding specialists extremely difficult.
- By tapping into existing candidate pools and relationships, we were able to suggest a range of suitable candidates with a wide range of experience, resulting in SunLife making two great Performance Analyst hires.
“ADLIB helped Sunlife hire two great analysts in record time , who are already embedded in to the team and delivering great pieces of work.”
Billy Mohan, Digital Performance Manager at SunLife