Big Data jobs and careers
As the use of data in business gets more and more prevalent the ability to quickly and effectively produce data algorithms that can analyse large complex datasets is crucial. In order to do this, the right data infrastructure needs to be in place.
Why consider a career in Big Data?
Data Engineers build databases, data warehouses and data lakes to service the needs of the Data Analysts and Data Scientists in their businesses. This area is fast growing and Data Engineers are in massive demand as so many companies race to unlock the value in their data.
Typical role titles include
Big Data Engineer
Business Intelligence Developer
Business Intelligence Engineer
Big Data Developer
Data factory Developer
Data Availability Engineer
Data Science Engineer
Big Data Software Engineer
Data Warehouse Engineer
SQL Server Engineer
Data Infrastructure Engineer
Data Infrastructure Programmer
Data Infrastructure Developer
Data Analytics Engineer
Big Data Architect
Big Data Specialist
Big Data Lead
Head of Big Data
Data Architecture specialist
Head of Data Architecture
Career progression example
Junior Data Engineer
Senior Data Engineer
Lead Data Engineer
Head of Data Engineering
At an entry level, junior/graduate Data Engineering professionals on average will be earning between £30k-£35k. At the mid-level you’d be looking to earn £50k-£60k and seniors would be looking to earn £70k+.
Is it right for me? The skills it takes...
The skills needed can be split into 3 categories; programming skills, data science understanding and problem-solving.
Programming skills: Java | Python | Golang | Cython | Software Design - A background in APIs e.g. web APIs and static/dynamic libraries | ML Libraries - e.g. TensorFlow, CNTK, MLib, Spark, Kafka, Kinesis, Redshift.
What qualifications does it take?
A BSc, Masters (MSc) or Doctorate (PhD) in a maths, physics, statistics, artificial intelligence (AI), software engineering, engineering or computer science field with technical skills in programming. Demonstrable experience working with large data sets or alternatively candidates will work in Business Intelligence and then do online courses /self-learning/ get trained up on Big Data Engineering and Architecture and can move into these roles through this route.
Our Data recruitment team
Our latest Data jobsSearch all jobs
— Growing Financial Services FirmLocation:SalisburySalary:Up to £50kLead reporting for the BI and Pricing Teams. Work with modern Cloud, BI and warehouse technologies. Define reporting standards for the Strategy Department. This firm is building out its Strategy Department to enable further data-driven decision-making across Product, IT and Pricing Teams. They require a Lead BI Analyst to lead a small team of analysts to optimise data and reporting activities. What you’ll be doing As the Lead, you will be responsible for all things BI and report into the Director of Strategy. This will require working with numerous teams to understand their data needs and deliver actionable insight. You’ll sit above three BI Analysts to provide support to the team ensuring that necessary data can be acquired and conveyed to the wider business. There won’t be line management responsibilities, instead, this position will be heavily focused on analysis, reporting and decision-making. As this is a brand-new position, you will be paramount in defining standards shaping BI. What you’ll need to apply Minimum of 2 years’ experience in BI or Data Analysis Experience with a modern BI tool (PowerBI, Tableau, QlikView or similar) Solid SQL and Excel Ability to mentor and guide junior analysts Statistical analysis experience preferred Degree in a numerical discipline preferred What you’ll get in return for your experience A great salary of up to £50k plus benefits and to work with numerous modern BI and Cloud technologies. What’s next Please get in touch with Scott with an up-to-date CV today, don’t hesitate to call/email to discuss the finer details.
— Big 4 Consulting firmLocation:BristolSalary:£75000Excellent opportunity to join a Big 4 global consultancy. Make a big impact on the data landscape. Play a leading role in shaping the future state of the organisation. This leading global consulting firm are setting up a Chief Data Office department and they're looking for an experienced data architect who has worked in large scale global enterpises. What you’ll be doing You will be one of the first hires into this chief data office with the responsibility to deliver dimensional models to enable this organisation to deliver more advanced reporting and analytics across their business. What experience you’ll need to apply - Data Architecture experience- Experience of working at enterprise level in a global business- Solid understanding and experience of working with dimensional data models- Deep understanding of data quality, master data management & data warehouses- Excellent understanding of business use cases for analytics and how to provide the data- Excellent stakeholder management and influencing skills What you’ll get in return for your experience A competitive package of up to around £70K - £75K + benefits + bonus. What’s next? Please get in touch with Alex with an up to date CV today. Don’t hesitate to call / email to discuss the finer details.
— Financial Services InstitutionLocation:BathSalary:Up to £45kDesign and build data warehouse solutions. Progress reporting and data standards. Shape BI including modern BI tool adoption. This firm require a BI Developer to help manage their data warehouse, support analytics and improve the availability of quality-assured data across the business. The team is looking to expand so anyone who joins will help shape what their roadmap in the coming months ahead. What you'll be doing As the BI Developer, you will ensure that the business can gain access to the data it needs. You'll build and maintain warehouse solutions to enable effective data management, data migration and analytics. This will require a combination of on-premises warehouse development and BI reporting. What you'll need to apply Minimum 3 years’ experience as a SQL/BI Developer SQL Server including SSIS and SSRS Data warehouse design and implementation Data modelling experience Knowledge of BI visualisation tools e.g. Power BI, Tableau preferred What you'll get in return for your experience A great salary of up to £45,000 plus benefits including the opportunity to implement and shape their modern BI suite. What's next Please get in touch with Scott with an up-to-date CV today. Don't hesitate to email/call to discuss the finer details.
— Technology FirmLocation:BristolSalary:CompetitveOptimise Big Data platforms. Collaborate with data architects, lead data engineers. Challenge Head of Data Engineering on technical architecture. This client is currently migrating Big Data platforms to utlise additional services. As a result, they require a Lead Data Engineer with experience in technical architecture and data strategy experience to optimise the platforms. What you'll be doing As the Lead Data Engineer, you will model, integrate and optimise scalable, cloud-based solutions. This position will require working with multiple Big Data Technologies. You will support the design, roll-out and execution of the data strategy across the business. As the lead, you’ll mentor the team and work data teams to ensure that the data is integrated and accessible. What experience you'll need to apply Solid ETL experience Python, Scala or Java development experience Cloud Data Platform experience (AWS, GCP, Azure or similar) CI/CD experience Experience of using either Spark, Hive, Hadoop, HDFS, Hydra preferred Experience of setting standards and supporting team members What you'll get in return for your experience A competitive salary with excellent bonus and benefits package. Grow with a leading company and have the potential to become the Head of Data Engineering. What's next Please get in touch with Scott with an up to date CV today. Don't hesitate to email/call to discuss the job in further detail.
— Technology FirmLocation:WiltshireSalary:CompetitiveDevelop, optimise and support Big Data platforms. Pipeline information from rich, global datasets. Collaborate with data architects, engineers and scientists. This firm has recently transitioned from batch-processing architecture to a streamed-based model and they require a data engineer to join their global analytics team. This team collates information globally from Hadoop and Cloud DBs to make it available to data scientists and analysts. They are open to junior, mid-level and senior candidates. What you'll be doing Working closely with the data architects and the wider data teams, you will ensure that the infrastructure is in place to analyse the huge amounts of data generated by this company. As a result, you'll drive improvements to the scalability, efficiency and the continuous deployment environment. This is a great opportunity for an ambitious data engineer to gain experience with a variety of modern data technologies across cloud ecosystems. What experience you'll need to apply Minimum of 1 years' experience in data or software engineering Python, Java or Scala programming experience Experience with Cloud and/or Big Data tools are essential (Spark, MongoDB, BigQuery, Hadoop or similar) Containerisation experience preferred Agile or similar methodology preferred What you'll get in return for your experience A competitive salary based on experience with excellent bonuses and benefits package. Grow with a global company and work with the most modern data technology on the market. What's next Please get in touch with Scott with an up to date CV today. Don't hesitate to email/call to discuss the job in further detail
— Leading CharityLocation:ReadingSalary:£70K - £75KHelp this great charity formulate their data strategy. Have a real impact by supporting their move to advanced BI, predictive analytics and more. Bring your knowledge of data technology and solutions. Excellent opportunity to come and shape the data strategy and move this charity to being data driven and utilising their data in an effective way. What you’ll be doing The Chief Data Officer will have full ownership of the data strategy for this charity. They are looking for someone who can bring a mixture of skills and experience across data strategy, data governance, data technology, predictive analytics and business intelligence. This is an excellent opportunity to really make a material difference for a leading charity for a great cause. They have some great ideas on what they want to do but they need an experienced data professional to help them on their journey. What experience you’ll need to apply Solid experience across data governance, data strategy and predictive analytics. Excellent ability to influence senior stakeholders and gain buy in for major change. Experience managing a mixture of data professionals including business intelligence specialists and data scientists. Solid experience with CRM systems. Experience in the charity sector is a nice to have. What you’ll get in return for your experience A competitive salary of up to £74,000, dependent on experience, plus benefits package and an overwhelming sense of doing a great thing for a great charity. What’s next Please get in touch with Alex with an up to date CV today. Don’t hesitate to call / email to discuss the finer details.
— Green Tech FirmLocation:BristolSalary:Up to £60k DOEJoin a leading green technology firm. Renowned engineering culture Great package, remote working, flexi-time. This client is building a new team as they continue to progress their technological capabilities. They are a data-driven organisation, it guides their strategic decision-making and service/product development. As a result, they need a Data Engineer to support the integration of various products. What you'll be doing As a Data Engineer, you will support the integration of several products, requiring you to work with a combination of serverless technologies. You will support the roll-out and execution of the data services across various business units. As a result, you'll collaborate with other engineering and data teams to ensure that the data is integrated and accessible. This role requires a range of data engineering activities including; prototyping, ETL pipelines, process automation and more. What experience you'll need to apply Minimum of 1 years' experience in data or software engineering Python, Scala or Java development experience Cloud experience (AWS, GCP, Azure or similar) Pair programming experience preferred Agile or similar methodology preferred What you'll get in return for your experience A base salary of up to £60,000 DOE with excellent bonuses and benefits package including flexi-time and WFH 1 day p/week. What's next? Please get in touch with Scott with an up to date CV today. Don't hesitate to email/call to discuss the job in further detail.
— High growth technology firmLocation:BristolSalary:Competitive• Unique opportunity to join a high growth international technology firm.• Work with an international team of talented Machine Learning Engineers and Software Engineers working on cutting edge algorithms.• Excellent time to join as this firm has a proven and successful business model that it is rolling out globally. Work with a bunch of like-minded technologists on a cutting-edge tech stack working on novel algorithm development. They are a firm with a start-up feel but backed by successful international parent company. What you'll be doingYou will play a major part in bridging the gap between the Machine Learning, Software Engineering and Site Reliability teams to build and rollout Machine Learning use cases to their mobile application and website. You’d also be working with the creation of data pipelines to enable the delivery of data through their platforms. This is a broad role with the chance to get stuck in with a range to tools and technologies. What experience you will need to apply• Experience as a Data Scientist, Data Engineer or Software Engineer• Strong programming skills in Python• Cloud computing experience with either AWS, GCP and Azure is key here. • Experience with container orchestration solutions like Docker or Kubernetes would be a real bonus. • Interest in working with an international team What you will get in return for your experienceA very competitive base salary as well as relocation assistance and benefits. As well as flexible working hours, regular socials, amazing central Bristol offices and the opportunity to choose your own gear and setup. What’s next?If this sounds like the role for you, get in touch with Adam with your updated CV and we’ll handle the rest.
— Green Tech FirmLocation:BristolSalary:Up to £80kJoin a leading green technology firm. Renowned engineering culture Great package, remote working, flexi-time. This client is building a new team as they continue to progress their technological capabilities. They are a data-driven organisation, it guides their strategic decision-making and service/product development. As a result, they need a Senior Data Engineer to support the integration of various products. What you'll be doing As a Senior Data Engineer, you will be influential in supporting the integration of several products, requiring you to work with a combination of serverless technologies. You will support the design, roll-out and execution of the data services across various business units. As a result, you'll collaborate with other engineering and data teams to ensure that the data is integrated and accessible. This role requires a range of data engineering activities including; prototyping, ETL pipelines, architecture, process automation and more. What experience you'll need to apply Minimum of 3 years' experience in data or software engineering Solid ETL pipelining experience Python, Scala or Java development experience Cloud experience (AWS, GCP, Azure or similar) Pair programming experience preferred Agile or similar methodology preferred What you'll get in return for your experience A base salary of up to £80,000 with excellent bonuses and benefits package including flexi-time and WFH 1 day p/week. What's next? Please get in touch with Scott with an up to date CV today. Don't hesitate to email/call to discuss the job in further detail.
Data case studies
Alistair Rogers, Data Scientist
- Alistair is an experienced Data Scientist with working with clients across government, healthcare and banking sectors.
- While successfully working in London as a Data Scientist for one of the Big Four professional service companies, he was planning to move to Bristol and approached us to get an understanding of the local market, especially opportunities and skills in demand within Data in the South West well ahead of the move.
- We advised on opportunities and businesses in and around Bristol as well as in-demand skill sets so he could gather relevant exposure in advance.
- Once ready to make the move, we were able to match Alistair perfectly with an open opportunity. Equipping him with tips and advice, we guided him through the interview process and negotiated the salary that was in line with his expectations. Alistair started as a Data Scientist with OnCorps, a start-up FinTech firm.
"Alex and ADLIB were fantastic to work with and absolutely made the daunting experience of finding a job in a new city a lot easier! They take the time to understand the fields that they are recruiting for and even go to developer meetups to learn more about the areas that they recruit in. Would definitely recommend working with Alex and ADLIB."
Alistair Rogers, Data Scientist
Two niche hires in record time
- SunLife are a financial services company and one of the largest life insurance companies in the world.
- As part of a small restructure, they were looking to expand their Performance Optimisation team, by adding two analysts.
- Low quality of applications and scarcity of top analytics talent can make finding specialists extremely difficult.
- By tapping into existing candidate pools and relationships, we were able to suggest a range of suitable candidates with a wide range of experience, resulting in SunLife making two great Performance Analyst hires.
“ADLIB helped Sunlife hire two great analysts in record time , who are already embedded in to the team and delivering great pieces of work.”
Billy Mohan, Digital Performance Manager at SunLife