Big Data jobs and careers
As the use of data in business gets more and more prevalent the ability to quickly and effectively produce data algorithms that can analyse large complex datasets is crucial. In order to do this, the right data infrastructure needs to be in place.
Why consider a career in Big Data?
Data Engineers build databases, data warehouses and data lakes to service the needs of the Data Analysts and Data Scientists in their businesses. This area is fast growing and Data Engineers are in massive demand as so many companies race to unlock the value in their data.
Typical role titles include
Big Data Engineer
Business Intelligence Developer
Business Intelligence Engineer
Big Data Developer
Data factory Developer
Data Availability Engineer
Data Science Engineer
Big Data Software Engineer
Data Warehouse Engineer
SQL Server Engineer
Data Infrastructure Engineer
Data Infrastructure Programmer
Data Infrastructure Developer
Data Analytics Engineer
Big Data Architect
Big Data Specialist
Big Data Lead
Head of Big Data
Data Architecture specialist
Head of Data Architecture
Career progression example
Junior Data Engineer
Senior Data Engineer
Lead Data Engineer
Head of Data Engineering
At an entry level, junior/graduate Data Engineering professionals on average will be earning between £30k-£35k. At the mid-level you’d be looking to earn £50k-£60k and seniors would be looking to earn £70k+.
Is it right for me? The skills it takes...
The skills needed can be split into 3 categories; programming skills, data science understanding and problem-solving.
Programming skills: Java | Python | Golang | Cython | Software Design - A background in APIs e.g. web APIs and static/dynamic libraries | ML Libraries - e.g. TensorFlow, CNTK, MLib, Spark, Kafka, Kinesis, Redshift.
What qualifications does it take?
A BSc, Masters (MSc) or Doctorate (PhD) in a maths, physics, statistics, artificial intelligence (AI), software engineering, engineering or computer science field with technical skills in programming. Demonstrable experience working with large data sets or alternatively candidates will work in Business Intelligence and then do online courses /self-learning/ get trained up on Big Data Engineering and Architecture and can move into these roles through this route.
Our Data recruitment team
Our latest Data jobsSearch all jobs
— Consultancy - 6 Months InitialLocation:Birmingham/BristolType:Salary:Up to £450 p/daySQL, AWS, Python. Birmingham and Bristol (paid expenses). 6-month initial. Large consultancy requires a Senior Data Engineer to work on client data migration as part of new app development. What you'll be doing Working on a client-driven project, you will develop data pipelines and data solutions for the client. What experience you'll need to apply SQL Server (Expert). Python (Proficient). AWS (Commercial Experience Essential). What you'll get in return for your experience The client is offering a daily rate of up to £450 p/day plus paid travel between client sites (Birmingham & Bristol). What's next Please apply with your up-to-date CV and I will be in touch to discuss further.
— Luxury Retail BrandLocation:LondonSalary:CompetitiveLong-standing Luxury Brand. Competitive Pay and Benefits Package. Warm Culture with Family Feel A luxury British retailer with over 130 years' of heritage is now opening its doors up for an eCommerce/digital CRO Analyst to join the team. This is a new hire for the business, as they look to utilise their data in a more efficient way to increase online conversions, decrease abandonment rates, and ultimately - increase profit! You will be the first hire within analytics and reporting into the Online Trader. As this is a standalone role, we're looking for a pro-active, experienced, confident individual who is going to promote best practices, and want to take the business forward in terms of the information you can get from data, and how it can be used to influence business decisions! What you'll be doing This role hasn't been entirely mapped out which makes it all the more exciting. There will be an element of adding certain responsibilities based on your experience and the tools you are able to use. You'll be confident in making suggestions to improve data capture, be that tools to use or models to put in place. You will raise relevant business questions alongside the rest of the team and senior stakeholders, and produce actionable insights. You'll be keen to get to really understand the customer base - what are they buying? Where are the up-selling opportunities? How do we engage our audience online? Getting to know your competitors will be really important to you so that you're able to keep up with trends within the industry. Your role will also see you managing the activity and relationship with 3rd party CRO partners in delivering programs relating to enhanced user experience. This role will fit nicely for someone who has solid Digital ability with a passion for data and Analytics. You will be utilising the GA platform, engaging with internal stakeholders to define and deliver output which is both analytically robust and commercially focused. Working alongside the Trading and Merchandising teams to capture Market, Competitor and Customer Insights. What experience you'll need to apply for the role Google Analytics / Adobe / IBM analytics experience. Solid A/B – MV Testing experience. Ability to use visualisation software (e.g. Tableau, Power BI, Data Studio) desirable but training offered. Excellent stakeholder management skills. Online luxury/fashion/retail experience is a huge plus. Strong Excel skills are advantageous. What you'll get in return for your experience You will be rewarded with a salary in line with your level of experience, as well as a competitive benefits package. All of these can be discussed on application. What's next? Get in touch with Marlie on 01179 269 530 for more information!
— High growth technology firmLocation:BristolSalary:Competitive• Unique opportunity to join a high growth international technology firm as they begin their entry into Europe.• Work with an international team of talented Machine Learning Engineers and Software Engineers working on cutting edge algorithms.• Excellent time to join as this firm has a proven and successful business model that it is rolling out globally. Work with a bunch of like-minded technologists on a cutting-edge tech stack working on novel algorithm development. They are a firm with a start-up feel but backed by successful international parent company. What you'll be doingYou will play a major part in bridging the gap between the Machine Learning, Software Engineering and Site Reliability teams to build and rollout Machine Learning use cases to their mobile application and website. You’d also be working with the creation of data pipelines to enable the delivery of data through their platforms. This is a broad role with the chance to get stuck in with a range to tools and technologies. What experience you will need to apply• Experience as a Data Scientist, Data Engineer or Software Engineer• Strong programming skills in Python• Cloud computing experience with either AWS, GCP and Azure is key here. • Experience with container orchestration solutions like Docker or Kubernetes would be a real bonus. • Interest in working with an international team What you will get in return for your experienceA very competitive base salary as well as relocation assistance and benefits. As well as flexible working hours, regular socials, amazing central Bristol offices and the opportunity to choose your own gear and setup. What’s next?If this sounds like the role for you, get in touch with Adam with your updated CV and we’ll handle the rest.
— FinTech FirmLocation:NewportSalary:CompetitiveWork on a variety of Big Data and Cloud technologies. Work in a team of Big Data engineers alongside other data engineering teams Design BI solutions, develop architecture and enable ML. This client is integrating a new service via the cloud as they move towards a single customer view. They are a data-driven firm, it guides their strategic decision-making and service/product development. As a result, they require a Senior Big Data Engineer to join the team of data engineers to enable this build. What you'll be doing As a Senior Big Data Engineer, you will be influential in building a modern, scalable, cloud-based Big Data infrastructure. You will support the design, roll-out and execution of the data strategy across multiple products and collaborate with other data teams to ensure that the data is integrated and accessible. This role requires a range of data engineering activities including; prototype development, pipelining, architecture, process automation, data strategy and more. What experience you'll need to apply Python/Scala/Java or similar Solid ETL experience Cloud Data platform experience (AWS, GCP, Azure or Similar) Experience of working on Big Data solutions Experience of using either Spark, Hadoop, Flink, HDFS, Kinesis Messaging broker platform experience such as Kafka//MQ’s/Airflow desirable Desire and ability to learn emerging technologies What you'll get in return for your experience A competitive salary based on experience plus bonus and benefits. What's next? Please get in touch with Scott with an up-to-date CV today. Don't hesitate to call/email to discuss the job in further detail.
— Financial Services InstitutionLocation:BristolSalary:Up to £70kJoin a Data Science Team comprising multiple data engineers and data scientists. End-to-end ML building and deployment. Work with a variety of Cloud, Ml and DevOps technologies. This client is building a Data Science Team in Bristol which will be working on a variety of machine learning projects. They require another Data Engineer to join this team to implement machine learning across the organisation to improve processes and analytics. The team is set to triple in size by the end of 2020. What you'll be doing As the Data Engineer, you will work with data engineers and data scientist to deploy ML algorithms to traditional methods of analysis. The production environment is already up and running so this role will be exploratory and research focused. You will work on end-to-end ML model building and deployment so this could be a great role for those new to/interested in this field. What experience you'll need to apply Minimum of 1 years’ experience in data/software engineering Degree in Computer Science, Maths, Physics or similar numerical field required Python or R programming experience Cloud Data platform experience (AWS, GCP, Azure or Similar) CI/CD desirable Previous experience of machine learning advantageous Desire and ability to learn emerging technologies What you'll get in return for your experience A base salary of £40k-£70k DOE plus bonus, benefits and the opportunity to progress machine learning skills and technologies. What's next? Please get in touch with Scott with an up-to-date CV today. Don't hesitate to email/call to discuss the job in further detail.
— Health Tech CompanyLocation:BathSalary:Up to £32kWork alongside a team of experienced developers. Design and build ETL solutions. Autonomous role in a collaborative environment. This established development company is continuing to grow and need a specialist ETL developer to work closely with their team of developers in Bath. This position will work on projects for external clients to deliver a wide range of data migration requirements. What you’ll be doing You will work alongside a team of experienced developers and external clients to scope out their data migration needs. This will include identifying, documenting and validating the migration scope by liaising with clients while also maintain the existing ETL set-up. You’ll shape the analytics at this firm by designing and implementing new ETL solutions and inform all relevant parties on the end-to-end process. Due to these responsibilities, you will be a well-organised developer, capable of working on different projects while engaging internal and external stakeholders. What you’ll need to apply Minimum 1 years’ experience in ETL Development (e.g Talend, SSIS, Informatica, Alteryx) Solid SQL Project Delivery and stakeholder management Experience with object-orientated programming desirable Agile/Scrum desirable Proactive and self-organised What you’ll get in return for your experience A competitive salary of up to £32,000 DOE plus bonus and benefits including flexible working, social events, training and development budget and more. What’s next Please get in touch with Scott with an up to date CV today, don’t hesitate to call/email to discuss the role in more detail.
— Green Tech BusinessLocation:WiltshireSalary:Up to £55kAward-winning sustainability business. Work with modern cloud infrastructure. Flexible-working, forward-thinking & modern approach to development. A Senior Data Developer is needed to join an award-winning tech for good business in Wiltshire. They’re already considered market leaders in ethical business and are showing no signs of slowing down. They require a Senior Data Developer to play a crucial role in the development of infrastructure, data acquisition and analytics. What you’ll be doing Reporting to the Data Lead, you will develop a broad mixture of data focussed solutions that are taken on by the team. This will include; integrations of data between internal and third-party systems, migrations, development of new databases and database solutions, new cloud data platform solutions and more. This is a hands-on role where you will also be expected to engage with stakeholders and mentor more junior colleagues across the Data and BI teams. What skills you’ll need Minimum of 5 years’ experience in data engineering/development Commercial experience with cloud platforms (Azure, AWS, GCP or similar) SQL Server Experience with source/version control e.g. Git Knowledge of Power BI Degree in Computer Science or similar technical field Agile/similar methodology experience What you’ll get in return for your skills A base salary of up to £55K plus a competitive benefits package which includes; 25 days annual leave with the option to buy more, company bonus, pension matched up to 7.5%, flexible working, travel & green travel allowances. What’s next? Please get in touch with Scott with an up-to-date CV today, don’t hesitate to email/call to discuss the finer details.
Data case studies
Alistair Rogers, Data Scientist
- Alistair is an experienced Data Scientist with working with clients across government, healthcare and banking sectors.
- While successfully working in London as a Data Scientist for one of the Big Four professional service companies, he was planning to move to Bristol and approached us to get an understanding of the local market, especially opportunities and skills in demand within Data in the South West well ahead of the move.
- We advised on opportunities and businesses in and around Bristol as well as in-demand skill sets so he could gather relevant exposure in advance.
- Once ready to make the move, we were able to match Alistair perfectly with an open opportunity. Equipping him with tips and advice, we guided him through the interview process and negotiated the salary that was in line with his expectations. Alistair started as a Data Scientist with OnCorps, a start-up FinTech firm.
"Alex and ADLIB were fantastic to work with and absolutely made the daunting experience of finding a job in a new city a lot easier! They take the time to understand the fields that they are recruiting for and even go to developer meetups to learn more about the areas that they recruit in. Would definitely recommend working with Alex and ADLIB."
Alistair Rogers, Data Scientist
Two niche hires in record time
- SunLife are a financial services company and one of the largest life insurance companies in the world.
- As part of a small restructure, they were looking to expand their Performance Optimisation team, by adding two analysts.
- Low quality of applications and scarcity of top analytics talent can make finding specialists extremely difficult.
- By tapping into existing candidate pools and relationships, we were able to suggest a range of suitable candidates with a wide range of experience, resulting in SunLife making two great Performance Analyst hires.
“ADLIB helped Sunlife hire two great analysts in record time , who are already embedded in to the team and delivering great pieces of work.”
Billy Mohan, Digital Performance Manager at SunLife