Big Data jobs and careers
As the use of data in business gets more and more prevalent the ability to quickly and effectively produce data algorithms that can analyse large complex datasets is crucial. In order to do this, the right data infrastructure needs to be in place.
Why consider a career in Big Data?
Data Engineers build databases, data warehouses and data lakes to service the needs of the Data Analysts and Data Scientists in their businesses. This area is fast growing and Data Engineers are in massive demand as so many companies race to unlock the value in their data.
Typical role titles include
Big Data Engineer
Business Intelligence Developer
Business Intelligence Engineer
Big Data Developer
Data factory Developer
Data Availability Engineer
Data Science Engineer
Big Data Software Engineer
Data Warehouse Engineer
SQL Server Engineer
Data Infrastructure Engineer
Data Infrastructure Programmer
Data Infrastructure Developer
Data Analytics Engineer
Big Data Architect
Big Data Specialist
Big Data Lead
Head of Big Data
Data Architecture specialist
Head of Data Architecture
Career progression example
Junior Data Engineer
Senior Data Engineer
Lead Data Engineer
Head of Data Engineering
At an entry level, junior/graduate Data Engineering professionals on average will be earning between £30k-£35k. At the mid-level you’d be looking to earn £50k-£60k and seniors would be looking to earn £70k+.
Is it right for me? The skills it takes...
The skills needed can be split into 3 categories; programming skills, data science understanding and problem-solving.
Programming skills: Java | Python | Golang | Cython | Software Design - A background in APIs e.g. web APIs and static/dynamic libraries | ML Libraries - e.g. TensorFlow, CNTK, MLib, Spark, Kafka, Kinesis, Redshift.
What qualifications does it take?
A BSc, Masters (MSc) or Doctorate (PhD) in a maths, physics, statistics, artificial intelligence (AI), software engineering, engineering or computer science field with technical skills in programming. Demonstrable experience working with large data sets or alternatively candidates will work in Business Intelligence and then do online courses /self-learning/ get trained up on Big Data Engineering and Architecture and can move into these roles through this route.
Our Data recruitment team
Our latest Data jobsSearch all jobs
— Financial ServicesLocation:Salisbury/remoteSalary:Up to £90kBe influential in the direction of the organisation. Collaborate with the CDO and Head of Data Science. Backed data strategy and investment from new CDO. A specialist financial service institution is looking for a Head of Data Engineering & Architecture to scale the data platform and the data platform team.What you'll be doingAs the Head of Data Engineering & Architecture, you will build and manage a team of highly talented cloud data engineers and data architects to enhance the data platform as part of the new data strategy. There are a number of sources that need to be integrated into the platform which you require you to lead on the architecture of said sources.Data governance will be the backbone for the platform so that effective data science and business intelligence activities can be carried out. Therefore, you will liaise with other heads of departments and teams to ensure that the platform is fit for purpose as it scales. What you'll need to apply Proven experience managing a team of data engineers and/or architects. Extensive data architecture and engineering skills Experience with SQL and NoSQL databases Excellent understanding of business use cases for analytics and how to provide the data Excellent stakeholder management and influencing skills What you'll get in return for your experienceA base salary of up to £90k plus bonus & benefits. The data team will triple in size over the next 12 months so this is a great chance to be a key part of enabling the business to become fully data-driven.The role is offered largely remote with the expectation to go into the office 1 or 2 days p/month depending on requirements. The role can be offered fully remote to UK-based candidates too.What's next Please apply with an up-to-date CV today. Don't hesitate to call/email to discuss the finer details.
— Cutting edge B-Corp CertifiedLocation:London/RemoteSalary:110,000Fantastic salary and benefits including life assurance, private medical insurance, numerous discount vouchers, gym membership, personal skill budget and generous holiday allowances, flexible and remote working options. Continuous learning and development plans for all staff. Constantly evolving products and highly collaborative team. A highly inclusive and supportive company leading the way in the insurtech sector. On a mission to create the best experience for business owners when it comes to buying insurance and pride themselves on their pioneering software platform and making things simple and easy to use. We are looking for a Solutions Architect with a background in software engineering to take on a new role within the business, this is a blank canvas opportunity. What you’ll be doing As Solutions Architect you will be reporting into the Head of Architecture and working very closely with cross-functional teams on the technical direction of the business. You will play a big part in the planning and contribute ideas to the teams and work collaboratively. The Solutions Architect will be working closely with the product teams and guide the implementation of the architecture whilst defining and owning the technical strategy and vison of the business. What skills you’ll be needing Strong technical background with experience designing and architecting solutions. Domain-Driven Design. Familiar working in an Agile environment. Ability to work collaboratively. Highly analytical and ability to understand complex IT solutions. Strong communication skills both written and verbal. Ability to liaise with stakeholders of all levels. Encouraging and positive leader. What you’ll get in return for your talents As well as a fantastic salary, you will be entitled to generous holiday with option to buy more (up to 5 days). Life assurance, health cash plan, private medical insurance, full gym membership, shopping vouchers, dental insurance travel insurance, discounts on meals out, cycle to work scheme, summer parties, weekends away, outside of work skills budget, social activities and events. What’s next? If you like the sound of this role, please apply or send your CV directly to firstname.lastname@example.org
— Fintech with FTSE100 CustomersLocation:Manchester / RemoteSalary:£70k & BenefitsReal-time Event-Driven Architecture Great chance for growth & upskilling. Well established, highly regarded business with a start-up vibe. This is the opportunity to join a small but international company as they further develop their automated real-time financial CRM.What you’ll be doing Working with cutting-edge technology, and a team eager to find even better approaches you'll help to further develop the platform and suggest new practices. You'll be reporting directly to the CTO & CEO who are very agnostic about your skills coming in. Predominantly working with Kotlin but are happy with any solid programming background as they're more than happy to upskill you in Kotlin. The system is a microservices architecture orchestrated via Kubernetes and hosted in Azure its an event-driven architecture with Kafka, enabling an event sourcing approach. So we are looking so someone who is an expert in either microservices or event-driven Architecture, although, a strong passion will suffice! The teams are small, cross-functional, DevOps teams working with pair programming with a strong focus on automation to reduce cycle time and a philosophy of keeping things as simple as they can be. What experience you’ll need to apply Previous knowledge of event-driven architecture or microservices. Modern Programming Language. 2 years of commercial development experience. What you’ll get in return for your experience A great salary of £60k with yearly reviews, flexi-time so you can choose when you work. Although they would like someone in the Manchester office more often than not, they realise for the best talent this may not be feasible so feel free to apply if you are out of the manchester range.You'll get to work with cutting-edge technology, and a team eager to find even better approaches so you'll have constant opportunities to upskill. What nextDoes this sound like the one? Send your CV to Will to find out more, or feel free to send over an email with any questions.
— Market-Leading BusinessLocation:South LondonSalary:Up to £45kJoin a data science team comprising multiple data engineers and data scientists. End-to-end machine learning model building and deployment. Work with a variety of technologies including Cloud, ML and DevOps tools. This client is building a data science team in Bristol which will be working on a variety of machine learning projects. They require Data Engineers to implement machine learning across the organisation to improve processes and analytics. What you'll be doing You will work with data engineers and data scientists to deploy ML algorithms to traditional methods of analysis. The production environment is already up and running so this role will be exploratory and research-focused. You will work on end-to-end ML model building and deployment so this could be a great role for those new to/interested in this field. What experience you'll need to apply Degree in Computer Science, Maths, Physics or similar Programming experience required (Python preferred) ETL experience Cloud experience preferred Desire and ability to learn emerging technologies essential What you'll get in return for your experience A base salary of up to £45k plus bonus, benefits and the opportunity to progress machine learning skills and technologies. Expectation is 2 days p/week in the office. What's next? Please get in touch with Scott with an up-to-date CV today.
— Market-Leading BusinessLocation:BournemouthSalary:Up to £40kJoin a data science team comprising multiple data engineers and data scientists. End-to-end machine learning model building and deployment. Work with a variety of technologies including Cloud, ML and DevOps tools. This client is building a data science team in Bristol which will be working on a variety of machine learning projects. They require Data Engineers to implement machine learning across the organisation to improve processes and analytics. What you'll be doing You will work with data engineers and data scientists to deploy ML algorithms to traditional methods of analysis. The production environment is already up and running so this role will be exploratory and research-focused. You will work on end-to-end ML model building and deployment so this could be a great role for those new to/interested in this field. What experience you'll need to apply Degree in Computer Science, Maths, Physics or similar Programming experience required (Python preferred) ETL experience Cloud experience preferred Desire and ability to learn emerging technologies essential What you'll get in return for your experience A base salary of up to £40k plus bonus, benefits and the opportunity to progress machine learning skills and technologies. Expectation is 2 days p/week in the office. What's next? Please get in touch with Scott with an up-to-date CV today.
— Financial ServicesLocation:Salisbury/remoteSalary:Up to £80kFull operational Azure ecosystem including data lake and DWH. Collaborate with the Head of Architecture on system design. Backed data strategy and investment from new CDO. A specialist financial service institution is looking for a Data Architect to design and integrate sources for business-wide analytics.What you'll be doingCollaborating with the Head of Data Architecture, you will map out the organisations' data sources and connect these to the data lake and DWH to enhance reporting. Currently, analytics are centred around marketing and customers so the CDO is looking to broaden this across Finance, HR, ERP systems and more.You will design the new reporting solutions in Power BI enabling stakeholders to tap into the newly discovered insight. There is a team of data engineers and BI analysts below you to deliver the pipelines and build the reports/dashboards, you will help piece it all together to form the bigger picture. What you'll need to apply Proven experience as a Data Architect or Engineer Cloud experience is a must, Azure preferred Solid understanding and experience of working with dimensional data models Deep understanding of data quality, master data management & data warehouses Excellent understanding of business use cases for analytics and how to provide the data Excellent stakeholder management and influencing skills What you'll get in return for your experienceA base salary of up to £80k plus bonus & benefits. The data team will triple in size over the next 12 months so this is a great chance to be a key part of enabling the business to become fully data-driven.The role is offered largely remote with the expectation to go into the office 1 or 2 days p/month depending on requirements. The role can be offered fully remote to UK based candidates too.What's next Please apply with an up-to-date CV today. Don't hesitate to call/email to discuss the finer details.
15th OctLocation:Bristol / RemoteSalary:£45k & BenefitsGet trained up in the latest data engineering technology. Excellent opportunity to join a large corporate. Work on a large-scale integration programme. This business is currently building a team to work on a large-scale integration programme following an acquisition. What you’ll be doing You will work in a small team of Data Developers to ingest data and create a single version of the truth. Once this project is completed, you'll move to a team building a new central data platform. What experience you’ll need to apply ETL development experience SQL Python What you’ll get in return for your experience A competitive base salary of between £30K - £45K (dependent on experience) plus a great benefits package including training on Cloud services. As such, this is a great chance for developers with an on-premise background to gain commercial experience in modern Cloud data engineering technologies. Remote and flexible working What’s next? Please get in touch with Will with an up-to-date CV today. Don’t hesitate to call/email to discuss the finer details.
— Global Tech FirmLocation:Bristol/remoteSalary:Up to £100kJoin a global tech firm. Work on a real-time platform. Great package, remote working, flexi-time. A Global Tech firm is looking for a Senior Data Engineer to support the data platform; building, maintaining and optimising functionality where possible. What you'll be doing As a Senior Data Engineer, you will be influential in supporting the integration of several systems, requiring you to work with a combination of serverless technologies. You will support the design, roll-out and execution of the data pipelines and storage solutions to enable new functionality on the platform. As a result, you'll collaborate with other engineering and data teams to ensure that the data is integrated and accessible. This role requires a range of data engineering activities including; prototyping, real-time streaming, ETL pipelines, process automation and more. What experience you'll need to apply Minimum of 4 years' experience in data or software engineering Solid ETL pipelining experience Scala/Java/Python development experience Streaming experience desirable e.g. Airflow, Dataflow, Kinesis, Kafka etc Cloud experience (AWS, GCP, Azure or similar) What you'll get in return for your experience A base salary of up to £100,000 (scope for more) with an excellent benefits package including remote working. What's next? Please get in touch with Scott with an up-to-date CV today. Don't hesitate to email/call to discuss the job in further detail.
— Highly successful Data Science servicesLocation:LondonSalary:Very competitive with bonusRobust and agile consultancy. Full ownership of sales and strategic sales growth. Outstanding in-house technical capability. This is an opportunity to take ownership of the sales for a highly successful and agile consultancy, offering services and deployment of Ai, ML, Bid Data, Quantitative Research and Cloud technologies.In line with sustained growth, they are looking to enhance sales by appointing a proven sales leader with extensive experience of selling consultancy services with a highly technical context (Data Science being of benefit).The company has superb in-house capability and a track record of successful completion and delivery for high-profile clients across banking, financial services, and risk amongst other sectors. What you’ll be doing: Create a sales strategy to connect with key stakeholders offering high quality services. Use your industry experience and contacts to build a robust and sustainable pipeline as a hands-on sales expert. Construct compelling bids and RFP’s. Lead sales and grow out the sales function, include coaching mentoring and recruitment of sales and lead generators. Implement strategies to widen the customer audience or reach into newer sectors, whilst seeking opportunities to increase existing accounts. What experience you will need to apply: Proven track record in winning contracts for solutions services with a highly technical context. Degree qualified with the ability to understand complex technical solutions quickly and effectively. A great communicator with high standards of sales acumen and training. Proven leadership, or confident in your ability to grow a function. What you’ll get in return for your experienceTo be part of a first-class company offering full ownership of sales, with the potential to grow into a Chief Sales Officer position.What's next?For more information or to apply, apply directly to this advert to speak with Roy Bennett.
— • Leading Financial Services FirmLocation:BristolSalary:40,000 - 50,000Excellent opportunity to work for a leading financial services firm Join at a time where the team is growing Work with insight professionals and learn new skills A great opportunity to join one of Bristol’s biggest success stories. You’ll be working alongside a great team, across varied projects as a go to professional when it comes to designing the insight strategies. Are you looking to get away from agency work and start a client-side role? Here’s your chance!What you’ll be doingIn this role you’ll be working to bridge the gap between data, insight reporting and marketing. You’ll be the go-to person to be able to design the plan and strategy across a product agnostic team; everything in this role is focused on the product and the customer approach. You’ll be working to implement, plan and design the marketing strategy, deliver strategic projects, and lead key data led marketing initiatives to ensure this is the priority. You’d be responsible for monitoring performance, flagging trends, spotting threats and growth through interrogation of data. Most importantly, you’ll need to be a strong communicator to be able to present these to stakeholders, work across various teams and more. Up to the task? What experience you’ll need to apply Experience with Adobe Analytics (or Google Analytics) Preferably someone from a digital/strategy/insight background Strong communication skills Analytical thinker Able to be collaborative and work across key projects Proven ability to deliver in a fast-paced environment Bonus for CRM experience What you’ll get in return for your experience A base salary reaching up to £50,000 per annum plus an annual bonus and unbeatable package! What nextPlease get in touch with Tegan with an updated CV today. Don’t hesitate to call/email with any questions.
Data case studies
Alistair Rogers, Data Scientist
- Alistair is an experienced Data Scientist with working with clients across government, healthcare and banking sectors.
- While successfully working in London as a Data Scientist for one of the Big Four professional service companies, he was planning to move to Bristol and approached us to get an understanding of the local market, especially opportunities and skills in demand within Data in the South West well ahead of the move.
- We advised on opportunities and businesses in and around Bristol as well as in-demand skill sets so he could gather relevant exposure in advance.
- Once ready to make the move, we were able to match Alistair perfectly with an open opportunity. Equipping him with tips and advice, we guided him through the interview process and negotiated the salary that was in line with his expectations. Alistair started as a Data Scientist with OnCorps, a start-up FinTech firm.
"Alex and ADLIB were fantastic to work with and absolutely made the daunting experience of finding a job in a new city a lot easier! They take the time to understand the fields that they are recruiting for and even go to developer meetups to learn more about the areas that they recruit in. Would definitely recommend working with Alex and ADLIB."
Alistair Rogers, Data Scientist
Establishing a Data Science function
- LV= are one of the UK’s largest personal insurers who announced earlier this year that they were partnering with the University of Bristol to establish a team of Data Scientists and Engineers with the aim of making advancements in the field of Data Science by sharing knowledge, skills and opportunities.
- We initially worked with Venura as a candidate, finding him the role at LV=. He soon needed another two sets of hands, planning on growing his team to 3.
- To help him with this, we worked with him to truly understand his style of working and management, gathering a detailed brief that outlined the ideal candidates.
- By tapping into existing candidate pools and relationships, we were able to suggest a range of suitable candidates with a wide range of experience and that had the vital mix of technical and soft skills needed, to be working alongside University of Bristol as a team.
- This resulted in Venura/ LV= making two great Data Engineer hires.
“From finding me my current role to helping place two fantastic hires, ADLIB has been a pivotal role in the development of our team here at Bristol. The guys are easy to work with, friendly and most importantly inspire trust. I look forward to working with them in the future.”
Dr. Venura Perera, Data Science Manager at LV=