research

Data Engineering is one of the fastest-growing job roles in the tech industry with LinkedIn Talent Insights categorizing demand for these roles as ‘very high’. This means that it’s harder than ever for firms to attract and retain talent in this pivotal role. Estimates on the number of unfilled positions last year range from as much as 33-50%.
One of the reasons for the shortage is the rate at which the discipline is moving, with tools and technologies emerging and evolving rapidly. This leads to the absence of a standardized toolset and means that the definition of the role can be dramatically different across companies.
Based on research with 50 Data Engineers, and in conversation with Dani Solà Lagares (Director of Data at Simply Business) research from technology recruiting firm Stott and May reveals what Data Engineers are looking for, and what potential employers can be doing to increase their chances of snagging top talent. Here are the four top tips to come out of the research.
1. Give them a clearly defined role.
When looking for a new role, Data Engineers need to see a detailed and realistic job description. 72% testified that this was the most important factor in whether or not they will apply. If an employer doesn’t have this nailed down, then Data Engineers will pass up the opportunity in favor of an employer who has a clear idea of what needs to be done. ‘Give candidates a sense of the projects they will be working on and the stakeholders they will be engaging with,’ says Dani Sola. ‘Even more importantly, provide some narrative on the type of impact you expect key initiatives to make.’
2. Provide the right technology stack.
48% of Data Engineers stated that the technology stack they will be working with is the most important consideration in accepting a role. Because the technology stack has so much to do with what their day-to-day work will look like, it’s important that the fit is right. ‘Technical skillsets could vary dramatically from Kafka, Kafka Streams, Scala, Kotlin knowledge, advanced SQL, data warehousing skills, Python, the list goes on,’ says Dani. ‘It is important, however, to paint a picture of your requirements without asking candidates to tick every skillset that’s ever existed in data engineering.’
3. Benchmark to ensure you’re offering a competitive salary.
According to the research, 42% of Data Engineers say they are most likely to jump ship because their salary and benefits are below market rate. It’s important to make sure you are benchmarking your salaries against your competitors, and offering a competitive compensation package if you want to retain in-demand talent. ‘In my view, one of the major reasons engineers move on is that the initial value proposition of the role in that organization has not lived up to expectations,’ says Dani. ‘Don’t sell a dream and deliver a nightmare. If you’re authentic and invest in your team’s personal development that can go a long way.’
4. Don’t wear them out with excessive interview steps.
Data Engineers’ time is very important, so if they are being asked to jump through too many hoops, they are liable to simply look elsewhere. If you’re looking to recruit a Data Engineer, try to streamline the recruitment process as much as possible so that you can make an assessment of their fit without losing momentum. ‘Keep talent engaged during the hiring process,’ says Dani. ‘Create a sense of your culture and values. Make great first impressions as a potential employer. Interviewing should be about making the candidate feel at ease and creating an environment where they can show themselves at their best.’
David Struth is Head of Marketing at Stott and May.

Share this post: More