Working with recruitment agencies: assessing effectiveness
Every company wishes that all potential candidates find them directly. However, that isn’t always the case. Although we want to keep recruitment spend to a minimum, we also need to face the fact that external agencies are one part of our sourcing strategy.
The scenario we found ourselves in about 6 months ago will be similar in other startups. We needed to grow fast and we saw numerous agencies come on board, either through contacts from hiring managers, referrals from other companies, or direct contact with the talent team. We suddenly realised we were working with about 25 agencies and we couldn't manage them effectively. It became frustrating for all parties involved.
Therefore, we decided to rethink our approach and put in place steps to get to know each agency properly, to understand how they will represent GoCardless, and how we can assess their performance. Moreover, we want to work with and manage a realistic number of relationships.
Getting to know the agency
Recruitment agencies can be a helpful additional source for finding talent. Although a good relationship can lead to mutual gains for both parties involved, a poor fit can be very detrimental, wasting time for the team, hiring managers and even the recruitment agency itself.
Therefore, before starting to work with an agency we want to understand if the way they work fits with our company values. We’ve created a checklist of key points about GoCardless (e.g., who we are, our interview process, other logistics) and what we need to know about the agency in our initial discussion. For example:
- What is the agency’s recruitment style & procedures?
- Do the agency’s recruiters understand us as a company?
- How do they usually prefer the interview process to be managed?
- Do they care about candidate experience?
- Will they be good ambassadors for our employer brand?
Sometimes, even if the agency is a good fit for a company we know well, it might not be a good fit for us and vice versa – it’s ok, not everyone is!
When we start working with an agency, we try to set up a short call every week or two. Although the call might not always be needed, it’s a useful reminder in case there are any issues we need to discuss, for example:
- Particular feedback – positive or negative – on a candidate that might be more appropriate over the phone.
- Particular feedback on profiles submitted, for example, if we’ve noticed a trend in the kind of profile that does/doesn’t meet the role requirements.
- General updates on role requirement, compensation, or the interview process.
Once we achieve a good working rhythm, we agree with the agency on a number of profiles submitted per month. Although setting a number is not always feasible, it gives us a rough idea of the amount of CVs we can expect to receive from this source.
Assessing agency performance: the performance index
We decided to create an easier way to objectively assess agencies’ performance and keep working with only those that perform well on the things we care most about. The assessment is based on 3 criteria, assessed quarterly:
- Rate of candidates advanced from CV screen to initial interview
- Conversion rate of candidates from CV to offer
- Rate of candidates rejected on what we call the GoCardless fit measure
The CV to initial interview rate: We expect that 70% of candidates submitted by the agency will make it to the initial screening call. We are more lenient with new agencies or new roles as we know initial CVs might be more exploratory.
The CV to offer rate: We expect to make an offer to at least 10% of candidates submitted by external agencies. Also, depending on the time and number of candidates the agency has submitted for a particular role, we expect to make at least one offer per quarter.
The GoCardless fit rate: One of the main advantages of using external agencies compared to job boards is that they will have pre-screened candidates. They should be able to evaluate whether candidates’ motivations and goals align with GoCardless’ values and what we can offer. Therefore, we expect that no more than 20% of candidates who make at least the first interview with us will be rejected based on these criteria.
The final index
Our index is split by department (e.g., engineering, marketing) and shows us how the agencies we’ve been working with in that quarter performed in these three criteria. Once the data is added, it updates the decision field to “Keep working with agency” if:
- The agency has positive scores on two out of three of the criteria, and
- The agency has a positive score on the GoCardless fit rate.
Looking at these criteria has allowed us to focus on the relationships we value most and end relationships that were not mutually beneficial (e.g., focus on quantity vs quality).
How can this system be beneficial to agencies?
We keep working with the agencies that are most successful in placing candidates. When we see a drop in performance, we’re able to look into it together and find ways to improve the relationship. We have done this in the past with two of our best performing agencies in which we noticed a drop in performance. We were able to discuss our and their challenges with the roles, understand where the profiles submitted were lacking, and have a general refresher on the role and teams.
How strict were we when implementing this system?
It took us a while to implement the assessment system, at least a quarter of trialling it to find if it aligned with our own empirical experience, and it is a process we're continuously working on. Initially we thought about making a “make-it-or-break-it” system in which the agencies should meet all 3 measures. However, we found that agencies might not meet all the criteria for extraneous reasons (e.g., changes in role requirements) and there is a level of flexibility that should be considered.
The only criterion we’re not flexible with is the ability to bring in candidates who demonstrate a good GoCardless-fit, giving candidates a good experience throughout the process, and building and maintaining a good relationship with our Talent Partners. The latter two fall outside the Performance Index, but are easily picked up either in our day to day work, or in the survey we send to all new joiners.
Assessing the agencies we work with is mutually beneficial. For the Talent team, it means working with fewer agencies and getting to know who we’re working with and how they can help us. For the external agencies, they know we might consider them whenever we’re in a position to reach out for external help, and we will invest our time to make sure the collaboration is productive.
When we implemented the measure we stopped working with 5 agencies. In the past 3 months of trialling this measure we’re now getting to the point in which we know the 1-2 agencies we want to work when we open a specific role and that we trust will fill it in a timely manner.
NB: this is not a promotion piece for working with agencies! We wish we could always hire directly and that is always our preferred way to find new talent to join our team. However, we work with some agencies we’ve gotten to know well. If you are from an external agency who we don’t work with, please do not call or email. We only work with agencies whose introductions are done by referrals from trusted sources.
What's up at GoCardless: November 2016
📷 Matt Heath @mattheath
2nd November - The DevOps Series: Insider Talks
The Insider Talks is the first event on GoCardless DevOps Event Series. Our own Chris Sinjakli presented "Lessons Learned the Hard Way: Postgres in Production at GoCardless" and Simon Vans-Colina presented "DevOps at Monzo". Stay tuned for more information on the next event!
16th November - Trans*Code London Remembrance Social
The Trans*Code community has had a busy year. It was one marked by incredible accomplishments from members of our community, but also touched by loss. One of our much loved community members, Emily Voukelatos, passed away earlier this year. Her coworkers, friends and members of the Trans*Code community created an event to gather and remember her, while looking back at the year that's ended and forward to the one ahead.
21st November - The Family. Stop Paying for It: How to Not Waste Money as an Entrepreneur, by Oussama Ammar
Join TheFamily for a talk by Oussama Ammar, Co-founder & Partner at TheFamily, to hear about how you can stop wasting cash. 💸
Wondering what we’re up to next month?
- 16th December - Model Westminster Pizza & Politics
(Re)designing the DevOps interview process
Interviewing is hard. Both the company and the candidate have to make an incredibly important decision based on just a few hours’ worth of data, so it’s worth investing the time to make those precious hours as valuable as possible.
We recently made some changes to our DevOps interview process, with the aim of making it fairer, better aligned with the role requirements, and more representative of real work.
We started by defining the basics of our DevOps roles. What makes someone successful in this role and team? What are the skills and experience that we're looking for at different levels of the role?
It was important that the process would work for candidates with varying experience levels, and so it needed to be flexible and clear to assess skills at each of these levels.
The skills we’re looking for fall into three broad categories: existing technical knowledge (e.g., programming languages), competency-based skills (e.g., problem solving), and personal characteristics (e.g., passion for the role, teamwork and communication skills). After defining these skills, we mapped out how we would assess them at each stage of the interview process.
The application review stage
At this stage, we want to check whether your background and interests align with our expectations for the role. Your CV and cover letter tell us about your experience and prior achievements; however, we also ask for a description of a technical project or problem that you’ve found particularly interesting. With this question, we’re looking to hear more about what motivates you and what kind of challenges you enjoy working on.
The phone interview
Assesses: motivations and communication
If we think there’s a match, the next step is a call with someone from the People team. In this call we’ll:
- Say hi! 👋
- Ask any questions we have about your application or background.
- Find out what you’re looking for and your motivations for the role.
- Guide you through how the rest of the interview process will work.
- Give you plenty of opportunity to ask questions about the role, the company, or anything else on your mind - interviews are two-way processes and it’s as important for us to know about you, as for you to know about us (we hope the answer is yes!).
The video interview
Assesses: motivations and interests, and web knowledge
This stage involves a one hour video call with two of our engineers. They will start by digging into your technical interests and background and finish with some questions about web fundamentals and troubleshooting. As with the phone interview, you will have plenty of time to ask our team questions, so don't be shy!
The live challenge
Assesses: problem solving, communication, logic, OS knowledge, troubleshooting
Research indicates that work sample tests provide some of the best indicators of future performance (e.g., Bock, 2015). For this interview, we looked through post-mortems for issues that we’ve debugged on the job, and built an EC2 instance that exhibits a number of these issues. We’ll give you remote access to the VM, and work with you to solve them.
The main aim of this stage is not to solve everything, but to gradually progress as you encounter different issues during the interview. For example, we want to know how you approach problems and why you might choose one process over the other. We’re interested in how you communicate with the team during the process and how you address the issues that you find.
The final stage
If everything else goes well, we’ll invite you to our office in London to meet the team. This final stage consists of three interviews.
The technical interview
Assesses: domains of technical knowledge (e.g., Linux internals, networking, storage)
The technical interview is usually held by someone from the Platform team, along with our CTO. Our engineers believe in giving people the best opportunity and will allow you to select the topics you will discuss during the interview. The level of difficulty within each topic will gradually increase until you decide to move the next one.
Pair-coding with our engineers
Assesses: collaboration, communication, logic and coding
This is another live exercise, but this time you’ll work directly with the team. We'll spend 90 minutes working together on a coding exercise that’s representative of the kind of task you’ll be doing on the job.
The soft skills interview
Assesses: motivation, team work and influence, and shared values
The final interview is a chat with someone from our management team. We’ll discuss your motivations and aspirations and whether your values match those of the company. Most of all, we want to know that you’ll thrive at GoCardless and within the Engineering team. This interview isn’t a personality test - we’re not looking to hire people who are the same as us. Instead, we’re looking to hire people that care about the same things as us. Making sure we add to our culture is as important as the tech skills you bring in; after all, we like where we work, but we love the people we work with!
Redesigning the interview process has hugely improved the experience of our engineering team and their confidence in making decisions after interviews. We've also received great feedback from candidates, especially regarding the live problem-solving challenge, which many find challenging and fun.
Overall, the redesign was a success in targeting the main issues that the team was facing. However, we're always looking for ways to improve and we're constantly adapting our processes based on feedback we receive from candidates and interviewers.
If you're curious about GoCardless, why not apply?