A XP-inspired hiring process
In this article I’ll show how I took inspiration from the Values, Principles and Practices from Extreme Programming (XP) to uncover our Team’s Engineering Values together with its Practices and how we later reflected back into our hiring process and tried a novel idea.
This article is organized in the following sections:
- Existing Hiring Process: how our process was previously organized, what we thought its problems were and discussion about common practices in hiring that we felt weren’t good fit for us.
- Using XP to uncover our Engineering Values: how I’ve ran workshops to uncovered our Enginnering Values.
- Looking back at our existing process: how we ponder over our existing process through the glass of our newly discovered Values and the changes we decided to do.
- Conclusions: what were the benefits and disadvantages we saw in the changes we’ve applied.
Existing Hiring Process
We were a small team, working on the development of a Software as a Service (SaaS) application. While 3 engineers (including myself and the CTO) were working from Berlin, there were also 2 other engineers working remotely. Since Remote Work was running great for us, we’ve decided we would look
for candidates interested in working completely remote. Worth to note was that the company didn’t have a People Team at the time and, thus, hiring responsibilities befell on the team.
Our hiring process, initially, was a pretty common hiring process in the industry:
- Candidate applies online at job ad the company posted with a Curriculum and Cover Letter
- Hiring manager reviews the application (Curriculum and Cover Letter) and have a screening interview
- Hiring Manager e-mails the candidate a home assignment document
- Candidates e-mails back their solution of the home assignment
- Hiring Manager and team review the submission and schedule a pairing review of the submission
- Hiring Manager sends an offer or reject response to the candidate
The candidate would only move to a following step if approved in the previous step. So, for example, if the screening interview went bad, the Hiring Manager wouldn’t e-mail the candidate a home assignment.
I’ve emphasised bad in the previous phrase because that was the first thing we thought was lacking in our process: we didn’t have, for any of the steps, a shared definition of what bad meant. And worse, we didn’t have a shared definition of what good meant either. The process was based entirely on the gut-feeling of whoever would be applying the given step.
Curriculum regex
Some companies have people that would simply scan curricula for languages and frameworks the company currently use and discard anyone that wouldn’t match those. We felt this practice rejects, rather unfairly, candidates with great potential simply because they haven’t used the same technology the company currently uses. It was specially unpractical for us because we were using technology that was uncommon in the market (Elixir and Phoenix).
Code-in-the-browser assignments
Other companies use code problems that should be solved in a Web Browser, normally with a time clock. We felt that, while this practice may save time for both the candidate and the company, it puts the candidate in a stressful situation that’s very different from how the daily work of software development works in the company. We believed a non-stressful environment makes people deliver their best work and, therefore, this practice was not a example of how we worked. We’ll get to why when we connect this part to the values we’ve uncovered.
Beside those points, the candidate can’t write tests nor have time to explore different approaches. These type of tests don’t help you analyse how the candidate think about Modularization, Abstraction and general Communication, something that even simple practices as well-written commit messages can tell us.
Overly complex home assignments
Finally, there’s the practice of creating overly complex home assignments. The idea was, we imagined, to filter out people that were not “smart” enough. We felt this practice only lead to overly complex solutions that were very hard evolve later.
On top of those problems, the assignments normally have nothing to do with the problem Domain the company is on. Assuming that software is always changing we strive for solutions that are simple to understand and extend and having the problem connected to the company’s Domain helps the candidate understand what’s the problem the company is trying to solve to its customers.
Using XP to uncover our Engineering Values
With all the above in my head I’ve hypothesized that we could use XP’s idea of Values, Principles and Practices to uncover our own Values and Practices and use those to come up with a definition of what we were looking for when hiring.
Workshops to uncover Values
I’ve started by having a series of workshops that were limited to 1 hour. The outcome of each workshop should be a living document we could iterate over. I’ve started with a shared Google Doc which was open to every team member and later moved to a Notion document.
After refreshing our minds and discussing through the team chat about XP’s Driving and Corollary Values I’ve decided to start the workshop by listing the XP values explicitly and trying to come up with our own Values. We’ve then discussed and shuffled the Values in and out of our list of 5 Values to start until the meeting time ran out. The Values we’ve end up were: Communication, Simplicity, Courage, Quality of Life and Feedback. Unsurprisingly, most of the Values were still XP’s original Values.
After we had those 5 values documented I’ve decided to run a workshop for each. In these workshops we’d be documenting the practices that the team already had that match the Value being discussed. Before our first workshop I had conversations with our CTO about the idea of career ladders and they suggested I could try to map the practices to different levels of skill, similarly to the Dreyfus model of skill acquisition.
I’ve decided to limit the workshop into finding 3 Practices for each Value, and for each Practice find 3 points for each skill level. The workshop was limited to 1h, so we either found the practices for every level and finish earlier or we’d run out of time. In either case we would have a living document that the team could improve outside of the meeting.
The skill levels we’ve decided were: Novice, Competent, Proficient and Expert. I won’t go into details but the idea is that for each higher level there’s more responsibility on mentoring other team members and even sharing knowledge with the community outside of the company.
Some examples of Values and Practices can be seen below:
It’s worth to mention that some practices may fall in multiple values. As an example I’ll compare 3 different ways to get Feedback: refreshing the browser to see the changes, writing tests after the fact and Test Driven Development (TDD).
If we compare refreshing the browser to see the changes with writing tests after the fact we would notice that writing a unit test and running it to get feedback is faster than refreshing the browser, not to mention that the tests will also work as documentation to other programmers so writing tests after the fact not only has a better Feedback but it also has better Communication.
Now comparing writing tests after the fact with TDD we would notice that TDD has an even faster feedback loop because it tells immediately when we can stop writing code: when the test are passing. It has the same Communication benefits that writing the test after the fact though, generally, teams that use TDD tend to also care more about the quality of their tests and how they work as documentation.
But TDD also leads to simpler code (Simplicity) because there’s no extra code that doesn’t have a purpose for what the application currently needs. Take as an example the codebase that we end up with by the end of Growing Object-Oriented Software Guided by Tests (GOOS) and compare with the current codebase you may be working with. You’d notice that in GOOS case every line of code in there has a purpose, there’s no Class that you’d look and think: I have no idea why this is here and what is this doing.
It is important to note that it demands lots of practice to get to the high level of TDD skill mentioned, specially if learning alone and not from an experienced team. The other practices can be started, mostly, right away.
We’ll get back to this point later in the article when we mention the changes we did to our process.
Looking back at our existing process
As mentioned back into the Existing Hiring Process section, the process involved emailing the candidate a PDF with a problem definition. From that point on the communication with the candidate was cut until they emailed us back (either with a solution, or declining to continue the process, etc).
Looking at the Values that we just discovered, Communication was still one of the most important Values. Sadly, the current process didn’t seem to value it that much.
Communication with the candidate
Now with our eyes open to our own process we felt that the candidate was very disconnected from us and from the way we work with the email-a-PDF problem statement. We lacked a open channel of Communication. One of the practices we used to improve Communication was User Stories. User Stories are conversation-starters and, thus, work very well as an open channel of communication. In our day-to-day work we wrote stories in Trello, so the first idea we had was: we can turn that PDF with the problem statement in different User Stories and share that with each candidate.
We then created a template Trello board that would be copied into each candidate’s board and the candidate would then be invited to join that board. Our whole team would be available at each board so if the candidate had any question they could directly make them in the Trello card. Each Trello card represented a small User Story and there was, at total, 5 of them.
The Trello board also had 2 other advantages: the candidate had an open window into the exact same way we work, daily, and our team could work together with the candidate directly, seeing the progress and trying to help if there was any issue or question with a card.
This was important since the whole process didn’t have a time limit. We noticed that some candidates would leave a card in the Doing column for a couple of days. By asking if they need help we could either unblock them or find out that they were not interested in the process anymore. Sometimes they didn’t have time to go back to the problem due other responsibilities and would, then, resume working on it.
That also allowed us to see the style of working of the candidate. Some would take multiple cards at the same time while other would focus into a single card until making it Done.
After a candidate would finish all the cards we’d schedule a pairing/mob session where the whole team would work together in the solution already sent by the candidate. The candidate would be the main driver of this meeting, starting by explaining their solution, the Concepts and Abstractions they’ve chosen and the decisions they took. We would them ask if the candidate would like to change some characteristic of the solution, otherwise we would propose a new feature to be built on top of the current solution. Again, the candidate was the main driver but our team would be there not to critic but to help, e.g. Suggesting TDDing an issue instead of trying different solutions by constantly refreshing the browser.
Values Interview
The next thing we noticed was that our evaluation was focusing solely on the technical solution but it was not focusing on our Values. We then decided to add an extra interview step where we would have a conversation with the candidate. All of our interviews were done by a pair of our own people.
The objective was to be able to answer the following question: does the candidate seem to value the same values we do, even if they were not using our exact practices?
With that in mind we’ve decided to create questions that would help the interviewers assess the candidates in relation to our Values. The objective of these questions were to start a conversation, unblock or move the conversation in a different direction, if stuck. We were not interested in specific responses.
For example, while assessing Communication some of the question we would ask were:
- How big is the team you’re currently on?
- Tell us about something you did that you are not so proud of. What would you do differently if you had the chance to re-do it with the information you have today?
- From the agile methodologies you’ve worked with, what would be the technique that helped you most in your job?
In the first question we were not interested in an answer like 18, rather, we used the question to start a conversation around communication problems that arise as teams grow and how the candidate dealt with them.
As another example, and since Quality of Life was not one of the original XP Values but we’ve decided to take it as one of our own Values, some questions we would ask were:
- Do you have any hobbies? What do you enjoy doing in your spare time?
- Have you ever had to work outside of your normal working hours to solve issues? How have you felt about it?
Again, we were not looking for specific responses but using the questions to start conversations. We used to think of them as different cards that we could play if the conversation got stuck.
The questions were in the same living document we were updating with the Values so whenever we would think about a new question that could help us in a conversation around one of the Values, we would add it there.
Also, and coming back to the point done at the end of the Workshops to uncover Values section, when we were interviewing a candidate and comparing the practices they use against the practices we use, we weren’t using our practice as the One True Way. We were trying to access if the candidate cared about our same Values even if they use other practices to achieve it. We would look, though, if the candidate would self-assess their practices from time to time.
Conclusions
The biggest benefit we saw in this whole process was that we had made explicit what we valued and used an living document that should evolve with time. It made us have whole-team involvement in processes that previously were responsibility of an individual and made the team stronger.
It’s interesting to think, too, that we could come up with some novel ideas by focusing in the Values we discovered and the problem we were trying to solve instead of simply copying what other big companies are doing. I’ve never seen a company share a Trello board with a candidate and have this direct line of communication.
As a hiring process though, it suffered from the same unconscious biases from the processes it was trying to replace. Gender and names were immediately available for the interviewers by the hiring tool that we used.
We took some mitigating actions like prioritizing the evaluation of applications from female engineers and engineers from historically discriminated minorities as well as anonymising the candidate information from the code applications. For the latter you can use git-anonymize.
This is a very complicated topic and those actions wouldn’t be enough. Having gone through the process of trying to revamp our hiring process forced us to face and understand these issues better than we would otherwise.
In the end, we felt that our process became more human and connected to the candidate than the alternatives. It also evaluated other characteristics that are as important, if not more, than typing code down in a keyboard. Clear communication, thinking about trade offs and coming up with alternatives to problems are as important as knowing how to write an algorithm or the latest JVM additions.
Ultimately I don’t recommend you blindly use any of these solutions in your hiring process, first and foremost due the biases problems.
Feel free to try to run the workshops and uncover your own Values since that works very similarly to a Retrospective, but with a different focus.
More resources
When writing this post I’ve found a blog post written by Evan Bottcher of ThoughtWorks that follows a similar idea.
