Homework assignments and personality quizzes won’t give you excellence
This year I was once again hiring junior level developers, and the same dynamic was at work, but I got a surprising reaction from the person I spoke to. I’ll call her Zareen, who had just come through the Grace Hopper program.
Zareen had been interviewing at a few different places, but I assumed she was still open to hearing about the specific job that I was hiring for, so we arranged a phone call. We chatted for 15 minutes, and then I suggested she should come by the office and meet the whole team. She was just absolutely stunned.
"Wow, I didn't expect things to move this fast!" she exclaimed.
Let's think about that for a minute. She wants a job, I might want to hire her, I ask her to come by the office for an interview, and so she is stunned. It says a lot about how broken our hiring processes have become if what used to be the absolutely boring, dull, and standard process now provokes the response, "Wow, I didn't expect things to move this fast!"
Apparently other companies were giving her homework assignments and personality quizzes and phone interviews. “Please go to this website and take this test, we are trying to figure out what your skills are.”
At two other companies, she had already done 20 minute interviews, but never with anyone on the tech team. Instead she got a call from someone in the HR department, who read a checklist of words, which the HR person did not understand, but they were all requirements: “Have you ever heard of HTML? Do you know how to use that? What about CSS? Do you have that? How many years of skill do you have with CSS? And Java? I mean, Javascript? Are those different? Yes? Okay, I think we want Javascript. Do you have that? Yes? How many years?”
What an empty ritual; reading words that are not understood.
If you are lost at sea, don’t drink the brine
Stories abound of people lost at sea, perhaps because their ship engine died, or their ship sank and they escaped in a life raft. Once the clean drinking water runs out, the temptation to drink the sea water is strong. But of course, that is a fatal step: the sea has more salt than than the human body can tolerate; drinking makes you more thirsty, not less so. And yet the sea is vast. What a paradox, to die of thirst amid such an abundance. “Water, water, everywhere, and not a drop to drink,” as Coleridge said.
I’ve had this conversation with tech team leads:
Me: Where are you looking for candidates?
Them: I’ve put a job posting up at LinkedIn and Indeed and StackOverflow.
Me: Did you get any candidates that you like?
Them: Well, the problem is, I’ve gotten several thousand. The problem is trying to narrow down the pile. I cannot possibly look through all of them. I’ve decided to post the job a second time, but this time I created quiz that people have to take.
Me: What is the quiz about?
Them: It’s 100 questions about the technologies we use, plus some algorithm questions that should help cut down the number of applicants who get through.
Me: You created the questions yourself?
Them: No, I copy-and-pasted them from various online sources. Most of them are about standard stuff. I put in a few trick questions to help filter out the weak candidates.
Me: Are you sure you want the candidates who get your trick questions?
Them: Well, if they get the trick questions, then they must know some fairly obscure stuff.
Me: Is that what you need right now, people who happen to know some fairly obscure stuff?
Them: Well, not really, our work here is fairly standard, but I need some way to screen the applicants because the first time I posted the job we got way too many responses.
Me: Are you sure the people who get your most obscure questions are the candidates who will work out the best once you hire them?
Them: No, but I’m not sure what else to do. You never really know about a person until you’ve seen them actually work.
Me: How many people do you need to hire?
Them: I need 3 developers for the frontend, and 2 more for the backend.
Me: That seems like a manageable number. Have you tried to find some good candidates from friends of friends?
Them: Oh, I don’t have time for that. We need to move quickly with these hires.
Me: Is that any slower than making up quizzes for Indeed?
Them: Well, I’m not sure how I would even start.
Me: Something simple. You could go through your email contacts and make a special list with the email addresses of those who you know work in tech. Software developers, project managers, designers, anyone who might be able to give a meaningful recommendation. Send out an email explaining what you’re looking for.
Them: Hmm, well, some people might feel that I’m trying to poach talent from their team.
Me: They don’t have to respond. They can simply remain silent.
Them: I worry it would feel like I’m asking for a favor.
Me: That’s exactly what you are doing, you are asking for a favor. And then later on, when they are hiring, you could potentially pay them back by recommending someone, if at that later time you know of a good software developer who is looking for a job, when you yourself are not hiring.
Them: Yeah, but, if they don’t know of anyone, then it’s just a big waste of time.
Me: But it sounds like your original effort with Indeed was a waste of time.
Them: Sure, but now that I’ve got a quiz in place, I’m hoping to filter out more of the applicants, so I’ll get a smaller number.
Me: But the process does take time, right? No matter what you do, you have to invest some time in this?
Them: Yeah, it really sucks. I’ve got a lot of work to do, but instead I have to waste time hiring.
Me: But hiring is work. This is the work that you are doing right now. This is important.
Them: Sure, I guess that’s true, but I’ve also got a lot of software that I’m supposed to write.
Me: But as your team grows, you’ll need to spend less time writing code and more time focused on management tasks, such as hiring people.
Them: I’m afraid so.
Me: Is that something to be afraid of? Your team is growing, it sounds like things are going well.
Them: Yes, I guess that’s true, but I’m worried, once I’m no longer writing code every day, and reading everyone else’s code, how can I be sure the quality of the work will still be strong?
Me: You could hire excellent people and then expect excellent results from them.
Them: Yeah, but that’s a lot easier said than done. You should see some of the resumes that I got from the applicants. Full of typos, obvious exaggerations, mistakes about dates, or dates that overlap, stuff that makes no sense. Some of these resumes are so bad it’s funny. I’ve started sharing them with the team so we can all have a laugh about it.
Me: It sounds like Indeed isn’t really giving you what you want.
Them: That’s true, Indeed sucks. The applicants we’re getting from these online sites, most of them are really awful.
Me: Are you sure you don’t want to reach out to your acquaintances and ask them for recommendations?
Them: Oh, gosh, no. Like I said, that would take too much time.
For many categories of job applicants, job sites like Monster or LinkedIn or Indeed or StackOverflow are basically like drinking the brine: they offer such a abundance of material that you are tricked into thinking “Surely we can drink an endless amount from this source.” And yet, they waste so much of your time that they are effectively a kind of poison.
It's pretty obvious that this whole "email everybody you know" method to find candidates has some serious selection biases. In particular, you're more likely to get referrals to people who are in the industry and have proven themselves, which clearly isn't a bad thing in and of itself, of course. However, I'd be concerned that this sort of thing just selects for people who are like the people that you know already. That doesn't seem like a solid strategy when what you want is actually the best available people who are willing to come work with you. How would you design a recruiting process to attract those people?
You don't need "the best available people" though unless your plan is to completely dominate an entire industry (maybe it is, but it's probably not). And even then unless you've got enough money to make your developers rich and enough interesting technical challenges to keep them there after they are, you're not going to get the best people at all.
Excellent question. If one accepts that no hiring process is 100% objective, then one is left with the conclusion that the only way to have a hiring process that is fair to everyone is to deliberately aim to overcome one's subjective biases. I try to address it in this part of the book:
---------------
Recommendations from friends or friends of friends should count for a lot
When hiring, I rely heavily on recommendations from people I know. To use some Silicon Valley jargon, I rely on "social proof." If a good engineer, who I've worked with, recommends some other engineer, that counts for a lot with me.
I wrote this on Twitter and James Youngman responded, “It seems to me that this kind of approach is what perpetuates the domination of the industry by an in-group (who know each other, directly or indirectly) at the expense of outsiders, to the detriment of both diversity and fairness.”
To be clear, this approach very much works when you are hiring novices who are straight out of school. Back in 2018, when I need some junior level frontend software developers, I hired several women from the Grace Hopper program for women that is run by Fullstack Academy. In that case I spoke with some experienced friends of mine who had either taught at the schools or volunteered as mentors at the schools. As such, they could point me to who were the best of the graduating class. As it was, I ended up with an unusually excellent team of novices.
We all have a professional and ethical obligation to be sure we hire a diverse workforce – the workforce must be open to anyone who has real talent. There is no contradiction between that obligation and a requirement that a candidate be recommended by someone we trust. Assuming you have friends who understand their professional and ethical obligations the same way you do, they should be able to give recommendations on people they know, or put you in touch with friends of theirs who can offer such recommendations. Within your extended social penumbra of friends, friends of friends, and friends of friends of friends, you should be able to find someone who can vouch for most of the candidates that you need to hire. You just need to put in the effort, chasing down those recommendations through extended chains of friendship.
Beware of fake, dishonest attempts at establishing an "objective" hiring process
In the tech industry, I've seen good people, full of good intentions, invest serious time in developing what they felt was an unbiased, fact-based system of hiring, but they were blind to extent their own preferences were being treated as facts.
A specific example of this: there was a CTO, a man I've worked with and who I largely respect, who put serious time into developing a code base, and then deliberately adding some errors and redundancy to it. This code base would then be given to job candidates, as a homework assignment. Could they find all the mistakes, fix them, and remove all of the redundancy?
Many people in the tech industry now rely on tests like this, which are seen as being more objective than older job interview techniques. But this overlooks how much of the assignment is actually subjective in nature. In particular, what does it mean to reduce redundancy in a code base? Anyone who's done much programming is aware that one can refactor a code base almost to infinity. One can create more and more abstractions. At the extreme, one can create a new programming language where the program is represented by just a single character, perhaps the letter "w"":
w
There it is, the whole entire program that runs a major website. Of course, you also need to create a compiler that knows how to parse that letter into a lower level language that would do the real work of running the website. But if you ask a software developer to remove the redundancy in your code base and they come back to you with code that has more than 1 letter in it, then they've clearly failed the assignment.
Am I being ridiculous? Yes, absolutely, because I'm ignoring nuance, subtlety, reasonableness and proportionality ALL OF WHICH ARE SUBJECTIVE. If you want a truly objective standard, one needs to accept absurd, surreal, and almost psychotic behavior, because that's what human behavior becomes when you try to remove all subjective factors.
Does this mean the hiring process can never be truly objective? Yes.
Does this mean the hiring process must be unfair, unjust, corrupt and cruel? No.
----------
Homework assignments and personality quizzes won’t give you excellence
This year I was once again hiring junior level developers, and the same dynamic was at work, but I got a surprising reaction from the person I spoke to. I’ll call her Zareen, who had just come through the Grace Hopper program.
Zareen had been interviewing at a few different places, but I assumed she was still open to hearing about the specific job that I was hiring for, so we arranged a phone call. We chatted for 15 minutes, and then I suggested she should come by the office and meet the whole team. She was just absolutely stunned.
"Wow, I didn't expect things to move this fast!" she exclaimed.
Let's think about that for a minute. She wants a job, I might want to hire her, I ask her to come by the office for an interview, and so she is stunned. It says a lot about how broken our hiring processes have become if what used to be the absolutely boring, dull, and standard process now provokes the response, "Wow, I didn't expect things to move this fast!"
Apparently other companies were giving her homework assignments and personality quizzes and phone interviews. “Please go to this website and take this test, we are trying to figure out what your skills are.”
At two other companies, she had already done 20 minute interviews, but never with anyone on the tech team. Instead she got a call from someone in the HR department, who read a checklist of words, which the HR person did not understand, but they were all requirements: “Have you ever heard of HTML? Do you know how to use that? What about CSS? Do you have that? How many years of skill do you have with CSS? And Java? I mean, Javascript? Are those different? Yes? Okay, I think we want Javascript. Do you have that? Yes? How many years?”
What an empty ritual; reading words that are not understood.
If you are lost at sea, don’t drink the brine
Stories abound of people lost at sea, perhaps because their ship engine died, or their ship sank and they escaped in a life raft. Once the clean drinking water runs out, the temptation to drink the sea water is strong. But of course, that is a fatal step: the sea has more salt than than the human body can tolerate; drinking makes you more thirsty, not less so. And yet the sea is vast. What a paradox, to die of thirst amid such an abundance. “Water, water, everywhere, and not a drop to drink,” as Coleridge said.
I’ve had this conversation with tech team leads:
Me: Where are you looking for candidates?
Them: I’ve put a job posting up at LinkedIn and Indeed and StackOverflow.
Me: Did you get any candidates that you like?
Them: Well, the problem is, I’ve gotten several thousand. The problem is trying to narrow down the pile. I cannot possibly look through all of them. I’ve decided to post the job a second time, but this time I created quiz that people have to take.
Me: What is the quiz about?
Them: It’s 100 questions about the technologies we use, plus some algorithm questions that should help cut down the number of applicants who get through.
Me: You created the questions yourself?
Them: No, I copy-and-pasted them from various online sources. Most of them are about standard stuff. I put in a few trick questions to help filter out the weak candidates.
Me: Are you sure you want the candidates who get your trick questions?
Them: Well, if they get the trick questions, then they must know some fairly obscure stuff.
Me: Is that what you need right now, people who happen to know some fairly obscure stuff?
Them: Well, not really, our work here is fairly standard, but I need some way to screen the applicants because the first time I posted the job we got way too many responses.
Me: Are you sure the people who get your most obscure questions are the candidates who will work out the best once you hire them?
Them: No, but I’m not sure what else to do. You never really know about a person until you’ve seen them actually work.
Me: How many people do you need to hire?
Them: I need 3 developers for the frontend, and 2 more for the backend.
Me: That seems like a manageable number. Have you tried to find some good candidates from friends of friends?
Them: Oh, I don’t have time for that. We need to move quickly with these hires.
Me: Is that any slower than making up quizzes for Indeed?
Them: Well, I’m not sure how I would even start.
Me: Something simple. You could go through your email contacts and make a special list with the email addresses of those who you know work in tech. Software developers, project managers, designers, anyone who might be able to give a meaningful recommendation. Send out an email explaining what you’re looking for.
Them: Hmm, well, some people might feel that I’m trying to poach talent from their team.
Me: They don’t have to respond. They can simply remain silent.
Them: I worry it would feel like I’m asking for a favor.
Me: That’s exactly what you are doing, you are asking for a favor. And then later on, when they are hiring, you could potentially pay them back by recommending someone, if at that later time you know of a good software developer who is looking for a job, when you yourself are not hiring.
Them: Yeah, but, if they don’t know of anyone, then it’s just a big waste of time.
Me: But it sounds like your original effort with Indeed was a waste of time.
Them: Sure, but now that I’ve got a quiz in place, I’m hoping to filter out more of the applicants, so I’ll get a smaller number.
Me: But the process does take time, right? No matter what you do, you have to invest some time in this?
Them: Yeah, it really sucks. I’ve got a lot of work to do, but instead I have to waste time hiring.
Me: But hiring is work. This is the work that you are doing right now. This is important.
Them: Sure, I guess that’s true, but I’ve also got a lot of software that I’m supposed to write.
Me: But as your team grows, you’ll need to spend less time writing code and more time focused on management tasks, such as hiring people.
Them: I’m afraid so.
Me: Is that something to be afraid of? Your team is growing, it sounds like things are going well.
Them: Yes, I guess that’s true, but I’m worried, once I’m no longer writing code every day, and reading everyone else’s code, how can I be sure the quality of the work will still be strong?
Me: You could hire excellent people and then expect excellent results from them.
Them: Yeah, but that’s a lot easier said than done. You should see some of the resumes that I got from the applicants. Full of typos, obvious exaggerations, mistakes about dates, or dates that overlap, stuff that makes no sense. Some of these resumes are so bad it’s funny. I’ve started sharing them with the team so we can all have a laugh about it.
Me: It sounds like Indeed isn’t really giving you what you want.
Them: That’s true, Indeed sucks. The applicants we’re getting from these online sites, most of them are really awful.
Me: Are you sure you don’t want to reach out to your acquaintances and ask them for recommendations?
Them: Oh, gosh, no. Like I said, that would take too much time.
For many categories of job applicants, job sites like Monster or LinkedIn or Indeed or StackOverflow are basically like drinking the brine: they offer such a abundance of material that you are tricked into thinking “Surely we can drink an endless amount from this source.” And yet, they waste so much of your time that they are effectively a kind of poison.