Ask HN: Why not have candidates fix bugs in open source code?

wool_gather · on May 1, 2020

Unless you've already diagnosed and fixed the bug yourself, you really have no objective idea of how minor the bug is. And therefore I can't see how you could meaningfully judge candidates. If you're using a fresh bug with each candidate, that also decreases objectivity because you're not comparing apples to apples.

Additionally, if the candidate gets totally mired you don't have any way to give them a nudge, because you don't know what to do either.

Unless this is just a test of pair programming -- and without expecting to reach resolution of the bug -- it doesn't really sound useful to me.

Someone1234 · on May 1, 2020

Because the task is typically the same across all candidates, so you can judge them comparable between each other. Giving them completely different tasks negates the point of the activity (and also negates historic knowledge from past candidate's solutions).

sdwedq · on May 1, 2020

Seems like a good idea but a bit too complex, first you need to find an OS project that is close to your company's projects. It may work for PHP or JavaScript but not sure if it would work in less popular languages.

Then many bugs that appear simple might be a lot more involved. I have looked for easy bugs in my favorite open source projects, there are not many. All low hanging fruits get taken care of immediately in popular projects.

Then there are many semi-abandoned projects, so even if candidate fix the bug, it may not go anywhere.

And if you do find actively developed project, how long before maintainers accept the bug. Are you going to wait and see how candidate answers maintainer's questions and get their changes merged in?

Are you going to pay candidate for the work? Otherwise, if this becomes common, companies may take advantage of candidates to have them fix bugs in softwares they use.

fred_is_fred · on May 2, 2020

1) Finding suitable first bugs in open source projects is actually a lot of work. I've joined several OS communities over the years and spent a long time doing this. Often times they can have a complex or unique process to get a merge committed (Openstack & K8s) and there's also a lot of competition to find said bugs - especially on projects people are paid to work on.

2) Given the above, people have day jobs, families and lives - and a limited amount of time for the unpaid games you invent for them.

3) As someone else mentioned, many company make OS work against policy - even on your own time. Might not get caught, also might not be worth it.

shaftway · on May 1, 2020

I've actually had two in-person interviews that did this. They were old bugs in open source libraries that had already been fixed, but obscure enough that I was unlikely to know them. You solve the bug sitting with the reviewer. I don't think I had to actually fix them (though we talked about potential fixes), just identify the issue. The only one I remember was in the Python requests library.

I felt like it was a pretty good real-world test.

duxup · on May 2, 2020

My experience was that whiteboard type interviews are often conceptual and fairly shallow (at least in terms of amount of code) to make it easy to review.

A bug in any piece of non trial software could involve a ton of investigation / research as the candidate doesn't know that code base...

You could accidentally give one candidate an absolute nightmare of a bug, another fixing a fairly lame typo.

sethammons · on May 1, 2020

One counter point not listed: there are candidates who would be violating an employment agreement to work on an open source project.

sergiotapia · on May 1, 2020

Because a 4 hour take home is way too much. Respect your candidates, please. When we hire, we give a very simple 1 hour take-home.

giantg2 · on May 1, 2020

"If [you're] hiring", do whatever you want. I would rather see a candidate contribute to opensource prior to the interview than be forced into contributing during the interview.

facorreia · on May 1, 2020

Because candidates usually have multiple offers and will probably not bother.

mtmail · on May 1, 2020

Will the candidate be paid for the work (the third 'win')?

phil-martin · on May 1, 2020

No they wont be I don’t think, based on the “work done for free” part at the end.

I think that is the point if the open-source angle if it, that it is socially acceptable to donate time to an open source project.

When we hire, we only do very short interviews, and so long as the person isn’t an abrasive jerk, offer 10-40 hours of work at contracting rates doing real work for us, at whatever capacity they have.

It’s not a perfect system, because it excludes people who are unable to take it on. But for us it gives us the most accurate picture into their work style and who they are, and it also gives them an insight into who we are. Most importantly they get paid because we value their time.