TLDR: We made a hackathon as part of the recruitment process. Results were mixed. The learnings are that prequalification cannot be skipped, that real-world code challenges can be great but are hard to rank.

On The Merits of Hackathons: At Peanut Protocol, we decided to experiment with a new hiring model: a real stakes hackathon.

1. Fair Play Over Fancy Play:

   • When it comes to hiring, one might argue that the act of doing often speaks louder than a paper qualification. A hackathon prioritizes output, the tangible results a candidate can produce under real-world conditions.

   • This method minimizes the weightage of potentially misleading credentials. Sure, candidates might boast impressive titles from illustrious companies, but that alone doesn't offer clarity on their individual contribution or the depth of their expertise. We believe in judging talent based on what they can achieve now, not what their resume claims they've done in the past.

2. The Double-Edged Sword of 'Hacky' Solutions:

   • As much as we appreciate the concept, the term "hackathon" does bring along certain baggage. By nature, these events can promote quick, 'hacky' solutions that prioritize immediate function over long-term viability.

   • In the professional realm, there's a stark difference between patching something together swiftly and crafting code that stands the test of time and scalability. We're in the business of building the software backbones of the future financial system, not temporary fixes.

3. Bridging The Gap - A Hackathon With Real Stakes:

   • Drawing inspiration from the merits and mindful of the pitfalls, we've arrived at a middle ground. We’re not just hosting any hackathon – we're introducing one that deals with real PRs, directly influencing our active codebase.

   • This approach aims to harness the raw innovation and problem-solving prowess of a hackathon while emphasizing the creation of production-ready code, rather than the idea or scope. After all, it's not just about solving the problem; it's about solving it right.

We hope this initiative showcases our commitment to not just finding the best talent but doing so in a manner that's both innovative and just.

The Setup

We created a quick Notion page that explains the basics, gives some resources, lets the participants know about communication channels etc. Here is an excerpt:

The Results

1. The Work-to-Outcome Ratio was surprising. We invested in senior FTE time, made several calls with a hiring agency to run the hackathon (it was their idea in the first place). While the results didn't completely match our expectations, the insights we gathered were invaluable.

2. From the 11 candidates who took part, we received 5 PRs. In contrast, just posting on web3.careers got us 170 applications in 2 weeks. Of these, about 100 took the time to answer our questions and fit our initial criteria. The questions took about 20 minutes to answer.

3. The PRs quality was very mixed but low overall. None of the PRs made it to production.

Key Learnings

1. Pre-qualification is key. For such a hackathon to make sense, the candidates have to be well-qualified. This is very difficult and still a lot of work on the recruiter’s side.

2. Negative selection. We assume that the best candidates would not even be bothered with so much effort.

3. The code reviews are still costly. It took more than 2h to review, rank and comment on the 5 PRs. In 2h, 3-5 short screening calls could have been made easily, 2 candidates chosen, and then code challenges given.

4. Comparing the PRs was difficult. Since scope and topics varied greatly, it was hard to rank them objectively.

Our conclusion: it seems like for this approach to be successful, good prequalification is needed anyway. Therefore, a mix of screening calls and code bounties will probably do a better job.