I got 7 job offers during the worst job market in history. Here’s the data......

Introduction

I got seven data science job offers during the worst job market in U.S history. I’ve completed over a hundred data science interviews, ranging from giants like Apple, Google, Facebook to four person startups. No, I don’t have a Masters or PhD. 

Sometimes, my imposter syndrome asks me what the hell these companies are thinking. I don’t deserve it. Other times, I quietly whisper in my ear that I’m superman and proceed to flex my chest muscles, ripping my shirt to shreds.

Through the process, as a data scientist, I hand-tracked all my data so I could play moneyball with the numbers. Here's what I found.......

High Level Statistics

This table maps out the interview funnel. The white rows, count and conversion, are my numbers. You can see I converted 46% of my onsites to offers and converted about 41% of my applications to phone screens.

The gray portion is the expectation. The expectation acts as a baseline to compare my performance. Stages that dip below the expectation is where I need to improve. You can see, that I beat the expectation at each stage. 

To compute the "expectation", I start with the number of offers I want to target, two. Then, work backwards to figure out what conversion rates I need to target on each stage to hit my desired offer amount.

Let’s dig a bit deeper into those first stages: Applications & Phone Screens. 

How much do years of experience impact my likelihood of getting an interview?

Intuitively, we know that more years of experience should improve our ability to get interviews. But by how much?

I defined interview rate as # of phone screens/# of applications. Basically, how likely was I going to get an interview after I applied to the position?

When I had zero years of experience, my interview rate was 17%. With three years of experience, it was 41%. Each year of experience had increased my interview rate by about 11.5%. Keep in mind, the economy didn't suck three years ago, so I suspect my interview rate would’ve been much higher during a great economy. 

So for anyone just entering a field and is struggling to get interviews, just know that it will get easier as you accumulate more experience. If years of experience matter, the group that’s struggling the most are the entry level applicants. So how many applications should they target?

How many applications should I be sending out to get an offer?

To answer this questions, I developed a new metric called Applications Per Offer or APO. You'll see here, that with 0 years of experience, it took me about 62.5 applications to generate each offer. Compare this to three years of experience, it took me 10 applications to generate each offer.

However, 10 applications per offer is ridiculously low and should not be interpreted as “Oh, I only need to send 10 applications to get one offer.” I’m making a false assumption of the data:

The problem with this metric, is that offers are not independent. Once you get your first offer, your chances of getting a second offer dramatically increase. 

The goal should be getting enough applications in to get to that first offer. Once that first offer rolls in, then we hit a tipping point where other offers start rolling in. Relationship looks more like this:

Unfortunately, I didn’t track timestamps so this curve is based off an eyeball, rather than an actual fit of the data. *Kicking myself…* If we do a rough eyeball of this graph, we see that the first offer hits at around application 50 ish which equates to about 70 to 80% of the total.

So how many applications should you target? As a ROUGH approximation:

1. Figure out the total number of applications you’ll need to send to hit your target offer amount.

2. Multiple that by 80% to get the required apps until first offer.

if you have 0 YOE, let's say you use my 62.5 Applications Per Offer metric. If you wanted to target 2 offers, you'd need 62.5 x 2 = 125 applications. 125 x 80% = 100 applications. Expect to send at least 100 applications to hit that first offer. More, when there’s a poor job market.

Caveats:

• 62.5 Applications Per Offer is based off my own data when the economy was good. It’s likely way, way higher when during a tough market. Later in this post, I talk about how to adjust for economic downturns.

• Every person's situation is different. I know people who've gotten an offer sending only a few applications and some who've sent hundreds of applications with no offers. There is high variance here.

One additional problem to point out, is we’re treating every application the same. In reality there now. Let’s figure out if we can improve our odds with a specific application strategy.

What's the most effective method of applying to positions?

If you're going to aim for 100+ applications, not every application should be treated equally. We intuitively know dropping your resume into the job board blackhole has a terrible response rate. But by how much? Let's look at the numbers.

We’ll use interview rate again (# of phone screens/# of applications) as our core metric. Unfortunately, I didn't record my data for years of experience = 0, so I only have data for the YOE = 1 and YOE = 3. But let’s first assume the interview rates across different methods of applications:

You can see here that the interview rate is around 43% for referrals, 29% for cold e-mails and 6% for direct. If you have 0 YOE, take a haircut off these numbers. Cold e-mail is finding the hiring manager's email address and emailing them your resume. I'll talk about this in a future post.

This graph says that yes referrals are the best channel. However, what happens when we have three years of experience?

With three years of experience, you'll see that across the board, my response rates are higher. But looking at the “Direct” channel, we see a dramatic increase in my interview rates. At zero years of experience, cold e-mails increased my odds by nearly 5x compared to direct. However, at three years of experience, cold emails only increased my odds by 1.3x, which is good, but not amazing.

This leads to the insight that the more experience you have, the better of a response you’ll get through direct applications. With more experience, you have more leeway to go the easy route.

So how should I spend my time?

Figuring out how to spend your time is an optimization problem. You want minimize the amount of time you spend to maximize your ROI (offers). We all know that getting referrals is the best strategy. For a referral, the cost to yourself is <5 minutes writing a message asking for the referral. In return, you get an interview 43% of the time.

If you know someone at the company, even as an acquaintance, it's almost always worth it to ask for the referral.

However, if we don't know anyone at the company, this is where it gets interesting. If I had a choice of either applying directly or through cold e-mails, what is my ROI for each channel? To do this, I built out a simple model:

Assumptions:

• Response Rate to Direct Applications: 2%

• Response Rate to Cold E-mails: 10%

• Probability of Offer given Phone Screen = 12.5%

• Direct Applications take 15 minutes to complete

• Cold Emails take 45 minutes to complete

Pure Direct is purely applying only through direct job boards. Pure email is finding key decision makers at the organization and sending a cold e-mail pitching yourself. 

If we applied 100% of a single strategy, using our simple model, here are the results:

With zero years of experience, writing cold e-mails has the best return on time spent. However, this changes when you have more experience, as the response rate to your direct applications increases. Now, if we were to update our assumptions with my actual response rates with 3 YOE:

You can see here that although cold emails require fewer applications, it requires significantly more time. This leads us to say that, as you accumulate more experience, if your goal is to maximize offer amounts, cold emails may be less and less optimal. 

However, we know that cold emails have a higher response rate. Since they require more time, if you’re more experienced, save cold emails for the companies you love. 

Note: This is a simplistic model that used basic algebra to make these predictions:

def compute_required_hours(time_per_app, response_rate, interview_offer_rate):
    expected = 1
    num_apps = expected/(response_rate * interview_offer_rate)
    
    hours_required = (num_apps * time_per_app)/60
    return hours_required, num_apps

I made it to the onsite. What are my chances of getting an offer?

Let's use some basic probability to figure out your chances of getting the offer.

You've made it to the onsite. Congratulations! Let’s assume, that the onsite has 4 interviews, ranging from behavioral to technical. Typically, you'll need at least 4 yes' to get an offer. However, a single No will result in a rejection.

To make computations simple, let's assume that I knew the right answer to a question 80% of the time and I didn't know 20% of the time. If I knew the right answer, I'd receive a Yes. If I didn't, I'd receive a No.

So with an 80% chance of passing a single interview, the probability that I pass all interviews is 0.80^4, which is 40% chance of getting an offer. Therefore, we'd expect 0.40 offers. To hit one offer, we’d need 2.5 onsites. (2.5 onsites x 40% = 1 offer). 

While this isn't an exact science, even if you're well-prepared, with a probability of 80% of passing each interview, your chances are still worse than 50/50 in getting an offer for a single onsite.

Now, let's throw in another variable: competition. During covid(or some other economic downturn), companies cut headcount so they only have 1 spot available. Now, let's say that you got 4 Yes' but two other candidates got 4 Yes'.

At this point, it turns into a game of chance between you and the other candidates. Since our previous expectation calculation is assuming that 4 Yes’ = an offer, we must normalize our expected offers by the number of competing candidates who also got 4 Yes’. 

So in our example, our expected offers goes from 0.40 to 0.40/3 = 0.13. So if you originally needed 2.5 onsites to get 1 offer, you will now need 7.69 onsites to get one offer. 

In simple terms, if its an ultra-competitive market (like during covid), expect to do 2 to 3x more onsites than usual. I always ask the recruiter how much headcount they have to get my odds. In scenarios with multiple headcount, no need to normalize.

So what’s the best strategy? To increase the total number of offers, your levers are:

1. Raise my probability of passing

2. Complete more onsites

If I raised my probability of passing from 80% to 90%, for a single onsite, this increases my probability of receiving an offer from 40% to 65%. 65% isn't too shabby but can we do better?

However, let's say rather than 1 onsite, we increase our number of onsites to five onsites. Assuming we still have 80% chance of passing, so 40% chance of an offer for a single onsite. This means we have a 60% chance of failing an onsite. The probability that we fail all five onsites is only 7%, which means you have a 93% chance of receiving 1 or more offers.

It’s better to take 40% chance of success five times versus 65% chance of success once. So the insight here is that first, get good enough. Once you’re good enough, your time is generally better spent getting more onsites than raising your probability of passing, unless it’s a company you love.

It’s not worth it to spend 2 hours studying(to improve from 80 to 85%) some obscure thing a company might ask you, compared to getting more shots on goal. 

Takeaways

In the case of job recruiting, it's better to be data-informed than data-driven. There's a lot of nuance in job recruitment. There are some companies you like more and some companies you don't care much about. There are factors such as the economy, job fit, competition, resume, interview skills that result in an offer.

So my recommendations based off the data and personal experience would be:

• If you have a connection for a referral, always ask for it. You have the best odds, even if there’s a chance your connection doesn’t respond to you. 

• If you don't have much relevant experience, your strategy should lean cold e-mails over direct applications. But never  100% direct apps or 100% cold e-mails strategy. Always have at least a small percentage in one channel as an exploration mechanism. 

• If you are on H1-B and have a time limit to find a job, maximizing the number of offers per unit of time spent is the ideal strategy:

  • With fewer YOE, the best strategy is to lean more cold e-mails, referrals than direct applications. If you need a rule of thumb, always go referral. Afterwards, go 80% cold e-mails, 20% direct applications. 

  • With more YOE, you can afford to send more direct applications and just reserve cold emails to companies you love. 

• Be dynamic in your strategy. Don’t rigidly stick to 20% cold e-mails or any type of hard rule. Always gauge the response and the market and dynamically update your strategy. Everybody has a different situation. 

• For companies you love but have no referral, always at least send a cold email. The response rates are still higher than direct applications.

• One area I didn't mention, was if a recruiter reached out to you. If it's a recruiter from the actual company, I'd say take the call. Since they made the move, you're in a position of leverage and you don't have to do any work and your interview rate is close to 1 (compared to 43% referral, 29% cold e-mail etc).

• Your odds are better if you increase the number of onsites, rather than increase your probability of passing. 

Anyways, I hope this post was an insightful look into the job recruitment process. Best of luck and feel free to reach out with any questions! Feel free to reach out on Twitter.