Job Search Engine Guide

Are Job Boards in Denial?

   

Several weeks ago, during the 2007 Top Sites for Job Search competition, an executive with one of the medalists sent me a message:

   

“Timing sucks. We are launching major search relevance enhancements on Thursday this week.”

   

Delaying the testing until this competitor could release their ‘improved search’ wouldn’t have been fair to the other contestants.  Instead, I committed to retesting the ‘improved search’ at the conclusion of the competition and posting an update if it appeared the results would have changed.

   

As it turned out, the ‘major search relevance enhancements’ were more akin to The Emperor’s New Clothes.

   

Consider the following example – a search for:

   

¾      ‘insurance sales’ jobs;

¾      within 'x' miles of city ‘y’;

¾      posted within 7 days; and,

¾      sorted by relevance. 

   

The 'new and improved search' returned 156 ‘matching’ jobs.  Sounds pretty good until reality intrudes.

   

Only 31% of the ads (49 of 156) focus on insurance in some way … not very good.  The search engine doesn’t seem to discern a difference between selling insurance and receiving insurance benefits.

   

Worse, most of these 49 are not at the top of the list.  They’re spread throughout the list of 156 results; and the average rank for the 49 insurance related ads is 78.1 … right on the average rank for the full 156.

   

Furthermore, the 49 insurance-related ads include positions that are unrelated to ‘insurance sales’, including:

   

Compensation Managers (3)                   (deal with comp. and ins. issues)

Attorneys (2)                                         (dealing with insurance issues)

Customer Service Reps  (2)                    (for an insurance company)         

HR Managers (2)                                    (deal with insurance issues)

Employee Benefits Assistant (1)             (deal with insurance issues)

Management Development Spec. (1)        (for financial firm)

Office Assistant (1)                                (for insurance agency)

Patient Account Reps (1)                        (for a hospital)

VP, Research and Development (1)         (for an insurance company)

AVP, Business Process (1)                    (for an insurance company)

AVP, Procurement (1)                            (for an insurance company)

Quality Assurance Manager (1)               (for an insurance company)

   

So, eliminating these non-sales positions leaves us with 32 insurance-sales-related positions … roughly a 21% accuracy rate. 

   

Surely these 32 positions have an average rank above the list midpoint?  Just barely … the average of 75.2 is just a hair better than the overall average of 78.5. 

   

Of course, not all of the 32 insurance-sales-related ads are equally relevant.  Some are for personal financial advisors; positions that focus more on investment planning/sales than on insurance sales.  Still others are support positions for the insurance sales process.

   

But eleven ads do fall within the ‘Insurance Sales Agents’ occupation … highly relevant matches for my search.  The average rank?  77.2 … argh!   Two are in the top-ten results, but then there are no more until a single ad appears at position 72.  For reference, here are the ads along with their rank in the search results:

   

1     Mortgage Life and Disability Insurance Sales with

2     Mortgage Life and Disability Insurance Sales with

72   Agency Producer Full Time

91   Insurance Sales Agent

92   Insurance Sales Agent

93   Insurance Sales Agent

95   Sales Representative/Sales Management - Insurance

96   Sales Representative/Sales Management - Insurance

97   Sales Representative/Sales Management - Insurance

98   Sales Representative/Sales Management - Insurance

112 Sales and Management Positions / Health Insurance

   

All in all, pretty dismal.  Delivering relevant search results isn’t rocket science; but it can’t be done with standard keyword search of unstructured data.

   

It is however possible to more than double the accuracy of current search technology.  To do so requires accurate interpretation of user search intent, and then combining this knowledge with occupationally categorized data.

   

As for the problem of intent, try searching www.onetsocautocoder.com for insurance sales.  The occupations deemed the best fit:

   

41-3021.00   Insurance Sales Agents  81

41-1012.00   First-Line Supervisors/Managers of Non-Retail Sales Workers  59

13-2053.00   Insurance Underwriters  46

11-2022.00   Sales Managers  42

41-3031.02   Sales Agents, Financial Services  38

41-3031.01   Sales Agents, Securities and Commodities  34

   

If job ads are categorized using the same occupational taxonomy, then preference can be given to ads accordingly … a process that can range from the simple to the exotic, with the more advanced algorithms returning the most accurate results.    

   

So is search relevance important to jobseekers and employers?  I think so; but one thing seems clear … there is a scarcity of highly relevant search results in the labor exchange, and wherever there’s scarcity, there’s opportunity.

   

The Essence of Blogging

   

Gems are sometimes hidden in surprising places.  Jason Lee Miller, in a controversial post Do Bloggers Need to Unionize?, does the best job I’ve ever seen of capturing the essence of the blogosphere:

   

“In the past two years, blogging, as a profession, has grown from geeky obscurity into a direct challenge to the journalism industry, even with bloggers' reputation for being unruly, unvetted, grammatically and syntactically insufficient, and above all, a disorganized mess.

   

But that is sort of what (okay, completely what) made the medium so appealing. They answered to no one and therefore were accountable to no one; the individualist, populist, no-truth-barred approach both what propelled it and what held it back. Abused, sometimes inaccurate, sometimes out and out wrong, but for the most part, a development for the greater good, for freedom of speech, for information exchange, for the free market of ideas.”

   

The whole concept of bloggers unionizing leaves me scratching my head.  I know that I enjoy my independence far too much to consider joining a union; and in reality, I’m not in the target audience … I blog for fun and influence rather than profit … and I’m guessing that’s true for most bloggers.

    

You Can’t Buy My Love

   

But you can buy a detailed report describing the 2007 Top Sites for Job Search competition. 

   

Interested in the testing methodology and detailed results for each round?  Curious why your site scored below your competitors?  Looking for specific usability and performance analysis that can help guide future development?  It’s all available to you in a format that uniquely compares your site to each of the tested sites in the 2007 Top Sites for Job Search competition.

   

If you follow the Job Search Engine Guide, you know that I refuse to accept advertising of any kind.  If I ever do, then I think you should question my objectivity.  But objective tests like the 2007 Top Sites for Job Search competition require considerable time; and one way to make this a sustainable evaluation process, providing guidance year after year, is to sell the detail-level findings to job sites as competitive intelligence.

   

Price?  $4,995 for a customized report for your site; or $9,995 for the report plus an onsite visit to discuss the findings and recommendations.

   

Let’s be clear though … at the end of the day, what I really want is your commitment to site improvement … the fee is simply a crude way to make sure that we’re both focused on the same goal.

   

Do you want to improve?

   

2007 Job Site Gold Medal Winner – SimplyHired

   

Fifteen of the very best battled through ten rounds of the 2007 Top Sites for Job Search competition – SimplyHired emerges as the gold medalist. 

   

Through the very early rounds, SimplyHired and Indeed ran neck-and-neck for the lead; Indeed enjoying a slight edge.   Overall positions at the top of the leader board remained unchanged through seven rounds – several sites winning a round or two, but none stringing together enough consistently high finishes to challenge for the outright lead.   At this point it looked like Indeed would cruise to the gold. 

   

The complexion of the race changed in round eight.  SimplyHired won the search agent round, and then went on to win again in the usability round. These two wins, combined with a last place finish by Indeed in the associated services round, assured SimplyHired the gold.

   

What makes SimplyHired great?  Five million jobs – more than twice Jobster’s total, and over three times what you’ll find on CareerBuilder.  Plus, SimplyHired is head and shoulders above the competition in seamlessly delivering features and content through a customer-friendly interface that is easy to use and understand. And while it may be a small thing, SimplyHired’s permanent links for job listings bring you to a landing page with info about the advertiser, additional jobs at the same firm, and related jobs in the area. Simply brilliant!

   

Weaknesses?  Search accuracy isn’t class leading (otherwise Indeed wouldn’t have led through the early rounds); and SimplyHired could do much better in disclosing their ownership, providing a phone number for customer assistance, and providing guidance on protection from fraud and scams.  Plus, when it comes to social networking, Jobster is a better choice; and no one beats CareerBuilder in the areas of advanced filters, sorts, search options, and supporting services.    

   

Bottom line … queue Tina Turner … SimplyHired, as the gold medalist in the 2007 Top Sites for Job Search competition …you’re simply the best, better than all the rest, better than anyone … Congratulations!

   

2007 Job Site Silver Medal Winner – Jobster

   

Jobster is the silver medalist in the 2007 Top Sites for Job Search competition; an impressive showing for the most aggressive innovator in the business. 

   

What makes Jobster great?  No question, if there had been an eleventh round in this year’s competition, focusing on the social networking component of job search, Jobster would have won the round.  No job site does it better. 

   

Jobster also has more job ads (approximately 2.3 million) than bronze medalist CareerBuilder; fewer scam ads; and is fearless in reinventing the job search process.

   

Weaknesses?  Search relevance could be better … leveraging the larger job ad database would be easier if the search engine was more precise.  There are also a few usability quirks … e.g. user search settings aren’t consistently retained. And several other sites have more extensive career guidance information.

   

Bottom line – many jobseekers will find that the social focus of Jobster is enough to make this site their top choice. These jobseekers will get no argument from me. 

 

The gold medalist?  Click here.

2007 Job Site Bronze Medal Winner – CareerBuilder

   

No Monkey Business here – CareerBuilder knows job search.   As the bronze medalist in the 2007 Top Sites for Job Search competition, CareerBuilder proved through ten rounds that it is the best of the big-three job sites; but not quite the best of our top fifteen.

   

What makes CareerBuilder great?  Wins in rounds five and nine show that no one beats CareerBuilder in the areas of advanced filters, sorts, search options, and supporting services.  Add in the exposure available to jobseekers who post resumes, along with search engine technology that offers suggestions based on resume matches, and it’s clear that CareerBuilder’s traffic dominance isn’t solely a result of Super Bowl ad spending.

   

Weaknesses?  Not many.  The biggest is a relative lack of job ads … roughly 1.5 million, compared to almost 5 million on SimplyHired.  Next is a willingness to accept questionable ads (e.g. work-from-home, make-a-fortune-working-part-time, upfront-fee-required, and tag-spam ads); and finally, a surprisingly mediocre performance in the search agent round.

   

Bottom line, if you’re looking for a job site offering comprehensive services plus a heavily trafficked resume system, you can’t do better than CareerBuilder. 

 

The silver medalist?  Click here.

2007 Top Sites for Job Search – SimplyHired Wins Round 10

   

SimplyHired continues to move up the leader board in the 2007 Top Sites for Job Search competition, notching a win in the Usability round to go with a win in the Search Agent round.

   

In the previous nine rounds we looked at the bits and pieces of the whole … individual bricks if you will.  In round ten, we look at the integration of the pieces.  Assembled correctly, the whole can indeed be greater than the sum of the parts; but too often, the electronic bricks of job search are assembled poorly, resulting in a substandard user experience.   

   

Usability is the mortar that binds together the great content and features offered by today’s electronic job sites.  SimplyHired is head and shoulders above the competition in seamlessly delivering features and content through a customer-friendly interface that is easy to use and understand.

   

To be the best – the most usable job site – each competitor was judged on the following design criteria:

   

¾      Interface simplicity;

¾      Intuitive navigation;

¾      Designed for accessibility;

¾      System performance;

¾      Consistent display of user path status;

¾      Key navigation above the fold;

¾      Placement of standard elements in standard locations;

¾      Predictable function of each screen element;

¾      Avoid distracting banner ads;

¾      Employ grid discipline in layout;

¾      Employ logical content grouping and chunking;

¾      Use standard names for standard operations;

¾      Design with clear focal point on each screen;

¾      Organic search results above the fold;

¾      Retain user settings throughout session;

¾      Suggest alternate spellings;

¾      Provide alternate paths when no search results found;

¾      Buttons look like buttons;

¾      Controls linked to user intention;

¾      Avoids primary navigation using drop-down lists;

¾      Able to refine search results with filters;

¾      Option to refine search results with additional words;

¾      No blind alleys;

¾      No dead-end paths;

¾      No unexpected results; and,

¾      No unnecessary info collected.

   

   

Each of the remaining three sites had strengths and weaknesses; and the overall scores were pretty close.  JobCentral was the fastest and paid strong attention to accessibility, but the interface is cluttered with banner ads and exhibits a lack of refinement that is understandable given JobCentral’s rapid expansion to meet the requirements of a replacement for AJB.  As Chad likes to say, Rome wasn’t built in a day.  True enough; but at this level of the competition, a little construction dust was enough to drop JobCentral from further contention.   

   

Best Job Sites – Round 10 Results

   

1. SimplyHired
2. Jobster
3. CareerBuilder
4. JobCentral

   

   

So we’re left with three medalists (in alpha order): CareerBuilder, Jobster and SimplyHired.  Who wins the gold?  Stay tuned.

   

Monster Changing its Stripes?

   

Perhaps Monster is listening after all.   

   

Joel’s made no secret of his feelings regarding Monster’s interstitial ads.  And last week, in round seven of the 2007 Top Sites for Job Search competition, Monster was “…a distant last-place finisher … booted from the competition … rife with misleading ads, and worse, uses interstitial ads to pad their revenue, ignoring the risk to consumers of phishing and identity theft.  Before using Monster, I think you need to ask yourself if you can entrust your personal data to a company that is willing to place you at additional risk if the price is right.”

   

Yet today, in Monster’s 2^nd quarter financial report, you’ll find the following nugget:

   

“The total revenue outlook for the balance of 2007 assumes that the rate of revenue growth in the third quarter will continue at approximately the same rate as in the second quarter, offset by planned reductions in certain interstitial ads and the elimination of "work-at-home" job postings, with a higher revenue growth rate in the fourth quarter.”

 

I guess there’s still hope for a reformed Monster.

   

2007 Top Sites for Job Search – CareerBuilder Wins Round 9

   

CareerBuilder took its second win in round nine of the 2007 Top Sites for Job Search competition by offering the most complete set of services in support of the job search process.  Frankly, the competition wasn’t close.

   

The best jobs sites offer the following support services to jobseekers – CareerBuilder offering more than the others:

¾      Personalized pages;

¾      Resume builder;

¾      Resume distribution to other sites;

¾      Resume match to job openings;

¾      Resume review service;

¾      Video resume service;

¾      Job recommendations based on saved info;

¾      Salary information;

¾      Expert career info;

¾      User-generated career info;

¾      Career assessments;

¾      Career fairs;

¾      Labor market information;

¾      Job trends from ad postings;

¾      Social networking features;

¾      Services for special communities;

¾      Company info on ad posters; and,

¾      Employer contact info for non-advertisers.

   

   

As we near the conclusion of 2007 Top Site for Job Search competition, it becomes crystal clear that this decathlon favors versatility over specialization. 

   

Indeed, winner of rounds one and two, and a personal favorite for basic job search, trailed the remaining contenders by a significant margin in delivering support services.  It’s as if Indeed is saying ‘we’re going to do one thing, basic job search, better than anyone else; and we’re not going to compete directly with the major job sites, like CareerBuilder, who we rely on for job listings’. It seems a sound strategy, helping Indeed become the leading job ad integrator; but this specialization also costs them a shot at this year’s title for best job site.

   

On the other hand, Jobster makes it to the final four without having won a single round to this point … but also, having not been the lowest performer in any round.  This is no small accomplishment since the later rounds feature only the best sites.

   

And it seems versatility comes in multiple flavors … in the final four we have a big-three job board, an employer membership site, an integrator with resume distribution, and a social networking job site.

   

   

Best Job Sites – Round 9 Results

   

1. CareerBuilder
2. JobCentral
3. Jobster
4. SimplyHired
5. Indeed

   

   

So it’s on to the tenth and final round.  One last site will be eliminated.  Three medalists will remain.  An ultimate winner will be chosen based on performance spanning all ten rounds.  The test?  Usability.  May the best site win.

   

2007 Top Sites for Job Search – SimplyHired Wins Round 8

   

Combining the advantages of passive and active job search, email alerts allow you to save job search criteria and receive matching jobs sent directly to you on a user-specified schedule – you can sit back and let the jobs come to you.  Problem is, the execution on most sites leaves much to be desired.

   

Scouts typically search the most recent content (although this is user configurable on some sites), limiting the volume of opportunities (fine on the surface), but unfortunately small data sets expose the weaknesses of today’s search engine technology. 

   

For you see, with rare exception (e.g. Just-Posted), today’s search engines determine relevance by ranking available content rather than comparing each ad against a standardized scale of relevance.  Predictably, when data sets are small, perhaps containing no truly relevant results, most search engines will still blindly rank the results, delivering poor or even ridiculous results.

   

So unless you like perusing daily emails containing mostly irrelevant results, size matters … the job sites with the most ads can return more relevant results.  And so it was in this test, with the three vertical integrators taking the top three spots.

   

The winner, SimplyHired, separated itself from the rest by scoring highest in relevance, and allowing users to setup alerts without first having to setup accounts. And as a fringe benefit, clicking on an emailed ad brings you to a landing page on SimplyHired with info about the advertiser, additional jobs at the same firm, and related jobs in the area. Brilliant.

   

In contrast, HotJobs requires users to setup accounts; and at least in this test, 50% of the ads sent via email by HotJobs were no longer available 45 minutes after the email was received.  So HotJobs exits the competition.

   

   

Best Job Sites – Round 8 Results

   

1. SimplyHired
2. Indeed
3. Jobster
4. CareerBuilder
5. JobCentral
6. HotJobs

       

   

Next up, the ‘associated services’ round.