Several weeks ago, during the 2007 Top Sites for Job Search competition, an executive with one of the medalists sent me a message:
“Timing sucks. We are launching major search relevance enhancements on Thursday this week.”
Delaying the testing until this competitor could release their ‘improved search’ wouldn’t have been fair to the other contestants. Instead, I committed to retesting the ‘improved search’ at the conclusion of the competition and posting an update if it appeared the results would have changed.
As it turned out, the ‘major search relevance enhancements’ were more akin to The Emperor’s New Clothes.
Consider the following example – a search for:
¾ ‘insurance sales’ jobs;
¾ within 'x' miles of city ‘y’;
¾ posted within 7 days; and,
¾ sorted by relevance.
The 'new and improved search' returned 156 ‘matching’ jobs. Sounds pretty good until reality intrudes.
Only 31% of the ads (49 of 156) focus on insurance in some way … not very good. The search engine doesn’t seem to discern a difference between selling insurance and receiving insurance benefits.
Worse, most of these 49 are not at the top of the list. They’re spread throughout the list of 156 results; and the average rank for the 49 insurance related ads is 78.1 … right on the average rank for the full 156.
Furthermore, the 49 insurance-related ads include positions that are unrelated to ‘insurance sales’, including:
Compensation Managers (3) (deal with comp. and ins. issues)
Attorneys (2) (dealing with insurance issues)
Customer Service Reps (2) (for an insurance company)
HR Managers (2) (deal with insurance issues)
Employee Benefits Assistant (1) (deal with insurance issues)
Management Development Spec. (1) (for financial firm)
Office Assistant (1) (for insurance agency)
Patient Account Reps (1) (for a hospital)
VP, Research and Development (1) (for an insurance company)
AVP, Business Process (1) (for an insurance company)
AVP, Procurement (1) (for an insurance company)
Quality Assurance Manager (1) (for an insurance company)
So, eliminating these non-sales positions leaves us with 32 insurance-sales-related positions … roughly a 21% accuracy rate.
Surely these 32 positions have an average rank above the list midpoint? Just barely … the average of 75.2 is just a hair better than the overall average of 78.5.
Of course, not all of the 32 insurance-sales-related ads are equally relevant. Some are for personal financial advisors; positions that focus more on investment planning/sales than on insurance sales. Still others are support positions for the insurance sales process.
But eleven ads do fall within the ‘Insurance Sales Agents’ occupation … highly relevant matches for my search. The average rank? 77.2 … argh! Two are in the top-ten results, but then there are no more until a single ad appears at position 72. For reference, here are the ads along with their rank in the search results:
1 Mortgage Life and Disability Insurance Sales with
2 Mortgage Life and Disability Insurance Sales with
72 Agency Producer Full Time
91 Insurance Sales Agent
92 Insurance Sales Agent
93 Insurance Sales Agent
95 Sales Representative/Sales Management - Insurance
96 Sales Representative/Sales Management - Insurance
97 Sales Representative/Sales Management - Insurance
98 Sales Representative/Sales Management - Insurance
112 Sales and Management Positions / Health Insurance
All in all, pretty dismal. Delivering relevant search results isn’t rocket science; but it can’t be done with standard keyword search of unstructured data.
It is however possible to more than double the accuracy of current search technology. To do so requires accurate interpretation of user search intent, and then combining this knowledge with occupationally categorized data.
As for the problem of intent, try searching www.onetsocautocoder.com for insurance sales. The occupations deemed the best fit:
41-3021.00 Insurance Sales Agents 81
41-1012.00 First-Line Supervisors/Managers of Non-Retail Sales Workers 59
13-2053.00 Insurance Underwriters 46
11-2022.00 Sales Managers 42
41-3031.02 Sales Agents, Financial Services 38
41-3031.01 Sales Agents, Securities and Commodities 34
If job ads are categorized using the same occupational taxonomy, then preference can be given to ads accordingly … a process that can range from the simple to the exotic, with the more advanced algorithms returning the most accurate results.
So is search relevance important to jobseekers and employers? I think so; but one thing seems clear … there is a scarcity of highly relevant search results in the labor exchange, and wherever there’s scarcity, there’s opportunity.
I agree.... Many of the Job Boards try to be everything to everyone and lose the specific niche of a particular job.
Posted by: Sales Jobs | February 20, 2008 at 07:21 PM
I agree. I think more niche based search sites would be great. A lot easier for the job seeker.
Posted by: Job Search Engines | May 12, 2008 at 04:14 AM
As a recruiter this brings in job candidates that would never have considered a certain field. Many times these people would have never considered anything outside of their own experience but often these people who are new to the insurance business prove to be better suited for it than those with a lifetime of experience because they bring with them new ideas and a fresh outlook on the business.
Posted by: insurance agent leads | April 18, 2009 at 02:59 AM
So why have you stopped writing your Blog?? You had some great content on here...
Posted by: Mario Gedicke | June 17, 2009 at 06:36 PM
Thanks Mario. :)
Producing quality content was more time consuming than I expected. Somewhere along the way, blogging shifted from something I did for 'enjoyment' to something that felt like 'work'.
Posted by: Bob Wilson | June 17, 2009 at 07:16 PM