I admit it, I don't use Bing often. In fact, I usually only go to Bing when someone triggers me to go there through prodding or a blog post.
After reading AJ Kohn's excellent blog post from earlier this year on Bing People search I did what anyone would naturally do and Google'd bing.com to stalk myself. I found something that I never thought I'd see; a cartoon image of myself.
This was unexpected, because in the many examples provided by AJ in his post the faces were of humans (with the exception of a lego face while searching for Hermione Granger). If Bing people search is so good, how did they show me as a cartoon?
Then it hit me:
I think that Bing People search might a part of Bing's algorithm and they are likely using facial recognition technology and a list of common names to display their people search results
Here is how I arrived at that theory.
Dissecting my Cartoonish Bing People Search result
I just overhauled my personal website, which I have had since college. I redesigned the site in March and gave it a much needed new look. On the new site I link to blog posts + projects that I have worked on over the years. There is a fair amount of text on the homepage, but mostly it is designed to link deeper into the site and other sites I have created.
In the top right hand corner of the page, you can see a picture of my ugly mug, but that picture is fed in through a CSS rule that makes it nearly impossible to grab that image file. It's literally buried so far down in the CSS that I had trouble finding out how to replace it. A search engine might have similar difficulty finding it as well.
The rest of the page features images of blog posts, projects, etc. One of those projects is for my travel blog, Jeffsetter, which is highlighted with a cartoonish version of my face.
While the image for Jeffsetter is a cartoonish representation of my likeness, it is detailed enough and the eyes/nose/mouth are in the “right” places. It could easily be viewed as a face by facial recognition software.
You may recognize the Jeffsetter image above as the image that is showing up beside my website in Bing people search.
These are the only two images of faces on the homepage of this site, and one is very small and buried within the CSS of the site, so not something that is easy to hot-link from a search engine.
Facial Recognition Technology
As we all know, Bing partners with Facebook to share data and technology between their respective products. Facebook information may be included in Bing results and Bing helps to power Facebook's graph search. The extent of the partnership is deep between the two companies, and it's likely that technology is shared between the two companies as part of that partnership.
With that relationship in place, doesn't it sound feasible that Bing may be licensing Facebook's facial recognition technology as part of this agreement? For those of you who may not be familiar, Facebook purchased facial recognition technology company Face.com last year to integrate into their core product.
With a partnership in place and facial recognition at their disposal, it seems feasible to me that Bing is crawling websites and looking for faces. It only makes sense that this is something they would be doing as part of their regular scans.
Putting this all together, I can only come to the following conclusion:
I believe that Bing is using facial recognition technology to identify the faces displayed in Bing People Search results
People Search Results – What is in a Name?
Now that I am satisfied with why my cartoonish face showed up in my search result, I want to examine how Bing knows me as a person in the first place. Name recognition technology is very prevalent within social networks, to the point where many of them won't let you sign up without having a real human name. Facebook and Google+ both have a real name policy that is tightly enforced through technology. IBM has a Global Recognition technology that can be purchased by businesses as well.
Bing likely can access Facebook's technology to determine if a search result is for a person. Even if they don't use that technology, they could simply verify that a first name shows up in a baby name directory and that the last name is on ancestry.com. Simply put, person recognition should not be a problem for Bing at this stage in the game.
I believe that Bing is able to quickly and (mostly) accurately identify when a search is for a persons name using a common name intelligence algorithm
How I Think Bing People Search Works: People Recognition + Face Recognition = Bing People Search Algorithm
Thinking logically, here is how I envision Bing People Search working in their algorithm (the order is likely not the exact order of operations).
- Crawl all websites and make index of the content topic, images, assets, etc. (i.e. basic search crawling)
- If a crawled image appears to look like a face, make note of that image and set flag to face
- For a particular page, assign the most likely image for the face that is showing up on the page
- When a query is entered, display search results
- If that search query is easily associated with a real human name, display the most likely image for that person beside the search results snippet
My Personal Result Starts to Make Sense to Me Now!
If it is truly a combination of face recognition + people recognition, I fully understand why my personal results behave the way they do.
Putting the Theory to the Test
Let's try a few examples of this theory to make sure that we are right.
For my personal SERP, the #2 result is a Wikipedia page for Jeff Sauer (not me, but rather a famous college hockey coach)
For this result there is no picture at all. If my theory is correct, there is likely to mean that on the Wikipedia page there is no face being shown. Sure enough, there are no images on the Wikipedia page.
Next is my LinkedIn page, which shows my LinkedIn profile picture
After that, another Jeff Sauer on LinkedIn
His image is pulled from his LinkedIn profile (only static image on the public page)
Jeffrey Sauer on Facebook (not me):
And this Facebook page (again, only image on the page).
Then after that, Bing doesn't decide to show any more images, which likely indicates some sort of governance over how many people results they will display on one page.
Is a Cartoon Really a Face?
It appears that way. The current #10 result on Bing for “Darth Vader” is a USA Today article with a cartoon drawing of Darth Vader and some children…
There is only one image in the body copy of the article, but is it a recognizable face? You be the judge:
Untested Theories Still Remaining
There are several more assumptions I have about this page, but can't prove:
- There is likely a de-duplication filter on images so they don't have the same picture more than once
- People results are likely an enhancement to results and not a ranking factor
- There is likely a limit to how many images will show up in an SERP, but I don't know that limit
- Sometimes Bing will just grab a picture of a face, any face, and display it on a snippet (I had an example of this, but it has since gone away)
Has anyone else tested these theories and would like me to share their post? I would be interested in hearing from you in the comments!
Concluding
This has been a fun theory to test that I likely would have glossed over if my own personal results weren't so cartoonish. I love the way Bing is enhancing their search engine in this way and it's a great way to take advantage of their partnership with Facebook.
It will be interesting to see where else Bing improves search results and I vow to start using it more often.