SourceCon

Sourcing News and Knowledge – Beyond the Obvious


Challenges, Technology & Resources

How to Build a Google Custom Search Engine


5 Comments

Screen shot 2011-06-07 at 12.05.54 AM

Custom search engines (CSE’s) may seem like a tool for advanced users — but in fact anyone can build one, even not knowing much about advanced Boolean operators. If you have never built a CSE, start with this: Create a Custom Search Engine on the fly. All you need is to input a site you’d like to search, such as your own website, or any site you like.

Let’s try zoominfo.com, for example. You can ignore the “code” section on the left – just go ahead and use your engine. Let’s try a search for “Vice president of business development.”

…This feels like magic, doesn’t it?

The Control Panel

Now let’s try to create something a bit more sophisticated. The start page allows you to specify the name of your engine, a description, and sites to search. Interestingly, you can include a template for your search using the asterisk *. (Note that the use of the asterisk in Google’s search engine is totally unrelated.)

Here’s an example: you can use the site templates *.gov and *.com – and your custom engine will search only in those domains. Or, you can use www.linkedin.com/in and www.linkedin.com/pub (sound familiar?). After you’ve added some sites, skip the “code” and the look-and-feel editing for now, and get back to the basics/control panel to add even more adjustments.

Keywords

On the basics page you can provide the “search engine keywords.” These will guide your CSE to bring pages with these words higher up in the list of results; pages without the words will be included, too.

Sites

The Control panel/Sites section lets you add and remove sites that will show up in the search results. To continue our exercise, let’s build a LinkedIn/USA profile search engine. This list of sites will give us a good start:

(Click on the image to try the engine)

By now we have learned how to include and to exclude sites, and to add keywords to help rank relevant pages higher. One last power hint for beginners is to define the “refinements.”

Refinements

A refinement makes your CSE automatically add your specified search substring to the search string that your user types.

As an example, create refinements with the strings filetype:pdf and filetype:doc, and your search engine will have options to search for files with the given type:

…These are the core capabilities of CSE’s, as I see them. Using the lists of sites and the refinements you can create truly powerful custom search engines. In fact, I’d say that with careful and creative use of these CSE building tips you should be able to build a super-CSE — and win the SourceCon Challenge!

Hints for CSE Users

To make your CSE’s easier and nicer to use, here are a couple of hints for both CSE creators and users.

  1. Add &num=100 to the search URL to see 100 results per page.
  2. Add &tbs=qdr:y to the search URL and your CSE will show a date range; as an example, try my Document Finder with the date range added.

Beyond the Basics

If you are still hungry for more technical stuff, here is a brief description of a few additional ways to enhance your CSE’s. To learn even more, read the documentation and experiment.

(By the way the mysterious “code” section that we’ve ignored for now is the HTML code for including the CSE on your own site. You don’t need it to share the engine with others — simply providing its URL will work.)

Here are some advanced hints:

  1. Use the synonyms to pick similar words as you see them fit. You can even create secret phrases or letter combinations to trigger special searches. Mike Notaro was able to force a CSE to look for the @ sign in some cases; you can’t expect to do a whole lot on this path though.
  2. Use the site templates to include sites defined by a pattern that are not possible to specify with the Google’s “site:” operator. This is a rich area to explore! I was experimenting and included some special characters in the template, forcing my CSE to (partially) search the Deep Web:
  3. Advanced users can save, edit, and upload XML files with lists of sites and the CSE parameters. You can edit these files to apply “weights” to sites to search, taking control of ranking. (You can make your site appear first in any search, for example, if you are so inclined).

Have fun building your CSE’s!

Irina Shamaeva is a Partner and the Chief Sourcer at Brain Gain Recruiting, an executive search firm that specializes in placing senior software development and management consulting candidates nationwide. In addition to full cycle recruiting, Irina does Sourcing projects across numerous industries and geographies. She shares techniques that she invents, while hands-on sourcing, in blogs and in presentations. Irina maintains an active blog, Boolean Strings, where she publishes sourcing advice, tips, and best practices, that has enjoyed 175K+ visits to-date. Prior to Brain Gain Recruiting, Irina held engineering and management leadership positions at biotech and high technology companies in the San Francisco Bay Area, including Applied Biosystems, MDL, and e-Motion. Irina holds a Master of Science Degree in Mathematics from Moscow University. To learn more about Irina, check out her LinkedIn Profile , follow her on Twitter, and circle her on Google-Plus.
  • Anonymous

    now.
    you need to make sure you code in an ability to find;
    Civil War Airplanes!

  • http://jpeek345.blogspot.com Jay Peek

    support is palid for this product and has been since 2006 http://goo.gl/jRf8T

  • Pravin

    Hi, how do we add specific locations to the CSE? I have created a CSE http://bit.ly/KEbMeV for LI Profiles across USA, still need to refine on specific locations( I would like to add CA,TX,IL,NJ,NY) and also would like to refine on specific skills(I would like to add Oracle & SAP).

    Advance thanks for educating me…
    Pravin 

  • Glenn Gutmacher

    Pravin, as you see from Irina’s screenshot for the Refinements feature above, you would need to add refinements for each metro area (or a group of continguous metros if you want a single statewide search, though for usability, you might want to do something in between like a NorCal refinement vs. SoCal one). Make sure to use LinkedIn’s nomenclature for metros: e.g., if your base search is something like (site:linkedin.com/pub OR site:linkedin.com/in) -inurl:dir then all you need for a refinement is (“albany, new york area” OR “greater new york city area”) and as for skills, that refinement is straightforward if you know what skill terms/synonyms you want to group under each refinement — again, use a parenthetical OR statement. Good luck!

  • Narayan Prusty