So you don’t have a LinkedIn Recruiter license? Don’t worry, neither do I! I typically do the bulk of my sourcing through search engines like Bing, Google, DuckDuckGo, and Yahoo. Each search engine has its own merits, as well as their own downfalls, but I recently started focusing most of my searches on Bing.
If you read my previous article, you know that I have been seeking a new search engine that will be the light of my life. I wanted to try out a few things with Bing to see if it was truly meant to be, and a large part of what I want is structured and accurate search results, both of which Bing provides.
I use a handful of tools and techniques to achieve my goals as a sourcer, so I wanted to share some insight into how you can use a combination of Bing X-ray search and Data Miner, one of the best free scraping extensions, effectively and efficiently to source candidates from LinkedIn FOR FREE. These same principles could also be applied to any other websites that contain structured information.
Choose your Tools
Data Miner is one of the most natural scraping tools to learn, and one that I would highly suggest for those interested in dabbling with scraping. Several sourcers are using it with plenty of documentation on best practices, so I won’t dive too far into the technical aspects but will instead demonstrate how to create an efficient scraping model to pull from LinkedIn x-ray searches on Bing. I’ve even shared a public recipe that you can use to scrape your LinkedIn x-ray searches yourself, look for the moniker “Stalkmaster Vroman” on the recipe name under public recipes while on the Bing search page.
It should be noted that for this walkthrough I will use the point and click method to identify the elements to be scraped, but Data Miner does offer support to scrape via Xpath. Since that is a more technical process, I won’t go over that here, but it is possible.
Step 1 – Conduct your X-ray search on Bing
Use “location (target city)” in your Boolean string – you can find the correct syntax by playing around with different variations. For instance, to search for Chicago or NYC, you would use “location Greater Chicago” and “location Greater New York City”. For Orlando, you would use “location Orlando”. From what I can tell, this field is pulled from the location field just under the headline of the LinkedIn profile.
Use “intitle:” to target the title contents – I use this to target the job title and company that are included in the title section when X-ray searching LinkedIn. I find it a more efficient way to source based on job titles, though you can use your own preferred Boolean to build your search.
- intitle:financial intitle:analyst -intitle:senior -intitle:manager -intitle:lead -intitle:lockheed
Step 2 – Open the Data Miner extension
Click on the Data Miner icon to open the extension. I’ve noticed that there is a little lag between clicking on the extension and it opening when using Bing, so you may have to reload (F5) your page for it to open correctly.
Step 3 – Select “New Recipe”
Step 4 – Select the Row container
This will be the “box” that contains all the information you want to scrape by row. Click on the “Find” button and hold the Shift key while hovering over the page to highlight the element, and then click on the to select it while holding shift.
Step 5 – Select your columns
Following the same directions as we did for the Row container, click “Find” and hover over each element and then click to select it while holding shift.
- Profile URL
- URL Title (For LinkedIn X-ray searches, the URL title contains the name, job title, and company)
- Title (different than the URL Title, it usually includes only the job title though it could be shortened with the end cut off due to character limits)
- Location
- Industry
- Connections (while mostly irrelevant, this field is necessary to ensure you pick up the complete table of all results, as some will have the location, industry, or title fields in this place)
Step 6 – Set up Pagination
Navigate to the Nav tab of Data Miner and select the pagination button at the bottom of the search results.
Step 7 – Run the scrape and the pagination feature
- Run the scrape from the Save tab in Data Miner.
- After running the scrape, in the window that opens automatically, click on the “Next Page” button.
- Enter the number of pages you would like it to navigate and scrape and the wait time to load each page.
- Hit Start Scraping.
Step 8 – Export the data in the format that you prefer
Once Data Miner has finished scraping, export your data into your preferred format.
Using Raw Data? NO WAY!
Now that you have your raw data exported into an Excel file, you can either keep it as is and work from the file, or you can clean the data and prepare it to be imported into your CRM, email marketing system, or any other system you may use in your workflow. In my next article, I will cover some essential Excel formulas to use to quickly clean and organize your data to prepare it for import into your systems.