Search engine webmaster tools
Using Webmaster Tools is a great way to see how the search engines perceive your site. You can create a Webmaster Tools account with either Google or Bing quite easily.
Google Webmaster Tools
You can find valuable data in the "not found" report. You can access this by selecting Crawl-Crawl Errors-Not Found.
You can see a complete list of URLs that Goolebot encountered somewhere on the Web where your web server returned a 404 error instead of a web page. Sometimes this happens because another publisher attempting to link to your site does not implement the link properly.
Another tool diagnostic tactic is to look at the error URLs in your Sitemaps file. (Sitemaps=>Issues)
Google Webmaster Tools also gives you an inside look at potential problems with your meta description tags and your title tags.
Of course, one of the most important reports available from Google Webmaster Tools is of the one that depicts the links to the site.
Fetch as Google Bot. This is Google's version of header checker, and is helpful to ensure that your pages are showing the correct HTTP header code (200, 302, 301, and so on). Go to GWT==>Crawl==>Fetch as Google. Test for Web, mobile and smartphone versions of Google Bot and ensure that the result is returned as success.
This report is the only way to get a good view of what links Google see to the site. In addition, you can download the report in spreadsheet form, which makes it easy to manipulate the data.
You can see a complete list of URLs that Goolebot encountered somewhere on the Web where your web server returned a 404 error instead of a web page. Sometimes this happens because another publisher attempting to link to your site does not implement the link properly.
Another tool diagnostic tactic is to look at the error URLs in your Sitemaps file. (Sitemaps=>Issues)
Google Webmaster Tools also gives you an inside look at potential problems with your meta description tags and your title tags.
Of course, one of the most important reports available from Google Webmaster Tools is of the one that depicts the links to the site.
Fetch as Google Bot. This is Google's version of header checker, and is helpful to ensure that your pages are showing the correct HTTP header code (200, 302, 301, and so on). Go to GWT==>Crawl==>Fetch as Google. Test for Web, mobile and smartphone versions of Google Bot and ensure that the result is returned as success.
This report is the only way to get a good view of what links Google see to the site. In addition, you can download the report in spreadsheet form, which makes it easy to manipulate the data.
Bing Webmaster Tools
With Bing Webmaster Tools from the Microsoft stable, which is also a free offering, you can get a meaningful set of numbers on many of the same functions that Google Webmaster Tools offers, but the difference is that you are now getting the feedback from the Bing Crawler. The viewpoint from a different search engine is tremendously important.
You can pick from several options:
The service available with Bing does not give full backlinks to any site. The service is provided only for sites owned by the user. Reports on other sites are restricted to 1,000 links.
As a result, it opened up space for other backlink discovery offerings from different vendors.
You can pick from several options:
- The 404 report lists pages that Bing tried to access, but for which your web server returned a 404 status code.
- You can get a list of URLs blocked by robots.txt
- You can find out where you have long dynamic URLs that may be troublesome to the Bing Crawler.
- You can get a report of the pages on your site that appear to be infected with malware. Hopefully, you never need this, but if you do, it could be a lifesaver.
- You can get a review of all the URLs on your site that are of an unsupported content type.
The service available with Bing does not give full backlinks to any site. The service is provided only for sites owned by the user. Reports on other sites are restricted to 1,000 links.
As a result, it opened up space for other backlink discovery offerings from different vendors.
Yahoo! Search
Never mind that Yahoo! Site Explorer is no more functional, Yahoo's normal query system actually provides far greater functionality. This includes the ability to get:
- A list of links to individual pages or entire domains (link: http://www.yourdomain.tld)
- Lists that exclude pages from certain domains or top-level domains (-site:domainx.com or -site:.co.uk)
- Lists that include/exclude pages with certain keywords (-keyword,-intitle:keyword, keyword, intitle:keyword)
- Lists that include/exclude pages/domains with certain attributes (-inURL:keyword or inURL:keyword)
- Lists refined to certain domain extensions (site:com.tr)