{"id":3845,"date":"2008-07-18T02:44:50","date_gmt":"2008-07-18T02:44:50","guid":{"rendered":"http:\/\/www.travelindex.com\/blog-posts\/?p=14"},"modified":"2022-05-05T10:49:46","modified_gmt":"2022-05-05T10:49:46","slug":"1-trillion-unique-web-pages-and-you-are-not-listed","status":"publish","type":"post","link":"https:\/\/www.travelindex.org\/media\/travelindex-blog\/1-trillion-unique-web-pages-and-you-are-not-listed\/","title":{"rendered":"1 trillion unique web pages and you are NOT listed!"},"content":{"rendered":"
\n

So Google has identified 1 trillion unique URL’s on the web but the estimate on the true size of the Google index is “only” 40 billion pages or so! Is your website in the 960 billion sites left out in the cold?<\/p>\n

Because of the sheer size of these indexes we, at travelindex.com<\/a> (www.travelindex.com<\/a>) believe that we need human edited directories to make relevant websites more visible on the net.<\/p>\n

Original post from TechCrunch, 25 Jul 2008<\/em>
\nIn a blog post today Google says they\u2019ve identified 1 trillion unique URLs on the web. It\u2019s actually more, they say, but some web pages have multiple URLs with exactly the same content or URLs that are auto-generated copies of each other.<\/p>\n

What they note way down in the fourth paragraph, however, is that they don\u2019t actually index all of those pages, so you can\u2019t find them on Google. Estimates on the true size of the Google index are a mere 40 billion pages or so.<\/p>\n

Why don\u2019t they index all the pages they\u2019ve found? Some of them are spam. But it\u2019s also very expensive to index sites. And the fact that Google indexes many news sites, blogs and other rapidly changing web sites every 15 minutes makes all that indexing even more expensive. So they make value judgment on what to actually index and what not to. And most of the web is left out.<\/p>\n

Google also says \u201cBut we\u2019re proud to have the most comprehensive index of any search engine.\u201d<\/p>\n<\/div>\n","protected":false},"excerpt":{"rendered":"

So Google has identified 1 trillion unique URL’s on the web but the estimate on the true size of the Google index is “only” 40 billion pages or so! Is your website in the 960 billion sites left out in the cold? Because of the sheer size of these indexes we, at travelindex.com<\/a> (www.travelindex.com<\/a>) believe …<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1859],"tags":[],"class_list":["post-3845","post","type-post","status-publish","format-standard","hentry","category-travelindex-blog"],"_links":{"self":[{"href":"https:\/\/www.travelindex.org\/media\/wp-json\/wp\/v2\/posts\/3845","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.travelindex.org\/media\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.travelindex.org\/media\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.travelindex.org\/media\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.travelindex.org\/media\/wp-json\/wp\/v2\/comments?post=3845"}],"version-history":[{"count":0,"href":"https:\/\/www.travelindex.org\/media\/wp-json\/wp\/v2\/posts\/3845\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.travelindex.org\/media\/wp-json\/wp\/v2\/media?parent=3845"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.travelindex.org\/media\/wp-json\/wp\/v2\/categories?post=3845"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.travelindex.org\/media\/wp-json\/wp\/v2\/tags?post=3845"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}