URLs in Web Index

I'm going to partially re-post a blog I wrote just a few weeks ago regarding web-pages included in Google's Index. I've only added a dozen pages in the last two months so the increase in the number of pages indexed has to be due the SEO changes being made to pre-existing pages already on the web. The Index History relates to an Engineering Portal on the web.

Index History:
6/30/10 = 1,504 indexed pages.
6/24/10 = 1,455 indexed pages
6/18/10 = 1,426 indexed pages
6/9/10   = 1,411 indexed pages.
5/31/10 = 1,400 indexed pages.
5/22/10 = 1,394 indexed pages.
4/7/10   = 1,309 indexed pages.
3/27/10 = 1,322 indexed pages. [Site-map loaded]
12/19/09 = 1,481 indexed pages. [Site-map loaded]
12/13/08 = 1,318 indexed pages. [Site-map loaded]


 That's not to bad, 12 new pages added, but 100 more pages indexed by Google.

Does the increase in indexed pages imply that all of a sudden the site will start receiving a great many more site visitors, not really. But it does mean that the newly indexed pages will show up in a Google search for that topic, maybe not first in the list but at least they will show up now.

 No Page Rank; Company index 'M'.

I think I'll run a new site map later tonight, once the traffic slows down. I'll get those dozen or so new pages included, and remove a few FAQ pages that have been de-linked over the last few weeks. I did have a few pages that are not being used but still had links pointing to them, but Google dropped them from the index long ago ~ time to removed the link and orphan the pages.
I'll add a comment to this post once Google has time to read the new sitemap.

Graphic; Number of visits to the web site 1/1/10 to 6/29/10 [mid-year update].
Year-to-date = 17.48% increase in visits over the same time period last year [192,858 more visits].

Web Page Hits per Hour

Here's one more graphic from the FAQ section of the web site. This one shows Page Visits by hour of the day.

 

Visits by Hour, May 2010

.

Comparing Site Vists

I figured I would take another look at how the site is doing compared to last year. As of last week the site is still getting 10% more site visits than the same week last year.

Over all the web site is still getting more than 17% more web visits year-to-date, down 1% from 3 weeks ago. The first week of June had the lowest increase with only 6% more visits to the site. The largest increase in visits occurred on the last week in Jan with a 26% increase.

Over all the site is showing a 15% increase over 2008 visits, and about a 14% increase over 2007.

The graph compares web visits for 2010 and 2009 ~ by week.

Page-views [not shown] are down 4%, year to date over last year.

FAQ-Incoming-Linked-Sites

I'm up-loading a graphic from the site to reduce both the size of the graphic that gets loaded and a separate DNS look-up that is required to load the graphic. The graphic is of Linked-sites, or sites linking to interfacebus.com from external sites. The graphic was stored out on Google Picasa, and not on the server, which required the extra DNS look-up.

So the page that held the graphic was deleted and a link pointing to this blog posting added to the pages that pointed to the old page. The point was to reduce that page from maybe 12k of html text to zero, delete the external look-up for the 100k graphic and reduce the over-all load time for the site.

Last month I did the same thing to 2 or 3 other FAQ pages to reduce the over-all loading time of the site. Remember Google rank page loading time as well as just key words.

Pages Crawled Per Day

Crawl Stats;

Looks like Google is finally slowing down checking my Engineering Site. After 2 months of heavy Googlebot activity the spider seems to be taking a break, which is good. There's really no reason Google should be reading 2,000 pages a-day. The numbers shown in the graph are High number of pages, Average number and low number of pages spidered each day.

HTML Suggestions;

I finally got down to the point that Google does not have an suggestions of fixing any meta-tags. It seems like there were a few duplicate pages that were unused that I never fixed ~ so they may have been de-listed.Here is what Google has to say;
"We didn't detect any content issues with your site. As we crawl your site, we check it to detect any potential issues with content on your pages, including duplicate, missing, or problematic title tags or meta descriptions. These issues won't prevent your site from appearing in Google search results, but paying attention to them can provide Google with more information and even help drive traffic to your site. For example, title and meta description text can appear in search results, and useful, descriptive text is more likely to be clicked on by users."

Sitemaps;

1,653 URLs submitted, 1,426 URLs Indexed.

Speed Performance Overview

Just a quick up-date to show the latest graph on site download time, or site performance. Google now indicates an average download time of 2.8 seconds, which is faster than 51% of internet sites. This is down from 3.7 seconds in the last posting. The previous posting was Web Site Performance, 4/11/10.

Website Health Check

Because of the drop in web visits this month, which is common for June, I figured I would compare the number of visits this year to last.
 interfacebus.com Stats:

So there's an 11.78% increase in sites visits between June of 2009 and 2010.
June 2009; 63,814 visits
June 2010; 71,334 visits

The year to date difference is even better, with an 18.15% increase in visits
1/1 - 6/10 2009 = 994,854 visits
1/1 - 6/10 2010 = 1,175,422 visits

interfacebus [Google Sites] Stats:
Year to date for my Google Sites pages have a 104.56% increase in visits
1/1 - 6/10 2009 = 3,223 visits
1/1 - 6/10 2010 = 6,593 visits

Serialphy.com Stats:
Year to date for my serialphy pages have a 33.93% increase in visits
1/1 - 6/10 2009 = 1,441 visits
1/1 - 6/10 2010 = 1,930 visits

Knol pages [Google Knol] Stats:
Year to date for my Knol Site pages have a 42.9% increase in visits
1/1 - 6/10 2009 = 3,481 visits
1/1 - 6/10 2010 = 4,974 visits

Blogs [This blog and the other] Stats:
Year to date for my Blog pages have a 104.7% increase in visits
1/1 - 6/10 2009 = 3,679 visits
1/1 - 6/10 2010 = 7,532 visits

I guess I have to conclude the health of the sites is good!

Why You Should Blog

At first I figured I was going to just insert a reminder post about some of the reasons a web master should also run a blog. But, as I was reviewing posts looking for the last relevant posting I found some disturbing data as it relates to the rest of this blog post. I'll address that data at the end of the post, with a comment plus a few related early blog posts.

The numbers below represent visits to the Engineering Site from this blog over the last thirty days.
The date represent when the blog posts were written with the number of visits from those posts.
The number of visits from blogger represent 84 visits from 15 different pages off blogger [although they are just shown by year of post].
The point of this post is to show that even after years of writing a blog posting, it still gets visits and provides referring hits to interfacebus.com. [Why Blog; Sep 12 2007]

Referral visits from Blogger [blogspot]
by page generation date:

42 referrals from 2010 posts
0 referrals from 2009 posts
3 referrals from 2008 posts
15 referrals from 2007 posts
10 referrals from 2006 posts
14 referrals from 2005 posts

The other blog site which only shows new pages included on the engineering site brings in even more traffic.
But in the last thirty days that blog sent over 82 referring visits from 26 different blog posts.
I did notice a spike in visits yesterday which has me wondering why.

I should note that this blog had 771 page views over the last thirty days, while the 'what's new blog' only received 354 page views.
That's page views in these blogs not sending traffic to the main site which is discussed about in this blog posting.
However you can read these blogs from a newsfeed, but these numbers don't reflect that.

The graphic shows incoming visits to the Engineering Site from these blogs over the last four years. Looks like there's been a steady decline in visits from mid last year. The middle of last year I stared a newsfeed, then near the end of last year I removed some permanent links from this blog that pointed to the web site. Now the newsfeed carries all these same posts, but if some one clicks over to the web site it doesn't appear to come from this blog.

Google Caffeine and Indexing

So Google came out with a new indexing system which updates the Google index much faster than in previous years. The new indexing system is called Caffeine [Google Caffeine], and allows for a change in its index each night.

It was common knowledge that once every thirty days Goggle would shuffle it's indexed pages, so if you were number 2 in the search engine position [index] one month that same page would change position the next month. Some times the position would get a higher position, and sometime the index change would leave the page in a lower position.

However at the same time many of the new pages I generated would show up in the index the next day because I would blog about the new page address in the New Engineering Pages Blog. Google operates these blogs [blogspot] so they read or spider them all the time. Any new page mentioned in blogger gets noticed by Google much faster before it reads any external web site [I assume]. Before I started blogging I would wait 2 to 4 weeks before a page would get indexed, but after I started blogging any new page would get picked up within several hours.

So for Search Engine Optimization [SEO], blog about any new page you generate because it gets picked up much faster that waiting for the spider [Googlebot] to find it, and much faster than having to generate a new site map for each new page addition.

So what is Caffeine? Well it's Google changing your search engine position ever day instead of every month. However; because your page position might change ever day as it was displaced by one of my new pages, I'm not really sure I see the difference. However Google does indicate a recent drop in the number of my page that are being displayed in the search engine listing, or really the number of pages that are being clicked on.

So this graph depicts the the number of times my page is shown in the search engine results [blue] and the number of times somebody clicks on one of my pages [yellow] in the search results. A drop in click-through rate [yellow] in the last few days should indicate that although my pages are still showing up in the search results, they now appear lower down in the results ~ maybe page 2 of the results instead of page 1.
But at the same time Google Analytics reports that there has been no drop in visits, and I trust the Analytics report more than I trust the report from Google webmaster tools [graphic above].

If I run the above report for the term Can Bus I see no real reduction in the click through-rate.
While the graph shows all search queries [all search terms] that relate to my website.

Indexed URLs by Google

I'm always watching how many of my pages are indexed in Google's search engine, and a few days ago it reached 1,411 pages indexed. Here's a few data points over the last few months [years];

6/9/10   = 1,411 indexed pages.
5/31/10 = 1,400 indexed pages.
5/22/10 = 1,394 indexed pages.
4/7/10   = 1,309 indexed pages.
3/27/10 = 1,322 indexed pages. [Site-map loaded]
12/19/09 = 1,481 indexed pages. [Site-map loaded]
12/13/08 = 1,318 indexed pages. [Site-map loaded]

The number of pages most go up just after I add a sitemap, and then drop down a few weeks later after Google finishes reading them. However as I 'fix' the pages they slowly start to re-appear as indexed, but at a slower pace. Fixing a page is adding more data so it no longer appears as the page it was copied from.
No page rank; Component Manufacturers. Just one example of a page not indexed.....

I'm not going to show the graph but Google indicates that on average Google will spider 862 pages per day ~ every day. What is up with that, the site only has 1,600 pages? Google used 537MB of server bandwidth as the spider Googlebot down-loaded from the website.

Site views and page ranking

So the numbers are in for last month. Site visits are down over the last month, but over-all page views have increased a bit. Now if the numbers follow the same yearly trend there should be an even bigger reduction in site visits next month. I think I'll wait until next month before I re-post the page visits graph, there's really no need to post the same graph every month.

After I noticed yesterday that a few more of my pages have been included into the Google Index, I figured I would try and find a few pages that are yet to be added. It didn't take to long to find a year old page with out a page rank. The links are Component page followed by the mention of the component in the 'whats new blog'.
How to Derate 2N4239 Transistor. [posted on 6/4/2009 When to Derate a Transistor]
How to Derate a 2N3743. [posted on 1/30/2009 Derate a BJT by Temperature]
How to Derate a 2N3485 Transistor. [posted on 2/12/2009 Recommendation on how to Derate a Transistor]
How to Derate a 2N4931 Transistor. [never posted in the blog].

I found these pages by looking at the report from Google Analytics, and clicking on a few pages that were getting almost no page views. None of these four pages have a Google page rank, and have only received a few page views this year. Of course re-posting their links here may not help these pages at all, but it may help them get re-spidered ~ as they were just up-dated.