Typically, since links rotate, or are displayed in random order, it is very hard to detect if any links are broken. It would take a lot of work on the part of the webmaster to check all the links.

Web Ring History

The web ring was born in 1994/5, created by Sage Weil, and is said to have died in 2001 under Yahoo’s reign. Ringmasters, the people who maintain and organize web rings, became more and more relegated to the background as “Web 2.0” emerged. Unfortunately, web rings were being associated with loud banner ads, annoying graphics, blinking text and horrible font colours. Read the whole story here.

Popular Web Rings

The most popular web rings are grouped by topic so you can “surf” after starting from one of these sites:

Search your favourite topic and explore web rings starting from these two sites:

• http://www.bravenet.com/search/sitering.php
• http://www.ringsworld.com

Hebrew web ring site from Israel:

• http://www.webring.co.il

More history and interesting facts about web rings:

• http://www.webringworld.org

Web Rings of the Future

It seems that web rings have an uncertain future. After 2001, there was a huge boom (after the bust) in the commercialization of the web. As corporate web presence grew, the homespun web got less and less popular. Web rings faded unless you were a Ringmaster with a strong community following. Corporations are a little less likely to bring a collaborative spirit to reignite the web ring concept. Lately – it’s all about keeping eyeballs on your site and converting them to sales, not surfing on a Sunday afternoon.

Link Exchange Scripts

Separated at birth, link exchange scripts are related to web rings. Link exchange programs have became very popular since the rise of web scripting languages. Many of link exchange scripts became fully automated which was a great time saver and greeted with enthusiasm. However, these automatic scripts cause many issues as well because webmasters forget to look after them.

Despite the fact that some link exchange scripts have the ability to “check the links”, these scripts are often abused by spammers because the webmasters are not as worried about them. Bad links get submitted daily as a result, ruining the user experience.

One of the first link exchange companies, LinkExchange, brought a little bit of “real-world” advertising to the warm, fuzzy world of Internet information sharing. LinkExchange was touted as a site that had an incredible reach to consumers looking for things and resources in a specific niche. Advertising on the site brought legitimacy as an ecommerce website. It also brought more eyeballs to whatever you were promoting – very powerful audience numbers before social networking came along.

Social Networking, Bookmarking Gadgets, Widgets and Scripts – oh my!

These methods have become the easiest way to share links when compared to other mentioned in this article. However, so may webmasters miss this aspect (isn’t StumbleUpon.com just a random web ring after all?) Despite that, URLs may look real in the browser but many of them are generated using JavaScript; this makes some links invisible or some of them broken to web robots like GoogleBot. The software mentioned above should not be avoided, but we’d like to emphasize that scripts and software usually cause linking troubles.

What should you do to be safe? “Nofollow” everything you find suspicious on your site. If social bookmarking has a specific URL, you can hide that from robots using robots.txt. Also, always check your site using LinkAider, our reporting module will provide the reports you need to fix broken or bad links.

The most popular server errors we’ve met while checking the web for broken links are:

• Request/server exceeded time limit. Surprisingly this is a very common error. Perhaps too many webmsters forgot to optimize their scripts or databases? Seeking into archive too deep? We can’t guess the real problem, but the lesson is check your old pages/scritps if they are still working, can handle the current load, and if your website’s archives are available.
• Can’t connect to SQL – is a very common problem. And this type of errors are fixed very fast, because they are easy to spot since they are affecting almost the whole site.
• Error in SQL query – is a less noticeable message than SQL server errors, but it is easy to fix as well.
• CMS error pages. These are the most informative ones. Developers usually forget to turn off debug messages. These are very dangerous, because they provide so much information for possible hackers.
• Down for scheduled maintenance. Such common and simple pages like this usually show random error numbers like 404, 500 and so on.

Worst solutions to the 5xx error pages

• Trying to show site’s main page. Actually sites like this try to show their main page everywhere: it’s for a 404 page, for any 5xx page, etc. Most of the time this appears as a big mess to a site’s visitor and does not provide any valuable information about what he can do next (if he wants to stay on the same page).
• JavaScript popups with debug messages. Common?!
• Showing return to the original page when the “original page” link points to the same page user is at.

5xx error screenshots we made for you

Clean and simple error page

Internal server error (problem description in Japanese)

Wordpress maintenance mode plugin in action. Website is down for maintenance

AT&T suggests users to return to the original page, but this time button leads visitors to the same page

Talking to aliens? Are BOF or EOF the names of your mothership? And what should your website’s user do now?

Nice looking server error page

Something blew up on digg’s server. One of the shortest error pages we’ve discovered.

Django gives a very detailed (and nice looking) report of what’s happened, but on production sites do not forget to turn that thing off in order not to tease hackers.

Drupal – site is down for maintenance.

Geocities is down. Nice and useful error page from Yahoo. But the title is a bit misleading. Isn’t it?

Server error. Because some of the text is bold red – we guess that something serious has happened.

Sweet and clean. In Russian.

Maxim tries to be funny.

An error happened in mod_python. The big snake is not happy.

MSDN tries to be helpful.

Even social networks go down.

Oh, database is not accessible.

Parallels – internal server error.

jQuery – no such file in repository (why not 404 then?)

Out of operation.

Being helpful and informative.

Ruby fails

Very nice design (actually we see this page for all errors on that site and we love how it looks)

Site offline. Gray color is back into fashion this year.

Oh, so we are the robot? And you are a brick without soul goddammit Gooogle!!!

Look! Robot’s lost its head!

Tomcat broke.

Site is under construction.

Lions!

Something wrong at Yahoo!

XSL file not found.

Some companies close. Here are four major examples of companies who have closed or probably shutting down their sites in the near future:

Vidoop is closing. They were an OpenID start-up that has run out of money. The story of Vidoop can be followed here. When an insider provides so many details about the company, people at the top, and how they were treated, it can start to set off alarm bells.

Trusera closed in May 2009, not quite surviving to its first birthday (June 2008). The funding just wasn’t there and the “economy” took this health and wellness sharing site down. There are a lot of social networks out there and journal-type sites. Keep in mind that there are only so many “niche” markets that can keep the momentum.

Portfolio is also closing. Information about the circumstances can be found here. There are approximately 5 million links to that website and an update states that it is soon to be “reborn” – most likely with new paths/links along with it. Despite the fact that the site is live for now, these links could become dead anytime now (hopefully not).

Geocities has closed. This shut down that was planned for quite some time. We are not sure if the content will stay there, but Geocities was the home of many personal websites (long before blogs exploded in popularity). Many people were self-taught and learned the basics of coding before their careers began using Geocities. There was a lot of Twitter traffic as well with many cyber-enthusiasts tweeting nostalgic RIP comments. More information about the site’s closure can be found here. The popular web comic, XKCD also had a tribute to Geocities’ design.

Some Content Management System plug-ins get discontinued.

WordPress is the most popular CMS software for blogging. It’s used extensively for personal and small business use. One of the reasons it is so popular is because it has thousands of plug-ins to bring specific functionality to your site or blog. For example, the Affiliate Store Plug-in provides a way to bring a large number of products to your site from several affiliate partners. This is powerful because you reap the rewards of being an affiliate and having links to your blog for popular products. However, this plug-in will be discontinued! You must handle the code yourself because the creator has decided to focus on other things. Sorry – just deal with it. The lesson? Before you use a plug-in, check out its history and do some research about the creator and its longevity. A little bit of research up front may save you a lot of fixes in the future.

Simple CMS by Pixopoint is another plug-in that has been discontinued. Paid custom development is now required to maintain this for your site. Unfortunately, many more get discontinued every day. For example, westonruter has announced that he is no longer working on these plug-ins.

Even though many of these plug-ins are working now, they could break on the next CMS update. Unfortunately, you may have to pay for a high-priced WordPress developer if this is the case. So be careful and do some research if you can – and use LinkAider’s automatic scheduler for checking for broken links. You never know how many headaches it will save you in the future.

Analyze your websites’ topology

Check who you really are linking to (we’ve written a great article on website topology analysis before). You should know that website’s structure matters in many ways: first, you want your visitor to be able to find and visit the pages he is looking for and second, you want search engines to know what the most important pages on your site are.

Use powerful and fast filters to condense and analyze web crawl results

Data filters gather a wide range of things: document types, source/destination paths, followed/nofollowed links, link types and more. Take a quick look at the screenshot below.

Filters can be saved for later use. And you can also share the filtered report with your colleagues or clients with a single click.

And by the way: size does not matter. You can crawl any size website with LinkAider and analyze a site of thousands of pages in a matter of seconds (this is a big advantage when compared to desktop tools. If you don’t believe us try sorting ten thousand lines in Excel or your favorite desktop link analysis software).

Check if you have dead ends on your site’s navigation path

You can see if you have pages with 0 outgoing links right in the site crawl summary report. Funny thing is that pages like this appear on so many sites. Pages with zero links give your users no opportunities. Thank god if your user came to such a page from somewhere on your site. Hitting “back” will bring him to the page he visited before. But if such user came from the search results – “back” button will lead him away from your site (and sometimes forever).

Your pages’ load time is important!

In a recent interview Google’s Engineer Matt Cutts stated that page load time will be a huge factor in seo starting in 2010. And according to eMarketer research 16% of people leave the page if it loads longer than 10 seconds and only every second person will wait more than 15 seconds. Also pages bigger than 150 kb sometimes (!) are not fully cached by search engines.

The good thing is that you can see slowest pages report in LinkAider. There is no need to check your site page by page with other tools – now you can get slow page report for the whole site.

Make your competitors link to you

Crawling your competitors site can reveal many hidden gems that you can use to improve your performance in search results. Here is a quick tip: find your competitors broken links and buy expired domains. You can export broken link report into an Excel file and start shopping. Don’t give up if you see that domain is not available at your favorite domain name registrar. The second place to check is the auctions. Take a peek at Sedo and the likes. Sometimes you can get a great domain name for a small $70 fee.

Take care of your duplicate content

Last but not least. Having duplicate content on your website often confuses your visitors and search engines. And you can find duplicate pages with ease with LinkAider.

So if you are ready – follow this link to signup for free. We promise, you’ll instantly find LinkAider extremely valuable for your seo campaigns.

We have received a bunch of great testimonials already and LinkAider is free to try, but let’s take a peek what LinkAider is capable of and how it can help you do your work (you can check all the shared reports for free, no signup required).

Our blog readers can see one theme come up on our blog from time to time: it’s broken link check. Despite the fact that LinkAider started as a broken link check tool its functionality is not limited to that. Let’s take a look:

Broken link report for code.msdn.microsoft.com

Msdn is a great resource for the developers. There you can find the latest (and the oldest, too) Microsft’s API documentations, code samples and tutorials. Being the one most trusted/official Microsoft documentation site it has only 376 broken links. 376 is really a small number and we should admit that Microsoft does a pretty good job keeping its site up to date.

Crawl report for 37signals

37signals is a famous web software company. The company is run by the web2.0 celebrities Jason Fred and David Heinemeier Hansson (the original developer of Ruby on Rails). You can find their blog at http://37signals.com/svn which is fun and inspiring to read. Since they are blogging for such a long time, no wonder they have many broken links (it is hard to keep up with many changes on the internet and it is not their primary business goal to keep their blog very clean).

Wonder what are the top things running through Jason’s head (we couldn’t keep ourselves from posting st. Jason’s picture here:) )? Perhaps the most linked hosts report may give you a clue:

And you can check the whole report here.

Smashingmagazine broken link report

Smashingmagazine is one of the greatest internet design/development sources. They write great tutorials themselves as well as post many links to great internet resources. You can see that they have more than 370 thousand links on their site at the moment. Writing lengthy tutorials and receiving a lot of feedback often produces big pages, however it is often worth to wait for a couple of seconds for their new great article to load.

Problogger link report

Problogger.net is one more internet celebrity we are huge fans of. Problogger.net is written by Darren Rowse with the main goal to educate other bloggers in how to blog and earn from what they are doing. This site has more than 6 million links at the moment. Let us spell that one more time: it has more than six million links with an average of 162 links per page. You should see this yourselves (check how fast LinkAider is organizing large amounts of data!).

If you like how LinkAider reports look like and what LinkAider does, signup now for free!

The rel attribute had its fifteen minutes of fame in 2005 when Google announced that they would be ignoring links with the rel=”nofollow” attribute when ranking websites in search results. Unfortunately, 2005 was the “golden age” in the rise of SEO marketing companies and webmasters took that announcement to heart – avoiding the rel attribute altogether or thinking it was only for “nofollow”.

Also, Google’s “nofollow” announcement was greeted with applause by famous blog software makers because it helped to fight spammers who used to abuse blog comments and create site trackbacks using the rel attribute.

However “nofollow” behavior has changed lately.

So, what is really wrong with the rel attribute? Can we use it without “nofollow” effectively?

A hyperlink connects two HTML pages; the rel attribute is supposed to identify the type, relationship or value of the link itself. Since the introduction of microformats and the XHTML Friends Network (XFN) many more rel values have appeared in the web developer’s toolbox. Multiple values can be used on the same link tag, like this: <a rel="home me contents" href="http://example.org">.

A few of the most popular/appropriate uses for the rel attribute are included here. But compare that list with what is available in HTML 5. So much more than “nofollow”.

What about rel’s cousin – the “rev” attribute?

Rev is less popular than rel and is about to die a slow death since it is being phased out in HTML 5. Since the rel attribute defines the link target’s relationship to the current page, rev’s purpose is to define the current document’s relationship to the link target (the reverse). For example, <a rev="help" href="example.org">. This example indicates that the current document is the help manual for the site/page specified in the href attribute.

These types of HTML features have existed for many years, yet webmasters are very slow to adopt and use them with confidence. Taking a more informed approach to HTML can have two main benefits: it would help many search engines to identify what’s really important and the end user would benefit with new and exciting features (that would not be blocked out automatically). For example, tags like rel=”payment” can be used in RSS feeds. If you are media/text publisher, you can ask your readers for a comment or tip without asking them to click away and visit your page. What features could you introduce by gaining control over your rel attributes?

A “permanent” broken link occurs when a domain name is completely inaccessible or has expired. The domain wasn’t renewed by the webmaster – abandoned only to be bought by spammers who now have the advantage of the site’s former Internet credibility. Other sites that have a link to that domain now have sent their users to the spammer’s site.

Obviously, it’s good practice to know when certain domains expire, especially your own. One of the more popular services is Go Daddy, who will monitor expiry date and domain information for a fee in return. This is good and bad: you might be a responsible and proactive webmaster, who renews in plenty of time, or you could be a spammer waiting to pounce.

If you find a domain inaccessible error, you should look for alternatives or cached content – refer to this LinkAider blog post for more information about how to deal with broken links.

“Temporary” broken links occur when sites go down for maintenance from time to time. Other sites experience network or software errors. In these cases, you’ll probably see a custom error message explaining what has happened. Twitter is the most obvious example of constant downtimes. Its infamous downtime has become the topic of Internet folklore and is a part of many jokes already. Furthermore, Twitter was the inspiration for funny projects like: istwitterdown.com (now defunct itself – how ironic!) August 2009 was rather tough for Twitter according to the chart from the pingdom monitoring service. Growing pains or maintenance can take a toll – but downtime on a social network is not as serious as downtime for your income-generating site. Although these days you may argue that Twitter now provides this type of service and should be making uptime guarantees like other hosting providers.

Hosting providers build contracts around guarantees for uptime. Typically, they are expressed using a percentage. Percentages put a context around the amount of time that a server/hardware/network is continually available (its availability). Percentages make the hosting contract look great at first sight. However, let’s do some math. If a hosting provider says:

• 90% uptime, it means the server may be down 876 hours a year
• 95% = 438 hours down
• 99% = 87 hours, 36 minutes
• 99.9% = 8 hours, 45 minutes, 36 seconds
• 99.99% = 52 minutes, 33.6 seconds
• 99.999% = 5 minutes, 15.36 seconds
• 99.9999% = 31.68 seconds

Of course, these are just network guarantees. Webmasters can also experience some downtime (on top of that) due to script mis-configuration, bad coding or human errors.

Also, when double checking and/or fixing your broken links manually, keep in mind that some websites may be inaccessible within a certain network location or country. So if you have some doubts, check services like these: downforeveryoneorjustme.com and dingitsup.com.

Downforeveryoneorjustme.com is a nice test to do when your users tell you that a site is down. You can check it online and then diagnose the problem – it might be their machine, not the server or the network.

Dingitsup.com sends a text message to let you know when a site is back up or has gone down. Be the first one to know if a site goes down before your end-users or customers start calling. The last thing you want to send out is a large email campaign and have a flood of complaints about a site that is unavailable.

In summary, if you have any doubts about a domain link, double-checking is always the safest route. If you’ve already signed up with LinkAider, you know that you can re-run your broken link report with a single click. If you haven’t signed up yet, consider doing it now. It’s free.

Google determines webpage importance according to the link count to that page. Links end up having a serious impact on your webpage link count and weighting for your page rank. Ever wondered where this came from? Do you think it was Google’s unique idea?


I’m sure they would like to take credit for it but the answer is no. If you have ever gathered a list of references for a paper at university or read any serious non-fiction or scientific book, you’ve probably done or have seen what Google has automated for the Internet world of end-users who search for information constantly.

Traditional references to other articles and book sources are provided in a list at the end of a piece of writing. These are called bibliography. This will lead readers to additional information and tell them what writings the author referred to when creating the book. You may also use it to evaluate the quality of the book you’ve just read – to make sure it has used legitimate information.

Bibliographic index or bibliographic database is a collection of these references which is used to find particular articles, magazines or books. And the reference count is used to determine legitimacy of a particular writing. Is this starting to sound familiar? Since Google uses linking references to rank its page results, this enhances a site’s “legitimacy” automatically, and improves your website’s rank on the Internet according to Google’s system.

An analog search engine was established in 1896. Index cards were used to build a database of books and people could mail in their requests for material. They would then have the information they needed to look up the book at the local library. Some professional indexers still use a method like this today to ensure key words are incorporated into an index. Indexing and inserting key word alternatives are critical to Search Engine Optimization as well.

Bibliographers, such as this one working in the 16th century, provided key historical texts that catalogued important writers in history. Without these types of reference lists, many people would not know that this kind of information was available.

The library that started on clay tablets is now going online. It should be interesting to see how search engines and webmasters cope with the information catalogs of the future. With hard copy formats disappearing, how does one reference a page number? How big is a page? What kind of linking will occur when new types of search engines, such as WolframAlpha, are developed to present information in a different way.

If your site’s reputation online depends on its links and the number of references you have, it would be interesting to keep up with new types of technology that have become available to streamline a basic need. Manage your links and satisfy the need for information – electronically and at your fingertips.

Typically, they have already developed a structure in their mind or sketched it out on a napkin during a lunch meeting. Seems unorganized but some of the structures remain in place today – but how effective are they for SEO and page ranking? Technology has evolved to be very out-of-the-box, but the extra help they may need is not available.

Get Informed

Watch Rand’s entertaining video about content categorization for large sites, such as blogs, news or enterprise sites. He provides some great examples and reasons for the model he selects.

Keep in mind that it is very easy to define site structure when starting a new site. Starting a fresh site is less complicated, everything is clear, organized and in just the right place where you’d like it to be. However, when your site gets a bit older, the content is growing in size and may be a little too haphazard. You add a page here and there. Some blog posts are moved to an archive section. You remove old pages, products. Content starts to get a little bit rushed and disorganized. The grand plans and structure you designed has now been lost along the way. When a year has passed – do you still know how your site is organized?

By the way, Bruce Clay refers to such technique as siloing. Siloing categorizes content on your site based on the theme. The theme and sub-topics are determined by target key phrases pulled from the content on your pages. Siloing categorizes the content both structurally and virtually, which means the content is categorized on the server (or at least how it appears to search engines), and virtually – it’s how contend is interlinked. Check out this article about siloing (a.k.a. page rank sculpting).

Link with Purpose

As you all know, search engine optimization is a lot about links. If you link much to a specific page, Google thinks that’s the most important page on your site. And yes, everybody links to their home page and top product categories, contact pages, and so on. Still – is this the right way for your site?

So – a site’s structure matters a lot. But keep in mind that page rank sculpting doesn’t work with the “nofollow” attribute anymore! However, you can achieve the same results with robots.txt, javascript links and the meta robots tag.

Definition: nofollow is an HTML attribute value used to instruct some search engines that a hyperlink should not be included in the link target’s ranking. It helps to reduce specific types of search engine spam.

Get Some Website Tune-up and Maintenance

Still, there is one question we didn’t answer: How do you maintain and follow your site’s structure when you have an established (grown up but messed up) site? Can you still draw it on your napkin? Don’t worry – because LinkAider will come to the rescue. With LinkAider, you can see your “most linked to” page with a single click. Also, you can manage and reorganize your your pages and links in any way you like.

Build your site’s structure wisely and if you haven’t subscribed yet, do it now. It’s free.

Webmasters remain unaware because the links are hidden from human eyes – only visible to search engine robots in order to grab higher positions in search engine ranking. MIT’s Technology Review recommends:

“that anyone running her own website regularly patch the Web server and any software running on it. In the same way that you wouldn’t browse the Web with an unpatched copy of Internet Explorer, you shouldn’t run a website with an unpatched or old version of WordPress, cPanel, Joomla, or Drupal.”

How can LinkAider help?

LinkAider now has a Smart Advisory module that detects excessive linking to a particular domain. And since LinkAider is a robot, similar to Google’s bot and other spiders, it can see the invisible and detect the content for human eyes. For example:

Links like this are invisible to human eye.

How do hacked sites and bad links hurt webmasters?

• Posts and tweets that contain links to your legitimate site can be rejected if malware is 
detected – reducing your exposure and reputation at the same time
• Sites get removed from search engine indexes completely
• Sites suffer a drop in rankings as a penalty for including spam links

LinkAider’s Smart Advisory module can perform a bad link check and report back to the webmaster on how to contain and correct these code violations.

Beware of Cloaked Spam

Seobook has an interesting story about cloaked spam and how hard it is to detect and remove. A text-only option displays the links and keywords that are hidden on the site. Google has indexed a list of this particular hack on more than 20,000 websites. This Google Reader discussion thread reveals how unsuspecting people are affected by the hack.

What You Can Do: Establish a Routine

1. Check if there are any suspicious links using LinkAider
2. Perform some manual checks:
   

   “Google, through some of its products, offers webmasters some ways of spotting if a site has been hacked or modified by a third party without permission. For example, by using Google Search you can spot typical keywords added by hackers to your website and identify the pages that have been compromised. Just open google.com and run a site: search query on your website, looking for commercial keywords that hackers commonly use for spammy purposes (such as viagra, porn, mp3, gambling, etc.)”

3. Perform your updates when you are notified – especially for the latest Content Management System currently in use.
4. Do not use unknown plugins or themes.
5. Monitor your site for new links or suspicious activity.
6. Ask all of your third-party developers to follow this routine.

Subscribe to LinkAider in order to catch all of the hidden hacks and bad links before they catch you off guard.