As I prepare to resurrect privateline.com, I am faced with my old nemesis: link rot. Links can be positive. External links help readers and they make a site more authoritative. Just as footnotes add credibility to the printed page, so do well chosen links. But some studies show 25% to 30% of external links die every year, as pages are pulled or changed. My experience in 11 years of running private line.com bore out that statistic. What to do?
There are software programs that will scan your site to show you what links have broken. Those programs, though, will not fix dead links. That must be done manually. Off you go with each broken link, hoping to find a new page that replaces the old one. If one exists at all. private line.com had hundreds and hundreds of external links over its 400+ pages and by 2006 I was done trying to repair them. It was a big reason I sold the site.
The problem is still rampant in 2016. This page gives an excellent presentation (external link) of the dilemma and offers ways to help reduce the number of dead links before you link over. The only partial solution I found was to archive vital material. That meant downloading and storing .pdfs, images, and entire pages in case a link broke. I could then restore the resource as an internal link. This is impractical for an entire site but manageable for crucial information.
One question that goes begging is that of copyright. Can you use someone else’s material, even if they have pulled it off the web? The short answer is probably not. You could ask permission but I have found that a dead end. No one replies. My response back then was to post the information, saying that a copyright holder could have it removed if requested. I think I got one request. No one seemed to care, anymore than they did in removing the information to begin with.
As I put the site back together I will put in far fewer external links. I may have the URL noted, but in static form like this: [http://www.raredata.com/info.html — accessed January, 2016.] Using Alexa or the Wayback Machine might permit a reader to find the page many years from now. But I myself will be spending less time documenting in the future.