These crawls are part of an
effort to archive pages as they are created and
archive the pages that they refer to. That way, as the
pages that are referenced are changed or taken from
the web, a link to the version that was live when the
page was written will be preserved.
Then the Internet Archive hopes that references to
these archived pages will be put in place of a link
that would be otherwise be broken, or a companion link
to allow people to see what was originally intended by
a page's authors.
The goal is to
fix
all
broken links on the web. Crawls of supported "No
More 404" sites.
