People love to say that things on the internet are there forever. While that’s true to some degree, it isn’t a guarantee. For instance, if a popular website goes out of service, it vanishes. The Internet Archive is working to ensure that the web truly is eternal with the Wayback Machine.
The unique tool allows people to view web pages as they looked at a certain date in the past. Obviously, collecting snapshots of the entire internet in this way is a daunting task. So, the Wayback Machine is teaming up with Cloudflare, one of the biggest cloud platforms in the world, to expand its mission.
Keeping the Lights On
The chaos of the internet can make it difficult to keep a site up and running 100 percent of the time. That’s part of the reason why the Wayback Machine has been such a helpful tool over the years. It maintains a catalog of long-forgotten webpages to ensure that sites aren’t really lost.
The Internet Archive accomplishes this in a few ways. It collects URLs by “crawling” lists of millions of sites much like Google does. It also accepts submissions from users who want websites that are special to them to be backed up.
Currently, the tool has more than 468 billion web pages in its archive. About one billion new URLs are added every day. The partnership with Cloudflare will increase that number, making more of the internet’s history available for public use.
Cloudflare has been offering a service called Always On since 2010. It helps customers keep their websites online by caching a static version of the site that visitors see if downtime occurs. In fact, Always On was one of Cloudflare’s original offerings.
Now, in 2020, the Cloudflare team is looking for ways to revamp the tool. John Graham-Cumming, the company’s chief technology officer, saw a partnership with the Wayback Machine as a perfect way to move forward. He says, “We worked with them to make sure they were okay with us using it in this way. It’s one of those things where it’s like, yeah, this works for everybody, so let’s do it.”
He goes on to note that visitors who arrive at a site that uses Cloudflare when it is experiencing an outage will be served the latest version of it from the Wayback Machine’s archive.
Given the fact that Cloudflare serves more than 25 million sites, there will also be some new additions to the Wayback Machine. Cloudflare’s Always On will continue to be free and domain operators will need to opt-in before using it. Additional queries and data pulls will be handled by the Wayback Machine’s infrastructure.
Internet Archive founder Brewster Kahle says, “We’d just like to make the web more reliable. We want a robust infrastructure out there and we can be part of it, but we’re not all of it. We want multiple participants to be working together in all different ways.”
The duo’s teamwork will do a lot of good for the internet and for everyone who uses it both now and in the years to come.