Brewster Khale: Universal Access to All Knowledge

A guided tour inside the imposing former Christian Science Church building is a nice surprise to visitors to San Francisco-based Internet Archive. On a lucky day, the one who shows them around could be Brewster Khale himself, the founder and digital librarian of the Archive. Khale loves to show off the Archive’s music collections. He challenges the visitors to yell out an obscure band’s name, pores over the giant computer screen in the auditorium for search results and instantly plays out a track to wow the admiring crowd. He wants the Archive to be educational and entertaining. And above all, the collections of easily-accessible open resources for creative minds.

 
(This is a condensed version of the interview with Brewster Khale based on the video recording. It’s not a verbatim transcript.)

 OHI  Team was interviewing Brewster Khale

OHI：Tell us a bit more about your career development following your graduation from MIT.

BK: Actually, my career has been made up by only one idea, which came up with in 1980, in MIT. A friend of mine posed this question to me: being a technologist and idealist, what’s something good you can do with your technology? I could only come up with this answer: to try to save people’s privacy, even though they are going to throw it all away.

The encryption of information was just started. I thought about that and worked on that. I tried to make chips that protect people’s privacy and phone calls. But I found I couldn’t make chips that are cheap enough to help the common men. So I went to Plan B, which is to build the library of everything, kind of the Library of Alexandria version two. So that was in the 1980, sort of my focus ever since. 

 OHI  Team was interviewing Brewster Khale

OHI: You are also a big supporter of Creative Commons?

BK: Yeah, back in the 1980s, computers didn’t work very well. They basically did accounting. So we need a big, fast computer. I helped build a connection machine and supercomputer that would be able to move through large amount of data. We ended up making search engine: Wide Area Information Servers, or WAIS. After we built connection machine and supercomputer, made search engine, then we make it available to anybody to be able to access.

So this new Internet thing was started really sort of coming out just being an ARPANET. We made a publishing system, a clan server publishing system called WAIS. It’s the first out there before we go further for the Web. And it’s open, and you could search corporate, wide-area information.

So with the connection machine working, then we need the Internet to work, so people can publish online, make everyone into a publisher. That was our model in the late 1980s. We formed a company in 1992, trying to get newspapers, magazines and book publishers to publish online.

They were successful, the magazines and newspapers. Web started to come around at that point. It’s an easier platform than WAIS. That took off. We sold that company to AOL. The idea was trying to help people get paid by publishing on the Internet. But they were having trouble just growing fast enough on their own board system. So I left to form Alexa Internet and the Internet Archive to try to catalog the world by Web.

By 1994, we had people publishing online pretty well. That was booming. So we had publishers. They weren’t paid very well. Because advertising was about the only model. That was a mistake. But once we had that going, I can build Alexa Internet and Internet Archive. Alexa is the name for the library of Alexandria. That was sold to Amazon.com in 1999. But it had a contract written into the soul of the company to donate everything that it collected to the Internet Archive nonprofit.

So what’s that was sort of stabling. And Amazon delivered what it is: that is looking for by moving over to the Internet Archive fulltime. Fifteen years later, we got good collections of the Web, television, pretty good books, music, and still not done yet, which is shocking how long these things are taking.

OHI: Part of the Archive’s mission is to preserve history through digital technology?

BK: Let’s at least make sure we have copy of it. Originally the life of a webpage was only 100 days before it’s deleted. We don’t want it to just evaporate. One of the blessings of the Web is very simple. The curse of it is that it’s very simple. They didn’t build in the system that Ted Nelson has imaged, the versions, back links, payment models.

It seems sort of just a fraction of what it is that many of us had hoped for. So the Internet Archive and many other things we’ve been doing have been trying to fix the Web and make it better.

OHI: From Alexsa and entrepreneurship to the Internet Archive, you seem to have been doing quite different things. How did you eventually run the Archive full-time?

BK: For me it’s not a big change. It’s always been part of the plan. It’s all trying to get publishers online and we have to get people making money. Capitalism is a pretty simple system. If you put enough care in front of the people, they follow it. If you have a path for making money, they would often do that.

We got publishing going. And once that’s going, we can shift to doing this library. It really should be non-profit, because it’s full of everybody else’s materials. So we didn’t think it should be a full profit. And it should be open for other people to use it to come up with all sorts of new types of services.

Google has been building the library much faster and on a bigger scale, with very different commercial content. But we’d like to see it done in an open way. So we go on to build in many ways parallel to what’s Google has been building and also at scale. It should be people first. Google has digitized 25 million books. We have done perhaps 2.5 or 3 million books. So we are 10 percent of what they’ve done. That’s good. They’ve done ten times more. We are gonna orient towards doing it in higher quality and making it integrate with the library system so that it’s the people’s system.

Just being the father of the search engine or to help them build new translation systems, I reckon everyone can do that. It’s just locked up in one company. It belongs to the world. In book digitization, there is a project called “The Million Books Project.” The government of China, the government of India and Carnegie Mellon, and big universities are big participants. Our Archive helped this project a great deal. But it didn’t follow through as well as it should. It’s under the second or third phase of it. The Chinese government is actually still involved with digitizing books through that program, even if it’s 15 years later.

OHI  Team was interviewing Brewster Khale

OHI: Tell us something about the financing of the Archive. How do you make sure it’s self-sufficient and sustainable?

BK: Libraries are usually part of the government or universities. In the United States, it’s an industry worth about 12 billion dollars a year. Worldwide, it’s about 31 billion. That’s pretty good. About one third of that money goes to buy publishers’ products. That really helps sustaining the publishing industry.

It’s a business model that fits very well in the Internet. Libraries are paid to give things away. It works very well in the Internet. The Internet Archive works with libraries and the system they use. They pay us to digitize books more effectively than collect Web pages. We don’t have a lock-in system. We are not part of the government or universities, but we are a library. So we see ourselves as being funded in the same way.

Even though there have been economic downturns, some of these popular open-sourced, open-access organizations have done very well. Wikipedia, mozello, The Electronic Frontier Foundation, the Internet Archive and the Public Library of Science, all have grown, even though there have been economic downturns. We may take a pause in growth. If we are valuable to people, people keep us alive. People want to have access to information. But the key thing is continuing to be relevant.

We are one of the 300 most popular websites. We used to be No. 5. So that’s a whole lot better. But there are 100 million websites out there. So there are a lot of people who want to have access to old stuffs. So I’m encouraged that there is this intellectual curiosity that strays outside of what’s up on television.

OHI: How did you build Alexa Internet?

BK: Alexa Internet was basically trying to catalogue the Web. I thought search engine is where it gives out steam. Ideally you need context around what you are looking at. Let’s make a card catalogue if you will for every webpage, to make a description of where it comes from, how old is the webpage, what else you want to see on this webpage.

So it’s metadata about the web. It must be a tool bar, basically on the leverage of all the paths that people on the web, to go and give you information about what you are looking at, and where you want to go. The idea of metadata, actually the term was coined by another mentor of mine, Bill Dunn, a fantastic guy. He came up with the term of metadata. He said the metadata is gonna be more important than the data itself. It’s in the mid-1980s.

Metadata is information about information. Bill was right, he was right! Metadata is more important than data itself. You can put it into machine form and be able to pull it apart and repackage it. Google, the big thing about Google, what made Google possible was anchor-text, it’s information about people putting in links on another site, to refer to the site without better information about the site, then reading the site itself. So Bill Dunn is completely right.

The paths through the web are really important, or how people move through the web. This idea of Alexa Internet was to go and leverage the paths of the people that saw this webpage. That’s why Amazon was interested. We know how to do this at scale.

BK: I have learned something from Marvin Minsky and also Danny Hillis, They said: keep track of the big ideas of what you are trying to do. Don’t get lost a little thing. Keep the big idea of what you are trying to do. For me, this is like to get access to universal knowledge. Let’s do that. For me, that’s my interpretation of what Marvin Minsky and several other guys did with artificial intelligence.

That’s a good term, because it allows you to work towards a goal. If your goal is to make a million dollars, so you go and make a million dollars. Then what? Who cares? Universal access to all knowledge is a big idea that lots of people can line up with. Then we all help each other to try to achieve that big idea. It’s the vision of the Internet that I signed up to. I want the Internet to make that possible. That’s what I’ve been pushing for throughout my career. “Universal access to all knowledge” is our motto. That’s what we are here for.

OHI: What’s your perception about the development of the Internet in the next 50 years?

BK: Well, we’ve seen that it’s become such that every organization in the real world had to have a virtual presence. But the virtual presence reflected in the real world. I think the virtual world is starting to dominate.

So if you take Amazon for example, it’s really a store of everything. It doesn’t exist in the physical world. So it’s a really born digital organization. Now things in the real world have started to pop out. There are some real stores, things like that in the real world. So the virtual world starts to dominate.

We start to have corporations with fewer and fewer people, and more machines, AIs. I think the best way of thinking where those are going is through where they came from. I think corporations are the first generation AIs in the sense that their people were acting like automatons inside corporations. Still act like an entity. But actually it’s better done by machines. And that’s happening.

So we are starting to see corporations. All these are because of Internet and computer resources that have been invented. So the next 50 years, we will see it sweep through the rest of the industry and displace a lot of roles that people used to do. I think we are going to see this sweep more and more.

Lots of things have happened since the Archive was founded in 1996. One incident remains fresh memory on Khale’s mind. A side building housing the scanning center went up in flames one night in 2013. The little building burned, along with some scanning machines and a small amount of materials that the Archive was going through. In retrospect, Khaler says it’s pure luck that they didn’t lose any data and that many materials. But the incident has really made him and his colleagues humbled. To prevent data losses in the face of a natural disaster, the Archive has been trying to create copies of its vast collections at more distant locations, including Egypt and the Netherlands. As of October 2016, its collections have exceeded 15 petabytes.