Gordon Bell might rightly claim the prize for one of the most digitalized life ever. Born in 1934 and coming of age after World War II, Bell witnessed and spurred on a societal shift from paper to digital, from paper print-outs and letters to digital documents, PDFs, email, and all-things digital. Bell was an American electrical engineer and innovator in the early days of computers. He worked on several ground-breaking technology products. But he is arguably most well-known for his quest to digitalize his entire life and develop products to take advantage of such a life.

Bell’s project, which was known as MyLifeBits, was part research project at Microsoft, part experiment in comprehensive lifelogging. In a sense, Bell and his team of engineers were taking up Vannevar Bush’s vision from 1945 for how technology might augment and help us our memories and attempting to build it. Bush, who was both a scientist himself and a key government official guiding scientific policy and research in post-ware America, laid out what many consider to be a prophetic vision for much of the technological creations that came to be in the decades ahead.

At the time, Bush was lamenting the struggle to keep abreast of so much research and publications and how to productively take notes, and use that knowledge in his own thinking and writing. He felt limited by piles of paper. So, he proposed several ways to overcome these limitations and developed a new paradigm of working and remembering he called the Memex. While Bush wasn’t able to realize much of his vision during his life, during Gordon Bell’s own lifetime various technological changes had made it possible to take up this quest again in the late 1990s and early 2000s.

The aim of MyLifeBits was to digitalize as much of a single human life as technically possible at the time. It also explored, researched, and built software solutions in support of this “lifetime store of everything.” The end result was two-fold: an assortment of software and integration solutions and Bell’s own lifestore or digital archive.

Admittedly, this quest to digitalize a life is an odd one. It sits at the intersection of a remarkable computational and historical change. For Bell there was a clear boundary between his early work and personal life in papers and the later one on computers, digital cameras, GPS, and smartphones.

Today most of our lives come already packaged in digital. We are more apt to get an email than a letter, more commonly read digital words than printed ones, and are more likely than ever to engage in large portions of our lives through digital tools and online spaces rather physical ones. And yet Bell’s somewhat quixotic quest to digitalize a life offers us an illuminating story on how one might live with and through technology both today and in the future. It gives us a number of lessons and even shows us some of the limits to such a technology-infused lifestyle.

Let’s dive into the tale of one engineer’s quest to go paperless!


NOTE: This post covers the historical and visionary aspects of personal digitalization. See a Guide to Digitalizing Your Journals with a Scanner App for a practical guide on how to digitalize your own papers and journals.

A Short History of the Epic Quest to Digitalize Everything

Source: Bush, V. (1945). As we may think. The Atlantic Monthly, 176(1), 101-108.

Bush’s Memex: A 1945 Technological Prophesy

Vannevar Bush’s 1945 article “As We May Think” remains a key document in the history of computing. It foretold many aspects of computing we take for granted today yet still rings oddly contemporary in his hope to make reading and knowledge management more manageable.

(NOTE: The Plain Text Life: Note Taking, Writing and Life Organization Using Plain Text Files is an in-depth write-up I did on knowledge management using plaintext file notes. Check it out for implementing a mesh of notes and knowledge management towards creative output.)

Bush, who at the time was the head of of the Office of Scientific Research and Development, lamented the difficulty of keeping up with the mountain scientific research and frailty of remembering and navigating that information over time. He proposed several technical solutions, including:

  • Cyclops Camera - a forehead worn camera that could record anything you see and want to capture.
  • Microfilm
  • Vodocoder - a machine that could do speech-to-text transcription
  • Thinking Machine
  • “an aid to memory. Like the brain, Memex would file material by association. Press a key and it would run through a ‘trail’ of facts” (p. 113)

Most famously he came up with a fundamental paradigm for how to organize and link data together, which he called Memex.

Bush’s Memex, which stood for memory extender, was a microfilm-based “device in which an individual stores all his books, records, and communications, and which is mechanized so that it may be consulted with exceeding speed and flexibility. It is an enlarged intimate supplement to his memory.” (p 121). Basically he was describing a knowledge management system. At the heart his insight is the idea of linking together of information by association. Today we would call these hypertext links.

As Bush foresaw in 1945:

“A record if it is to be useful … must be continuously extended, it must be stored, and above all it must be consulted…The difficulty seems to be, not so much that we publish unduly … but rather that publication has been extended far beyond our present ability to make real use of the record”

Much like how the brain links ideas together through association, Bush imagined a system “ready made with a mesh of associative trails running through them.” He envisioned complete storage, full-text search, text and audio annotations, and hyperlinks. Furthermore this way of connecting was not just for organization, the memex through these associated trails was intended as a way to amplify that knowledge. It wasn’t just knowledge capture and storage; it was designed as extension of our creativity, our learning and thinking too.

Unfortunately, for Bush, the technology available at that time of his life never allowed for Memex to see the light of day. It remained a futuristic vision. By contrast, by the time Gordon Bell was dealing with similar challenges and conceiving his own e-memory project, various technologies had come together to make this type of quest not only possible but increasingly feasible using cheap consumer technologies at hand.

Gordon Bell’s Quest to Digitalize Everything

Gordon Bell. Source: Wikipedia.

The MyLifeBits project began as an effort to digitize the books Gordon has written, and it has turned into a pioneering endeavor to record and preserve in digital form everything he sees, hears, learns, and experiences. (p 8)

Going under the singular title of MyLifeBits, the project itself was multifaceted. Along with his long-time collaborators and fellow software engineers, Jim Gemmell and Roger Lueder, Bell and company’s initial efforts were two-pronged: First, they wanted to record and digitalize everything from Bell’s paper archive from the past. Second, they wanted to assemble tracking tools that allowed them to record everything in the present and going forward, including emails, biometric data and even a stream of photos taking from a first-person perspective. Eventually though digitalizing and tracking were key inputs, they realized capturing wasn’t going to be enough; they need a way to intelligent use all this digital information.

Bell, having started work in a pre-digital era and seen the rise of personal computing, felt a pull to bring his file cabinets of papers and boxes of photos into the digital age too. So, the first stage of their project was aimed at scanning all of his papers.

Using a commercial scanner and a research assistant, they converted this paper archive from the past into a digital one. In total, they scanned virtually all of his:

  • Books written (and read when possible)
  • Personal documents (correspondence including memos and email, bills, legal documents, papers written, …)
  • Photos
  • Posters, paintings, photo of things (artifacts, …medals, plaques)
  • Home movies and videos
  • CD collection

They would eventually hire a team to scan and annotate every scape of paper in Bell’s life, a process that would take a number of years. His paper filing cabinets emptied as he digitalized everything, filling the storage capacity of external drives at the time.

MyLifeBits’s Vision: Capture Everything, Bring It Together

Beyond just digitalizing everything he had on paper, Bell and company also strove to proactively digitalize his life going forward.

In what is known as “lifelogging,” the stated goal is to digitize everything that happens to someone. This includes recording images, video, and audio from everything around us. As Wired Editor Kevin Kelly puts it, “Lifestreaming can be thought of as an active, conscious tracking.” It is one of the most extreme forms of self-tracking.

Working in the early 2000s, for Bell to engage in lifelogging, he was forced to build many of his own tools to keep a log or record his life. For example, using some custom software, they recorded his computer activities, like websites visited and the text on those sites; they experimented with GPS trackers to monitor geospatial movements; and, in the spirit of Bush’s head-mounted version, they even developed a camera to take first-person perspective photos as Bell went about his life. In fact, the camera they developed eventually became the Microsoft product, SenseCam.

Ironically, while the term “lifelogging” has lost popularity, today various commercial products, apps and tools now exist to make tracking your life easier than ever, and it can largely be done passively using a smartphone, a wearable, and a few self-tracking apps.

(NOTE: I’ve written extensively on this topic in series on how to track a life. For example, among other things, RescueTime can be used to record your computer usage, Instapaper can log your article reading, most phones can tell you your screentime, and a wearable like Apple Watch can help you know your steps, movements, workouts and even sleep. You can even use an app like PhotoStats.io to know more about when, where and what you take pictures of on your phone.)

Eventually though the MyLifeBits team realized that digitalizing and lifelogging were only part of the challenge; they needed a way to make this collected information useful and usable. Simply having all of this information in an unorganized or unstructured format created more problems.

In a series of research papers culminating in their 2010 book, Your life, uploaded: The digital way to better memory, health, and productivity, Bell and Gemmell’s focus went to the actual system, MyLifeBit, that would store, link and annotate all this information together.

Here is an early sketch on the MyLifeBits store:

At the core of this vision is a database. This is a singular place where everything gets linked together and/or stored. It includes various digital files, photos and videos already there. Additionally, going into this database are various lifelogging or tracking streams. This includes data like voice annotations, internet usage, TV watching, email, radio and many, many more. Everything would have meta-data allowing you to contextual the time of creation and where it happened.

You might think about this software as personal internet or a form of e-memory where everythinig is linked together. This allows you to search your history on every digital and digitalized aspect of a life.

In one of the most interesting and insightful asides from the entire project, Bell remarks that amongst all the lifelogging and tracking, the most informative media he found was audio from conversations.

While it might seem like visual information and photos are the best thing to track and learn from, for him our conversations are one of the most powerful lifelogging data sources we have. In fact, if he could only lifelog one thing, it would be his conversations. He preferred to have the audio or even better text from his daily conversations rather than just a collection of photos or visual data.

Sadly, this is the one feature that quantified self technology has yet to provide, though tools like Alexa, Google Assistant and Siri might one day provide this.

Total Recall’s Three Streams of Data and Technologies that Make It possible

The MyLifeBits project offered us a new paradigm for living they called “Total Recall.” Essentially instead of being limited to our biological memory, all of this lifelogging enables e-memory, a digitalization and extension of our “wetware” brains through digital storage and retrieval.

According to the team, this “Total Recall revolution is being birthed on the strength of a few key devices: cell phones, digital cameras, and pocket-size GPS units.” (p 186). Much like the Quantified Self Movement, they define three streams of technologies that make this vision possible: recording, storage, and sophisticated recall. It’s worth quoting this in full:

Three streams of technology are coming together to make the world of Total Recall a reality.

First, and perhaps most important, we are recording more and more of our lives digitally without even trying. Digital cameras, e-mail, cell phones, and personal digital assistants (PDAs) are the vanguard of technology that is generating an explosion of digital records of our daily lives. Digital sensing and recording will become ubiquitous.

Second, this mountain of new personal digital records can now be stored more cheaply than can easily be imagined—for about two hundred dollars you can own enough memory to store everything you read, everything you hear, and ten pictures a day for your whole life.

Third, technologies enabling you to search, analyze, and present all kinds of reports from such large mountains of data are being developed, with astonishing results. Google will by no means be the last extraordinarily successful company to be built on new search technologies. So, we live in a world with more digital memories, more space to store them, and better and better technology to recollect them. The world of Total Recall is inevitable for these three reasons. (p 11)

In summary, various tracking technology makes capturing easier and more ubiquitous from our devices, from our bodies via wearables, and from our environment via remote sensors; computer and cloud storage is also cheaper than ever, meaning it isn’t cost prohibitive to never throw away anything digital; and since we have a nearly omnipotent ability to index and search all of this data, personal and universal, we can recall, pull out and put together the information and data.

In the same spirit as Kevin Kelly’s wonderful tech-future vision and book, The Inevitable, Bell believes e-memory and total recall are here to stay and will be a dominant aspect of the technology landscape going forward.

The MyLifeBits Prototype: Making Life Data Usable and Useful:

One key feature of MyLifeBits was to be an e-memory tool that could augment and extend our biological memory. As they put it:

Software will allow you to sort and sift through your digital memories to uncover patterns in your life you could never have gleaned with your unaided brain. Your work habits, your leisure habits, and your spending habits; your emotional response patterns in various situations and around certain people; the numerous subtle factors that affect your mental well-being and your physical health; and just about anything else you care to know about yourself can be chronicled, condensed, cross-correlated, and plotted out for you in useful and illuminating ways. (p 12)

So, after years of efforts digitalizing one man’s paper and photo archive, developing tools to lifelog, and integrating that data together, a major challenge remained for Bell and the team at MyLifeBits:

What to do with all of this data? Could they make it useful and usable? What sort of product or service it might be?

It was time to take this lifelogging and digital archive and make it usable and useful:

The challenge now is to make the stored material useful. MyLifeBits implements a number of features originally proposed by Vannevar Bush, such as links, annotations, and full text search. This demonstration highlights the importance of augmenting these ideas with typed links and database features. Visualizations, complex queries, and pivots enable the user to find or browse many items that would otherwise remain unused.

It was time to build a prototype.

Here are some early prototypes of the product:

Underlying this product was a database of annotated and linked metadata. It wasn’t arranged into folders, but interconnected collections of data. To use it, you could do a search and use filters to refine it by periods of time and various other facets.

Additionally, various charts and data visualizations could be created, like which websites were visited during certain periods:

These graphs show that once you have a collection of life data, you have an opportunity to see trends and display patterns toon. Bell’s charts are also reminiscent of my own efforts at visualizing tracking data using Tableau or with QS Ledger. For example, see my recent A Year in Data.

Based on these prototypes and examples, MyLifeBits represented an epic product to capture everything, bring it together, make it searchable, and provide a data-driven tool for a well-tracked life. For anyone hoping to digitalize their life, this was the dream product many of us had been hoping for.

So, what happened? Where can I get a version of MyLifeBits?

What Happened to the MyLifeBits’s Product?

Unfortunately, at the time of writing this post, it appears that neither the software for MyLifeBit nor a public version of Bell’s archive ever saw the light of day. One of the products created in conjunction, SenseCam, which was a wearable lifelogging camera, is also no longer available, though alternative wearable lifelogging cameras are still around.

It’s been reported that aspects of the MyLifeBits’s technology was applied to Microsoft’s desktop and cloud search tools. While Bell might have succeeded in his epic, albeit quixotic, quest to go paperless and digitalize everything in his life, it should be admitted that much of the hype and promise of a comprehensive e-memory remains unfulfilled, especially if judged by the fact that no user-facing product ever got released.

What happened to the MyLifeBits’s product? Was it too hard technically? Did the system require too much manual work to integrate and annotate information? Was usability too tedious? Was it just a service no one really wanted? What the heck happened?

For one thing, I think the vision of a comprehensive tool may not be what people wanted or needed. Consider this: Google already makes informational searches seamless and magical. File search on your computer or in the cloud has become robust and makes finding what you need relatively easy. Various note-taking tools now help you store and find your notes too. Similarly, Apple, Flicker, Google and even my own photo app, PhotoStats.io have all significantly improved aggregating and searching our personal photos. So, while all of these services are fragmented and siloed and none really bring it all together or make it data-driven, several major services provide incredible indexing and searching functions for public information, files, notes and photos.

There remains one area that no service has really solved and that’s aggregating and using various self-tracking and lifelogging data. While it’s easier than ever to track a life, it’s still a challenge to bring this data together and make it useable and useful. This means that if MyLifeBits’s software was to find a meaningful and impactful use case, I’d argue that it would most effecive in this situation of aggregating and augmenting our personal tracking data towards the data-driven life.

Admittedly, even though a software product never emerged, Bell has written a few books that act as a surrogate for the vision he proposed. Much like Bush, he offers a mix of technical predictions and prophetic mash-ups for how technology and a way of living might emerge. I don’t necessarily subscribe to every claim, but it does provide good fodder for futuristic thinking.

In my own work, while I didn’t intentionally plan it, my open source, data aggregation and visualization tool QS Ledger is also a step in the same direction as MyLifeBits and Bell’s vision. For example, QS Ledger allows you to pull your data together from 20+ sources and visualize it. It allows you to contextually see your data and to tell data stories. You can use it to ask and answer questions a whole host of questions on your life using your life tracking data.

Admittedly, much work remains on that project and others if we hope to translate all of this tracking into products that allow us to have and augment a data-driven life.

Conclusion: A Quixotic Quest to Digitalize Everything?

Over 75 years ago, Vannevar Bush’s manifesto, “As We May Think,” offered a futuristic vision called Memex for capturing and extending our minds through technology. Ironically he foresaw this as being done using microfilm. Computing pioneer Gordon Bell took up this same vision and attempted to fabricate a version of this using computers, sensors and other innovations. Through a series of projects under the umbrella of MyLifeBits, Bell digitalized all of his own papers and photos, developed technologies for self-tracking and lifelogging, and built software to make it searchable. While MyLifeBits software failed to materialize, many parts of Bell’s vision and implementation have to come to be in dozens of technologies and companies that help us track our lives, find digital information, and lead a data-driven life.

In the book, Your life, uploaded: The digital way to better memory, health, and productivity, Bell and his co-author and lifelong collaborator Jim Gemmell summarize their project thus:

Our goals were twofold:

  1. To create software for lifelogging, and the subsequent recall and usage of one’s e-memories. We wanted software to record a diverse array of information about one’s life and activities, from a variety of sources and devices, and to do so as easily, as unobtrusively, and as automatically as possible. The software would have to give people powerful tools for searching, organizing, annotating, and pattern-mining their ultimately huge e-memories.
  2. To identify the benefits, drawbacks, technical issues, sticking points, and usability of Total Recall in real life. We wanted to try it out (as much as we could) and see what it was like. (p 40)

Many of us already live, at least to some extent, in a world of total recall. When our biological memory slips, tech can help us. We can check our social media feed for life events, we search our calendar for past trips and meetings, we can scan our expansive email, photo libraries, and chat logs for play-by-play, and even look at biometric data on movement, sleep and heart rate to visualize our hourly health journey.

Looking at Bell’s project holistically. I’m struck by two things:

First, Bell’s world existed at the borderland between the materiality of a paper and printed past and today’s already digitally-infused worldview. He spent several decades developing computer technologies while also seeing his paper file cabinets grow. For this reason, digitalization was an obvious goal for him and many others like my parents who lived through the transition from physical papers and photos to cloud storage and social media. Unfortunately, this kind of half-digital life has become increasingly rare. We largely no longer need to digitalize, since much of life already comes packaged in digital. Many of us spend most of our non-sleep hours looking at a screen or carrying around a mini, ever-present tracking device.

For this reason, much of Bell’s discussion of digitalization rings dated even a few years after publication. For many of us today, paperless and digital is the norm, rather than the exception. There are still areas and parts of our lives that could use digitalization, especially how to digitalize a journal or hand-written letter. This is a topic I’ll take up in Part 2 on the practical side of going paperless using scanning apps.

The second thing that strikes me about Bell’s epic quest to digitalize everything is how individualistic it comes off as on one level. You see elements of this in how his children comment on the project too. It’s not something for everyone. And for as much as Bell couches his work as a step towards technology and software that will help humanity digitalize and lead a data-driven life, the failure to deliver any products makes you wonder if about his underlying mission and obcession.

For example, Bell talks about the role self-tracking data might play in learning, health and more. But did Bell really care about helping society evolve towards a useful ubiquity of lifelogging and personal data products? Or was it more about a single human’s drive to create an artifact and memorial to his own temporary existence? Was it epic or quixotic?

Either way, it appears Bell did largely go paperless, and he can rightly be called a trailblazer in the lifelogging, quantified self and data-driven life. Bell’s efforts and projects are indeed a unique curiosity and tale in the history of humanity and of computing as he strove to become the most digitalized life ever.



Sources
  • Bush, V. (1945). As we may think. The Atlantic Monthly, 176(1), 101-108.
  • Bell, G., Gemmell, J., & Lueder, R. (2004). Challenges in Using Lifetime Personal Information Stores based on MyLifeBits. Proceedings from Alpbach Forum.
  • Bell, G., & Gemmell, J. (2010). Your life, uploaded: The digital way to better memory, health, and productivity. Penguin.
  • Bell, G. (2014). Supercomputers: The Amazing Race (A History of Supercomputing, 1960-2020).
  • Cherry, S. (2005). Gordon Bell, A Life Digitalized. IEEE Spectrum.
  • Gemmell, J., Lueder, R., & Bell, G. (2003). The MyLifeBits lifetime store.
  • Gemmell, J., Bell, G., Lueder, R., & Drucker, S. (2002). MyLifeBits: fulfilling the Memex vision. Communications of the ACM.
  • Gemmell, J., Bell, G., & Lueder, R. (2006). MyLifeBits: a personal database for everything. Communications of the ACM (CACM).
  • Selke, S. (2016). Lifelogging: Digital self-tracking and Lifelogging - between disruptive technology and cultural transformation (1st ed. 2016 ed.). Springer VS.