‘As We May Think’ – part 3

Besides creating audio records instead of writing information down, an image says more than a thousand words. Vannevar Bush already envisioned an entirely different experience for researchers.

One can now picture a future investigator in his laboratory. His hands are free, and he is not anchored. As he moves about and observes, he photographs and comments. Time is automatically recorded to tie the two together.[1]

Although still images already provide a lot of information together with their tags, commenting while photographing might be easier using a video camera. Most devices such as laptops, tablets and mobile phones already contain cameras, but a Go Pro or other hands free camera increases the ease of use. However video production consists of planning during pre-production, capturing during production and processing in the post-production phase.[2] Bush was already aware of these phases.

As he ponders over his notes in the evening, he again talks his comments into the record. His typed record, as well as his photographs, may both be in miniature, so that he projects them for examination.[3]

The process of recording information is constantly improving, especially after the invention of an optical head-mounted display such as Google Glasses. It displays information and communicates with the internet using natural language voice commands. Furthermore the glasses include a touchpad and a camera which can take photos or record videos just by voicing the command “record a video”.[4] This technology can improve the research process since it “1) provides workflow guidance to the user, 2) supports hands-free operation, 3) allows the users to focus on their work, and 4) enables an efficient way for collaborating with a remote expert”.[5] Although the glasses were already tested in the healthcare and industry maintenance fields, the Glass Explorer program ceased to exist. Ivy Ross and Tony Fadell took over, but the release date of their new product remains unknown.[6]

[1] Bush. “As We May Think,” Chapter 3.

[2] Dustin Freeman, Stephanie Santosa, Fanny Chevalier, Ravin Balakrishnan and Karan Sing, “LACES: live authoring through compositing and editing of streaming video,” Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2014, 1207-1216, doi: 2556288.2557304.

[3] Bush, “As We May Think,” Chapter 3.

[4] “Google Glass,” Wikipedia, accessed March 7, 2016. https://en.wikipedia.org/wiki/Google_Glass.

[5] Xianjun Sam Zheng, Patrik Matos da Silva, Cedric Foucault Siddharth Dasari, Meng Yuan, Stuart Goose. “Wearable Solution for Industrial Maintenance,” Proceedings of the 33rd Annual ACM Conference Extended Abstract on Human Factors in Computing Systems, 2015, 311-314, doi: 702613.2725442.

[6] Nick Bilton. “Why Google Glass Broke,” The New York Times, accessed March 7, 2016, http://www.nytimes.com/2015/02/05/style/why-google-glass-broke.html?smid=nytcore-iphone-share&smprod=nytcore-iphone&_r=1.

‘As We May Think’ – part 2

In the first few chapters of “As we may think” Bush discusses the difficulties on how to create a record. He already envisioned solutions to shorten the process of creating and spreading research.

To make the record, we now push a pencil or tap a typewriter. Then comes the process of digestion and correction, followed by an intricate process of typesetting, printing, and distribution. To consider the first stage of the procedure, will the author of the future cease writing by hand or typewriter and talk directly to the record?[1]

After the invention of the Voder which emitted recognizable speech when typed to, the Vocoder did exactly the opposite. “Speak to it, and the corresponding keys move.” Another technology already in existence at the time Bush wrote his influential article, was the stenotype “which records in a phonetically simplified language”. “Combine these two elements, let the Vocoder run the stenotype, and the result is a machine which types when talked to.”[2] The technology that enables devices to respond to spoken commands called speech recognition, exists in both mobile phones and tablets, as well as laptops. Most common uses include dictation, search and giving commands to computers. Another example of speech recognition is Apple’s Siri, a personal assistant on their smartphones.

In the Humanities, and Digital Humanities especially, researchers have shown an interest in non-text materials because of the “massive increase in the quantity and availability of audiovisual (AV) materials and a rapid development in technology for handling such materials”.[3] Speech recognition is mostly linked to transcribing audio-visual materials, especially speech-to-text transcription.

Schematic model of humanities research with AV materials

Since speech recognition systems are so domain specific, they cannot handle every domain and “even speech from the same domain that differs from the ‘training’ data may be problematic”.[4]

Speech-to-text transcriptions have historically comprised an unpunctuated and unformatted stream of text. There has been considerable recent research into generating ‘richer’ transcriptions annotated with a variety of information that can be extracted from the audio signal and/or an imperfect transcription. […] Investigations have often used speech from only a small set of domains, such as broadcast news and conversational speech. Emotion-related work in particular is very preliminary.[5]

Strategies on how to expand the role of audiovisual media in Digital Humanities was discussed during the Digital Humanities Conference 2014 in Lausanne, at a workshop by researchers involved in AXES.[6] Research fields which might benefit from speech recognition include: film, television, radio, and oral history.

[1] Vannevar Bush, “As We May Think,” Atlantic Monthly, July 1945, http://www.theatlantic.com/magazine/archive/1945/07/as-we-may-think/303881/, chapter 3.

[2] Bush, “As Wy May Think,” chapter 3.

[3] Alan Marsden, Adrian Mackenzie and Adam Lindsay, “Tools for Searching, Annotation and Analysis of Speech, Music, Film and Video – A Survey,” Literary and Linguistic Computing, no. 22,4 (2007): 469-488, accessed January 9, 2017, doi: 10.1093/llc/fqm021.

[4] Marsden et. Al., “Tools for Searching”.

[5] Marsden et. Al., “Tools for Searching”.

[6] “AV in DH Special Interest Group,” Access to Audiovisual Archives, February 12, 2015, http://www.axes-project.eu/?p=2419.

‘As We May Think’ – part 1

When Vannevar Bush wrote As we may think in 1945 he could never have imagined the technologies existing today.[1] A few of his ideas became reality in some or other form but where he “urges that men of science should then turn to the massive task of making more accessible our bewildering store of knowledge”, his ideas actually affected a much larger population.[2] He came across one particular problem, namely that “publication has been extended far beyond our present ability to make real use of the record.”[3] Even in 1945 machines were relatively cheap and dependable, so according to Bush they would provide a solution. In his opinion a record could only be useful to science when it is continuously extended, stored, but “above all it must be consulted”.[4] First he discusses multiple solutions for recording knowledge, such as a combination of a vocoder and a stenotype to create a “machine which types when talked to”. He also imagined advanced arithmetical machines that perform 100 times present speeds or more.[5] Bush even describes what we would call data mining or machine learning.

In fact, every time one combines and records facts in accordance with established logical processes, the creative aspect of thinking is concerned only with the selection of data and the process to be employed and the manipulation thereafter is repetitive in nature and hence a fit matter to be relegated to the machine.[6]

Furthermore, he explains the difference between simple selection that examines every item and the selection mechanism of a telephone that narrows down its selection by classes and subclasses, represented by each digit. However both these selection methods use indexing, while the human mind “operates by association”.[7] He figures his memex will provide the solution, for it creates trails that tie multiple items together.[8]

Screen Shot 2017-05-08 at 13.57.57

Finally Bush concludes that not only new forms of encyclopedias will appear, with trails running through them, he also asks himself this: “Must we always transform to mechanical movements in order to proceed from one electrical phenomenon to another?” Imagine that instead of typing this document, a machine would intercept the electrical impulses and type, without the interference of the mechanical movement of my hands on the keyboard.[9]

[1] Vannevar Bush, “As We May Think,” Atlantic Monthly, July 1945, http://www.theatlantic.com/magazine/archive/1945/07/as-we-may-think/303881/.
[2] Bush, “As We May Think”, Introduction.
[3] Bush, “As We May Think”, Chapter 1.
[4] Bush, “As We May Think”, Chapter 2.
[5] Bush, “As We May Think”, Chapter 3.
[6] Bush, “As We May Think”, Chapter 4.
[7] Bush, “As We May Think”, Chapter 5-6.
[8] Bush, “As We May Think”, Chapter 7.
[9] Bush, “As We May Think”, Chapter 8.

Social (Media) Data: a gold mine for Digital Humanities?

Lev Manovich identifies two types of data used in social and cultural studies during the twentieth century: “‘surface data’ about lots of people and ‘deep data’ about a few individuals or small groups” (Manovich, 2012). An intermediate method is used in statistics, where a researcher chooses a sample to represent an entire country for example. Comparing this aproach to Photoshop, Manovich goes on to say:

A “pixel” that originally represented one person comes to represent one thousand people who are all assumed to behave in exactly the same way. (Manovich, 2012)

This is exactly what happened in polls preceding the recent Presidential Elections in the United States of America, and predicting that Hillary Clinton would win. Nate Silver’s FiveThirtyEight 2016 general election forecast predicted Donald Trump would only have a 28.2% chance of winning, although they estimated a 10% chance that Clinton would win the popular vote and lose the electoral college vote. Their model containing three versions needs quite a lengthy user guide, their polls-plus version of the model combining polls with an economic index and each version following four major steps:

  1. Collect, weight and average polls. – based on Pollster ratings.
  2. Adjust polls.
  3. Combine polls with demographic and (in the case of the polls-plus) economic data.
  4. Account for uncertainty and simulate the election thousands of times.

Nate Silver later defended his model in Why FiveThirtyEight Gave Trump A Better Chance Than Almost Anyone Else saying that:

People mistake having a large volume of polling data for eliminating uncertainty. […] the polls sometimes suffer from systematic error: Almost all of them are off in the same direction. (Silver, 2016)

The four objections Manovich states with regard to the rise of social media and new computational tools that can process massive amounts of data (Manovich, 2012) can help explain why The Polls Missed Trump.

The first objection Manovich describes, is the lack of availablity of data outside of the social media companies, specifically for transactional data (Manovich, 2012). In the case of the election polls, one of the recurrent errors is the nonresponse bias, or “failing to get supporters of one candidate to respond with the same enthusiasm as supporters of his opponent” as Carl Balik and Harry Enten stated in their article asking Pollsters Why.

The second objection Manovich formulates, is the lack of authenticity, since communications over social media and digital footprints are often carefully curated and systematically managed (Manovich, 2012). However, “several pollsters rejected the idea that Trump voters were too shy to tells [sic] pollsters whom they were supporting” (Balik and Enten, 2016). However, automated-dialer calls which used a recorded voice registered more Trump voters as opposed to live-interviews (Balik and Enten, 2016).

Manovich also raises a third objection, referring to the size versus depth issue since different data leads to different questions, patterns, and insights (Manovich, 2012). In the aftermath of the elections, many explinations for why the polls were off took the stage in several articles. Even on the FiveThirtyEight website, I found at least three articles offering a different point of view, even contradicting each other: Jed Kolko explaining Trump Was Stronger Where The Economy Is Weaker, Carl Bialik stating that Voter Turnout Fell, Especially In States That Clinton Won, but also claiming No, Voter Turnout Wasn’t Way Down From 2012, whereas Clare Malone blamed the outcome on the sentiment of Americans Don’t Trust Their Institutions Anymore. The differing approaches even amonst the same redaction team shows how some refer to individuals’ emotions, while others state voter turnout or a weak economy.

Finally Manovich’s fourth objection points out the need for specialized expertise especially in computer science, statistics and data mining, needed to work on large data sets and especially combining the data as Nate Silver did for his general election forecast. Even though he clearly has a well-defined method, adding several factors and ranking pollsters based on historical data on their accuracy, polling needs to “get more comfortable with uncertainty” (Balik and Enten, 2016). One of the people they interviewed even went as far as to state that “the incentives now favor offering a single number that looks similar to otheer polls instead of really trying to report on the many possible campaign elements that could affect the outcome. Certainty is rewarded, it seems” (Balik and Enten, 2016).

If Digital Humanists want to make sure that:

The rise of social media, along with new computational tools that can process massive amounts of data, makes possible a fundamentally new approach to the study of human beings and society. (Manovich, 2012)

We need to change how students in humanities are being educated, something the Advanced Master in Digital Humanities of the KU Leuven is certainly trying to achieve.


Manovich, Lev. “Trending: The Promises and the Challenges of Big Social Data.” Debates in the Digital Humanities. Minneapolis, MN: University of Minnesota Press, 2012.

Silver, Nate. “2016 Election Forecast.” FiveThirtyEight, November 8, 2016. https://projects.fivethirtyeight.com/2016-election-forecast/#plus.

Balik, Carl, and Enten, Harry. “The Polls Missed Trump. We Asked Pollsters Why.” FiveThirtyEight, November 9, 2016.  http://fivethirtyeight.com/features/the-polls-missed-trump-we-asked-pollsters-why/.