‘As We May Think’ – part 4

Bush moves on from describing several recording techniques to problems of storage and of processing data. As he puts it: Much needs to occur, however, between the collection of data and observations, the extraction of parallel material from the existing record, and the final insertion of new material into the general body of the common record.[1] There exists a distinction between creative thought and repetitive thought, which can be taken over by machines. Bush already envisioned such a powerful and fast electrical machine.

Moreover they will be far more versatile than present commercial machines, so that they may readily be adapted for a wide variety of operations. They will be controlled by a control card or film, they will select their own data and manipulate it in accordance with the instructions thus inserted, they will perform complex arithmetical computations at exceedingly high speeds, and they will record results in such form as to be readily available for distribution or for later further manipulation.[2]

First of all, the control card could be considered computer hardware and seems to refer to the Central Processing Unit or CPU. The control section “determines the sequence of operations”, whereas the Arithmetic-Logic Unit or ALU is responsible for the basic arithmetic and logic operations. In some way the CPU selects its own data and manipulates it according to instructions such as arithmetical computations and finally stores the result. This is called the “fetch-decode-execute” cycle. [3]

Since Vannevar Bush intended his memex as “a future device for individual use, which is a sort of mechanized private file and library”, it can be compared to what we call a Personal Computer.[4] In order to store “all his books, records and communications” his memex would use improved microfilm. The main memory of a computer has however evolved massively over time. In 1952 the magnetic-core memory appeared where two wires changed the direction of the magnets and a third wire detected the orientation. Only in 1971 the first integrated circuit memory chip appeared with a transistor-capacitor combination. The random-access memory or RAM could be either dynamic (DRAM) or static (SRAM) depending on whether the values had to be refreshed periodically or not. Since SRAM is more costly but faster, it is used for the cache memory of the CPU. Computers also store data and programs that are not in use all the time in secondary memory such as magnetic discs, CD-ROMs and DVD-ROMs. [5]

The memex also had to be “mechanized so that it may be consulted with exceeding speed and flexibility”.[6] To continue the comparison between the memex and PC, let’s take a closer look at the description of this new machine Bush envisioned.

It consists of a desk, and while it can presumably be operated from a distance, it is primarily the piece of furniture at which he works. On the top are slanting translucent screens, on which material can be projected for convenient reading. There is a keyboard, and sets of buttons and levers. (…)

There is, of course, provision for consultation of the record by the usual scheme of indexing. If the user wishes to consult a certain book, he taps its code on the keyboard, and the title page of the book promptly appears before him, projected onto one of his viewing positions. (…)

As he has several projection positions, he can leave one item while he calls up another. He can add marginal notes and comments, taking advantage of one possible type of dry photography, and it could even be arranged so that he can do this by a stylus scheme, (…)[7]

Most the functionalities Bush enumerates would now be called computer peripherals “used to input information and instructions into a computer for storage and processing and to output the processed data”. One important input device is the keyboard that uses mechanical or electromechanical switches to change the flow of current. A microprocessor then needs to interpret the changes and send a signal to the computer. In the memex levers are used to browse books, which can be compared to scrolling nowadays. These levers could be considered today’s mouse, whether mechanical or optical. A mechanical mouse uses a rubber to measure horizontal and vertical movement and translates it to cursor movement on the monitor, whereas an optical mouse uses a light beam and camera to translate motion. The monitor or display serves as the primary output device and it employs the same cathode-ray tubes or CRTs as television and radar systems.[8] The stylus Bush envisioned became widely used only after the invention of touch-screens which are now not only employed in portable devices such as smartphones and tablets, but also in 2-in-1 laptops. Finally, to access records by indexing could be similar to the file structure in computers that refer to memory locations.

[1] Bush. As We May Think. Chapter 3.

[2] Bush. As We May Think. Chapter 3.

[3] David Hemmendinger. 2016. Central Processing Unit. Encyclopaedia Britannica. Britannica Academic. Consulted 3 march 2016. http://academic.eb.com.kuleuven.ezproxy.kuleuven.be/EBchecked/topic/130429/computer.

[4] Bush. As We May Think. Chapter 6.

[5] David Hemmendinger. 2016. Main Memory. Secondary Memory. Encyclopaedia Britannica. Britannica Academic. Consulted 3 march 2016. http://academic.eb.com.kuleuven.ezproxy.kuleuven.be/EBchecked/topic/130429/computer.

[6] Bush. As We May Think. Chapter 6.

[7] Bush. As We May Think. Chapter 6.

[8] William Morton Pottenger. 2016. Peripherals. Encyclopaedia Britannica. Britannica Academic. http://academic.eb.com.kuleuven.ezproxy.kuleuven.be/EBchecked/topic/130429/computer.

Advertisements

‘As We May Think’ – part 3

Besides creating audio records instead of writing information down, an image says more than a thousand words. Vannevar Bush already envisioned an entirely different experience for researchers.

One can now picture a future investigator in his laboratory. His hands are free, and he is not anchored. As he moves about and observes, he photographs and comments. Time is automatically recorded to tie the two together.[1]

Although still images already provide a lot of information together with their tags, commenting while photographing might be easier using a video camera. Most devices such as laptops, tablets and mobile phones already contain cameras, but a Go Pro or other hands free camera increases the ease of use. However video production consists of planning during pre-production, capturing during production and processing in the post-production phase.[2] Bush was already aware of these phases.

As he ponders over his notes in the evening, he again talks his comments into the record. His typed record, as well as his photographs, may both be in miniature, so that he projects them for examination.[3]

The process of recording information is constantly improving, especially after the invention of an optical head-mounted display such as Google Glasses. It displays information and communicates with the internet using natural language voice commands. Furthermore the glasses include a touchpad and a camera which can take photos or record videos just by voicing the command “record a video”.[4] This technology can improve the research process since it “1) provides workflow guidance to the user, 2) supports hands-free operation, 3) allows the users to focus on their work, and 4) enables an efficient way for collaborating with a remote expert”.[5] Although the glasses were already tested in the healthcare and industry maintenance fields, the Glass Explorer program ceased to exist. Ivy Ross and Tony Fadell took over, but the release date of their new product remains unknown.[6]

[1] Bush. “As We May Think,” Chapter 3.

[2] Dustin Freeman, Stephanie Santosa, Fanny Chevalier, Ravin Balakrishnan and Karan Sing, “LACES: live authoring through compositing and editing of streaming video,” Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2014, 1207-1216, doi: 2556288.2557304.

[3] Bush, “As We May Think,” Chapter 3.

[4] “Google Glass,” Wikipedia, accessed March 7, 2016. https://en.wikipedia.org/wiki/Google_Glass.

[5] Xianjun Sam Zheng, Patrik Matos da Silva, Cedric Foucault Siddharth Dasari, Meng Yuan, Stuart Goose. “Wearable Solution for Industrial Maintenance,” Proceedings of the 33rd Annual ACM Conference Extended Abstract on Human Factors in Computing Systems, 2015, 311-314, doi: 702613.2725442.

[6] Nick Bilton. “Why Google Glass Broke,” The New York Times, accessed March 7, 2016, http://www.nytimes.com/2015/02/05/style/why-google-glass-broke.html?smid=nytcore-iphone-share&smprod=nytcore-iphone&_r=1.

‘As We May Think’ – part 2

In the first few chapters of “As we may think” Bush discusses the difficulties on how to create a record. He already envisioned solutions to shorten the process of creating and spreading research.

To make the record, we now push a pencil or tap a typewriter. Then comes the process of digestion and correction, followed by an intricate process of typesetting, printing, and distribution. To consider the first stage of the procedure, will the author of the future cease writing by hand or typewriter and talk directly to the record?[1]

After the invention of the Voder which emitted recognizable speech when typed to, the Vocoder did exactly the opposite. “Speak to it, and the corresponding keys move.” Another technology already in existence at the time Bush wrote his influential article, was the stenotype “which records in a phonetically simplified language”. “Combine these two elements, let the Vocoder run the stenotype, and the result is a machine which types when talked to.”[2] The technology that enables devices to respond to spoken commands called speech recognition, exists in both mobile phones and tablets, as well as laptops. Most common uses include dictation, search and giving commands to computers. Another example of speech recognition is Apple’s Siri, a personal assistant on their smartphones.

In the Humanities, and Digital Humanities especially, researchers have shown an interest in non-text materials because of the “massive increase in the quantity and availability of audiovisual (AV) materials and a rapid development in technology for handling such materials”.[3] Speech recognition is mostly linked to transcribing audio-visual materials, especially speech-to-text transcription.

Picture1
Schematic model of humanities research with AV materials

Since speech recognition systems are so domain specific, they cannot handle every domain and “even speech from the same domain that differs from the ‘training’ data may be problematic”.[4]

Speech-to-text transcriptions have historically comprised an unpunctuated and unformatted stream of text. There has been considerable recent research into generating ‘richer’ transcriptions annotated with a variety of information that can be extracted from the audio signal and/or an imperfect transcription. […] Investigations have often used speech from only a small set of domains, such as broadcast news and conversational speech. Emotion-related work in particular is very preliminary.[5]

Strategies on how to expand the role of audiovisual media in Digital Humanities was discussed during the Digital Humanities Conference 2014 in Lausanne, at a workshop by researchers involved in AXES.[6] Research fields which might benefit from speech recognition include: film, television, radio, and oral history.

[1] Vannevar Bush, “As We May Think,” Atlantic Monthly, July 1945, http://www.theatlantic.com/magazine/archive/1945/07/as-we-may-think/303881/, chapter 3.

[2] Bush, “As Wy May Think,” chapter 3.

[3] Alan Marsden, Adrian Mackenzie and Adam Lindsay, “Tools for Searching, Annotation and Analysis of Speech, Music, Film and Video – A Survey,” Literary and Linguistic Computing, no. 22,4 (2007): 469-488, accessed January 9, 2017, doi: 10.1093/llc/fqm021.

[4] Marsden et. Al., “Tools for Searching”.

[5] Marsden et. Al., “Tools for Searching”.

[6] “AV in DH Special Interest Group,” Access to Audiovisual Archives, February 12, 2015, http://www.axes-project.eu/?p=2419.

‘As We May Think’ – part 1

When Vannevar Bush wrote As we may think in 1945 he could never have imagined the technologies existing today.[1] A few of his ideas became reality in some or other form but where he “urges that men of science should then turn to the massive task of making more accessible our bewildering store of knowledge”, his ideas actually affected a much larger population.[2] He came across one particular problem, namely that “publication has been extended far beyond our present ability to make real use of the record.”[3] Even in 1945 machines were relatively cheap and dependable, so according to Bush they would provide a solution. In his opinion a record could only be useful to science when it is continuously extended, stored, but “above all it must be consulted”.[4] First he discusses multiple solutions for recording knowledge, such as a combination of a vocoder and a stenotype to create a “machine which types when talked to”. He also imagined advanced arithmetical machines that perform 100 times present speeds or more.[5] Bush even describes what we would call data mining or machine learning.

In fact, every time one combines and records facts in accordance with established logical processes, the creative aspect of thinking is concerned only with the selection of data and the process to be employed and the manipulation thereafter is repetitive in nature and hence a fit matter to be relegated to the machine.[6]

Furthermore, he explains the difference between simple selection that examines every item and the selection mechanism of a telephone that narrows down its selection by classes and subclasses, represented by each digit. However both these selection methods use indexing, while the human mind “operates by association”.[7] He figures his memex will provide the solution, for it creates trails that tie multiple items together.[8]

Screen Shot 2017-05-08 at 13.57.57
THE MEMEX

Finally Bush concludes that not only new forms of encyclopedias will appear, with trails running through them, he also asks himself this: “Must we always transform to mechanical movements in order to proceed from one electrical phenomenon to another?” Imagine that instead of typing this document, a machine would intercept the electrical impulses and type, without the interference of the mechanical movement of my hands on the keyboard.[9]

[1] Vannevar Bush, “As We May Think,” Atlantic Monthly, July 1945, http://www.theatlantic.com/magazine/archive/1945/07/as-we-may-think/303881/.
[2] Bush, “As We May Think”, Introduction.
[3] Bush, “As We May Think”, Chapter 1.
[4] Bush, “As We May Think”, Chapter 2.
[5] Bush, “As We May Think”, Chapter 3.
[6] Bush, “As We May Think”, Chapter 4.
[7] Bush, “As We May Think”, Chapter 5-6.
[8] Bush, “As We May Think”, Chapter 7.
[9] Bush, “As We May Think”, Chapter 8.

Social (Media) Data: a gold mine for Digital Humanities?

Lev Manovich identifies two types of data used in social and cultural studies during the twentieth century: “‘surface data’ about lots of people and ‘deep data’ about a few individuals or small groups” (Manovich, 2012). An intermediate method is used in statistics, where a researcher chooses a sample to represent an entire country for example. Comparing this aproach to Photoshop, Manovich goes on to say:

A “pixel” that originally represented one person comes to represent one thousand people who are all assumed to behave in exactly the same way. (Manovich, 2012)

This is exactly what happened in polls preceding the recent Presidential Elections in the United States of America, and predicting that Hillary Clinton would win. Nate Silver’s FiveThirtyEight 2016 general election forecast predicted Donald Trump would only have a 28.2% chance of winning, although they estimated a 10% chance that Clinton would win the popular vote and lose the electoral college vote. Their model containing three versions needs quite a lengthy user guide, their polls-plus version of the model combining polls with an economic index and each version following four major steps:

  1. Collect, weight and average polls. – based on Pollster ratings.
  2. Adjust polls.
  3. Combine polls with demographic and (in the case of the polls-plus) economic data.
  4. Account for uncertainty and simulate the election thousands of times.

Nate Silver later defended his model in Why FiveThirtyEight Gave Trump A Better Chance Than Almost Anyone Else saying that:

People mistake having a large volume of polling data for eliminating uncertainty. […] the polls sometimes suffer from systematic error: Almost all of them are off in the same direction. (Silver, 2016)

The four objections Manovich states with regard to the rise of social media and new computational tools that can process massive amounts of data (Manovich, 2012) can help explain why The Polls Missed Trump.

The first objection Manovich describes, is the lack of availablity of data outside of the social media companies, specifically for transactional data (Manovich, 2012). In the case of the election polls, one of the recurrent errors is the nonresponse bias, or “failing to get supporters of one candidate to respond with the same enthusiasm as supporters of his opponent” as Carl Balik and Harry Enten stated in their article asking Pollsters Why.

The second objection Manovich formulates, is the lack of authenticity, since communications over social media and digital footprints are often carefully curated and systematically managed (Manovich, 2012). However, “several pollsters rejected the idea that Trump voters were too shy to tells [sic] pollsters whom they were supporting” (Balik and Enten, 2016). However, automated-dialer calls which used a recorded voice registered more Trump voters as opposed to live-interviews (Balik and Enten, 2016).

Manovich also raises a third objection, referring to the size versus depth issue since different data leads to different questions, patterns, and insights (Manovich, 2012). In the aftermath of the elections, many explinations for why the polls were off took the stage in several articles. Even on the FiveThirtyEight website, I found at least three articles offering a different point of view, even contradicting each other: Jed Kolko explaining Trump Was Stronger Where The Economy Is Weaker, Carl Bialik stating that Voter Turnout Fell, Especially In States That Clinton Won, but also claiming No, Voter Turnout Wasn’t Way Down From 2012, whereas Clare Malone blamed the outcome on the sentiment of Americans Don’t Trust Their Institutions Anymore. The differing approaches even amonst the same redaction team shows how some refer to individuals’ emotions, while others state voter turnout or a weak economy.

Finally Manovich’s fourth objection points out the need for specialized expertise especially in computer science, statistics and data mining, needed to work on large data sets and especially combining the data as Nate Silver did for his general election forecast. Even though he clearly has a well-defined method, adding several factors and ranking pollsters based on historical data on their accuracy, polling needs to “get more comfortable with uncertainty” (Balik and Enten, 2016). One of the people they interviewed even went as far as to state that “the incentives now favor offering a single number that looks similar to otheer polls instead of really trying to report on the many possible campaign elements that could affect the outcome. Certainty is rewarded, it seems” (Balik and Enten, 2016).

If Digital Humanists want to make sure that:

The rise of social media, along with new computational tools that can process massive amounts of data, makes possible a fundamentally new approach to the study of human beings and society. (Manovich, 2012)

We need to change how students in humanities are being educated, something the Advanced Master in Digital Humanities of the KU Leuven is certainly trying to achieve.

Bibliography

Manovich, Lev. “Trending: The Promises and the Challenges of Big Social Data.” Debates in the Digital Humanities. Minneapolis, MN: University of Minnesota Press, 2012.

Silver, Nate. “2016 Election Forecast.” FiveThirtyEight, November 8, 2016. https://projects.fivethirtyeight.com/2016-election-forecast/#plus.

Balik, Carl, and Enten, Harry. “The Polls Missed Trump. We Asked Pollsters Why.” FiveThirtyEight, November 9, 2016.  http://fivethirtyeight.com/features/the-polls-missed-trump-we-asked-pollsters-why/.