1.5 Failing Productively
We have found that students are very nervous about doing digital work because, ‘what if it breaks?’ and ‘what if I can’t get it to work?’ This is perhaps a result of high-stakes testing and the ways we as educators have inculcated all-or-nothing grading in our courses. There is no room for experimentation, no room for trying things out when the final essay is worth 50% of the course grade, for instance. Playing it safe is a valid response in such an environment. A better approach, from a pedagogical point of view, is to encourage students to explore and try things out, with the grading being focused on documenting the process rather than on the final outcome. We will point the reader to Daniel Paul O’Donnel’s concept of the unessay; more details behind the link.
Our emphasis on open notebooks has an ulterior motive, and that is to surface the many ways in which digital work sometimes fails. We want to introduce to you the idea of ‘failing productively’ because there is such a thing as an unproductive failure. There are ways of failing that do not do us much good, and we need - especially with digital work - to consider what ‘fail’ actually can mean. In the technology world, there are various slogans surrounding the idea of ‘fail’ - fail fast; move fast and break things; fail better; for instance.
When we talk about ‘failing productively’ or failing better, it is easy for critics of digital archaeology (or the digital humanities; see Allington (2016) but contra: B. Greenspan (2016)) to connect digital work to the worst excesses of the tech sector. But again, this is to misunderstand what ‘fail’ should mean. The understanding of many technology startup folks that valorizing failure as a license to burn through funding has caused a lot of harm. The technology sector failed to understand the humanistic implication of the phrase, and instead took it literally to mean ‘a lack of success is itself the goal’.
Perhaps a better understanding of what ‘fail’ should mean is as something akin to what Nassim Taleb called ‘antifragility’. The fragile thing breaks under stress and randomness; the resilient thing stays the same; and the anti-fragile thing actually gets stronger as it is exposed to randomness. Kids’ bones for instance need to be exposed to shocks in order to get stronger. Academia’s systems are ‘fragile’ in that they do not tolerate fail; they are to a degree resilient, but they are not ‘antifragile’ in Taleb’s sense. The idea that ‘fail’ can break that which is ‘fragile’ is part of the issue here. So silicon valley really means ‘fail’ in the sense of ‘antifragile’ but they frequently forget that; academia sees ‘fail’ as the breaking of something fragile; and so the two are at loggerheads. Indeed, the rhetorical moves of academe often frame weak results - fails - as actual successes, thus making the scholarship fragile (hence the fierceness of academic disputes when results are challenged, sometimes). To make scholarship anti-fragile - to extract the full value of a fail and make it be productive, we need remember only one thing:
A failure shared is not a failure.
Not every experiment results in success; indeed, the failures are richer experiences because as academics we are loathe to say when something did not work – but how else will anybody know that a particular method, or approach, is flawed? If we try something, it does not work, and we then critically analyze why that should be, we have in fact entered a circle of positive feedback. This perspective is informed by our research into game based learning. A good game keeps the challenges just ahead of the player’s (student’s) ability, to create a state of ‘flow’. Critical failure is part of this: too hard, and the player quits; too easy, and the player drops the controller in disgust. The ‘fails’ that happen in a state of flow enable the player to learn how to overcome them. Perhaps if we can design assessment to tap into this state of flow, then we can create the conditions for continual learning and growth (see for instance Kee, Graham, et al. 2009). As in our teaching, so too in our research. Presner writes,
Digital projects in the Humanities, Social Sciences, and Arts share with experimental practices in the Sciences a willingness to be open about iteration and negative results. As such, experimentation and trial-and-error are inherent parts of digital research and must be recognized to carry risk. The processes of experimentation can be documented and prove to be essential in the long-term development process of an idea or project. White papers, sets of best practices, new design environments, and publications can result from such projects and these should be considered in the review process. Experimentation and risk-taking in scholarship represent the best of what the university, in all its many disciplines, has to offer society. To treat scholarship that takes on risk and the challenge of experimentation as an activity of secondary (or no) value for promotion and advancement, can only serve to reduce innovation, reward mediocrity, and retard the development of research. PRESSNER 2012 cite
1.5.1 A taxonomy of fails
There are fails, and then there are fails. Croxall and Warnick identify a taxonomy of four kinds of failure in digital work:
- Technological Failure
- Human Failure
- Failure as Artifact
- Failure as Epistemology
…to which we might add a fifth kind of fail:
- Failing to Share
The first is the simplest: something simply did not work. The code is buggy, dust and grit got into the fan and the hardware seized. The second, while labeled ‘human failure’ really means that the context, the framework for encountering the technology was not erected properly, leading to a failure to appreciate what the technology could do or how it was intended to be used. This kind of failure can also emerge when we ourselves are not open to the possibilities or work that the technology entails. The next two kinds of failure emerge from the first in that they are ways of dealing with the first two kinds of failure. ‘Failure as Artifact’ means that we seek out examples of failures as things to study, working out the implications of why something did not work. Finally, ‘Failure as Epistemology’ purposely builds the opportunity to fail into the research design, such that each succeeding fail (of type 1 or type 2) moves us closer to the solution that we need. The first two refer to what happened; the second two refer to our response and how we react to the first two (if we react at all). The key to productive failure as we envision it is to recognize when one’s work is suffering from a type 1 or type 2 fail, and to transform it to a type 3 or type 4. Perhaps there should be a fifth category though, a failure to share. For digital archaeology to move forward, we need to know where the fails are and how to move beyond them, such that we move forward as a whole. Report not just the things that work, but also the fails. That is why we keep open research notebooks.
Lets consider some digital projects that we have been involved in, and categorize the kinds of fails they suffered from. We turn first to the HeritageCrowd project that Graham established in 2011. This project straddled community history and cultural history in a region poorly served by the internet. It was meant to crowd-source intangible heritage via a combination of web-platform and telephony. People could phone in with stories, which were automatically transcribed and added to a web map. The first write-up of the project was published just as the project started to get underway (Graham, Massie, and Feuerherm 2013). What happened next, is of interest here.
The project website was hacked, knocked offline and utterly compromised. The project failed.
Why did it fail? It was a combination of at least four distinct problems:
- poor record keeping of the installation process of the various technologies that made it work
- computers talk to other computers to persuade them to do things. In this case, one computer injected malicious code into the technologies Graham was using to map the stories
- Graham ignored security warnings from the main platform’s maintainers
- Backups and versioning: there were none.
Graham’s fails here are of both type 1 and type 2. In terms of type 2, his failure to keep careful notes on how the various pieces of the project were made to fit together meant that he lacked the framework to understand how he had made the project vulnerable to attack. The actual fail point of the attack - that’s a type 1 fail, but could have been avoided if Graham had participated more in the spirit of open software development, e.g., read the security warnings in the developer forum! When Graham realized what had happened to his project, he was faced with two options. One option, having already published a piece on the project that hailed its successes and broad lessons for crowdsourcing cultural heritage, would have been to quietly walked away from the project (perhaps putting up a new website averring that version 2.0 was coming, pending funding). The other option was to warn folks to beef up the security and backups for their own projects. At the time, crowdsourcing was very much an academic fashion and Graham opted for the second option in that spirit. In doing this, the HeritageCrowd project became a fail of type 3, an artifact for study and reflection. The act of blogging his post-mortem makes this project also an instance of type 5, or the communication of the fail. It is worth pointing out here that the public sharing of this failure is not without issues. As we indicated in the Open Notebook Research & Scholarly Communication section, the venue for sharing what hasn’t worked and the lessons learned is highly contingent on many factors. Graham, as a white male tenure-track academic on the web in 2012, could share openly that things had not worked. As the web has developed in the intervening years, and with the increasing polarization of web discourse into broad ideological camps, it may well not be safe for someone in a more precarious position to share so openly. One must keep in mind one’s own situation and adapt what we argue for here accordingly. Sharing fails can be done with close colleagues, students, specialist forums and so on.
If we are successful with ODATE, and the ideas of productive fail begin to permeate more widely in the teaching of digital archaeology, then a pedagogy that values fail will with time normalize such ‘negative results’. We are motivated by this belief that digital archaeology is defined by the productive, pedagogical fail. It is this aspect of digital archaeology that also makes it a kind of public archaeology, and that failing in public can be the most powerful thing that digital archaeology offers the wider field.
We implore you to do your research so that others can retrace your steps; even a partial step forward is a step forward! When you find a colleague struggling, give positive and meaningful feedback. Be open about your own struggles, but get validation of your skills if necessary. Build things that make you feel good about your work into your work.
1.5.2 Exercises
What are the nature of your own fails? Reflect on a ‘fail’ that happened this past year. Where might it fit on the taxonomy? Share this fail via your open notebook, blog, or similar, with your classmates. How can your classmate convert their fails into types three or four?
References
Allington, Brouillette, D. 2016. “Neoliberal Tools (and Archives): A Political History of Digital Humanities.” LA Review of Books. https://lareviewofbooks.org/article/neoliberal-tools-archives-political-history-digital-humanities/#!
Greenspan, B. 2016. “The Scandal of Digital Humanities.” Hyperlab Blog. https://carleton.ca/hyperlab/2018/the-scandal-of-digital-humanities/.
Graham, Shawn, Guy Massie, and Nadine Feuerherm. 2013. “The Heritagecrowd Project: A Case Study in Crowdsourcing Public History.” Writing History in the Digital Age.