1 Introduction

This morning, during my regular coffee and newspaper ritual, I read an interesting article: ‘Amazon scraps secret AI recruiting tool that showed bias against women’ (Dastin 2018). In one of its development centres, Amazon’s team of experts has designed an Artificial Intelligence (AI) software ‘to review job applicants’ resumes with the aim of mechanizing the search for top talent’ and ‘taught’ the software using 10-year archive of recruitment at Amazon. After it started working, at the surprise of researchers, the software showed strong bias against women. To resolve the problem, ‘Amazon edited the programs to make them neutral to these particular terms. But that was no guarantee that the machines would not devise other ways of sorting candidates that could prove discriminatory’. Therefore, they scraped the AI completely (Dastin 2018). This borderline sensationalist story, which seemed to amuse to death a group of freshers at the next table, carries a deep message. These days our society is increasingly reliant on collecting large amounts of data (the so-called Big Data) and processing this data using automated systems of various hues and shapes. However, probably for the first time in history, these systems (which are often called AIs), function in ways which cannot be predicted even by their designers and makers. After they designed and ‘taught’ their AI, Amazon’s engineers could not predict its later behaviour or even avoid (perhaps a different type) of bias. Amazon AI’s bias against women is not a technical glitch, or even an error in design – AI’s ‘independent mind’ is a feature built in the very essence of its workings.

A television set may reproduce many programs, but the choice of program is firmly in the hands of the viewer; during commercial breaks, everyone watches the same advertisements. Streaming services such as YouTube use recommendation systems to direct us towards watching certain content and offer personalized advertisements, yet we can always choose to watch something else. Critical media literacy is important, because it helps us navigate and produce these media. However, once we know what we would like to do, we are technically in full control. As viewers, we can change TV program, delete our browsing history, open a fresh YouTube account, etc., and escape the long arm of recommender systems. As producers, we can make a video, upload it to YouTube and make sure that it will show exactly what we wanted. However, Amazon’s AI has gone one significant step further – shaped by people, ‘taught’ by data about people, it makes ‘own’ choices and decisions. If YouTube videos worked similarly to Amazon’s AI recruitment tool, they would self-edit after we upload them and use recommender systems to offer themselves to audiences which they find appropriate. When similar systems are put in place, say, for determining our credit scores or health insurance, then AI’s get to make significant impact to human lives. This is the essence of what Jeremy Knox calls the age of algorithmic cultures, ‘in which automated computer operations process data in such a way as to significantly shape the contemporary categorizing and privileging of knowledge, places, and people’ (Knox 2015: 5).

Critical media literacy addresses some of these challenges in rapidly developing areas such as data literacy (D’Ignazio and Bhargav 2015; Koltay 2015, just to mention a few). However, functioning of computer systems is based on two equally important pillars: input data and system architecture. This article examines recent advances in data literacy with an accent to Big Data literacy. It moves on to internal workings of contemporary AI’s and associated concepts such as machine learning, neural networks, and deep learning. Finally, it explores the complex interplay between Big Data and AI and teases out some challenges for postdigital critical media literacy.

2 Data Literacy

Data literacy is a relative newcomer in literacy studies; these days, it is ‘in search of a name and identity’ (Koltay 2014: 401). Data literacy ‘cuts across disciplinary boundaries’ and traditional workplace roles; in the academia, for instance, ‘it is often difficult to separate data-related skills needed to become a successful researcher and to work as a data specialist’ (Koltay 2014: 404). In the context of critical media literacy D’Ignazio and Bhargav argue that ‘Big Data has an empowerment problem’ because of its four main characteristics:

1. – Lack of Transparency: The data about people’s interactions with the world is generally collected with only token approval, if any at all, from the user. This denies the subject awareness that their actions are being recorded at the time the actions occur.
2. – Extractive Collection: The data is collected by third parties and is not meant for observation or consumption by the people it is collected from (or about). This denies the subject agency in the data collection mechanism and interaction opportunities with the collector.
3. – Technological Complexity: The data is analyzed with a variety of advanced algorithmic techniques, and discussed with highly technical jargon. This denies the subject an understanding of how any results were achieved, and how they might be critiqued.
4. – Control of Impact: The data is used by the collector to make decisions that have consequences for the subject(s). This denies the subject participation in decisions that affect them. (D’Ignazio and Bhargava, 2015, emphasis from the original)

This list can be expanded by recent advances in educational research. Standardized tests, which are now routinely used in various contexts from admissions to final exams, produce large datasets about students – and these datasets suffer from various biases. Looking at ‘the role of various psychometric practices and testing theories, in particular item response theory, and their ability to link literacy practices and calculable psychological constructs’ Cormac O’Keefe suggests that large-scale digital assessments such as piaac do not merely produce data about ability – more importantly, they ‘perform the concept of ability into being’ (O’Keefe 2017:133). Ben Williamson looks into power relationships behind the development of educational data science. He asks an important question, ‘who owns big data?’ and shows its direct links to the broader question ‘who owns educational research?’ (Williamson 2016). In his analysis of ‘the Lytics Lab, Stanford University’s laboratory for research and development in learning analytics, and the Center for Digital Data, Analytics and Adaptive Learning, a big data research centre of the commercial education company Pearson’, he makes ‘an important central argument that educational data science has moved from non-profit academic laboratories to for-profit companies’ (Williamson 2017: 105).

In our recent book, Michael Peters and I argue that big data is crucial for enabling the digital logic that drives the single technical system of ‘algorithmic capitalism’ (Peters and Jandrić 2018: 32; see also Peters 2012; Peters and Bulut 2011; Braidotti 2015). Christian Fuchs describes a similar notion of ‘transnational informational capitalism’ (Fuchs 2011), and Jodi Dean speaks of ‘communicative capitalism’ in which ‘big data is interesting because of the way in which every kind of communicative interaction generates metadata consisting of location data, different layers of contacts and networks, and links between them. Now all our social substance is available to be enclosed, analyzed, and sold off’ (Dean, Medak, and Jandrić 2018). Astrid Mager examines ‘algorithmic ideology’, which is inscribed in data produced and used by computer code and broader computational logics, and concludes that ‘a fundamental debate about where to draw boundaries between the state and the market, how to set limits for corporate players, and how to sustain social justice is needed’ (Mager 2014: 37). Today’s (big) data is far from neutral, as ‘the complex systems of data production and representation co-constitute the very systems they purport to describe, and in this process, they often embed, replicate or reinforce pre-existing attitudes and prejudices’ (Jones 2018: 49). In the postdigital age, data actively co-constitutes our reality.

Emerging attempts in this area, such as digital humanities, have already shown some good results (see Jandrić 2017: 131). However, understanding of the Web as a dynamic system of humans and machines has also brought about many questions about the intersections between the material and the social worlds ‘where the human teacher’s agency comes up against the workings of data to conduct another, and different, kind of teaching which is neither human not machinic but some kind of gathering of the two’ (Jandrić 2017: 206). This gives rise to various sociomaterialist approaches which ‘conceptualise knowledge and capacities as being emergent from the webs of interconnections between heterogeneous entities, both human and nonhuman’ and ‘offer the prospect of being able to integrate the material technologies and media found in networked learning into a framework that encompasses people and machines in a symmetrical way’ (Jones 2018: 47). However, not everyone agrees with attempts at creating such frameworks. In June 2002 Steve Fuller and Bruno Latour staged a popular public debate with the following motion: ‘A strong distinction between humans and non-humans is no longer required for research purposes’ (Barron 2003: 78). By this day, the debate has not arrived even close to resolution (Fuller and Jandrić 2018). However, it clearly shows that the question of Big Data reaches all the way to fundamental questions about the changing relationships between humans and machines while they co-create the world as we know it.

Big Data is educators’ friend, because it creates the new window of opportunity including but not limited to educational Web science. Big Data is also educators’ foe, because of its in-built problems of representation including but not limited to the tendency to replicate and reinforce ideologies while presenting itself as fair (as in the case of standardized tests) and ideologically neutral (as in the case of educational research). Big Data is the current research frontier in areas from assessment theory to epistemology and ontology, and Big Data literacy – through its direct impact to the ways we produce, collect, and structure data – is an inherent part of this frontier.

3 AI Literacy

These days, AI is often called bombastic names such as ‘the next digital frontier’ (Bughin et al., 2017). However, computers have been around for a while, so we first need to establish what makes AI so special in relation to older digital technologies. According to Liza Daly, ‘artificial intelligence is the umbrella term for the entire field of programming computers to solve problems. I would distinguish this from software engineering, where we program computers to perform tasks.’ (Daly 2017) This simple definition describes an important paradigm change in inner workings of the computer. Traditional computers, including the most sophisticated expert systems of yesterday, consisted of long lines of code which determined their behaviour: for every input, such systems would do predetermined calculations and provide an output. In contrast, AI systems are provided with some initial rules of behaviour, and then they are ‘taught’ by large datasets. Then, computer independently establishes various connections between input data and produces ‘intelligent’ solutions to new problems in non-predetermined ways. This is the essence of machine learning, which is broadly defined as ‘the science of getting computers to act without being explicitly programmed’ (Ng 2018). Recent developments in machine learning are predominantly in areas of neural networks and deep learning. Neural networks are specific organisational model for machine learning, loosely modelled on neurons in human brains, where different parts of the network specializes for different tasks; deep learning is a class of neural networks with layered networked architecture.

AI systems do not only embed, replicate or reinforce attitudes or prejudices found in data – more importantly, they also recombine them and produce new biases. Creators and researchers of AI cannot directly predict or interfere with these processes; they can only change input variables such as architecture of neural network or input dataset and hope that their results will improve. However, this is easier said than done, and non-predictability remains an important in-built feature of AI.

As I write these words, AI’s non-predictability arriving from combinations of data bias and system architecture is a hot research topic. ibm research claims that ‘bias in AI system mainly occurs in the data or in the algorithmic model’; therefore, they claim, ‘it’s critical to develop and train these systems with data that is unbiased and to develop algorithms that can be easily explained’ ( ibm Research 2018). However, using unbiased data is often not enough for developing unbiased AIs, and companies are trying out other solutions. In September 2018 Wired’s Jessi Hempel has written about a new trend: ‘an auditing process that asks companies to open up their technology for evaluation’ (Hempel, 2018). Algorithm auditing does not follow any standard procedures, and ‘an audit doesn’t prove that a company has avoided all the unintended pitfalls of an algorithm. The auditor might not look at the right set of stakeholders, or pose the right set of questions.’ (Hempel, 2018) Nevertheless, concludes Hempel, ‘it’s a baby step toward a more transparent data future: If we cannot strip algorithms of all their bias, at least we should rid them of the bias we can identify.’ (Hempel, 2018) At the moment, AI literacy cannot be thought of without (big) data literacy and information literacy in more general. However, AI brings about a unique challenge of constantly changing and never completely disappearing biases arriving from two dialectically intertwined sources: system architecture and input data. For the time being these biases cannot be predicted or supressed analytically, and solutions such as auditing software are becoming increasingly similar to educational assessment of human beings.

4 The Postdigital Challenge of Critical Media Literacy

In place of conclusion, I will now build upon this definition using insights into data and AI literacy. In the age of traditional mass media such as radio and television, technological affordances could be taken for granted in our studies of relationships between information, popular culture, ideology, and power. However, we are now entering a new stage of technological development, where human attitudes, ideologies, and power relationships are not only expressed through technology, or even built into technology – these days, they also inform and direct technology’s autonomous behaviour. AI systems are obviously technological / digital, but their functioning is dialectically intertwined with the non-technological / human. In result, our postdigital reality ‘is hard to define; messy; unpredictable; digital and analog; technological and non-technological; biological and informational. The postdigital is both a rupture in our existing theories and their continuation.’ (Jandrić et al. 2018: 895) The postdigital challenge does not make earlier forms of critical media literacy irrelevant; instead, it updates them for the digitally saturated world. In words of Peters and Besley: ‘The postdigital does not describe a situation, condition or event after the digital. It is not a chronological term but rather a critical attitude (or philosophy) that inquires into the digital world, examining and critiquing its constitution, its theoretical orientation and its consequences.’ (Peters and Besley 2018) The crucial contemporary challenge of critical media literacy, therefore, is to accept the arrival of postdigital reality and reinvent existing theories and practices.

Looking more practically, it is a no-brainer that critical media literacy needs to involve deep understanding of issues pertaining to data and AI. Referring again to Peters and Besley, ‘the critique of digital reason has two elements: first, the mathematico-technical control systems that are part of the emerging global digital infrastructure within which we now exist, and second, the political economy of these systems – their ownership, acquisition, structure and ownership.’ (Peters and Besley 2018) These elements are dialectically intertwined: Big Data carries inevitable political, economic, and ideological baggage in the form of in-built biases, and AI algorithms exacerbate existing data biases and produce new ones. Postdigital biases are not only carried by technology, or disseminated through technology, or produced by people using technology. They are built into technology, and developed through technology, in new and fundamentally non-predictable ways. This brings about the second postdigital challenge of critical media literacy: finding a new balance between the technological aspects of data and AI literacy with the political aspects of data and AI literacy, and learning how to deal with non-predictability.

Whether we philosophically subscribe to one or another type of sociomaterialist symmetry between humans and non-humans, or we decide to follow more traditional understandings of humans as masters of technology, there is no doubt that AIs have fundamentally altered the nature of human control over technologies. Traditional practices such as software testing are now being replaced by far less analytic practices such as AI auditing – in place of mathematico-technical processes where humans control software, we now have techno-biological processes where humans teach software and then assess its independent behaviour. Thus we arrive to ‘the convergence and marriage of the two dominant world historical forces of digital and biological systems and the ways in which together they constitute the unsurpassable horizon for existence and becoming – the species evolution of homo sapiens and life in general.’ (Peters and Besley 2018) This brings into the fore various posthumanist accounts of human and social development, which translate into some deep philosophical questions. ‘While we speculate what kind of future world we will inhabit in coexistence with new forms of intelligent life, we should firmly focus on the questions what forms of intelligent life should be included in our collective decisions about the future and how we might raise them’ (Peters and Jandrić forthcoming 2019). The third postdigital challenge of critical media literacy is to find ways of active engagement with these questions. In the postdigital context, the question of literacy does not relate only to people; these days, we also need to start thinking what makes AIs literate and develop ways of raising literate thinking machines.

The postdigital challenge is a rupture and continuation in our existing approaches to critical media literacy. It is about the practical challenge of producing (more) balanced datasets, about understanding of inner workings of AIs, about grand philosophical questions such as equality and / or symmetry between human and non-human actors, and about conceptualisation, development, and understanding of new forms of intelligence we would like to live with in the future. We are now at the very beginning of the postdigital epoch, at its very infancy, and critical media literacy has an urgent duty to take an active role in all aspects of its development.