How will they shape you?
Algorithms are everywhere, from simply suggesting online search results or friends ‘you may know’ on social media, to more critical matters like helping doctors determine your cancer risk, to decide whether you can have a mortgage, or to predict crimes such as gang violence and burglary. Because it is so hard to spot, you might not have even noticed how much of our life is influenced by algorithms. The content we consume on Facebook, the music we listen to on Spotify, the movies we watch on Netflix – it relies on predictive modelling by algorithms. With each click, the algorithms learn to personalise their ‘feed’ and the marketing for our best experience. Although many of these examples are driven by commercial interests, they are also used in the public sector, such as in health care, education, criminal justice and tax administration. Public organisations increasingly use new forms of data analysis in order to improve public services. We find algorithms used by judges to decide whether a criminal defendant is likely to reoffend or not. We see algorithms used by municipalities to identify optimal routes for waste collection. We encounter algorithms used by teachers for assigning students to schools. Algorithms are here to stay and – the story goes – to help us. The idea is that algorithms are a solution to the mayor challenges of our time, such as security, public services, health care, and environmental protection.
We are living in the midst of a significant transformation of our lives, and while it is an incredible time and place to be in, we must be wary of the effects that come along with it. Mind-boggling amounts of data are generated regarding our daily actions with algorithms processing and acting upon these data to make decisions that manage, control, and nudge our behaviour in everyday life. The use o f algorithms not only expands the possibilities of current control and surveillance, but also introduces a new paradigm characterised by an increased rationality of governance, a shift in the functioning of power, and closure of decision-making procedures. We can refer to this by using the term ‘algorithmic governance’ – the replacement of human, legible and accountable judgements with ‘black-box’ algorithms – or, as sociologist Aneesh Aneesh dubbed it, ‘algocracy’ (2006, 2009; Engin & Treleaven, 2019). The term ‘algorithmic governance’ signifies three distinctive, but related elements with a direct bearing on our behaviour. While automation in decision-making is not particularly new, the impact of algorithms is increasingly becoming systemic in terms of (1) automation; (2) architecture; and (3) anticipatory applications.
Full automation means that human agency can be almost completely designed out of decision-making processes – even though the extent to which this happens varies in practice. Algorithms collect information (input), process it (throughput), apply it (output) and learn to improve output (feedback) (Zarsky, 2013; Citron & Pasquale, 2014; Danaher et al., 2017). Artificial intelligence, databases, websites, and automated procedures are replacing human agency from actual decision-making (Henman, 2010; Peeters & Widlak, 2018; Van der Voort et al., 2019). Moreover, decision-making becomes a matter of classification rather than judgement of individual cases (Peeters & Schuilenburg, 2018). As a consequence, new actors or experts are now entering the game (usually not trained in social sciences). The discretionary space shifts to the IT professionals that design algorithms, to the data analysts that identify behavioural patterns, and in a certain w...