Will Brehm  0:00
Janja Komljenovic, welcome back to FreshEd.

Janja Komljenovic  2:17
Thanks for having me, Will. It’s great to be here again.

Will Brehm  2:19
So, let’s start by unpacking the meaning of datafication in education. We hear a lot about that today. What does that even mean?

Janja Komljenovic  2:28
Okay. So, datafication in academic literature normally refers to so-called quantification of human life, which involves representing social and natural worlds in machine, readable, digital format. Then in the literature, you would get that in education, specifically, datafication, consists of collecting and processing data at all levels, from individual to institutional, national and beyond, and then impacting education stakeholders’ practice. Now, concretely, that would mean trying to capture artifacts of our lives, right? So, you know in your everyday life, outside of the education, you could datafy your sports activity. You know you could track like your movements on Fitbit and so on. As a student in your learning life, you might want to datafy your time studying, or the patterns of your studying, or your activities maybe as an academic, you’re datafying your research, you know, through various metrics such as citations and so on and so forth. So, datafication, in a sense, is more or less all around us where we are sort of noticing and seeing different kinds of data artifacts perhaps, I would say. So, essentially, we’re trying to turn events, processes, things, into data. Now what I think is important to add here also is acting on data. So, not just collecting, you know, and processing, but then also what happens with that data. So, I think it’s really important to understand that in this broader perspective of not only the political, economic embeddedness of data, which I think we will probably talk about later, but also once the data is collected and processed, turned into some sort of data product, or data output, such as analytics, you know, or various displays like, what happens to it? Like, who acts on it? What are the consequences of it? So, to me, datafication is this, overall process of how we turn various things and processes of our lives into data, how it’s processed, and then what happens with it.

Will Brehm  4:37
So, can I give an example, and can you tell me if this is like, an accurate example of the datafication of not just our life, but particularly education? So, would it be something like, as a scholar, I am supposed to make a contribution to knowledge, and so to do that, I write publications, and those publications, then, you know, potentially get picked up by other scholars and used in other work. And you can then start counting the number of citations that a scholar receives, and that can be counted over time. And then there’s ways of, sort of measuring how many citations you receive in a year. And there are certain indexes to sort of compare you to other scholars, which then might get used in promotion applications, you know, your employer might say, Oh, you only have this number of citations, and therefore, we don’t think you’re ready for promotion. Is that sort of this notion of datafication?

Janja Komljenovic  5:32
Well, I would count that as part of the datafication movement in higher education, for sure. You know, it’s part of datafying the research process. It’s part of this sort of metric that is collected around, I’m not sure, I would say research success, but at least the metrics we developed to behave as if this is research success. I would call that as a part of datafication of the whole sector and academic life. Yeah.

Will Brehm  5:57
But isn’t that sort of the phenomenon? Then it is when that data sort of gets created and then used and then acted on, it also then becomes sort of valorized and changes the human practice of being a professor in higher education?

Janja Komljenovic  6:12
That’s exactly it. So, datafication is such, and various data products and data outputs that we see integrated in digital projects are performative, in a sense, you know. So, how does different kinds of analytics of your activity impact the way how you perceive your own activity individually? How does it impact the institution’s decisions about you and on you? How does it change the way we think about particular things? You know what matters? And that’s exactly part of the story, like, what does datafication include and what does it not? You know? So, we might again get to that in detail later but the question that you hear often around things like learning analytics and similar is, does it actually represent learning you know, or does it represent other behaviors? Because what we have to understand with datafication is that data can only capture specific things that happen on platforms or things that are able to be digitalized and digitized. It doesn’t capture everything that happens. It doesn’t capture our actual thinking, right? So, you know, what does datafication and data outputs actually say about our lives and our social processes? That’s a really important part of understanding the datafication process and many complexities and clashes that are going on in this space, you know, between different stakeholders within institutions and across the sector.

Will Brehm  7:53
So, maybe we should dive into higher education, and you know, before potentially critiquing data like maybe it would be good to really try and understand what people who support the datafication of all the activities in higher education, what they think they can get out of it, like the positive outcomes, the aspirational outcomes. You mentioned learning analytics, you know, but learning analytics for what? What’s the sort of goal by being able to analyze learning with data. But you know, what else do proponents of datafication in higher education sort of hope to achieve?

Janja Komljenovic  8:29
I wouldn’t want to understand datafication as a process where people are either for or against it, you know? So, I think that’s a really destructive view on what’s happening. I see it as a contemporary phenomena that is around us. But then what we should discuss is how it is happening. You know, all the possible kind of opportunities, but also risks, and listen to each other. You know, when you said, what do proponents of datafication thinks, maybe it would be better if we would actually be more specific and say people who think that learning analytics in particular is very useful, would argue that learning analytics is useful for, say, online large classes so that teachers can manage such large classes better, or that various kinds of student engagement scores are useful for identifying potential students at risk and organize early interventions and things like that. So, you know, there are these ideas around how data can help us make teaching and administration at universities better, but there are also people who raise concerns and questions and identify risks which are equally legitimate to consider. What are, then, the problems with particular parts or kinds of datafication, and you know, what are the potential threats that we should tackle? And sometimes one can get a feeling as if these different kind of camps, or, you know, different parts of the conversation are not really sitting together well, or as if people are not really listening to each other. And it just sometimes feels that this is, perhaps this part of the bigger kind of pressure, you know, the acceleration of academia. Yeah, the pressure universities are under for marketization purposes, recruitment pressures, financial pressures. So, you know, there’s this idea as if, oh, we really need to do something with technology. We really need to find value in data and use it and so on and so forth, and so it almost kind of feels we need just more time to actually properly reflect on, you know, what’s happening, and listen to each other, like within one university and across the sector. So, yeah, I think, you know, it’s not about pro versus against, it’s more about how is it happening, all the nuances, all the use cases. I like to talk about use cases, you know, because these might be very different as well at different institutions. You can hear examples how something works at one university doesn’t in another. Why is that? You know, what’s the kind of local difference, or what happened in one institution, not another? So, I guess it’s to say that that’s also okay. It’s a normal part of how processes develop, right? So, I’m just making things even more complex. Sorry about that.

Will Brehm  11:31
And I totally appreciate it. Now that you mentioned it, I totally agree with how you’re sort of positioning this. At the same time being in the sector, I feel like I constantly get emails from companies that are sort of giving me the new technical solution to some problem I supposedly have. And so, you know, although I agree that we shouldn’t dichotomize it between proponents and opponents, there are certainly people in this space advocating for a very big take up of datafication in our processes as well. Wouldn’t you agree with that?

Janja Komljenovic  12:02
Definitely I would agree with that. I think that’s part of the story around general sort of building and expanding higher education digital ecosystems, where universities cannot do this themselves, so obviously they procure proprietary digital platforms and infrastructures. So, I should probably give a little bit of a context here. So, with my collaborators, we held a two and a half year research project funded by the ESRC (Economic and Social Research Council), which is the British Social Science Research Council, where we were looking at the value of digital products and digital data in higher education. So, we were following the ed tech startups and how they are developing and innovating in that space. And an important part of how they are driving up the value of their digital products is what we call datafying their products, so they are integrating various kinds of data outputs into their digital products. So, different kinds of analytics, insights, every new metric, experimenting with that. It’s really a very lively space where there’s a lot of testing happening. You know, what sticks, what doesn’t, what’s useful, what’s not, what universities want to pay for, what not, you know. So, it’s really trying to find, so essentially, working with data that’s there and how you can make that valuable, rather than thinking about, okay, we need to kind of get an understanding of a phenomenon, let’s see how we can collect data on that. You see what I mean. So, it’s a lot of like commercial interest there, around just monetizing data that’s available, rather than thinking carefully with the sector. You know what kind of data is really useful? So, I’m not saying that nobody’s doing that. I’m just saying there’s a lot of kind of this experimentation with data monetization. So, that’s happening at the startup scene and with ed tech incumbents, or even with big tech, we see that maybe even more aggressively, you know, just kind of new data products being released or launched in the new updates of the systems, even if universities didn’t ask for it. You know, academic students’ kind of start getting emails and information about their activities, even if they don’t need it, you know. Even if those measures and insights are not really useful for their studies and work. So, there’s a lot of experimentation, testing with sort of finding economic value in data by many, but then also just using data that’s available with automatic roll outs of new software when companies are already present in the sector. And yeah, so I see you nodding. You probably experienced that as well yourself.

Will Brehm  14:54
I certainly have, and I would imagine a lot of other lecturers and professors that are listening to FreshEd, but also a lot of students. Because I think you’re right; everyone involved in higher education is being sort of targeted as a potential customer. You know, I want to get a little clarity on how data can actually be monetized, right? Like, how does that actually work? How do you earn a profit from data?

Janja Komljenovic  15:16
Yeah. So, when you’re talking about or asking about monetization of data, we refer to the economic value of data, which you know would be something connected to, but perhaps at least analytically distinct to, the use value of data. So, I would say that, you know, universities and regulators and higher education stakeholders would be more interested in the use value of data, and you know how they can benefit from data to improve quality and things like that, whereas commercial providers of technology would then be interested in the economic value of data through data monetization. So, based on our research, the way how they are monetizing data is through, as mentioned before, datafying their digital products and then increasing subscription fees for their products and software. So, that would be quite a typical move. Alternatively, it could be that data products would be charged as an add on, also the universities could pay subscription fees for like a basic service and then pay for like data add-ons. We also see companies that are emerging as -if we could call that data as a service. So, universities would be, in fact, sending user data, digital data, to those companies. They would process them and then come up with different kinds of data outputs, such as student engagement metrics, you know, and then universities would pay for these data outputs. So, we see, you know, it’s a dimension. We see different ways of how data can be then operated and processed for economic purposes.

Will Brehm  17:02
It’s quite interesting, and it sort of raises a whole bunch of issues. I mean, I guess the first one that comes to mind is, what is the implication of all of these different companies working with data service, you know, providing different services and subscription fees? What is the implication on, say, university budgets because of this?

Janja Komljenovic  17:18
It’s interesting because we found in our research at least quite a big paradox. You know, on the one hand, at least discursively, from the ed tech industry, universities hear a lot of promises around efficiency, you know, driving down costs with technology and with the help of data, probably helping staff, you know, and kind of with the idea of reducing the labor costs and things like that. At least, what we found in our research; the reality is quite different. So, this sort of digitalization and introducing more and more technology, and especially datafication and getting to the stage where we can talk about data powered universities is very, very costly and resource intensive. So, universities are talking about reporting on really, really importantly, increased costs, not only, you know, for like, subscription for all of the software that they need to get, or storage is getting really expensive for all of the data that they host, and, you know, all the content and so on. But then it’s also just about the needed sort of skill and labor reorganization. So, there is a myriad of new jobs that are needed to support datafication of universities such as data scientists, project managers, you know, new sort of IT departments need to grow. Also get instead of system architects, now they need developers. Legal departments need to grow. So, vendor management is getting really complex. You know, a typical university will need to manage between 100-200 vendors, just for, like, the core software. And universities need to stay on top of this, and that’s only just to sort of run it administratively and technologically. But then, if you’re talking about actually using the data and acting on data in everyday practices, right, you know, at the top leadership level, or middle sort of management level, and at all departments and units, you really do need to upskill people and offer them proper support as well to interpret and use data. So, all of this is incredibly costly. So, it’s not about saying, Okay, now we’ll have data outputs, and we will be able to save all this money. It’s, in fact, well, we need to spend more money to introduce all of this and to act on it. We might do things better, but, you know, we’re not at that stage yet. We’re just at the stage where universities are really working hard, incredibly hard, you know, also experimenting, trying to manage all of this properly, work with their digital ecosystems, technologically, legally, you know. And it’s just very, very complex. So, I think we’re in a stage now where we are at sort of a transition almost, or at least on some sort of fundamental dynamic that is happening, but it’s just very complex and costly for them.

Will Brehm  20:24
And I think you’re highlighting such an interesting paradox. You know, this notion of one of the potential prospects of datafication is efficiency, and is becoming more effective in our practice, because we’ll have a better sense of what’s going on, and we can crunch the numbers and figure out how to do it better and potentially save money and get better learning outcomes in the end. But actually, to get there, the paradox is that it actually costs a huge amount of money to do this, and so those efficiency gains might not be either available in the short term, or maybe are not available ever. And we don’t know yet, because as you said, we’re in this transition. So, it’s a really, really fascinating sort of paradox, and sort of begins to question some of those value propositions that might appear at the beginning of the show, where you’re talking about, you know, why datafication might be seen as valuable. You mentioned a few times issues of the legal department. What are some of the legal issues that come out of datafication, but I guess a lot of it has to do with giving university data to third party companies to sort of create data packages that get sold back to the university, right? Like, what sort of legal issues have you uncovered in the research?

Janja Komljenovic  21:39
Perhaps I want to answer this by saying that there’s like, internal care of data processing from the legal perspective and external care of managing relations with, you know, tech and software providers. So, internally, obviously, universities collect large amounts of data on students and staff, especially if you’re talking about digital data. As you know, we students and staff as digital users leave behind on platforms. So, universities need to respect all of the legal requirements about protecting this data and are responsible for data they process, as well as potentially share with others for processing. So, that’s a big chunk of it. But then also with these relations and relationships with external providers. You know, procurement is just such an important process, also from the legal perspective. In the past and not so long ago, different departments and units or even individuals were allowed to procure software they thought they needed. So, you know, if you would just take a typical university in the UK, there might have been hundreds of different kinds of maybe even small software that would be just used across the university, and nobody knew exactly what was used. Well, that’s not possible anymore with the new legal requirements. So, universities have centralized procurement, not only from the you know, economic perspective to try and get the best deals, but also from the legal perspective. So, they have to ensure that they are compliant with data protection laws, also all of these providers of technology that universities are using. So, procurement for large software, for example, the whole process may take a couple of years. It’s really astonishing you know; you don’t think about these things in your everyday life, but then when you actually learn about this, it’s just so complex. So, sorting out relations between universities and vendors is really a lengthy process, and then also the contracts -they need to set up, the contracts, they need to manage these contracts. Another layer of complexity is also that we have more and more acquisitions in the ed tech industry. So, companies can then also be acquired which might have not only legal but also ethical consequences, you know. Say, if a company is acquired by another, bigger company that already has other services in their portfolio. Are we talking about merging different kinds of data sets? You know, what are the consequences for the future service provision? So, that’s another layer of unknown and ambiguity. But that was, I think, quite interesting. It seems very mundane, but what was quite interesting that we picked up in our research is just the statutory reporting that universities in the UK have to do. So, they have to report data. Typically, we’re talking about administrative data, not necessarily, you know, this sort of digital data collected by platforms, but more like traditional administrative data. Nevertheless, these requirements are expanding, and universities are also more and more sort of using various kinds of analytics outputs from their digital data pool. And our interviewees and participants in focus groups were reporting that this just takes so much of their time, and they just need to report -it feels like more and more without actually having the time to reflect on all of the data that is being reported. So, it’s this sort of you need to fulfill, like a legally required demand, but then the purpose of this demand is not met because you’re overburdened to actually consider it and action it.

Will Brehm  25:25
So, I mean, it’s such a complicated and complex, sort of changing, dynamic environment that universities are sort of going through both, sort of organizationally, they’re changing, but also, sort of, you know, as Ben Williamson said last episode about the digital infrastructure sort of being built beneath our feet. And all of that is sort of changing so the very ecosystem in which universities sort of operate in is sort of in flux. And I guess you know from your research, how do you find universities managing this change, right? Because these are some big fundamental changes. Are they handling the change alright? What are some of the approaches that they are using? Are there voices that aren’t being sort of heard in this process? Being someone who’s actually done the interviews and spoken to different people around, particularly the UK, what’s your assessment on this?

Janja Komljenovic  26:16
My first thought when I heard your question was, I’m just sympathetic. I’m impressed with just how hard they’re trying and how hard they’re working. And the universities are in very different positions, you know, not only with their size, but also with their resources and just what they can actually do and what they can afford. So, they are in very different stages of so-called data maturity, and you know, just how well-developed digital ecosystems they have. But everyone, it seems is striving, you know, for this sort of ethical datafication. Now, some universities are more entrepreneurial than others, so they might kind of see the ways, how they achieve this very differently, you know. So, we’ve heard examples of universities where they want to scale their online provision and use AI to support that kind of provision, you know, to really reduce the number of academics and sort of grow the personalized learning with AI. So, those universities are choosing a particular way, how they are growing their datafication, or how they want to make use of datafication, if you put it this way. But not everyone’s obviously doing that, you know. So, you also have very different kind of approaches and views of how they want to use data and for what purpose. So, I would say, you know, everyone’s trying hard, but in my view, and based on our research, one of the biggest problems is just this sort of mismatch of internal beliefs and values and strategies. You know, it seems that there’s very different ideas about datafication at the very top leadership, you know, where people really need kind of solutions, and they’re trying hard. Our participants reported that especially at the top leadership, there’s a lot of sorts of tech and data solutionist thinking present. But then, you know, people who are actually working with data are more critical and realistic as well in terms of we need to understand what particular data output is actually saying. What does it actually mean? You know, how do we interpret this? And when we make decisions based on these data outputs, who is in the room? Are people who are making decisions actually understanding the data that they are basing their decisions on. And then you also have critical voices. So, in our research project, our participants mainly said that it’s mainly academics who would be critical about datafication. And then you also hear the discursive sort of messaging, as if they do this because they’re afraid for their jobs, or they’re against progress and things like that. And it doesn’t seem that that’s the case. It’s really people have legitimate questions, legitimate concerns. And to me, it just seems that until universities, like within themselves and then across the sector, actually nourish the honest, democratic conversation and democratic relational decision making of, you know, what kind of data products do we want, with what effects, what they will do, what kind of rights do we have about it, who manages this, what if we change our minds once things are integrated in our daily lives, who measures the impact, what happens after? So, it seems to me that this is lacking, right? So, you do have, you know, very kind of thorough thinking through kind of legal, you know, even legitimate kind of concerns and things like that, but this really practical getting together seems to be missing.

Will Brehm  30:06
Well, Janja Komljenovic, thank you so much for joining FreshEd again. Really a pleasure to talk, and I feel like you’re going to have to come back on to talk us through what happens in this transition in the years ahead.

Janja Komljenovic  30:17
Well, thank you so much for having me, Will, it’s been a pleasure.

Want to help translate this show? Please contact info@freshedpodcast.com

Have any useful resources related to this show? Please send them to info@freshedpodcast.com