OverviewTranscriptTranslationResources

What is the connection between education and the economy? For many neoclassical economists, the connection is found in Human Capital theory.

My guest today, Professor Steve Klees, thinks human capital theory and rates of return analyses are very problematic.

In our conversation, Steve talks about his new article, “Human Capital and Rates of Return: Brilliant Ideas or ideological dead ends?”, which can be found in the latest issue of the Comparative Education Review. He takes us through human capital theory, its internal logical fallacies, and proposes a set of alternatives.

Steve Klees is a professor of International Education Policy in the College of Education, University of Maryland.

Citation: Klees, Steve, interview with Will Brehm, FreshEd, 54, podcast audio, December 5, 2016. https://www.freshedpodcast.com/steveklees/

Will Brehm  1:45
Steve Klees, welcome to FreshEd.,

Steve Klees  1:52
I’m very glad to be here. Thank you for having me.

Will Brehm  1:55
You are an economist by training. But you have spent most of your career in the comparative and international education field. How do economists typically think about, or look at education?

Steve Klees  2:12
That’s a huge question, but the answer really depends on what kind of economist you’re talking about; they’re different schools of thought. But the the main dominant school of thought is called neoclassical economics. And neoclassical economics is really about competitive capitalist market systems. And within that education is a very important piece of understanding education, economics and development. In particular, neoclassical economists have developed something they call “human capital theory” that is a framework for understanding education’s role in the economy and in society.

Will Brehm  3:10
And how is human capital theory measured? How do economists see human capital?

Steve Klees  3:22
Well, human capital is a latecomer to economics, to neoclassical economics. Neoclassical economics goes back to Adam Smith, and the Wealth of Nations in the 1700s and the term “neoclassical” actually was coined at the end of the 1800s, and it’s about how a competitive market system operates. Human capital theory wasn’t developed till the late 1950s, early 1960s. Prior to human capital theory, economists understood the economy in terms of supply and demand, you’d always see economists talking about supply and demand – of small companies, small firms, small households competing with each other – and trying to understand how that competition worked, what you got out of it, how a market system worked.

Prior to human capital theory, economists had a lot of difficulty understanding labor and work. Labor prior to human capital theory was an anomaly. It wasn’t something you could talk about in terms of supply and demand. The economists in those days, in the 50s, look more like sociologists; it was a whole field of labor economics where they studied real world labor, they studied strikes, they studied unions, they studied how large firms operated. But education didn’t really fit into that structure at all – that way of thinking. And there were odd people out in neoclassical economics because they were more like sociologists, and they weren’t talking about competitive market structures and supply and demand, and human capital changed all that

Will Brehm  5:28
How so?

Steve Klees  5:29
Well, it really made economists able to talk and think about education and labor. Labor, especially, as a commodity like any other commodity that’s bought and sold on a marketplace, that has a price, that’s determined by supply and demand in the marketplace. Human Capital theory developed because it was explaining puzzles. People were trying to understand how economies grew. And they understood that there were more workers, and there was more capital investment, but they didn’t really have any idea about quality of work. And the whole idea of human capital, was it explained better, to neoclassical economists anyway, why some countries grew faster versus slower. They called it a revolution in thought, and the idea behind it was essentially simple – that education wasn’t just a consumer good, it was an investment. It was an investment in individuals, and it was an investment by society and societal development.

Will Brehm  6:48
So in a sense, it would be that if an individual were to receive education, or more education than another, they or he or she would be more productive in an economy, and maybe measured through income? Is this the way the neoclassical economists were seeing this?

Steve Klees  7:12
Yes. They looked at two outcomes of education in particular They looked at earnings, and they weren’t interested in private benefits as much; earnings were a benefit to you, income is a benefit to you. But they were interested in, within their framework earnings as a proxy for people’s productivity, like you said. And so, they were trying to get a handle on education’s connection to individual productivity. And secondly, education’s direct influence on economic growth, its effect on gross national product. So you saw starting in the 60s, lots of studies of the “rate of return”, they called it, the return on investment. So education in terms of earnings as a proxy for productivity, and in terms of economic growth measured by gross national product.

Will Brehm  8:15
So based on the rate of return methodology is is some education better than other education for foreign economy or for foreign individuals’ productivity?

Steve Klees  8:27
Yes. I should explain a little bit about rates of return. Rates of return are a measure of benefits and costs. In neoclassical economics, the private sector is motivated by profit. Profit is a signal that this endeavor is valuable. Adam Smith talked about the invisible hand of supply and demand working in the public interest. That’s the profits supposedly representing where peoples’ benefits exceed their costs; where the outcome of whatever you’re making, tables or software or whatever, the benefits exceed the costs. And so economists were looking for something as analogous to that in the public sector. So the idea was to explicitly study the benefits and costs of public sector activities, whatever field, education, health care, environment, transportation, and rate of return is a summary measure, after you figure out what are all the benefits to an education investment, what are all the costs of that investment, and it’s a summary measure to try and get at  – gosh, you know, you’re making 20% on your investment, the benefits exceed costs by 20%. And so that’s applied to lots of different types of educational activities, and other sector activities, to study the returns to education, of various types of vocational education versus academic education; of higher different levels of education, higher education versus early childhood, or primary education; different programs of education. Anything where you can find reasonable monetary measures of outcomes. Sometimes you can’t do that, you’re just looking at test score differences between different programs. And then economists do a more limited array of what they call “cost effectiveness” analysis. But mostly economists really like to go after cost-benefit analysis, because that gives them a metric that they can compare with returns in the private sector: Is this a better investment to take your money out of the private sector, tax it and put in an education or health care or environmental protection?

Will Brehm  11:13
This sort of thinking of cost-benefit analysis of education to an economy, do you see this is problematic in anyway?

Steve Klees  11:24
Yes, the paper you mentioned that I did, and actually much of my work over the last – I hesitate to say it – 40 years (I’ve been working in this field for a while) has been with the problems of neoclassical economics, generally. And more specifically, with the internal dynamic, the internal problems with that field that gives you measures like benefit-cost studies of rates of return. My work has been recently basically saying that even not taking in a critical outside neoclassical economics look, which we can talk about in a little bit, of political economy perspective, for example. But even if you take the neoclassical economics perspective, there are so many problems within that framework, that for me, the benefit-cost analysis/rate of return type measures just fall apart; that they become almost meaningless.

Will Brehm  12:41
How, so? Let’s dig into it, human capital theory, rates of return analyses. If you’re saying that there are problems of the internal logic of neoclassical economics for human capital theory, and for rates of return analysis, can you can you dig more into that? Like how, so? What are some examples of this?

Steve Klees  13:05
I don’t know whether to start with the details or the broader picture. Let me just start with the broader picture, because I think, then the problems with the details become clearer. And the broader picture really revolves around one central idea of neoclassical economics, and that’s the idea of economic efficiency or societal efficiency. They sometimes called it Pareto efficiency after an Italian economist a century ago called Pareto, and it’s a complex idea that I find completely unsound and unreal. And I’ll try and explain the idea briefly, explain why I think it’s unsound, and then give you how it manifests itself in this cost-benefit/rate of return type studies. So efficiency is something, you know, it’s a common sense concept. So to us, people talk about efficiency of this or that; it’s an engineering concept, it’s a physics concept of, you know, you can do more with less somehow. But in education, you can talk about it sensibly, right? Limited ideas of efficiency, like you can talk about an educational system as inefficient because it has a lot of dropouts, or a lot of people repeating grades, or a lot of people who aren’t learning much. So there’s a common sense idea of efficiency that makes sense to all of us. And I have no objections to that. It’s the economist concept of efficiency that’s problematic. And that’s not about an individual sector or individual project, as much as it’s the overall society is deemed economically efficient if it operates according to the assumptions of a very highly competitive market framework that in abstract neoclassical economics discussion is called “perfect competition”. Perfect competition is a competitive system that is so highly competitive that you’ve got many buyers and many sellers of identical products: that nobody’s big, nobody can influence prices, they’re all taking prices in the market, they’re all small potatoes. Consumers and producers are the two major motivators and movers of the economy. Consumers are just out there maximizing their happiness, and producers are just out there maximizing their profits. And if everything functions according to, and information is perfect, you know everything about everything. If you operate according to these simple assumptions, the whole economy is deemed efficient. And what they mean by that is that somehow, not only is there no waste, you’re you’re doing everything as cheaply as you possibly can. But you’ve got the right balance of everything, the correct balance of everything. So you’re producing the right amount of chairs and tables, and movies and hamburgers, and software. There’s something called “correct balance”. And that’s what’s efficient in this. And it’s completely separable from their other major concept, which is equity or fairness or the distribution of these things. So the distribution is sort of irrelevant to efficiency. You can have an efficient society in which half the people in the world are starving. That’s that can be efficient, because efficiency is just about those people who have effective demand, meaning they have money. And they can wave that money in the marketplace and demand goods and services. And so efficiency is really to economists about forgetting equity, forgetting distribution. Are we producing as much as possible with the inputs of land/labor/capital/technology that we have? In the theory, and in practice, this is just simply absurd.

There’s actually in theory neoclassical economists have something called “second best theory”. And second best theory says that if you don’t live in the first best world of perfect competition with all those tight assumptions – unreal, impossible assumptions – but let’s say have one monopoly in one sector in which everything else is highly competitive, second best says in the second best world with just one imperfection, you have no idea if the economy’s efficient at all, there’s no idea if it’s close to efficient at all. Because this framework is so tight that you only get this overall efficiency of the correct balance of things if prices are the accurate signal sending benefit and costs signals to producers that act in the public interest. With one price off, all of the prices are affected. So in practice, efficiency demands, for example, that you have the correct balance, the correct inputs balance of producing yachts for rich people and rice and beans for poor people. Well, that’s just a distributional issue to me. That’s an equity issue to me. There’s no right balance of yachts and rice and beans; there’s no right balance of computer software, higher education, early childhood education, nutritional programs, roads building. There’s no correct balance of that. And in practice, there’s just no vantage point in the sky. That’s what this efficiency idea is. Where you could separate what we produce from who gets it. They’re all integrally tied in practice. And this is what in practice they’re trying to do with cost-benefit analysis of rate of return: get an idea of whether something in particular is efficient or not.

Will Brehm  20:06
Just hearing that, it just makes me think that the theory of the world in neoclassical economics doesn’t match the reality that I live in. I mean, certainly people do not have perfect information when it comes to to buying anything. But at the same time, I also think that this separation of equity and distribution from efficiency seems to have actually happened. The world I see today, there seems to be a huge gap between the rich and the poor, that those eating rice and beans and those on their yachts.

Steve Klees  20:48
Yes, absolutely. The real world today is very problematic in terms of distribution, as we all know. In terms of equity.

Will Brehm  21:04
How is this embodied in rates of return?

Steve Klees  21:08
The whole efficiency framework is translated into guidelines for the public sector through cost-benefit analysis. And not costs and benefits to private individuals they’re after, they’re after costs and benefits to society as a whole. Because they want to correct the market, to account for all the costs and benefits to society as a whole. And so if you’re thinking about education, you think about the benefits of education. They buy benefits in terms of earnings to an individual. That’s a benefit to society if earnings reflect productivity. That’s problematic because earnings are a price, and prices are distorted in real world economies. So there’s no reason to believe that earnings reflect productivity at all. Earnings are determined by market power, by the vagaries of who’s got skills and who doesn’t have skills, on where firms do their business. So the idea of earnings as a proxy for productivity is a problem.

A second problem is that even if you wanted earnings as a proxy for productivity, that’s just one individual benefit. That is a social benefit, because it measures productivity, but there are what they call externalities. There are benefits to other people who were not included in the market transaction, you decide how much education you’re getting, some supplier gives it to you, a public school or private school, a training program, the benefits to education go way beyond you. And those are not taken into account in the market. So when there are benefits beyond the individual, they’re called externalities, because the market doesn’t take them into account. And therefore, the market is making inefficient decisions, because it’s not counting all the benefits. So in education, you can think of lots of benefits that aren’t just to you, your education benefits other people through your coworker productivity, through your family, through household health decisions, through helping your children, through lowering crime rates, through lowering welfare rates. It’s got lots of these external effects. And the second problem with rates of return is measuring those all are very problematic.

And the third problem with rates of return is that even if we were trusting earnings as a good measure, it’s very hard, I would say impossible, to figure out the effect of education on earnings. This goes to our problems, not neoclassical economics, but our problems with research methods generally. Separating our causes from effect, impact evaluation is extremely difficult to the point where I think it can’t be done quantitatively. If, for example, you took 1000 people and you ask them what’s their income, and then you try to figure out what are the dozens of factors that make those incomes different. And then you’ve actually tried to build a mathematical model that would separate those dozens of factors, so that you could say, “Well, their income went up, because they were a union this much, because they had another year of education this much, because they were in a high demand field this much, because they were healthy that much.” I mean, it boggles the mind. I’ve done another paper on the economist statistical procedure called regression analysis that tries to do that – it tries to take the dozens of factors affecting some outcome and separate them out. And my view is that we just can’t do that. So that even the minimal idea of looking at the impact of education on individual earnings is problematic. Taking them all together, I find rates of return and cost-benefit analysis, generally not a good basis for decision-making.

Will Brehm  25:48
But yet, it has been. These methods and this particular theory have been dominant and have been used to make decisions in education systems, among other sectors. So what’s the scholarly track record of those using rates of return and human capital theory? All of the critiques that you put forward seem very plausible to me, but yet rates of return and human capital theory has had quite the long longevity in academic research.

Steve Klees  26:23
Yes, it’s certainly has. And just two things to say in response to that. One is you asked about their track record, and in terms of track record, there’s no testing this. This isn’t something you can predict and then find out was a true. Because, I say the rate of return to expanding higher education is 12% in your country at this time. Is that a good investment, if you decide 12% is a good return, and you put your money in. But there’s no validation of whether you got 12% or not, so there’s no track record in terms of these predictions. They’re making predictions now, for example, about education and GNP. And I just find those scary and absurd. There’s some very interesting economists, very competent economists doing this. Eric Hanushek and [Ludger] Woessmann, and they tried to do regression analysis to say if your PISA scores go up (the PISA being that international test that people take as a proxy for cognitive achievement), how much will your GNP go up? And they come to these conclusions like a 10% increase in cognitive skills gives you a 2% boost on GDP and if everybody moved a standard deviation on PISA, your GNP would grow seven times in the next 30 years. I mean, this is carrying this framework to an absurdity to me. They can’t separate out the impact of education from the dozens and dozens of other factors that influence GDP or GNP, and then to take that out as its influence now and project that 30 or 40 years in the future is just the height of irrational use of a framework of this kind. And I understand why they do it. These are reasonable people, you want good information for decision-making, and to economists, this idea of efficiency, separable from equity is the touchstone.

But the real question for me is the second question you asked: basically, why has this framework been dominant for so long? And neoclassical economists would say at the very simple answer – it’s because of its explanatory power. It explains differences in investments in health and differences in individual behavior. And it’s true, I mean, in differences in individual behavior, this is useful framework because your decisions are affected by the returns to you, and you make decisions about your educational investment versus your investment in health care, versus your decision to go to a movie or your decision to buy a house, about returns to you. And that’s fine. So there is some use for this framework in terms of understanding people’s motivations. But in terms of societal’s efficiency and investment preferences, this framework is bankrupt and it’s empty. So to me, and to many critics, it’s not the dominant framework because of its explanatory power, it’s the dominant framework because it fits. Human capital theory is embedded in neoclassical economics, and that’s embedded in capital market – it fits with a capitalist market economy. The critics would argue the reason there’s so much attention to efficiency and rates of return and technical views of whether you invest in this thing, or that and how much do you do is because it makes sense in terms of efficiency. If you lose that efficiency framework, you realize that this is just a way to support a market system. This is neoclassical economics is an ideological justification for capitalist market systems to be efficient. To act in everybody’s interests aside from equity. If you question that, then you can see neoclassical economics generally, and human capital theory, as basically an ideological framework and ideological bulwark.

The whole skills discourse today comes from human capital theory and a skills discourse seems like common sense. It says if people only had better skills, they would be better off, and their countries and societies would be better off. That skills discourse based as it is on human capital theory and neoclassical economics is very problematic. People today have talked about the “triple economic challenge” that we face, and they talk about the three things: job creation; poverty elimination; and inequality reduction. Human capital theory and neoclassical economics generally gives one simplistic answer to all three challenges: lack of skills or equivalently, the mismatch between what education is producing and what businesses in the economy need. For the critics, lack of skills is not why people are poor, are not why jobs are scarce, and not why societies are so unequal. The culprit for the critics is that the very structure of the world system in which we are living, capitalism most particularly, but patriarchy, racism and other structures. Those very structures are problematic. While capitalism has increased our ability to produce material goods tremendously, so it looks very productive in that way, in another sense, it’s one of the most inefficient and destructive structures that you can imagine. Why? Because almost half the world – the World Bank says 3 billion people – are relegated to the margins of society. Capitalism has not created jobs for them, livelihoods for them. For the vast majority of our global population, if capitalism was an efficient system, we would be taking advantage of the skills and develop the skills of the 7 billion people on the planet and produced a lot more. Capitalism in its 200, 300 years hasn’t done that, and isn’t doing that. You know, some people talk about we live in a meritocracy. What nonsense! These 3 billion people are relegated to the margins of society because they’re not meritorious? It’s not that at all. It’s poverty, unemployment and inequality, not to mention environmental destruction and other problems, are not failures of capitalism, as they’re sometimes seen, but the logical outcome of its inherent structure. So that that in many ways, contrary to prevailing economic views, human capital has been a very destructive discourse. This is contrary to what the majority of economists think as it’s been brilliant, but it’s been a destructive discourse, because it’s really blamed individuals for their lack of skills, their lack of investment in the right skills, the lack of good choices. And so instead of understanding problematic structures that we need to do something about, we’ve been directing attention towards the supply of individuals and how to fix that. And we’ve been fixing it for decades. And the payoff with poor countries is abysmal. And the payoff even within rich countries is abysmal. The inequalities within the US, the level of hunger in the United States, the level of marginalization, the level of poor dead end jobs, the level of insecurity, the level of environmental destruction, this is not an efficient system.

Will Brehm  36:04
Turning to alternatives. I mean, is it even possible, or can we even have an education system in a capitalist economy without human capital theory? It almost seems like many of these problems that we see in education in terms of equity that you were just mentioning stem from the capitalist economic systems that that are pervasive in most countries. So how can we envision and create education systems in alternative ways that account for equity while still being in capitalist economies?

Steve Klees  36:48
Yes. All you’re asking today’s tough questions.

Will Brehm  36:55
I apologize.

Steve Klees  36:57
That’s okay. Let me preface my response to education with a little bit on how these alternatives are viewed to the very structures in which we live. Because education can only be successful if it’s a part of a challenge to those structures in fundamental ways. And there’s a lot of alternatives. Everything is contested terrain in this world; everything is up for grabs, up for debates with different views. So I view the alternative to neoclassical economics as what I call “political economy”. Political economy is a contested term and people on the right use it as well as people on the left. I’m using it more from the left of center point of view of critics of capitalism, critics of other world system structures. And for me, a political economy perspective today raises questions about the structures of the world system in which we live.

It’s the intersection of feminist perspectives, of post- perspectives, postcolonial, post structural, neo-Marxist perspectives, queer theories about heterosexism in society, disability theories, critical race theories. Not that these theories are identical, not that these approaches are identical, but all of them see marginalization as central, and all of them see marginalization not as failures of the world system -they’re failures for sure – but more is a logical consequence of the structures of patriarchy and racism and capitalism in which we live. And while there’s agreement that reproduction is pervasive, that is this marginalization is not an aberration, systems are out there that reproduce and legitimate marginalization and inequality. And the education system is part of that, as are all of the systems in which we live.

But the critics, the political economists, as I label them agree that while reproduction is pervasive, there are lots of spaces for progressive action. Through exercising individual and collective agency. You have the ability, and especially collectively, we have the ability to challenge these structures. And collective challenge is perhaps the watchword of political economists. Social movements like the women’s movement, like the civil rights movement. These are worldwide now. Like the landless movement in Brazil and now other countries. The poorest people in the world are organized and having an influence on policy. The untouchable movement in India, not anti-globalization, but the alter-globalization movement and in human rights movements and the children rights movement. And so there’s lots of examples of contestation at the systemic level and in education.

There’s lots of examples in every education in every city, in every country, and in every school system of what political economists call more progressive approaches to education. The legacy of people at Paolo Freire, the famous Brazilian educator who founded a field that we call today “critical pedagogy”. Critical pedagogy is a political economy approach to education, arguing that while reproduction is pervasive in schools, there’s lots of ways to challenge that. And so people, individual teachers challenge that all the time. They close their doors to their classroom, they use different learning materials, they teach their students differently about fairness, about equity, about the structures in which they live, they raise questions on that. And it’s not just individual teachers, there are systems of it. In Brazil, the landless movement which I just mentioned, have their own schools that are Freirean, that are participatory, that are so different from the technicist technical approach to education that we have today throughout the world. In Brazil, something called the citizen school movement that, again, is very participatory, that involves the community. We say “community involvement” all the time, but this is serious community involvement. This is serious democracy for students, for teachers, for administrators, participating and directing curriculum, directing grading, making decisions at a local level together, and sometimes very explicitly challenging the types of feeding education into work and into the labor market that dominates so strongly. On the alternative, for most political economists, when you reject the sort of functionalist view of sociologists, of society, of efficiency of markets, and say, “This is not something in which everybody is benefiting, there’s conflict here, there’s different interests, and the only way that’s going to change is through struggle, through individual and collective struggle.”

What that means in terms of alternative system-wide is difficult to say. At a minimum, where we’re not a neoliberal form of capitalism. Capitalism in the 60s, and 70s was a much more liberal capitalism in which government intervention was recognized as necessary to correct the ills that were essentially built into the structure of capitalism. Neoliberalism starting in the 80s with Reagan and Thatcher and Kohl in Germany said: Government is the problem, the market is the solution. We need to get away from that. We need to restore the legitimacy of government action, we’ve got the sustainable development goals of the United Nations on the table. Goals that are very ambitious about improving the world, we’re not going to get there under neoliberal capitalism. We’re not going to get there when we think it’s illegitimate for government to direct action. We’re not going to get there if everything is a public-private partnership and depends on corporate profitability in order to direct that system.

And maybe we have to move beyond capitalism. At the local level there’s lots of alternatives. And  broadly speaking, it’s the subject to a whole another conversation. And I have a paper coming out next year on capitalism in education that talks about alternatives, so maybe we’ll do another podcast there. But the broad answer is, you need to build towards a more participatory democracy and more towards a workplace democracy. The problem with capitalism is that our workplace is authoritarian. We teach democracy in the political sphere; we don’t have a lot of that at a very participatory level either. But we need democracy in the workplace as well.

Will Brehm  45:28
It seems like a lot of what you’re saying is that we have to think beyond the connection of education as being for the development of human capital, and having a different value of education. And there can be many it sounds like, and many different ways of achieving those values or putting those values into action. But it seems like that’s the first step: decoupling, or de-linking the connection between education and human capital development.

Steve Klees  46:05
No, I would agree. And to be fair to human capital theorists, some of them recognize that broad connection. It’s got narrowed in practice so much that all we’re looking at as the connection to education to the workplace. But citizenship can be subsumed in that human capital framework. The problem is its basis in efficiency. You want to talk about the many things education does, the many more things we want it to do. We don’t want to just make education about workplace. We don’t want to make education just about literacy and numeracy. We need education for peacebuilding, for people to not be aggressive, for people to be fair with each other, for people to have resilience, and people to be creative. So there’s lots of purposes of education, and human capital theory and practices just narrow the field too much. And more broadly speaking about this political economy framework versus a more mainstream dominant human capital neoclassical framework, the political economy framework doesn’t offer the technical policy guidance that rates of return give. For neoclassical economists, policies are a dime a dozen. You just do your cost-benefit analysis and this year, vocational education is better than academic education, higher education is better than primary education.

If you reject that framework, what you have is human capital theory, neoclassical economics – our ideologies masking as science. It’s absurd to think that there’s some way to assess technically, the tradeoff between higher education and primary education, between education and health and the environment. All we have is a messy, participatory democratic struggle of individuals and groups with some common interests and some different interests. And for me, what we have to do is find ways to facilitate that struggle and in doing so, economics and the dominant scientific perspective says you need to stay neutral and objective. For me, I’ve learned that you always take sides. That when I write a paper, when I teach a class, when I’m doing research, when I engage in policy, when I engage in my life, I always have to take a side. And if you don’t think you’re taking a side, you are because this is a struggle. This is contestation. And I guess my concluding point is that for me, this is what I said in the paper you cited, neoclassical economics and human capital theory are ideological dead ends. But fortunately for all of us, there are lots of alternatives.

Will Brehm  49:29
Well, Steve, Klees, you gave us a lot to think about in this conversation. Thank you so much for joining FreshEd, and you’re definitely welcome back when that new paper comes out on capitalism in education.

Steve Klees  49:41
Thank you very much for having me.

Want to help translate this show? Please contact info@freshedpodcast.com
Have any useful resources related to this show? Please send them to info@freshedpodcast.com
OverviewTranscriptTranslationResources

The Globalization and Education Special Interest Group of the Comparative and International Education Society will be hosting a public webinar on December 12 entitled “Puncturing the Paradigm: Education Policy in a New Global Era.” The webinar will bring together the four co-editors of the newly published Handbook of Global Education Policy, Karen Mundy, Andy Green, Bob Lingard, and Toni Verger.

During the lead up to that event, FreshEd will interview the co-editors to set the stage for the webinar. Today I speak with Professor Andy Green about the global education policy of social cohesion.

Although we often think of education policy as primarily concerned with economic development, it also has been historically connected to the idea of creating a cohesive group of people who share certain norms and customs. Benedict Anderson called this “imagined communities.”

Andy Green has looked at the effect from education on social cohesion across the globe.

Andy Green is Professor of Comparative Social Science and Director of the Center on Learning and Life Chances at the Institute of Education, University College London.

Citation: Green, Andy, interview with Will Brehm, FreshEd, 52, podcast audio, November 21, 2016. https://freshedpodcast.com/andygreen/

Will Brehm 2:01
Andy Green, welcome to FreshEd.

Andy Green 2:04
Well, Will, it’s pleasure to be here to talk to you.

Will Brehm 2:07
What is social cohesion?

Andy Green 2:11
Well, social cohesion, fundamentally, is all those properties which bind societies together. So, it might well be common values, common lifestyles, common identities. It may simply be the rule of law. You can define it in quite a number of different ways, and I would say you need to define it quite broadly, though, as the set of attitudes, values, and behaviors that binds societies together. Bearing in mind that social cohesion actually works quite differently in different places. So, I wrote a book some years back called ‘Regimes of Social Cohesion’ with my co-author Jan Janmaat, and we were looking at countries across the developed world, basically, at what factors seem to be holding their societies together. What was the nature of social cohesion in those different societies? And it transpired there were groups of countries basically, in different regions which were quite distinctive. So, for instance, English-speaking countries -which aren’t a region, of course, but have a common cultural history- tended to have a core set of key values, not a strong emphasis on a broad set of common values, because they’re very diverse societies generally. And the key value tends to be about opportunity, and rewards for merit, if you like meritocracy, some people would say, which is a little bit different from what you’d find in countries with Republican systems like France, where there’s much less emphasis on social mobility, but more emphasis probably on equality of outcomes. And where a dominant lifestyle traditionally, with a broad set of values, has been rather more important. Although this is clearly under strain these days. And then again, you get Scandinavian countries where identity and social cohesion revolves very much around their particular forms of welfare state, and a very, very strong belief in those countries in equality of outcomes. And you can find different things again, if you go to East Asian countries, for instance, where you can provisionally sort of identify a form of social cohesion based very much on Confucian values. So, here it’s respect for elders, respect for the state, and so on, as well as a common sense of cultural identity are extremely important. So, it would seem that social cohesion does work somewhat differently in different countries. People do try to develop a single definition, which is okay, in my view if it’s a very broad definition. They also try to measure it with a single set of measures, which is rather difficult because it’s basically a cluster of different things. And the cluster of different properties may be different in different countries. And so, having a single scalar measurement is rather difficult. Very often, though, the key measure is taken to be levels of social trust. That’s to say how far we trust other people, and particularly how far we trust people we don’t know.

Will Brehm 5:43
What’s the connection between social cohesion and education?

Andy Green 5:48
Social cohesion has always been a primary aim along with economic development, and so on and so forth. It’s almost always been there somewhere in the big visions for education systems.

Will Brehm 5:59
So, governments, when they create policy, they would not only see their education system as producing future laborers in the national economy but also creating citizens and members of society that would come together in some form, broadly defined notion of social cohesion with these different values being emphasized in different countries.

Andy Green 6:25
They would. It’s been typical of all newly created public education systems from the early 19th century onwards, that forming of citizens was one of the main purposes of an education system as well as you know, developing skills and so on and knowledge for the economy. It tends to be particularly important in new states, which don’t have a common identity firmly established, or institutions firmly established. Young states always emphasize the forming of citizens is a primary role of education. In older democracies, it’s tended to take second seat to economic development. I suppose skills formation becomes more important than citizen formation, but it’s not disappeared, that’s the important thing. Almost every document that you’ll find coming out of national governments, and indeed, transnational organizations, will refer in some way to the importance of cultivating social cohesion and citizenship and so on through the education system.

Will Brehm 7:34
So, would you be able to say that social cohesion is, in a sense, a global policy of education?

Andy Green 7:43
Yes, I would, but it’s referred to in different ways, perhaps in different international organizations depending on their remit. So, the OECD, the European Commission will use the word social cohesion quite frequently. OECD likes the term social capital as well. If you look at the outputs of, say, the World Bank, or UNESCO, where they’re looking at a wider range of countries, including less developed countries, peace education might well be the main focus, or post-conflict education, education in conflict societies or post-conflict societies. They’re all talking, though, about different forms of social cohesion, really, with different emphasis.

Will Brehm 8:30
So, what sort of education initiatives exist for peace education, or citizenship education, or social capital? Like, concretely, what are these educational initiatives that are either being promoted at the global level or within the national level or perhaps both?

Andy Green 8:51
Well, at the national level, most countries have citizenship education as part of the curriculum. It may be cross-curricular, it may be a separate subject, but it’s there in almost all countries. There may be allied to that a number of procedural things around schools, say, which are designed to boost cooperation amongst children and common understandings and so and this may take the form of schools, councils or these kind of things -so structures will be part of it as well as the curriculum. Other initiatives have been tried in countries in the throes of conflict or in post-conflict societies. Common education across divides, say in Northern Ireland during the troubles, there was quite a strong initiative to build a set of schools which would cross the Catholic-Protestant divide and so on. This kind of thing has been tried in various different societies. Education about the values of peace and so on is something that’s often tried in many conflict-ridden and post-conflict societies. But you could add to that broader concerns about the social society, if you like, and how education could contribute and include environmental education. Notably, education on relationships, on birth control, and family planning, and so on in poorer countries, would be seen as part of education for social cohesion because of its proven very beneficial effects in raising the esteem of women and in reducing population growth, and so on, which it can have all sorts of, you know, social benefits. So, I would extend the range of social cohesion policies quite widely in that sense if you’re talking about developing countries.

Will Brehm 10:59
What has been the efficacy of some of these initiatives? Like, are they creating more social cohesion?

Andy Green 11:08
Well, this is where you have to start making this difficult distinction between what benefits individuals and benefits society as a whole. Politicians tend to start with the easy part, which is measuring the benefits to individuals, and certainly for developed countries -and it probably applies for developing countries as well, although we don’t always have such good data to show it- the general pattern is that more educated people tend to be more tolerant of other values and lifestyles, they tend to be more politically engaged, they tend to show better signs of health, and fewer signs of depression and negative aspects of health. They’re less likely to be overweight as children, and the list goes on and on and on. They tend to vote more in countries as well. So, at the individual level, higher levels of education, particularly up to degree levels, are certainly associated with social outcomes, which most people would consider good. But there are questions about that way of looking at it. There are two sorts of questions. One is ‘was it the education which actually caused them to have those social attributes, or did they have them already? ‘ Is it to do with who is selected what you call selection effects in statistical analysis, which is an important issue because it may be education is not adding that much, it’s the people themselves who go into higher education, for instance? The evidence, though, tends to suggest there is an effect from learning, as well as any effects from selection. So, at the individual level, yes. There are clear benefits to individuals from higher levels of education. They’re likely to be more engaged politically, they will trust more generally, they’re more likely to vote, and they’re likely to be healthier.

Will Brehm 13:21
So, how do we apply that from the individual to the social benefit? Can we aggregate? Can we just look at all of these individual benefits and say, we know if most people in society have these individual benefits, then we have some level of social cohesion? Is that possible? Is that how the policymakers do it?

Andy Green 13:43
That tends to be the start of the reasoning, but it rather quickly breaks down because many of these properties don’t aggregate for one reason or another. So, I mean, to take a quite commonly cited example: generally speaking, more educated people in most countries are more tolerant. They’re more tolerant of other sexualities, other lifestyles, other religions, and so on, so forth. That doesn’t apply in every country, but in most countries, there is a relationship there. However, better- educated countries are not necessarily more tolerant in the aggregate. And what we see in many countries now is increasingly educated societies judged on people’s qualification levels and declining levels of tolerance. So, something else is intervening…It’s not that education is not promoting tolerance; it probably is, but other things are working against it. So, you can have increasing levels of education at a societal level, but it’s not showing up in increasing levels of tolerance. So, you then have to start asking difficult questions about what else is involved in the context which may be working against it, and how far the individual’s tolerance anyway aggregates at the social level to more individuals being tolerantly to a more tolerant society, generally other things being equal. That’s where the difficult questions start to enter in.

Will Brehm 15:22
And so, what sort of theories have been put forward to explain social cohesion at the group level rather than at the individual level?

Andy Green 15:33
Well, you have to start talking about what social scientists call “mechanisms.” So that was the transmission mechanism for individual values, affecting societal values, and so on. And the research that’s done in this area more or less identifies four kinds of effects. The simplest will be what I’ve described already, for tolerance. It will be what we call absolute direct effects. So, a direct effect is where education through learning, or socialization, or whatever’s happening, is having a direct effect on somebody’s attitudes, which carries through into later life. That’s a direct effect. It would be an indirect effect if education was increasing people’s employability, got them better jobs in later life. And if it were the better job that were raising their levels of tolerance. So, an absolute direct effect is one that occurs without any mediation from anything else. And being absolute, it means that it should aggregate. So, most of the research on tolerance suggests that actually, in large part, it is an absolute direct effect. And the argument made by psychologist would be along the lines of higher education raises your cognitive abilities, which makes you better able to disentangle poor arguments, to see behind stereotypes, to challenge the logic of prejudicial kind of thinking. But the second part of it will be education simply, as they used to say, broadens your horizons, you get more experience of different types of people in different countries, different living situations, even if at a distance, you know. You have a wider experience, and that in itself is said to make you more sympathetic to difference and, therefore, you know, the other person. So, education there is making a direct contribution immediately. It may make further contributions later through helping people get jobs, which may make them even more tolerant. It may. But the complication there is that there are countervailing effects from other things. Which may mean that even though education is increasing amongst individuals, and at the societal level, generally, you don’t see rising levels of tolerance. But it’s still a direct effect. It doesn’t mean that education is not helping, it just means it’s not helping enough to counteract things which are having a negative effect. So, that would be the simplest kind of aggregation mechanism. So, you could say that education here is certainly contributing to an aspect of social cohesion. Now, of course, in some societies, such a high value isn’t placed on tolerance as in others, and social cohesion may not rest on it to such a degree. But in most Western societies, tolerance would be fairly key. Another situation would be where you have these direct effects, but in addition to that, you have other effects, which result further down the line from the impact of your education on employment. So, to take the tolerance example, again, you may have been socialized and learn towards to being and having more tolerant attitudes, your higher qualifications may also get you a higher-level job, which may in itself promote more tolerant attitudes through various psychological mechanisms, maybe you feel less threatened by others and so on and so forth. So, that would be a kind of a cumulative process. And then, rather different is what we call relative effects or positional effects. And this is where education is having an effect on something, but it is not the absolute level of education you achieve that has the effects. It’s the level of education of yourself in relation to others. So, it may be that some of the benefits of education result from the fact that you are better educated than others, you get a better job than others. And it’s that better job compared with others that has the social benefits for you. And in that kind of situation, it can be a zero-sum game. You can have more and more people being educated, but it’s only the best-educated who get the benefits. And the classic illustration of this would be work that’s been done by Nie and his collaborators for the US actually on political engagement. And their finding is really that the effects of education on being engaged politically in key activities, and they’re talking about belonging to parties, campaigning for parties, voting, of course, having particular influential roles in parties. They’re saying that actually, this is promoted by education, but it’s strictly positional, in as much as only some people can be at the center of the action, and only the people at the center of the action are going to have real influence. And that, again, is a positional matter. So, it’s only the very best educated, who will get those key roles in party campaigns, that will get to advise people, who will get the ear of important people, who will influence policy and because of all those opportunities they’re having, they’re more likely to be interested and engaged to do those things. And the argument these authors make is that it’s a zero-sum game, basically. There are only a certain number of what they call network central positions where you can have an influence. And twice as many people may have degrees, but it’s still only going to be that small percentage who are best connected and best educated, who will have the effect. So, this is how they would explain why in a country like the US, where people are more and more educated, actually levels of political engagement are going down in actual fact. Voting levels are going down, and so on and so forth. And what’s more, the younger generations, who are more educated than older generations, are less prone to be politically engaged. And that’s why it’s a relational thing, it’s positional. So, not everybody can be the best educated. So, in this case, the effects of education on individuals is not translating into societal benefits at all. It’s translating into some benefits for some people. That’s true probably have quite a lot of the things we think of in terms of the social benefits of education. They are probably of that nature, they are positional. And one other possibility, which is really quite different, actually. But it’s quite important nevertheless, is what you might call or what I would call distributional effects. And this is not to do with individuals. Individuals don’t have distributions. It’s to do with how education and skills, the outcomes of education, are distributed across society. And whether it’s that very distribution that affects social cohesion. And we have done some research on this, which follows a similar logic to a lot of the research done by people like Wilkinson and Pickett on the effects of income inequality on social outcomes. So, the very popular and widely disseminated work of Wilkinson shows that if you look across a range of societies, societies, where incomes are more equal, generally have better health, lower childhood obesity, lower suicide rates, lower mental health problems, they have higher levels of political engagement, and the list goes on and on and on. I mean, so many social benefits, some of which you’d associate with social cohesion, are related to lower levels of inequality. Trust, by the way, is primary amongst those. So, it would seem that it’s hard to explain exactly what’s happening. But it would seem that lower levels of inequality and income, anyway, is important. While in the same way, lower levels of inequality and skills may be important to achieving certain social benefits from learning. And there is a psychological theory behind this, which is quite plausible. It’s quite hard to prove statistically, but certainly, there’s a theoretical model of its believable, and that basically is about in societies with very unequal levels of education, very unequal levels of skill, the social distance between groups of people at different levels tends to be greater. Therefore, compensation tends to be more difficult and suspicion is higher, and trust is lower. But at the same time, in unequal societies, you have more high-stakes competition. There’s much more at stake in any given competition over resources, jobs, housing, whatever it is, there is more at stake quite simply because the top end is a long way away from the bottom end. And the argument that social psychologists make about these situations is that -and actually, you can find it in behavior of animals as well- if you put a lot of people in high stakes competitions, where there’s high levels of inequality, there’s high levels of stress and anxiety, and stress and anxiety is associated with all sorts of negative social outcomes, particularly negative health outcomes. So, those would be distributional kind of relationships which is education skills and positive social outcomes. And here, it’s not how much you educate any individual that matters, it’s really how you spread the education around.

Will Brehm 26:51
So, of these four approaches, the absolute direct effects, the cumulative effects, the relative effects, and the distributional effects, which approach do you normally think is the correct way to approach social cohesion?

Andy Green 27:09
I think it depends on the case. It’s horses for courses. You’re looking at particular values in each case, things you can measure. And in the case of tolerance, I think the argument is pretty strong that it is a direct effect of education, which can be affected by context, as well. But it is an effect of education. If you’re looking at, as I said, something like political engagement, it is pretty positional. And there may be quite a lot of other effects, which are in the same way positional. If you’re looking at trust, I will put that in amongst those things where you have to look at the distributions, how education is spread around. It seems just if not more important than the actual levels of education in the averages in a country in cross-country studies. And this is where policymakers lose touch slightly, I think, because they’re inclined to only look at individual direct effects. It’s easier to comprehend. They’re a bit suspicious of distributional matters even though the evidence is there.

Will Brehm 28:22
Talking about social trust, how is that even measured? How do you go about measuring social trust in a society?

Andy Green 28:30
Well, social trust has been measured, actually, for quite a long time. Going back -for Western societies- at least going back to the 1960s. There was a study by Almond and Verba about civic culture, which had some of the first measures. And it’s been measured, going through with the European value survey and the world value survey starting not long after that, right through to the present day. And they generally asked the same question, which is, how much do you think other people can be trusted? Or would you never be too careful? Or variance on that? So, there’s a pretty standard question they ask people, and it does seem to be tapping into the core aspect of what you’re looking at, which is ‘do people trust other people they don’t know?’ So, social trust is not about trusting members of your family. It’s not about trusting people you know well; it’s not about trusting institutions. It’s about trusting people you don’t know. And it can be, it’s been measured. And it varies across countries very substantially. And it changes over time rather slowly. But the differences are very marked. So, in Scandinavian countries, typically 70 or 80% of people will say they generally trust other people. In some Latin American countries, it goes down to 20%. And we can do various tests to see if they are answering the question in the way we are meaning it to be asked. And it does seem that they are talking about the same thing, which is do they trust strangers. And it turns out that this is fairly crucial to -certainly is central to social cohesion. It’s the only thing that everybody would agree probably is a measure of social cohesion. But it seems to be very important to economic life, GDP growth has been related to social trust, clearly, it’s much easier to conduct business if you can trust people, legal costs are low, and so on. It’s also been linked to innovation and economic life because people cooperate together better. And it’s linked to a whole range of positive social outcomes, not least good health, and, of course, low levels of conflict. It’s almost the inverse of conflict. So, trust is a definable and a measurable phenomena. You could also measure it ‘trust for institutions,’ political trust, which is slightly different. It is important, it has consequences, and it is a key part of what most people mean by social cohesion. And it varies a lot across countries.

Will Brehm 31:30
And so, what would be the connection of social trust to education? Do people learn social trust in schools, or are there other factors learning social trust in families or in religious institutions?

Andy Green 31:48
Well, that is a very good question actually and not easily answered. I mean, a lot of people who write on this will say that trust is a very fundamental attitude, it’s learned early in life, it’s about childhood socialization, you basically learn to trust through trusting your parents, and other members of your family. And I’ve no doubt there’s some truth in that. So, we’re talking about really fundamental socio-psychological properties of people. And in one respect, trust is almost synonymous in that sense with what you might just call normally optimism. You know, you generally expect to get the best out of any situation. Because you feel in control, because you trust others not to cheat, because you think they won’t cheat with you, which has a lot to do with your position and social status, but also your just levels of confidence. So, those childhood influences are extremely important. But the research also shows that circumstances in adulthood do change people’s levels of trust. So, you know, if they have bad experiences as adults, they can move quite a lot on the trust scale. And social contexts then are making a difference, whether it’s childhood contexts or adult contexts. And there is a remarkable and quite wide shifts going on. So that, for instance, in England, something like I think 50%, 60% of people used to say they trust other people, say when I was growing up in the 60s. This is now down to below 30%, which you could argue is a real culture shift. You know, we live in a different type of society now where people are much less trusting of other people. Why does education affect it? Well, that’s not so easy to answer either. It does seem to be true in most countries that more educated people are more trusting. It’s probably partly to do with the fact that they’re more confident and more optimistic. It may be because they’ve been able to rid themselves of some prejudices which might otherwise stop them trusting people. It may be because they’ve been through a school system that really puts a very strong emphasis on people cooperating. And one of the interesting things about the Nordic education systems is that they have children that that cooperation is absolutely a central purpose of learning with young children particularly. And they keep children in the same classes through the whole of school with the same sets of children, the same teachers. And the purpose of this is very much to promote a group of people who learn how to work with each other to share and to cooperate. So, probably it’s both the behavioral things you learn in education which affect it and also the more general effects of education on your levels of confidence. As well as maybe some cognitive things, some knowledge, which is beneficial. So, it’s quite complex, but there’s no doubt the more educated people in most countries are more trusting.

Will Brehm 35:11
So, if social cohesion is a global policy, and most national governments are thinking of something about social cohesion in their education policies. And if social trust is a good measure of social cohesion, what sort of advice would you give policymakers about accurately thinking about social trust and education?

Andy Green 35:41
Okay, well, it is, as we’ve discussed, a key aspect of social cohesion. It should be at the center of concerns and possibly more than it is in policymaking. I would say two things. I mean, behavior is important. Cooperation in schools is important. So, you know how children relate to each other in school, how they’re taught to relate to each other, is building a foundation of cooperation and trust. So, it’s not just about the syllabus, it’s about the hidden curriculum, and so on. It’s about the rules, and the way schools are organized, which is very important. But the other point I’ve made, which arises out of our research, is that you really have to address not only the average levels of education as a society, which is the obsession of most policymakers because of the PISA test and all the rest of it. You’ve got to look at how education is distributed. Countries with very, very unequal distributions of skills are going to be unequal societies, and they’re going to be less trusting societies. We know this almost certainly applies across the developed world in any case. So, you have to look at both. You have to try to raise the general standards of education and skill, but you have to look at how these skills are distributed and try to reduce the rather large gaps that are actually increasing in social gaps in skills in many countries.

Will Brehm 37:17
Well, Andy Green, thank you so much for joining FreshEd.

Andy Green 37:21
Thank you.

Want to help translate this show? Please contact info@freshedpodcast.com
Have useful resources related to this show? Please send them to info@freshedpodcast.com

 

OverviewTranscriptTranslationResources

We often think of international assessments as being synonymous with PISA, the OECD international assessment that has been the focus of many shows in FreshEd’s mini-series on global learning metrics.

But international assessments have a history far beyond PISA. In fact, it was the International Association for the Evaluation of Educational Achievement, known as the IEA, that first introduced large-scale comparative studies of educational systems in the late 1950s.

This history is important to consider when thinking about global learning metrics today.

My guest today is Dirk Hastedt, Executive Director of the IEA. He’s spent many years working with the IEA, seeing the development of assessments in new subjects, such as citizenship and computer literacies, and the emergence of league tables, which rank education systems and have become popular today. Drik offers valuable insight for any discussion on the feasibility or desirability of global learning metrics.

Citation: Hastedt, Dirk, interview with Will Brehm, FreshEd, 49, podcast audio, November 7, 2016. https://freshedpodcast.com/dirkhastedt/

Will Brehm 1:54
Dirk Hastedt, welcome to FreshEd.

Dirk Hastedt 1:56
Thank you, Will. It’s a pleasure for me.

Will Brehm 2:00
The history of international assessments reveals many different actors and debates about the role and purpose and presentation of assessments that we sometimes forget today. I mean, today, we often talk about PISA or TIMSS, but we don’t remember the history of those tests and international testing in general. Can you give us a brief kind of background to the history of international assessments?

Dirk Hastedt 2:26
Sure, Will. It’s a pleasure. I think it’s also very important to go back and think about what it’s all about. Well, the IEA’s origin dates back to 1958 already. And it started as a collaborative venture between measurement scholars, educational psychologists, sociologists, and psychometricians from different countries. They first gathered at the UNESCO Institute for Lifelong Learning in Hamburg, and then also in London, they met a couple of times. And the idea was to exchange ideas about educational processes in order to improve educational systems. And at that time, little was known about the process of education across the world. And the founders of the IEA proposed to engage in cross-national comparisons to student achievement, as they believe that the country’s educational systems can better be understood when comparing educational systems of different countries. And their opinion was that surveys must take into account not only inputs but both input and outcomes of educational processes. And outcomes were defined quite broadly, as achieved knowledge but also as attitudes and participation in education, which now is also another focus that we have in developing countries. And their interest was to collect data that would enable to better understand educational systems and to identify relevant factors that had an impact on student learning. So, the target of their analysis was notably the educational system, not the individual student. And to facilitate further cooperation of researchers, finally, the IEA became a legal entity in 1967. And that’s when all of this started.

Will Brehm 4:25
A legal entity of a particular country?

Dirk Hastedt 4:29
Well, that’s an interesting question. At that time, the researchers tried to found an umbrella organizations to do international research with international people from around the world. And so, they finally decided that this is only possible under the Belgium law. So, it first was emerged as a Belgium association with board members from different countries actually.

Will Brehm 4:59
And you said that originally it was focused on improving education systems, not the achievement of individual students. So, has that changed over time?

Dirk Hastedt 5:12
No, not really. The focus is still on the system level. But the focus always was on comparing systems, and not as we sometimes see today, the competition of systems. So, who is first in league tables? That was never the intention, and I think it still should not be. The question was, what can we learn from other countries’ practices? And in that sense, at that time, it was unclear if cross- cultural and cross language assessment at the beginning were feasible at all. So, they first conducted a study just to prove that this kind of assessment makes sense at all in general.

Will Brehm 6:05
Can you tell us a little bit about that first test? What exactly was uncovered and found?

Dirk Hastedt 6:11
Oh, that’s a good question. The first study was the pilot 12 country study. And that was done in the 1960s beginning. And the subjects included mathematics, reading comprehension, geography, science, and also non-verbal abilities. And the first study, however, started with mathematics because, at that time, the researchers also believed that the subject mathematics would be most language and cultural independent, and consequently, easiest to assess in international study. Later on, the six subject study that was conducted in 1970, and 71, expanded also the scope to science, reading comprehension, literature, education, English as a foreign language, and civic education. So, you can see that the target was very broadly defined in the beginning.

Will Brehm 7:18
And is that target still broadly defined today?

Dirk Hastedt 7:23
Well, there are different projections taking place. One is it’s always a question of what’s a focus in a particular time. So, for example, in the 1990s and late 80s, computers in education also became a focus. So, the IEA, at that time, started a ‘Computers in Education Study,’ COMPED, which was conducted in 1989 and 92, and provided data on the educational use of computers. This trend also was followed later on with our computer information study size and also the current International Computer Information Literacy study. And also, political changes in the world of the 1970s gave rise to the subject of citizenship. As a response, the IEA conducted, at that time, the civic education study, CIVED, and it investigated civic knowledge and engagement and policies and practices.

Will Brehm 8:39
So, different subjects or topics become of interest to the IEA depending on what’s going on in the world. So, today, what’s going on in the world? And what are the topics that are kind of trending?

Dirk Hastedt 8:56
Well, actually, there’s a little shift also in the 1990s, which we sometimes called the empirical shift also. And at that time, international large-scale assessment started to be used increasingly by policymakers, also outside the domain of education. And also, at that time, an extension to more developing countries took place. Before, these studies were more academic studies, and support from policy side was not that big. But at that time, this whole environment changed. And on one hand, this also had a change of the focus in skills like reading and numeracy, which are seen as preconditions for further learning of students. So, without reading abilities, textbooks and other areas can’t be read and understood. But also, economists were interested in relating educational outcomes to economic wealth. And this also resulted in a focus and change of interest of subjects to subjects that are useful for employability and regarded as preconditions for economic wealth. And here again, reading, numeracy, and also science became a focus of international assessments.

Will Brehm 10:32
So, it sounds like these assessments, they change for all sorts of reasons over time, and different actors maybe get more influence inside the testing agencies. And one of the moments that you referenced earlier was when league tables came about. And you were part of the IEA when this debate was happening. Can you give us a little insight about what were the different sides of the debate to either include or exclude league tables?

Dirk Hastedt 11:08
Oh, that’s a very good question. League tables were highly debated also in the studies, and that most of these debates took place in the 1980s, actually already, and beginning of 1990s. The IEA was always interested in understanding educational processes, and which education system is first or second or seventh doesn’t matter too much, actually. And it’s also not very informative for most policymakers. So, consequently, in the early studies, the IEA produced reports and related background instruments and background information to achievement but did not produce league tables. But then, the IEA realized that if they don’t do it, other people created league tables. And we found league tables in magazines, newspapers, and also in some academic journals. But these league tables were also then created in a wrong way. So, they were wrong. So, at that time –

Will Brehm 12:21
How were they wrong?

Dirk Hastedt 12:23
Well, we have a very complex system for assessing the achievement. For example, we use rotated booklet. That means that we want to cover different areas, and consequently, not all students are getting all items, because then every student would have to sit down for six or 10 hours to cover all the areas, which we, of course, do not do. So, that’s one thing. And secondly, also, we are taking a random sample of schools. And then, within the schools, we are sampling one or two classrooms with students. And the researchers call this a cluster sample. And a cluster sample is very different psychometrically from having a simple random sample. And we use different mathematical procedures to calculate, for example, standard errors, or if differences are statistically significant different. And when not considering this, you get wrong results, and you consider results different that are statistically not different. So, there are some parts that really require a good and deep understanding of the projects and its procedures.

Will Brehm 13:46
So, these league tables that weren’t necessarily considering all these statistical and methodological considerations as the IEA was considering. These league tables were basically being produced with potentially wrong information. And so, the IEA decided that it needed to jump in and actually produce accurate tables?

Dirk Hastedt 14:08
That is right. So, when we saw that other people are producing, obviously, these league tables anyway, because of interest. Then the IEA decided, well, if it’s done, then it should be done correctly. So, we decided to include this also in the report.

Will Brehm 14:25
So, over the last 30 years or so, when these league tables have been included in the report, what sort of effects or outcomes have you seen being used of these league tables? Because this is, for the most part in like the popular press, this is what we read about. We read about which school system or country is ahead of another country. And so, these league tables have really, in a way, have become the defining feature of these international assessments.

Dirk Hastedt 14:56
Yeah, well, I think there are different components and different things that our international large-scale assessments are good for. On one hand, of course, this international large-scale assessment describes the status quo in terms of how educational practice in a country is conducted. And this helps researchers and policymakers to compare a country to other countries. And then there’s a comparative perspective. So, identifying differences in ways in which education is organized and practiced across cultures and societies. Then this gives you an understanding of where you are actually. Secondly, I think international assessments like our Trends in International Mathematics and Science Study (TIMSS), or the Progress in International Reading Literacy Study (PIRLS), or the ICILS study, and the Civic and Citizenship Education study that I mentioned, they are measuring trends. And this helps policymakers and researchers to understand changes in educational outcomes and processes, also in comparison to other countries’ development. And especially when curricula or educational policies are changed, it is important to monitor changes. Imagine a captain of a big ship on the ocean. Which captain would not look at his instruments or look outside the window to see where his ship is heading to? And I think there is also the value of maybe also the league tables if you take them and interpret them correctly, in its context, and also looking at trends over time.

Will Brehm 16:42
So, looking at these trends and monitoring and describing the status quo, have these helped policymakers, do you think?

Dirk Hastedt 16:53
Oh, sure. I think policymakers also have seen different impacts from these studies, actually. And it depends very much on the country itself, actually. Since we are research organizations, we are conducting these studies, and we also help countries understanding and interpreting the data. But we do not give any policy recommendations from our side. But we help the countries to do that. And the focus in different countries is very different. We have some countries which, even before the actual assessment start, when we look at the curriculum, they realized that their curriculum is not in line too much with other countries curricula in terms of curriculum expectations. So, some countries, when they looked at this, changed the curriculum expectations and have now higher expectations for the students to be also international comparable. Other countries looked at different sub-populations. For example, immigrants, or they looked at boy-girl differences. So, there’s a lot of questions around equity in education. And policymakers, after looking at the results, took measures to provide more equal learning opportunities to students. One example that we can see is that there was always a discussion about differences of boys and girls in mathematics and science in particular. And 20 years ago, in most countries, boys were performing much better than girls actually. So, researchers and policymakers asked, is this a natural given that boys are better in these areas? And actually, it is not. But it depends very much on motivation, and what are the textbooks, and what are the examples that they used in textbooks? Are they also engaging for girls? So, in a lot of countries, there’s now a shift also to have more relevant and interesting materials that also engaged student girls more in science and mathematics education with the outcome that now the girls are doing as good as boys in most countries. And in a lot of countries, girls are even outperforming the boys in mathematics and science. So, there’s a lot of different conclusions that policymakers have drawn from looking at the results of these international large-scale assessments like the IEA studies.

Will Brehm 20:01
Have there ever been any misuses by policymakers of large-scale international assessments?

Dirk Hastedt 20:10
Misuses. Well, I think it’s not only maybe a question of misusing the data, but maybe also misunderstanding the data. What we do is we do statistically analysis. And there’s always a certain error margin around it. So, coming back to this league tables, again, some of the misunderstandings around these league tables by policymakers, or also the general public, are that if a country is one score point better on a scale with a mean of 500 and a standard deviation of 100, it is better. But we always have a measurement error and sampling error around it. So, there’s some variance around it. So, if one country’s achievement is one score point better than another one, or if one country’s achievement increased by one score point, that doesn’t mean anything, actually. But it might just be a question of the error term around it. So, actually, no change has happened, but it appears as if there would be an increase. But this is statistically not significant, and consequently, there are some misinterpretations of these results if you look only at league tables.

Will Brehm 21:48
So, in your experience, how many policymakers actually understand the statistics behind these studies?

Dirk Hastedt 21:55
Well, probably not many policymakers. And actually, I don’t think it’s something that they need to understand, actually. But I think it’s also something that researchers in the field have to explain to policymakers. And this is probably one of the crucial aspects to communication between researchers and policymakers. Policymakers have their perspective, and they want to have a clear and straight answer. But educational systems are quite complex. And if you look at the outcome of the studies and what researchers think about it, their answer is usually not that simple. And simplifying the results of educational studies in a way that it’s not over-interpreting the results of the studies is very difficult and challenging. So, I think we need more and more to pay attention about how to communicate results and a good communication between researchers who have the technical terms, and policymakers and the general public.

Will Brehm 23:17
But of course, we also have to be realistic that policymakers are navigating domestic politics and may use these international assessments to further their own interests, even if they use a finding in a wrong way or something like that. That will happen, and I think many researchers have shown that that happens.

Dirk Hastedt 23:41
I think you’re right. There’s also, maybe some misuse from policymaker side. But my experience, actually, is that this happens sometimes in a way that policymakers want to look good from the outside. So, they want to have good results for their educational systems, which I think is very natural. Everyone wants to have good results from the work that you do. But I think when it comes to making use of the results, there’s a strong focus from a lot of policymakers that they really want to improve their system. And this can be only done if you really interpret the results correctly. So, maneuvering blind and pretending you’re good might be good for that day, if there’s a day of election, but in a long-term process, you need to understand the educational system. And I think this is also what policymakers understand. That they use the data, and we call it “data-driven policies,” to look what are the strengths and weaknesses of their systems, and then, by that, also try to improve their systems. And if you look at policies today, they really make use of the data with the aim to improve educational systems. And this is, I think, what policymakers are mostly doing.

Will Brehm 25:20
What I find so interesting about the IEA and the history that you’ve recounted is that in the beginning in 1958, doing these cross-national assessments was very much an academic pursuit. You know, is it possible? Can we statistically compare education systems in different countries? And it seems, over time, it’s shifted to being not only an academic pursuit but also very much an issue of policy, and it seems like the role of the IEA, in a way, has slightly changed.

Dirk Hastedt 26:03
That’s a good point, I think. I don’t know if the role of IEA has changed. But it’s clear that we have member institutions, and these member institutions are sometimes research institutes from different countries, and we have member institutions from more than 60 countries. But it’s also people from ministries of education, or institutes that are connected directly to the ministries of education. And surely, they want to learn which policies can help improving achievement in their countries, which is very different from maybe the researchers point who for purely academic intents how … want to understand how educational processes are working and to understand these processes more in-depth.

Will Brehm 27:05
It’s also interesting that originally the test was -mathematics was the subject. Because, as you said, it was, in a sense, “easier” to control for context and local variation, and language variation, for that matter, around the world. But today, you’ve now talked all about all these different subject areas that are tested. How has the issue of context and variation been controlled in these various tests?

Dirk Hastedt 27:40
Oh, that’s a good question. What we are using in IEA is, we are looking at what we call the curriculum model. So, we are looking at the intended curriculum in the different countries. So, what they have written as policy documents, what should be taught in schools, and what are the aims. Then we look at the implemented curriculum. So, what teachers are actually doing in the different countries. And finally, we look at the attained curriculum. So, what have students learned. And this is what we measure with our assessments. But there are also these other fields like the intended and implemented curriculum, and we look at this whole process. And of course, when curricula are changing in participating countries, then also, there is a slight shift in what we are measuring in our international assessment. And I think what’s also very important is to understand that in these studies, we are not only having an assessment, but we also have a huge amount of background variables. So, when we are doing one of this assessment, we are looking at system- level information. So, the curriculum and other relevant information. We have a questionnaire to the school principals of the schools where the students are in about how education is organized, what’s the school’s size and background instruments on that. We have questionnaires to the teachers of the actual students so that we can directly relate teachers’ attitudes, perceptions, and background to student achievement. And then, we also have background questionnaires for all the students. And in grade four, we also have questionnaires to the students’ parents. So, we have huge amounts of background information to analyze what are the factors related to student outcome. And outcome are still not only achievement but also opinions, self-confidence, etc.

Will Brehm 30:14
And this wide range of measures allows IEA, or the tests, to compare across nations and account for the cultural variation?

Dirk Hastedt 30:34
Well, the cross-country cultural comparisons are, of course, always challenging. Education always takes place in a different culture, in different societies, with a history and with a background. But what we can see is that there are a lot of similarities across countries. Sometimes more than you would expect at first glance. Let me cite one of the results from one of our recent studies, which was the Computer Information Literacy study, ICILS, where we looked at what matters most for the usage of computers and computer information literacy of the students. And we had 22 educational systems taking part in that study. And we found that in all these educational systems, the crucial points were the teachers and the teacher education and what they think about computers, and if they want to use computers, and how self-confident they were in using computers in their teaching. And that was the same across all 22 different educational systems. And we had participants from Europe, from Latin America, from Asia. So, for more or less all around the world, and you see that you find the same patterns across different cultures, which was a surprise to us. But also, probably, to a lot of countries because we often hear that, “Well, our educational system is very different. And we have a problem in our country that’s probably very unique.” And when you look at these international assessments, you can see that a lot of countries are facing very similar problems.

Will Brehm 32:45
So, you’ve been involved with the IEA for quite some time, and you’ve mentioned all of these different shifts that have happened. So, in the 1970s, the focus became citizenship, and in the 1980s, it became computers, and in the 1990s, you talked about this empirical shift, I wanted to see -kind of reading the tea leaves- do you see any shifts occurring now? Or do you envision any future shifts about what the focus will be for these international assessments?

Dirk Hastedt 33:20
Well, on one hand, I think that the current process will continue. But on the other hand, I think a huge influence currently is with the UN and the Declaration for the Sustainable Development Goals, where the UN declarations set under target four is concerning with education. So, there’s also in all member countries also UN which means mostly all the world. There’s a common agreement that education plays a more and more important role. And when we look at the target four of the Sustainable Development Goals, one is universal primary and more and more secondary education. And here I see, for example, also a shift on the UN side, which before looked mostly on enrollment rates, now also target minimum standards of outcomes in literacy and numeracy for all countries. So, I see that there will be an expansion also to more developing countries, but still with a focus on numeracy and literacy abilities. But under the target 4.7, it also targets, for example, global citizenship and sustainable development. So, it’s also seen that this is an increasingly important aspect, which we see. And I think that’s a very, very important thing to understand that education is not only with means of educating the future workforce, but education is much broader and also helps us to live together in peace and to also understand the needs of future generations. And there I see also a shift on looking also at global citizenship competencies around the world.

Will Brehm 35:54
Well, Dirk Hastedt, thank you so much for joining FreshEd.

Dirk Hastedt 35:58
Thank you very much. It was a pleasure to meet.

Want to help translate this show? Please contact info@freshedpodcast.com
Have useful resources related to this show? Please send them to info@freshedpodcast.com