Will Brehm 2:12
Eric Hanushek. Welcome to FreshEd.

Eric Hanushek 2:14
Will, thanks so much for having me.

Will Brehm 2:16
What are global learning metrics?

Eric Hanushek 2:19
Well, I don’t know if there’s any standard definition of global learning metrics. But I think what people use it to imply is how do we compare the education that we see around the world. And that for my own work, I’m interested in ways to compare one country to another.

Will Brehm 2:41
And so, what sort of metrics exist currently that people or researchers like yourself use to compare systems of education across the world?

Eric Hanushek 2:53
Well, perhaps the most common is just how many years of schooling does somebody complete in different countries. That turns out not to be a very good metric. Because a year of schooling in Peru is not the same as a year of schooling in Japan. So, in my opinion, better metrics involve what people actually learn when they’re in school, and in when what they learned when they’re out of school.

Will Brehm 3:23
And so, what metric would be used to measure what students learn in school?

Eric Hanushek 3:30
Well, the most straightforward is a series of tests that we now have to compare countries. Most people today know the PISA test – Program for International Student Assessment. But there’s historically been, what is now the TIMSS test – Trends in International Math and Science Study, which actually started out with a prior version in the mid-1960s.

Will Brehm 4:06
So, this endeavor has obviously been going for quite some time, the ability or the attempt to measure student learning cross-nationally.

Eric Hanushek 4:16
Absolutely, as I say, a certain mid-1960s, when some people asked the question ‘can we compare learning across different countries?’ and it was really rather experimental, but they gave math and reading test and science test to people in different countries, at various times they’ve applied to different ages and different subject matters. But it’s basically taking a set of questions of knowledge and learning or analytical abilities, translating them into languages of the home country and marching in them around the world and see how kids do of the same age and schooling levels in different countries.

Will Brehm 5:03
So, you say that the metric for student learning is more significant? Or is a better predictor of, I guess, we’re talking about economic growth in the future? Is that what these metrics tried to predict?

Eric Hanushek 5:22
I don’t think they were designed in that way. They were designed by some psychometricians, who are interested in just testing learning. But I’m an economist, and my interest was in finding out whether these tests were good measures of the capacity, intellectual capacity, or cognitive skills of people in different countries. And if so, if they explain some of the economic differences that we see across countries.

Will Brehm 5:52
So, can you give us a few examples of your research and what you found using these metrics cross- nationally?

Eric Hanushek 6:02
Well, I think that the most important part of the reason of my research has been look at differences in economic growth of countries. If we look around the world, we see some countries are a lot richer than other countries. What this richness largely is related to is the economic growth rates of these countries over time, you know, at some pre or prehistoric time, everybody had about the same wealth. But as time is going on, and in particular, over the last few centuries, differences in economic growth rates dictate income levels and economic wellbeing of people in different countries. So, my interest has been, can we explain those, that’s also been the interest of a number of economists, but they haven’t spent so much time looking at learning or these international tests as much as looking at just years of schooling and things like that, that they think might be proxies for the knowledge of people in different countries.

Will Brehm 7:15
So, why is looking at learning a better metric for or connected to economic growth than years of schooling?

Eric Hanushek 7:27
Well, I think it tells a lot more about the skills of people in different countries and the quality translated into the labor force and the scientists, and the ability to improve productivity. Economic growth is essentially doing more with the same amount of time and resources, its productivity, and it’s the growth in productivity over time that determines economic growth. Turns out that people with both more skilled scientists and engineers, and more skilled labor forces, those countries grow faster.

Will Brehm 8:07
So, what sort of skills have you found to be good indicators of economic growth?

Eric Hanushek 8:15
Well, that’s not quite the question that I’ve asked. Because these tests, if we take a PISA test run by the OECD, the PISA tests measure a variety of different math problems, for example, of different skill levels, but is not that I would suggest that people learn how to solve all the PISA test that those are the skills that are important. What these PISA scores are is simply a proxy for the cognitive abilities, the analytical skills of people in different countries, and the skills that come across in the labor force are developed both in society and in schools. So, universities take people with basic abilities and try to give them specific skills. They teach them how to be mechanical engineers, or how to be scientists, or how to be a good artist, or how to be a good economist. And those are the skills that ultimately play out in the labor force.

Will Brehm 9:35
So, you’re not saying that we need to identify particular skills that should be taught to all children across the globe for the best, quote, unquote, the best economic performance in the future?

Eric Hanushek 9:48
Absolutely. I’m not saying that. What I’m saying is that when we measure skills of, say, 15-year-olds in the PISA test or eighth-graders in the TIMSS test, they give us an index of the basic fundamental background that people that schools can build upon. And the simple answer is that colleges and universities develop more skillful people when they have people that come in knowing more. So, Stanford University has a very selective admissions program, and it allows them to produce some spectacular outcomes, some students with spectacular knowledge and innovative capacity. That wouldn’t be quite so good if they took people that were less prepared because the universities and earlier grades basically take some fundamental analytical capacities, cognitive skills and mold them into knowledge in different subject areas and different abilities to solve problems and do new things.

Will Brehm 11:41
So, you have this idea of the knowledge capital, which is related to human capital, which is pretty popular theory connecting economics and education. Can you talk a little bit about what you mean by knowledge capital?

Eric Hanushek 11:56
Sure, we use that term to denote aggregate cognitive skills of a nation. So, we take test scores and use those as an index of the cognitive skills of a nation, we had to change the terms from human capital, we changed the term largely because some early work in economics said, well, skills are important. People invest in skills, we’ll call that human capital, like investing in machines, you invest in yourself, and you develop new skills that allow you to do more. But very early on, when people tried to apply those concepts, they’re looking for ways to measure the skills. The easiest measurement that was readily available and still is most available is years of schooling. The problem is that years of schooling is a very poor measure of differences in skills. We know that if we look into individual countries like the United States as worried about the quality of its primary and secondary schools, what really is being talked about is how much do people know after they’ve had 12 years of schooling. That’s the quality. And so, just saying 12 years of schooling is not the answer, because we know that it varies by what school people went to, what their family background was, what state they’re in, their neighborhoods, and a whole variety of other things. Schooling is one element of developing these skills, what people synonymously refer to as human capital, but that’s what we’re trying to get away from, we’re trying to say that the skills that are important are developed not just in schools, but in the family and in the neighborhoods. And moreover, all schools aren’t the same. There are some schools that are better than others. So that if we really want to know what the capacity of individuals might be, we have to go deeper than just measuring how long did they sit in a classroom.

Will Brehm 14:15
And how good are these tests to measure learning?

Eric Hanushek 14:18
What turns out measuring the learning that’s important for economic growth. These tests are extraordinarily good. The simple answer comes from looking at the work that economists have done to explain economic growth. About a quarter of a century ago, economists got very enthusiastic about trying to explain differences in economic growth, and to have empirical models that actually explained why Korea has grown so fast relative to Algeria. And the first thing that people said was, it’s gotta be human capital, it’s gotta to be skills. And then some other things. Well, it turns out that if we measure human capital by just years of schooling, and call that the differences in human capital, that and almost anything else we want to include can explain about a quarter of the variation in growth rates across countries. If alternatively, I just take an index of their test scores in math and science, which are readily comparable across countries, I can explain over three-quarters of the differences in growth rates across countries. Now, there’s still some that’s not explained. And we see a lot of that in terms of questions about regulations of economies, the economic institutions, and other things. But in simplest terms, the majority of differences in growth rates across country can be explained by these indexes of what we call knowledge capital.

Will Brehm 16:11
Now, why is it that math and science are the subjects that are commonly used for these tests?

Eric Hanushek 16:22
We can develop questions that are pretty straightforward that are directly comparable. One example is that I, in the past, and still try to convince people at the United Nations that is talking about development goals, that they ought to have a quantitative measure of skills in their human capital and education component of the Sustainable Development Goals. One measure that we use is what we call basic skills. Basic skills can be easily explained by a simple math problem. If I fly to Tokyo to be with you, I would pay $3200 for my ticket from California to Tokyo, the exchange rate between the dollar and the euro is 1 to 1.1 euros, how much did my ticket or would my ticket cost in euros? So, it is a very simple math problem. It turns out that among the developed countries of the world, 20% of the 15-year-olds cannot answer that kind of question reliably. In the US, it’s 23%. In some countries, it’s much less. And, in developing countries, it’s much more. So, I was recently in Honduras. And in Honduras, 84% of 15-year-olds who are in school cannot reliably answer that question. What turns out to that? If we had something like that, it would provide a simple goal that could be applied internationally to say, ‘how are we doing?’ The Honduran government could apply that, the US government could apply it, the Japanese government could apply that if we wanted to say, ‘how are we doing toward meeting an education goal?’ Now, that’s not all we want to do; of course, we want to eventually have people learn a variety of other specific things. And we don’t want to drill them relentlessly on exchange rate problems in order to get them prepared. But we want to prepare them to answer simple mathematics problems, along with being in school. You know, the caveat I gave you before was, in Honduras, 84% of those currently in school could not answer that problem. But Honduras still doesn’t have universal access of 15-year-olds to school.

Will Brehm 19:16
So, it would be harder, I would imagine, to have a metric to measure something in the subject of history or social sciences that can be used cross-nationally. Is that right?

Eric Hanushek 19:32
Well, I think it’s almost impossible to do something like that. And in fact, I have my own questions about whether we can test reading capacity across countries. The international tests, the PISA and the TIMSS, or the relative of attempts include test of reading ability, but I myself can’t see how you can reliably compare people’s reading ability when they’re being tested in different languages with different language structure in different difficulty. And so, I generally think more about science and math. Now, it turns out, that it probably doesn’t matter all that much. Because if we look at individuals or at the country level, where math problems can be solved, reading is possible. People do both reading, history, geography, and math, pretty consistently across the board, the curriculum of a country will affect that a little bit. But in general, if we talk about skills across math, science, and reading, as tested, they’re all quite highly correlated.

Will Brehm 20:55
So, I do a lot of work in Southeast Asia. And I’ve noticed that the trend of late is for governments to really focus on STEM, on science, technology, engineering, and math, as being, I think in their minds about being the subjects or the skills that are needed to drive economic growth in the future. Is this, you know, policy connection of focusing on STEM connected to the ability for these global metrics to measure math as being a good predictor of growth, of economic growth?

Eric Hanushek 21:39
Well, I think that having a portion of your population adept in STEM fields is important that lots of innovation, and changes in production, and the operations of economies flow from having good engineers and good scientist. But it’s also very important to, we find, to have strong basic skills across the entire population. So perhaps an example is contrasting India to Finland. India has a very highly developed competition to get into their very best engineering schools, and they produce spectacular engineers. We hire a lot of them here in Silicon Valley. On the other hand, their general education, the basic skills for the other billion people in India, is not very well developed. And in Finland, they worry about both having top-end people and having a broad general skill base. It turns out that both having rocket scientists, people that are really at the top of the STEM fields, is important, but so is having a broad, general population with broad general skills. And interestingly, scientists are more productive if they have a good workforce to use their products. So, if they have a highly skilled workforce, their output in terms of economic gains is stronger. So, I think that across the board, having both fields is important. Whether for individual countries emphasizing STEM fields or not helps them to develop their broad general skills, I don’t know, I think that’s a matter of country strategy. To the extent that it’s done the way India has traditionally done it by only developing high levels of STEM fields, I think that is probably a mistake. It’s a mistake in two ways. One is that it doesn’t give as much overall economic growth. But it also leads to tremendously difficult income distribution problems. This is not inclusive growth, when in fact, a large portion of your eighth graders cannot read a simple paragraph, they are not going to be able to keep up in an advancing, growing society.

Will Brehm 24:39
So, you said that, you know, a lot of the country-specific strategy would determine what policies and what skills and subjects are pushed. But you’re saying that there’s a basic skill set that you think needs to be, you know, generally accepted at maybe the primary and secondary levels. Can you talk a little bit more about what basic skills you’re talking about? We’ve talked a lot about math and the ability to solve simple math problems. But are there other basic skills that you would look at, to be kind of … or you think are the essential sort of skills that societies need?

Eric Hanushek 25:24
Well, it’s hard to speak very specifically about these. What I have emphasized is primary and secondary school learning. Basically, consistent with the development goals of the world that we’re talking about, what you tell me know at the end of lower secondary schooling, or the end of middle school, in the US, the kinds of skills, that analytical skills, inferential skills, general reading and comprehension skills, comprehension in both reading sense across different fields, but also comprehension in basic math skills. These are the kinds of things that further education always builds on. And further education will be much more powerful if we have people that are better prepared early. I mean, what we found, in simplest terms, is that early learning has tremendous gains subsequently because everything builds upon what you knew coming in. Basically, you don’t study history if you aren’t pretty good at reading. You don’t study economics if you don’t know a little bit of mathematics and statistics. And that all builds upon having being able to solve that exchange rate problem, being able to read the basic instructions that are available. And this all builds together. So, I’m not advocating sort of a set of testable, final outcomes that we can write into our curriculum that we have to know how to integrate by parts. That’s not the kind of thing that I’m talking about, I’m talking about more, building up skills over time, and that this starts all generally in primary and secondary schools.

Will Brehm 27:35
So, basically, for you, the Global Learning metrics are valuable because they have the potential of improving the quality of schooling at particularly the primary and secondary level.

Eric Hanushek 27:48
That’s true. And then that has pays dividends both in individual productivity, turns out that, if we look across countries, we find that people who know more, earn more. It’s simply as simple fact that if people who test higher on some of these basic math and science test, systematically earn more throughout their career. We also, what that means, among other things, is if you have more balanced learning in society, so there’s people as are learning at their capacity, you’re going to help all the equity and inclusion and distribution issues that are currently being discussed. But you’re also going to help the overall economy.

Will Brehm 28:41
So how would you factor in inequality inside education systems in the sense that, yes, there are, you know, the more you know, the more you’ll earn. But there are a lot of people that know, a lot more than, say, some Stanford undergraduates, but aren’t able to get into Stanford, for all sorts of different reasons maybe related to poverty or where they came from. And so, privilege and kind of the safety net that that entails and the networks that that entails sometimes disadvantages, students who, although may be smart, but simply don’t have those connections. So how do you factor in that sort of the structural inequality that we can find inside education systems?

Eric Hanushek 29:31
Well, I think I take that as a fact that we have to deal with and that we have to develop policies, to, in fact, provide more equitable outcomes. But if I look at, if I look at admissions to Stanford University, large, private university, expensive private university, the real constraint is not in income and money to get in to pay the tuition. The real constraint is whether they have the skills and achievement to, in fact, get in. Now, that’s going to be related often to earlier family backgrounds. And one of the roles of schools is to try to ameliorate some of these differences that we see early on in people’s careers that come from the opportunities, through their family, through their neighborhoods, and so forth. But these are issues that every society faces to some extent. The economist thinks that, and a very important part is providing the skills for people to compete, as opposed to just trying to redistribute income or opportunities across groups, independent of the skills they have. And so, in the long run, if people don’t have the skills, they’re going to be disadvantaged, and they are going to be the ones who are left behind. Now we can deal today with the disadvantages we see by taking from the more wealthy, taxing them more, putting them into government programs, giving it to the poor. But if we don’t adjust the skills and the productive capacity of individuals when they’re young, we’re going to keep facing that same problem over and over and over again. But at the same time, we might also disadvantage our ability to do this redistribution because if we take policies that lower economic growth, we have less capacity to help out those that don’t make it.

Will Brehm 32:00
I want to return to the idea of the Global Learning metrics. Because it occurred to me that, you know, issues of inequality need to be addressed at the policy level of particular nation-states. And the subjects that should be pushed inside education system should be done through government strategies, also at the nation-state level. So, the question I have is, why do we need to do tests like PISA on the global scale? Could this not just be done at the national level?

Eric Hanushek 32:34
So, many countries have national tests, and more should have national tests. In simplest terms, you can’t improve a system if you don’t know where you’re at. And so, having within-country metrics are often crucial to making good policy decisions to improve outcomes. On the other hand, if I’m sitting here in Stanford, and my kids are going to the Palo Alto schools, I have no way to know how they compare to my co-author’s children, who are going to school in Munich. I have no way to know what is possible. And I’d use the international test, first and foremost, is telling everybody what is possible, how much skill can a 15-year-old have in terms of math problems. And secondly, providing some clear goals and things to aim for to local, national schools. So, I want the Palo Alto schools to be competitive with the Munich schools, as opposed to the East Palo Alto schools next door.

Will Brehm 33:56
But the issue there, of course, is that the PISA test measures national systems of education. So, are you trying to say that we need more of the PISA for schools, which I think I’ve talked about on the show before, which is about measuring school systems? And I think this only takes place in the US at the moment. And then allowing it to compare a particular school system to national systems of education, is that that’s what we need? Basically, we need more testing at the school level?

Eric Hanushek 34:27
Well, I think the PISA for schools is probably okay, it did not too much in terms of making policy because, in simplest terms, if you have a set of school districts had voluntarily agreed to take these tests. And the ones that volunteer are not a random sample of the schools, in any state, or the country, it makes it very difficult to use them to guide policy. So, if the Palo Alto schools participate in PISA, for schools, and we find out that we are or are not comparable to Munich, we might be able to puff out our chest a little bit and say, yes, we’ve got a good system or not. But it doesn’t really help in making policy. What we want are systematic measures, such as the accountability measures that are in the United States across the states, that allow us to compare schools that are similar. So, if I look at schools that have similar family backgrounds, in terms of safety, education levels of parents, or the income levels of parents, is my school doing better than the one next to me? But I can also translate all of these tests into the international studies, it takes very little actually to have the capacity to take the tests that are given routinely in the United States into PISA test because I know where the country stands, and psychometricians can, in fact, do these comparisons.

Will Brehm 36:21
Well, Eric Hanushek, thank you so much for joining FreshEd.

Eric Hanushek 36:24
Thanks so much for having me. I hope this has been an interesting discussion for our audience.

Coming soon.

Coming soon.