Will Brehm  1:59
Karen Mundy, welcome to FreshEd.

Karen Mundy  2:02
Thanks, Will. It’s a real pleasure to join you.

Will Brehm  2:04
So, what are the sustainable development goals? And how are they related to education?

Karen Mundy  2:09
So, the SDGs are a set of international development goals that were the product of several years of consultation under the United Nations. There are 17 of them. Each one focuses on a specific area of development, and one of them is designed for education. I want to just start by saying that including education in the SDGs was not a foregone conclusion. And in fact, at the early stages of consultation, there was some sense that education might not be included. But there was overwhelming support for including education, both from citizen surveys, from surveys of the leaders of international organizations, and from world political leaders. So, we ended up being the fourth of 17 goals. And other goals, including health and poverty alleviation, make reference to education.

Will Brehm  3:04
So, what specifically about education is included in this SDG 4?

Karen Mundy  3:11
So, it’s a really interesting set of targets. 10 in total under SDG 4. And it is, in comparison to the Millennium Development Goals, important to note that these are a much broader set of goals. The first goal and the goal that most people are familiar with focuses in on the achievement of universal primary and this time, secondary education. And you might remember that the MDGs focused primarily on basic education -completion rates of basic education. And one thing that’s new to this goal on achieving universal primary and secondary education is a focus on learning outcomes. So, this time round, it’s not only about completing school but actually completing the school and having learned.

Will Brehm  4:02
So, that obviously brings up a question of measurement. How on earth are they measuring learning outcomes, and the other nine targets across the world? I mean, it seems like such a huge endeavor.

Karen Mundy  4:17
You know, the answer to that is quite interesting. And I remember once somebody saying to me, make sure that you don’t adopt any goals that you can’t measure. But actually, under SDG 4, there are many indicators for which we do not have methodologies, that is measures. And some that have methodologies but no data. And among the indicators where there’s a target but no data or methodology, learning outcomes is one. So, we’ve established a goal and the proxy indicator for that goal, for the target, is that children will learn math and literacy in the primary level and the lower secondary level. But we actually don’t have a global indicator that can measure that target at the moment -at present.

Will Brehm  5:22
So, the Global Partnership for Education, which is where you’ve been working for the last few years, are they working towards coming up with a global metric for the math and literacy? For learning outcomes?

Karen Mundy  5:39
The Global Partnership for Education is a multi-stakeholder partnership that provides financing primarily to lower middle- and low-income countries. And as a partnership, we focus on strengthening education systems. One of the ways in which education systems can be strengthened is around a greater focus on the achievement of learning outcomes. I think all the 65 developing countries in our partnership would agree with that statement. But as a partnership, we don’t have a direct role in the delivery of a global learning metric. We are, on the other hand, very keen to support countries to have really good solid learning assessment systems. So, our focus has not been on the metric but on the assessment systems that will foster better learning.

Will Brehm  6:36
And in this sense, when you’re working with these national governments, each national government could, in effect, come up with their own assessment system that is measuring learning outcomes slightly differently. So, there’s no one global universal metric for these learning outcomes.

Karen Mundy  6:56
I think what’s so interesting here is that at the global level, there is now a strong commitment and urge to develop a single set of metrics that are cross-nationally comparable. And there’s a pretty mighty debate going on about that. I think we’ve heard in this series, from David Edwards, about some of the reasons we might not want to move so quickly on that effort. And we will hear at the conference from others who have been given the responsibility to come up with that global metric. I want to just stand back because I think it’s a really important question about how to get the metric. It’s not really only about whether or not we should have it, but also what kind of metric is the best type of metric at the global level. At the moment, it’s clear from the targets that we need a metric that focuses on literacy and numeracy and at three levels in the education system. And that is the task at hand for global actors like the UNESCO Institute for Statistics and other UN organizations. But there are different ways of ensuring that learning metrics are comparable. And I’m very keen to see learning metrics that are nationally owned, and they may be technically robust but not necessarily entirely similar across venues. And I think from the GPE point of view, the important feature of a global metric is that there’s just enough comparability to show trend and to make broad comparisons across countries. I would suggest that some approaches to a global learning metric may be focusing a little too much on empirical purity. And if we look at other areas, this is something that Amber Gove has written about, for example, measuring in the health sector, maternal and child health, under-five mortality rates. All of these allow governments to pick their own approach to measuring the outcomes, and then draw broad comparisons between the outcomes that countries themselves have identified their own measures of. So, for example, are children reading in your country? I think that there are some broad technical reasons to follow a kind of robust set of procedures to answer that question at the national level but that countries might want to answer it a little differently. And the important thing is: is the country focusing on children reading? Are they keen to know about the trends, the inequalities in reading outcomes in their countries? And I’m just using reading as one example but I think a very important one.

Will Brehm  9:54
Yeah. It’s interesting, right? Because a lot of times this idea of statistical purity in these measurements and getting this cross national and global indicator that we can use. And it’s made the focus on data. It’s like, we just want to get data but we’re not actually thinking about how these assessments are supposed to actually increase learning outcomes, or like you said, increase the ability for children to read. And those sorts of goals will depend on the local context, it seems like. And so having that ability for national governments to have their own sort of take on these metrics seems to be like that’s more feasible than just doing data for data’s sake.

Karen Mundy  10:51
You know, the national ownership, the ability of people on the ground to make use of the data, I think is, for our global partnership, for the partnership of countries that belong to GPE, is much more important than the global comparability. There are some reasons that you want global comparability. One of them is to leverage political will. And so, we know that in the health sector, the ability to capture stark statistics about children dying under the age of five, or maternal and child mortality overall has driven an enormous amount of financing and expertise into the health sector internationally and supporting countries where those statistics are very weak. In education, we simply haven’t had the same kind of compelling data. So, to my mind, that’s a very good reason for wanting some measure that is better than are kids in school or not. But I wonder myself, and now I’ll speak without my international hat on but more as an academic, I wonder whether we strive for too much granularity in that kind of metric in the education sector. And perhaps we could make do with something that’s a little less perfect but still capable of playing that political role. So, that’s one reason you want a global metric, is political will, is an ability to channel, to advocate attention towards education. And internationally, I think, education has not been a well-funded area for international development. Certainly not well funded to the degree that health has been well funded.

You have another global reason that you might want a learning metric. And this one is the one that was encapsulated in your podcast with Eric Hanushek. He believes you need to have a global learning metric because it enables governments to compare themselves in a globalized economy and to figure out what kinds of skills will drive success within that economy. And I think actually, many leaders of government do think that that’s an important measure to have. But that is an outcome that would not necessarily be in keeping with the driving forces behind the Sustainable Development Goals. So, I think it’s interesting to just parse out what the purposes of these global metrics are. And then ask ourselves, what do we need to really have a solid metric? Now, one of the things that I think has been so important about the drive towards global metrics, metrics like PISA, for example, has been the ability to highlight key ways in which the equity of investments in education impact learning outcomes and vice versa. So, I think this is a very key piece. And I’m not myself convinced that it can only happen through global metrics. But I am convinced that having data about learning outcomes that is correlated with other kinds of social variables helps governments to see that learning outcomes are not equitably distributed, and that resources need to be targeted to specific groups and populations. So, I think that’s a very important rule. Will a global metric allow us to play a role in that type of distributive justice? I don’t know. Because it seems that the metrics we’ve had on out of school children also closely mapped on to other forms of inequality and in fact, knowing that children are out of school hasn’t actually driven funding to the poorest countries. So, I’m not sure that we’ll have that level of impact with our metric, but I think we should be hopeful about it.

Will Brehm  15:13
Yeah. I want to just shift gears to actually looking at the Global Partnership for Education since it’s working with national governments. And I just want to hear a little bit about how are they working with national governments? What are the challenges? What actually happens, in a sense,
“on the ground”, when the GPE is working with national ministries of education to develop these assessments and try and capture measuring learning outcomes and trying to provide data to help them reform their education systems?

Karen Mundy  15:53
Well, GPE has a very particular theory of change. And it’s interesting. One of the things that someone pointed out to me recently was that actually the SDGs are not set up as a theory of change. And you have to parse it out to understand how people think specific actions are going to lead to specific outcomes. For GPE, we make an investment at the country level in the implementation of a national education plan, and we make an investment in supporting governments to design that plan. And then we also make an investment in mutual accountability. We support civil society, we support a multi-stakeholder policy dialogue table at the country level in the hopes that governments will produce really good evidence-based plans, they’ll be costed, they’ll be targeted, and they’ll focus on learning outcomes and equity. That’s our top-level objective as a partnership. And we want to make sure that we provide reinforcing financing so that the sector plans also include capacity development strategies. They have really strong analysis of some of the key challenges and barriers to achieving equity and inclusion in their education sector, and they have strong policy plans. The learning outcomes or metrics that are generally being supported under GPE are quite varied. We have not prescribed what governments should spend either their sector planning dollars or their plan implementation grants on. So, many countries have adopted early-grade learning assessments using GPE funds. Some have improved their national assessments, curriculum-based assessments at higher levels of their education system. We haven’t prescribed it. But we do ask countries when they receive our funding to select a learning outcome measure that we will finance on a results-based basis. So, we will finance them when they achieve that outcome. So, 30% of all of our grants is delivered through that results-based tranche. 70% is given upfront to support the broader development of their education system around their sector plan. So, we’re kind of an interesting organization in that we are very focused on country ownership. We don’t define country ownership as only involving government. We would like to see a policy dialogue table that includes national stakeholders and that is capable of driving alignment across international donors so that they come to the table to support a nationally owned plan. But we don’t prescribe what’s inside the plan. We have some broad guidance about what would make a good plan. We want to know that governments have paid attention to learning and to equity. But we don’t prescribe the specific policy solutions because we think it’s through that policy dialogue table that those solutions should be debated and discussed, that evidence should be considered, and that plans should be developed. So that they are truly owned and, of course, then executed on.

Will Brehm  19:29
I’d like to ask about -you said that the governments get to create a learning metric that then they would measure. And the measurements would then lead to this results-based financing 30% of the money GPE is financing.

Karen Mundy  19:50
They actually -it’s interesting. They have to create a target that supports learning outcomes. And we actually don’t prescribe that these targets have to be at the level of a learning measure. So, some countries will choose a process related or an output related variable rather than a test related variable.

Will Brehm  20:15
Can you give an example of some of these different types of variables?

Karen Mundy  20:20
Some countries will invest in assuring that a certain ratio of textbooks are available to children, or they will focus on improving teacher development, teachers’ capacity to deliver a curriculum for example. And that’s the target that they will select as the metric against which would deliver our results-based tranche. So, we’ve been quite hands-off in allowing governments to select learning metrics. On the other hand, there’s a couple of countries that have selected a specific learning metric. And they’re quite varied, often in their early grades. And in these cases, we aren’t delivering the entire 30% of the tranche to just the learning outcome metric, but up a theory of change in the country’s plans for improving learning outcomes. So, some part of the tranche is delivered when the country introduces the methodology for measuring learning, and another part might be delivered when the country first conducts the learning assessment, and another part might be delivered when the country makes that data publicly available. So, we’re quite transparent about the fact that we don’t want to prescribe. On the other hand, we are content when countries select fairly sophisticated theories of change in the delivery of learning outcomes.

Will Brehm  22:11
You said that the education sector doesn’t receive as much money as, say, the health sector? How big of a gap are we talking about?

Karen Mundy  22:23
You know, now you’ve asked me a hard question, Will, because I don’t have the health versus education data. And what I would just say broadly, is that if you looked at a diagram, and you said, “Well, how much of the effort to achieve the EFA goals came from domestic financing, and how much came from international financing?”, you’d find that in the education, the vast majority came from domestic financing, and that the level of domestic financing for education went way, way, way up after 2002, especially in low-income countries. For health, the level of international financing went way, way, way up after 2002. And domestic finance was quite slow to follow. It’s now catching up. But at first it was not. So, you really saw, with the creation of the GAVI and the Global Fund, a huge new surge of international investment in health. In education, the growth was much more modest in international financing but domestic financing grew very, very rapidly, especially after the highly indebted poor countries initiative kicked in.

Will Brehm  23:41
And to return back to the Sustainable Development Goals where we started. You said that they replaced the Millennium Development Goals, and the Millennium Development Goals were not necessarily achieved. And that’s why the Sustainable Development Goals perhaps needed to -you know, the timing ran out in 2015. And we needed new goals, and now they’re Sustainable Development Goals, the target is 2030. Do you expect that these goals will be achieved? Or are they more or less goals that we should be aspiring to regardless of actually getting there?

Karen Mundy  24:20
I think that’s a great question. And it’s really important to remember that well, not all aspects of the MDGs were achieved, millions and millions of children went into school after 2002 that without that extra level of attention, financing, and effort might not have got into school. A very good report from the International Commission on Financing for Education produced its estimate of what it would take for us to achieve SDG 4 and in particular the coverage in early childhood education, in primary education and lower secondary education. And the numbers were -well, mind boggling to be frank. But I think overall, the attention of the world is on us today and on education in ways that it perhaps hasn’t been even in the MDG era. There is a sense that education’s time has come on the international stage, that the Sustainable Development Goals place a much stronger emphasis on education and on a broader range of educational needs that encompasses the need for citizenship education, and the need for adult literacy. So, I think it’s quite exciting, and the question is: is it achievable? I can tell you that, I think at the primary level, and lower secondary level, the goals reaching to 2030 are just barely achievable if we put all our shoulders into it. Pre-primary education, I’m less confident of and some of the other goals strike me as being unlikely to be achieved, at least by 2030. The UNESCO GEM report has recently given their estimates and they place the achievement of full cycle of education to the end of secondary school universally, sometime towards the end of this millennia. So, yes, they’re very aspirational. I’m myself very pleased to see them be more comprehensive and to have a wider scope and definition of education baked in. Having a focus on training and skills but also a focus on the need to have global citizenship education values related to sustainable development and equality and so on. I think that’s a welcome change from the more narrow MDG goals. But it does mean that we’ve raised the bar, on some items, raised the bar before we were able to meet the first bar.

Will Brehm  27:24
I guess, as a last question: I’ve asked a few other guests on FreshEd about wearing multiple hats. So, you are an academic, but you also are working with GPE, and I just want to know, how do you balance that? How do you manage having multiple hats and having to be an academic but also a development practitioner?

Karen Mundy  27:48
Well, I haven’t been really wearing my academic hat for two years and one month. I think one of the nice things about having been an academic is that you enter situations with a truly questioning mind. And I’m a sociologist by training so I also treat what I’m doing today in the Global Partnership for Education as a kind of sociological ethnography. But it’s truly fascinating to see how institutions work when you’ve been studying them and writing about them over many years. And it’s also quite fascinating to see the things you can get done, and to realize some of the things that are so hard to get done. And I just want to emphasize one of those things, which is coordination, which is achieving an international architecture that truly is less fragmented and more effective. Today, the way I see actors interfacing, I sense that whatever gains were made after 2002, in terms of aid effectiveness and alignment and harmonization, are rapidly being eroded. And it’s a great privilege to work in an organization, a partnership, that has baked into its DNA, a focus on aid effectiveness. But it’s quite disheartening to see how rapidly momentum in that direction has eroded.

Will Brehm  29:33
Well, it’ll be very interesting to see what happens at the symposium where a lot of the questions that you raised through this interview, I think, hopefully will be discussed and debated at greater length. So, I think you’ve really set us up for the beginning of the symposium. So, thank you so much, Karen Mundy, for joining FreshEd and I look forward to seeing you at the symposium.

Karen Mundy  29:56
Thanks, Will. See you in Arizona.