Silvia Montoya
Measuring and Monitoring the SDGs

Will Brehm 1:46
Silvia Montoya, welcome to FreshEd.

Silvia Montoya 1:48
Okay, thank you so much and thank you for having me and it’s just a great opportunity for us to share what we’re doing and trying to see if we could help to have them -the SDGs.

Will Brehm 1:58
So, what does UIS, or the UNESCO Institute of Statistics, do vis-a-vis the Sustainable Development Goals?

Silvia Montoya 2:07
Yes, I mean this is a good opportunity to explain the UNESCO Institute for Statistics. It is the statistical branch of UNESCO. What we do is that we are the repository of data mainly in education but also, we work in science, culture, and communication, and innovation. So, we have a much broader mandate. And the UIS not only collects data on a comparable basis we will collect data, we transform into indicators that are comparable because they need some extra work to see what are the definitions that each country has and define the standards to actually achieve that comparability. This is a very relevant issue because if we want really to talk and to understand each other we need to have a common language. Without the common language and understanding what we’re talking about, it’s very difficult to reach agreement or to understand where we should go. So, these comparable indicators demand that that extra effort of how can the data be transformed into something comparable for talking about the same criteria. And we do that for these different areas and we have been doing that since the year 2000 and basically at the end of 1999. The UIS has been set and since then has been actually having this role. They happened the Millennium Development Goals that have three global indicators but then they were focused on actually having the kids in school, and gender equality, they were basically the main two drivers and the right to education, to the number of years in education that were the main three focuses. Now, with the Sustainable Development Goals, it was a kind of a wake up call about the fact that everything is related to the environment, education, health, science the way you live in the cities and the way you actually treat the oceans and life under the sea and life in the, so you have different aspects of poverty level, everything is interrelated so for the reason we came with the Sustainable Development Goals. The president committed to this broader agenda that everything is connected, and education comes as a catalyzer of this, that is the SDG 4. And the SDG 4, UNESCO has a very strong political leadership is the goal is eventually the only one who has such one driver that this convening all the stakeholders and the different interests of work. And this is at the political level. At the technical level, the UIS has a branch of a UNESCO. We have taken the technical leadership in this broader agenda that is not only about having the kids in school. But it’s also about the kids learning, it’s about the kids integrated in the labor market, it’s about the equity in the different moments of this lifetime, and it’s also about how you interact with the rest of the members of the society. How you treat the environment. How are the different aspect that have to be with the sustainability of the world? So, the UIS has actually seen the agenda world being multiplied by different factors that are related to all these new areas. And these new areas are difficult because they are new. And because they are new we’re not really understanding for everybody what we talk about when we actually talk about some concepts and we think that in general they are good on the concept or idea, but then we need to define what they are. Even if we think a little bit for instance the indicators are contemplating different areas in different moments in time. So, we have in early childhood, we have in primary and basic education, we have indicators related to skills and labor market, and you have indicators related to youth and adult, and we have also indicators related to the expenditures and related to the condition of the school environment as are relevant. But we also have to define different areas and there is there are concepts that even if we really think we understand the concept, they are not that clear.

Will Brehm 6:14
Such as?

Silvia Montoya 6:15
Reading and mathematics. What is reading and mathematics at the end of primary in different continents and in different countries. It depends a lot on the variety of possibilities, the variety of languages that the country has at the national level, and what are the speed at which some of the subjects are taught. The written and oral tradition that operates in the countries because depends on the on the tradition. I come from a Spanish speaking country and the written tradition is much more powerful than the oral tradition that is dominated in some other cultures, like, in Africa, for instance. So, the concepts are in general a little bit more difficult to define but still reading and mathematics that are kind of supposedly easier has taken three years of time to reach an agreement of what are the concepts that every child has the right to learn and what are the minimal level that they should achieve.

Will Brehm 7:13
You can actually agree to have a comparable indicator on something like literacy?

Silvia Montoya 7:19
We have agreed on this.

Will Brehm 7:21
And who’s agreeing? Who actually comes into an agreement here?

Silvia Montoya 7:24
Okay, who is coming to an agreement. This is really very interesting question because in the previous mandate of the MDGs, the agreement was between the major agencies that were working. With the SDGs, the agreement comes from a group of countries and stakeholders. So, what we do is that we have in a table attending a competition group that has representatives from all the regions in the planet. You have stakeholders, you have civil society, you have donors. So, we actually work, and we have in the case of learning assessment, we have also organizations that administer learning assessments. You have OECD with PISA, you have IEA that administer the test in science and mathematics and you have different regional initiatives. So, all these people, it has taken a while -three years actually- to come to an agreement of what is reading and mathematics in each of the point of a basic education and to agree on the minimal level. But this is not ending there. We really need to go to other areas, we need to go to youth and adults and for instance, what is literacy right now? Is it enough with reading your name? Or knowing how to write simple words like apple or orange or something like that? Or do you need to have a much complex management of basic concept to live in society? Because nobody today is enough with reading a simple word or a simple phrase.

Will Brehm 8:53
And then how do you end up testing that to see if there actually is learning taking place in literacy, right? Because the very way in which a question is even asked assumes a particular idea of literacy. In some cultures, or in some countries, you may show, or the very question used to sort of monitor the level of learning of literacy, there may be some cultural differences there and that might skew the results.

Silvia Montoya 9:20
No, that’s a definitely question. And it has been a concern for this recently has taken a time. This is very adequate, because I mean, the challenges that you have to measure the set of skills in a particular context. And without taking into account the context, anything that you measure is totally unfair. So, we have actually taking and focused more on the type of skills that they need, in the way exactly how you measure. It is oral? It is written? What type of items you use or questions. Because many people think, “Oh, I need to have the same question”. And ask the same question to different people, it is clear that this is not the way. Because I mean, the text, even the texts are meaningful in some cases, and there are not meaningful into some other cases, if you live in a city and you’re testing a child that is in the city and you talk about activities that are mainly outdoors and aren’t actually part of your life is, it’s really very likely that the child is not used to the situation on itself, a challenge is different, because you’re asking him or her to put in a situation that is not familiar to them and they cannot and they get distracted. So, it’s a whole lot. So, we have actually focused into trying to define at the broader level, what are the skills that they need to have? But we have reached consensus in basic education. For youth and adults, it has become more complicated because the literacy levels or the need to understand some instructions for the way of integrating the society in this moment, depends a lot on the country. It depends on the reality depends on if it is rural or urban. So, this is actually challenging in this moment, we haven’t reached an agreement yet about how to measure other than just literacy. The only metric that is there is a metric that is good for very developed countries that are more or less at the same level of structure of the economy. So, you could assume that the instrument or the test is reasonably measuring differences in skills. We are really working into trying to see how we develop on other tests that are adequate, or context specific and this is one of the challenges. But there you have one of the challenges you are facing in these new SDG era. Because this is a lot of research and areas of work that needs still to be done. So, there is a need to try to understand better. And I think that this is one merit of the SDGs that are raising attention or actually focusing attention of these kinds of issues and trying to actually to focus also in the differences of the skills, because at the end of the day, we shouldn’t be in some moment, we’re going to converge and everybody’s going to progress. And we’re going to assuming we’re going to progress towards equitable levels of skills and learning. And supposedly, we are going to have to have the same set of skills. But what happened if we’re in such different points, and we really need to be ready to go into that. We also need to be fair, so we need to generate tests that are allowing for these differences in this moment. And this is a kind of a brand idea of work that needs to have funding, that needs to have research and then needs to have also some understanding that the contexts are different. I think that in in to some extent, in the last three years, the whole community has reacted in a very positive way to understand the broader differences. There are still challenges related to funding, there are challenges related to research, and how we could bridge towards research how research also could be more operational, to be started embed the measurement challenges and could serve better tools to the policy challenges that the country’s face at the time of implementation. So, these are the kind of things that the UIS has actually been facing in the last three years is an agenda that is incorporating learning is not only learning in reading in mathematics is learning across the life cycle, it’s early childhood also as well, it’s incorporating the discussion about global citizenship and education for sustainability in a context that there is no agreement on what it means.

Will Brehm 13:34
I’d actually like to talk a little bit about that the SDG 4.7 the global citizenship education, the education for sustainability, I think human rights is also included in this. I mean, it’s this big, long list of very difficult terms to understand and they all seem to be mixed into one target. And so, the question I have is, you know, what sort of conversations go on among the stakeholders to even come into agreement about how that target should be monitored? Like, is that even possible?

Silvia Montoya 14:11
Yeah, I mean, this is really, I think it’s most difficult eventually, of the, of the ones to work on, because, as you said, everything has been kind of embedded there and encrypted in one concept. And the problem is that when we try to unfold a little bit and unpack what is there is not clear what global citizenship is and it’s not clear what education for sustainability is. Some people agree that they could be a split and some others on the other side said no, it’s just a concept that is so intertwined and connected, that you cannot split the two aspects. So, so far, the target in itself is I think that we shouldn’t forget that the SDGs are formative. So, there is an intention to signal what issues we should be concerned about. So, in that sense it has installed in itself the target. As a desire of the countries, I mean this is conversation about being willing to commit to different aspects related to what the target says is something that seems that the country have reacted, and the community also as well in general to the issues that how they implement and how we measure is very complicated. We have only one indicator at the global level in that target that is related more to policies. Is that how the countries are incorporated in four different aspects? Concepts related to global citizenship and inclusion for sustainability, they shouldn’t be the same concept and it should be different degrees of implementation. So, it’s about the curriculum, it’s about the policies, laws, it’s about the teacher’s profession it’s about if you have some learning, some learning assessments and way of measuring. But the concepts are not yet defined because we have been these three years. There have been a lot of conversation and work, but it has not been possible to say this is global citizenship, this is vocation for sustainability. Instead of that there seems to be aspects or symptoms of those concepts that are over there. So, there are issues related to gender equality, to respect for diversity, for human rights, for ruling for discrimination concept of these type, environmental sciences, how you treat the environment and some concepts of biology. But it’s not clear. In this moment it’s not clear, there are more aspects or expressions than a clear definition that makes it really makes it a little difficult to guide countries about what to do and about the concept. So, in itself, what has happened is that UNESCO has, by the end of the 70s, a kind of a discussion about a convention trying to a deal to a kind of a recommendations about these concepts. Obviously, recommendation for the last 70s, 50 years, 40 years later, it is good and bad, it is really bad, that still the convention is valid is not a good time from the world, that still the convention is an umbrella is just a little bit depressing. I would say on the other hand, is clear that the convention is an umbrella that you can come with a way of making it operational. It still was a declaration -a big declaration- that needs to be defined at an operational level. And then we have colleagues UNESCO headquarters in the division for sustainability has been working on a question to try to get a sense of what’s going on and what the countries are doing because we have to report for this convention. So, at least, there has been agreement or report in every four years that the system forward in these and it has been a first attempt of a questionnaire that is more asking countries about the series of aspects of global citizenship and education for sustainability, that could be actually observed in the countries and there is an indicator that is going to improve in the next wave of the consultation. That is still under consideration for the group that decides if the indicator is approved or not, but you collect information and at least for this first step, I’d love to map what the countries are doing according to their own words. I mean, not necessarily everybody needs to have the various aspects, we are really not clear what aspects need to be actually put into place because the concept is not clear. I have been asked in the last meeting of the group that is deciding on these indicators that is at the level of the UNESCO Statistical Commission, a specific group on SDGs. The countries were asking, but what is the concept and they said, “No, there is no concept, what we have actually agreed is that there is a menu of aspects”, and then the countries could have the different aspects, and this is something that we really need to take a deeper look, this seems to be very difficult.

Will Brehm 19:09
I mean, it sounds like it would be very difficult to then compare countries of global citizenship if they take different aspects of that concept.

Silvia Montoya 19:17
Yes, I mean, I think that in this moment, what we could compare is the progress towards incorporating aspects related to these two broad concepts that compare the countries between them. So, if your country if you agree, or more or less, we saw a clear definition: If 10 concepts need to be there, like gender equality, human rights and other ideas and vitamin and things, eventually, the country is doing a good job with having three or four only address and they don’t need to have the 10 but in this moment, we’re not sure because the concept has not been defined what is even not acceptable. So, we’re just trying to progress as much as the countries are incorporating the concepts or more concepts in different areas. This is a little bit of the proposal of the indicator in trying to land in some space that allows to collect data, that allows us to get a sense of what’s going on. I would love to see if there is progress. And it’s about the same time is it that’s not allowed to have a comparison because you don’t have that.

Will Brehm 20:24
And maybe that’s impossible, right? I mean, it just seems like even coming up with an agreed definition to global citizenship education might not even be possible by 2030 when the goals are supposed to be achieved. I mean, do you think you’ll be able to actually come into an agreement on the meaning, let alone figuring out ways to measure it and operationalize it?

Silvia Montoya 20:46
No, I think that hopefully, there will be at least some kind of agreement, I would say, of at minimum, what minimum level of global citizenship and education for sustainability. I mean, we don’t know, I don’t know the penalty or the codes or the laws in particular even of my country at such a detail to know what the kind of an offense to some law could be, but I know if I kill someone with no reason, it’s bad. So, I think that we could aspire at least to know what kind of issues at least need to be at the minimum or respected by that moment, it has proven to be very difficult this one. Because we have some attempts in terms of the World Value Survey that is measuring, we have the European attempt. And you see that there are differences even in in areas of the world that are more homogeneous and has actually faith through history, processes even of trying to define common areas like the European area. And we have also some attempts in terms of assessment, like there is a test on civic education that is administered on kids from the first year of secondary education. This test is very difficult. The countries are reluctant to intervene, because it’s very difficult. I actually tried to administer in Argentina twice. And it has been the only one that gave me a terrible headache.

Will Brehm 22:14
Why?

Silvia Montoya 22:15
Because I mean, protesting. I mean, they are some expression of citizenship. I mean, in some cultures, unfortunately, would you protest, you make a big noise, or you cut the streets, or you burn tires, and this is part of the culture or the way of expressing disagreement. And in other cultures, the correct is not expressing any disagreement. So, it’s bad one or the other, eventually, you could agree that in the long-term it’s better not to cut the streets and burn tires, because there are some other concepts, but at the same time you get a sense of people replying to a test and saying this is bad to burn tires to cut the street because they are used to that way of acting.

Will Brehm 22:58
Different ways of civic engagement.

Silvia Montoya 23:01
Civic engagement. I mean, you see the also issues of the of the school system, I have lived in the United States for almost 10 years and when your kids go to the school, as parents you are supposed to have a major engagement and to go to meetings and to go to participate and you see that it’s a driving force that everybody. That’s a good side, that is correct because it creates a lot of ownership. In many areas of the world, in many countries, it is not part of the culture in general because of the culture. And I come from one that is evolving but not too many years ago, you assume that the teachers and the schools knew what to do with the kids. You just leave the kids in the door of the school, in the entrance, and then you come and pick them, and you don’t get involved into internal into the decisions and business and then through time this is changing but actually these are the different ways of actually of expressing and engaging also as well. So, what are the expression, what are the way of operating depends on the culture, depends on even violence or some ways and not we do different but that there are different ways. When you come from different areas, I mean you have half Spanish, half Italian, you know that the ways of many things are different between the two cultures and you see the cultural differences as you go, and you travel around the world.

Will Brehm 24:24
But yeah, I mean it reminds me of the climate strikes that students are participating in in Australia today. And you know, on the one hand these are students that are taking action into their own hands saying we’re fed up with fossil fuel production out of Australia and we want to make a stand to say you know we want to fight against climate change, which sounds to me to be very much part of the Sustainable Development Goals -they’re including global citizenship education, environmental protection, climate change- so all of these things that they’re sort of embodying in their very strike, or in their protest, but the government of Australia comes out and says you know, these students are bad, we should send them back to school, they shouldn’t take days off, so it almost seems -it’s not necessarily cultural in that regard but- it’s this political issue and you can see you know, for me those students in a sense are representing global citizens. They’re actively engaged in a topic of global importance

Silvia Montoya 25:28
I think that definitely and actually this is a planet that they are owning and if we see from what we were when were children to now, you see the issue of the climate and how it changes is not too foreign to us. I think that this is the way of protesting of how you express and also, I don’t know what are they having the channels institutionalized actually to find another way of protesting? I mean, I don’t know eventually they don’t know or there are no channels to actually express the discontent and I think that also in the SDGs you have youth here seated and representing youth and, in the morning, there were expression of this. This is actually part of the laws is because in some moment is not because eventually they are not we are not right in everything and everybody and we know all this but there are different voices and there are spaces. I mean, I think my question will be, are we generating the spaces for these at the end of the day is much more theirs than ours. But I think that these are the questions that actually they should become to an agreement, and the way of actually how these actually final and how we actually express our different views. In PISA from OECD and the test for 15 years older, they tried to agree on a model on global citizenship and they couldn’t actually find an agreement after four or five years and they have come to say, Okay, this is not going to be part of PISA, it is going to be a model and see who is coming. And I don’t know how many countries but there were -if they were it were a few only willing to participate in the model. So, I think the concept is really complicated. It comes to values, to religions, to cultures, to political, it’s policy, it has all this sensitive also at the same time. But to some extent, I think that the way we’re trying to solve for the time being is trying to see at least try to mark what do people talk when they talk about global citizenship and education for sustainability and I think that the first step is also going to give an understanding of what are the different kind of concepts that are there. We are really worried in the sense that also would be good, if we could converge as soon as possible, at least to agree on a set of minimal issues that we consider that they should be incorporated in everybody. I mean, I don’t think that anybody will fight too much about some issues of gender equality, or some basic human rights and I think that in that sense, if the SDGs are a formative tool that we are trying also to learn, we don’t have to have everything ready right now. It’s just a collective process of learning and also learning from each other is that how much are we guiding what the countries are doing and what they should be incorporating. I think that eventually you have to take another shortcuts, like mapping policies are mapping what type of policies they are taking and trying to see how they express in different ways and implementation of these concepts in their understanding and try to see what are really going beyond what there are already like the Declarations of Human Rights or major issues that you know that are wrong. You don’t need to contest this too much. But in itself this indicator needs still a lot of convening, a lot of conversations and obviously we’re worried because I mean there are issues that are related to sustainability of the world and the planet itself and it is really very dangerous. And it’s going to be very dangerous and it seems to be that some of the issues need to have a training and retraining very quickly because we see how much it changed. I am really amazed about the global warming and things of this type is really desperating to see that the world is coming there, I mean it’s really incredible

Will Brehm 29:23
Is that an indicator for you, I ask? On the Sustainable Development Goals, a measure of the earth’s warming?

Silvia Montoya 29:30
We don’t have. We have indicators, the UIS -because of the mandate- we have indicators on education, on science, a number of research, and we have indicators on innovation and in heritage at the city level. UNESCO has other indicators on oceans, but we don’t have indicators on vitamins, they are not falling. And these I actually find it very difficult to defining. Because actually, the indicator in Goal 13 that is related to climate and climate change while working with a UNFCC, the indicator 13.1 that is related to respect for the sustainable aspects of the planet is based also on the curriculum and the policies they are going to use, global indicator of UNESCO if we succeed in upgrading in making it operational this mapping. And the second indicator that is in the set there is related to how you strengthen the capacity of the individuals and of the societies to respect climate and sustainability and these. And we have proposed now to use the organization that deals with climate changes at the global level in the UN to see if they could use some of the tests that are related to sustainability. They have science that are PISA teams to see if they could actually take us an indicator for building the capacity at the individual level. But the issue is that the indicators is about building the capacity at the society level, at the community level. So, I think that there is a further development there to try to use and these are the kind of things we are concerned. We are not the custodian, we are not in charge of the indicator, we’re trying to help to see if we can find shortcuts, but I think that this could actually work a little bit because you already know that there are some messages that you don’t need specific indicators to deliver. And eventually we’ll have to try to implement some way of asking governments to make a stronger commitment in their policies to some aspect related to sustainability of the environment.

Will Brehm 31:35
I mean, so it’s been three years since the Sustainable Development Goals have been adopted. And I want to just ask your opinion, do you think in these three years, do we see a qualitative difference from what the global community is doing towards these global goals, than what was happening during the Millennium Development Goal period?

Silvia Montoya 31:55
So, I think that it has been a huge progress. My view has been in terms of the UN system and the other multilaterals that are related -World Bank, OECD, and the rest- there is a huge attempt to not duplicate jobs, to do partnership, to actually improve the quality, to see how the questions can avoid to be duplicated when it comes to data collection. The issue of the understanding that this is an agenda that affects everybody, because the Millennium Development Goals, it’s true that there were countries that needed to have that help, but actually it was not the focus in the interconnected world, if in the developed countries, they don’t respect the environment, everybody’s actually affected by that. And if they have more money, they have actually the same way that you have taxes, they have some way they need to speed and to help the rest of the world because it’s for everybody’s benefit that we all improve. So, I think that that understanding has been already started. The first year it was a discussion about still with this idea of the developing countries and trying to see and now we have all the discussion about universality of the agenda is off the table. Everybody understands, and also has been in that phenomenon, the issue of the refugees, immigrants and all these displaced population is very relevant and has also faced many countries with the reality that this problem is universal. And also, they have to deal and they have to commit to solve overall. And I think that overall has been an adoption and a sense of urgency on this one. And I consider it has been very positive in terms of the take. I see as major commitment at the global level and hopefully I eventually we obviously we’re anxious and we’re in a hurry to see the actions if especially we are action oriented, but I think that overall it has been very positive for us.

Will Brehm 33:50
Well, Silvia Montoya, thank you so much for joining FreshEd.

Silvia Montoya 33:53
Thank you so much for giving the opportunity and good luck and I hope it helps to actually to be a kind of stepping stone towards saving the environment.

Will Brehm 34:02
I hope so. Maybe you can come back in another three years and tell us how it’s all going.

Silvia Montoya 34:06
Okay. Thank you so much.

Want to help translate this show? Please contact info@freshedpodcast.com

Have any useful resources related to this show? Please send them to info@freshedpodcast.com