SCORE Ukraine 2016
In 2015, SeeD was invited to implement the SCORE Index in Ukraine, by the USAID/OTI supported program “Ukraine Confidence Building Initiative” (UCBI). To address the issues underpinning community tensions and cohesion, SeeD in partnership with UCBI implemented the Wave One of the SCORE Ukraine between August 2015 – December 2016, with a nation-wide sample of over 7,700 respondents, over 300 interviews per oblast plus Kyiv city, as well as a booster sample of 1,600 Internally Displaced People (IDPs). SCORE Ukraine Wave One (General Population 2016) focused on investigating citizens' views on public policy reforms and the peace process, their sense of identity, access to services and civic engagement as well as the challenges faced by IDPs. Wave One succeeded in developing evidence-based and participatory policy briefs and communication messages, and mobilized significant interest among a wide range of key stakeholders (e.g. senior policy makers, journalists, civil society).
SCORE Ukraine 2018
USAID and SeeD acknowledged that it is crucial to conduct longitudinal and comparative research that can provide temporal analysis in order to monitor key indicators as well as to better understand of socio-political dynamics and trends. SCORE Ukraine Wave Two was calibrated in the summer of 2017 with stakeholder consultations and 8 regional focus groups conducted in Kherson, Kharkiv, Kramatorsk, Kyiv, Ivano-Frankivsk, Lviv, Sieverodonetsk and Odessa. Following the calibration process, the Wave Two questionnaire was designed and fieldwork was deployed in January 2018 by Growth from Knowledge (GfK) Survey Company. In total 9,018 face-to-face household interviews in 24 oblasts and Kyiv city were conducted via stratified random sampling. 6,102 respondents from Wave One also participated in Wave Two as part of the panel sample, which allowed for more robust and causal analysis. SCORE Ukraine 2018 General population study focused on the following areas of investigation:
SCORE Ukraine Schools 2019
In 2018, SCORE Ukraine Schools Wave Two, commissioned by UNICEF, was expanded both its scope and its reach. The data were collected during the first semester of the 2018-2019 school-year. The sample consisted of 7,846 adolescents aged between 12-19 years old. Adolescent respondents were students from 200 education institutions in Ukraine who resided in the government-controlled areas of Donetsk and Luhansk, Dnipropetrovsk, Zaporizhia, Kharkiv, Mykolaiv, Lviv, and Kyiv oblasts.
SCORE Ukraine Schools 2020
In 2019, the study was recalibrated in partnership with by UNICEF and the Wave Three was launched. The data were collected during the first semester of the 2019-2020 school-year. The sample consisted of 8,643 adolescents aged between 13-19 years old, as it was decided that the reliability of the data from 12 year olds was limited. Adolescent respondents were students from 215 education institutions in Ukraine who resided in the government-controlled areas of Donetsk and Luhansk, Dnipropetrovsk, Zaporizhia, Kharkiv, Mykolaiv, Lviv, and Kyiv oblasts, and Kyiv city.
Compared to the resources dedicated to war and conflict, the financial and human resources available to invest in peacebuilding is significantly limited. Effective, efficient and strategic use of limited resources to influence positive change in conflict transformation and sustaining peace necessitates a deep grasp of conflict drivers, as they manifest in each specific context, in order to design appropriately targeted and cost-effective programmatic interventions. Peacebuilding and development programs are all too often designed on the basis of limited inputs from a small number of in-country experts or conflict analysis consultants, with insufficient data-driven insights. Evidence-based decision and policy making is absolutely essential to improve peacebuilding and conflict transformation efforts, and in fostering social harmony both vertically (with governance institutions and citizens) and horizontally (between different groups in society). Yet, existing assessment approaches are often challenged when it comes to empirically linking potential drivers of conflict with desired peace outcomes in a way that can help resolve policy and programmatic dilemmas. As such, genuine efforts and investments can often miss the mark in producing the outcomes and societal change they seek to influence.
SCORE seeks to improve the effectiveness of peacebuilding efforts based on evidence, and was designed to address the abovementioned deficits in informing sustainable processes of conflict transformation. As a customizable, flexible and evidence-based diagnostic and predictive assessment instrument, SCORE can be utilized to identify programmatic entry points which are most likely to have a positive impact on peacebuilding outcomes. From this perspective, SCORE speaks to the concerns of the peacebuilding community, often comprising national and international actors who are still uncertain about the real efficacy of their investments. SCORE offers governments, donors and peacebuilding organizations the opportunity to systematically design and test conflict transformation theories of change before sponsoring and endorsing new peace-building programs.
SCORE draws inspiration from multiple scientific disciplines such as sociology, psychology, international relations and security studies and is flexible enough to incorporate new research findings, global policy guidelines and the realities of each local and regional context. It was developed in 2012 in Cyprus by the Centre for Sustainable Peace and Democratic Development (SeeD) in partnership between UNDP and funded by USAID. The backbone of SCORE is founded upon participatory research based on mixed-methods, where multi-level stakeholder consultations, focus groups and interviews are conducted to inform the calibration of the SCORE questionnaire, which draws from the extensive SCORE library of measurement instruments and indicators. Figure 1 below illustrates the SCORE process cycle.
Figure 1. SCORE Process Cycle
The SCORE Index eschews the one-size-fits-all approach that often characterizes cross-national indices. To ensure that each SCORE Index reliably captures the societal dynamics of the specific country where it is being implemented, the process typically begins with inclusive consultations with a broad cross-section of national stakeholders such as civil society, academia, government, business leadership and grassroot communities. These contribute to an initial in-depth understanding of societal dynamics in relation to outcomes of interest, while the technique of system mapping (detailed and integrated conceptual models) is utilized to capture and validate complex inter-relations between different societal, economic, political and individual dynamics and components. With such a conceptual framework designed, using inclusive and participatory dialogue techniques, appropriate measurement instruments are then selected, adapted and designed – either from the existing library of SCORE instruments or custom-made for the specific country.
The SCORE Index can flexibly integrate different modalities of data collection as required, including surveys, discourse analysis, expert assessments and draws its strength from advanced analytical and statistical toolkits. Large SCORE sample frames are then designed in a way to ensure that results can be reported with a high level of confidence for different sub-regions within the country, but also for distinct societal groups of interest (e.g. specific ethnic communities or social demographics such as internally displaced communities). The actual fieldwork is usually conducted in collaboration with established national researchers or research agencies, who can display the needed level of cultural and contextual awareness and sensitivity to assure a reliable data collection process. Results are processed using advanced data analysis techniques (e.g. factor analysis, anova analysis, regression and structured equation modelling) from which robust metrics are designed for multiple indicators, which are translated into network analysis and predictive models that can reveal the intricate relationship between different indicators and groups. This modelling process is used to suggest effective entry points to design evidenced-based projects and policies.
Using participatory research principles, preliminary SCORE results are interpreted through multi-level stakeholder consultations and dialogue groups, which informs further data analysis and design of participatory policy briefs. Such a close cooperation and communication with the key stakeholders as well as grassroots not only adds richness, depth and value to the SCORE findings but also builds ownership and triggers constructive public debate ensuring that the final SCORE products are as contextual and as impactful as possible.
For a more detailed explanation of SCORE methodology, please read the short methodology paper here. For information on how to use the platform, you can watch the short video on our Facebook page here or read the How to Read SCORE manual here.Outcomes of interest: Identified desirable and high priority normative objectives that relate to assessing and fostering social cohesion in a given context (i.e. intergroup harmony, mitigating violent tendencies, fostering a human rights ethos). These outcomes are listed under the dimensions section of the SCORE Platform but should not be confused with categorical/thematic groupings. The outcomes of interest that make up the key components of social cohesion for Moldova are illustrated in the figure above. The other dimensions listed on the SCORE platform are categories that group different indicators thematically.
Indicators: The components of the conceptual model are translated into metrics and indicators that are quantifiable and measurable via public opinion polls. Each indicator that is measuring a particular phenomenon (e.g. economic security, discrimination towards out groups, belief in human rights, support for certain policy options, post-traumatic stress disorder, etc.) is usually assessed with minimum 3 questionnaire items, scaled following reliability tests, to ensure that the SCORE can robustly capture different dynamics underlying the given indicator.
Drivers/Predictors: Indicators that have a strong positive or negative impact on the outcome of interest are called drivers or predictors. They provide strategic entry points that hold the most likelihood of impact on the desired outcome of interest. These can be seen under the path analysis section.
Heatmaps: A score is calculated for each indicator. The scores range from 0 to 10, where 0 means that the phenomenon the indicator is measuring is not observed in the context at all, and 10 means that it is observed strongly and prevalently. Heatmaps demonstrate the regional differences of these scores in order to identify areas of concern and tailor interventions more precisely. For example, the Personal Security indicator is measured through the following questions:
To what extent do you feel safe from violence in your daily life?
To what extent do you feel confident that the police or other institutions can protect you from violence?
To what extent would you feel safe walking alone in the street at night?
A score of 0 for personal security would mean that no one in a given region feels secure at a personal level, while 10 would signify that every person feels absolutely secure. As such, a heatmap of personal security for Ukraine illustrates the scores for each oblast, allowing for an understanding of the regional variance in terms of personal security across the country.
Path analysis (predictive models): Based on advanced statistical analysis including regression, network analysis and structural equation modelling, predictive models investigate the relationship between different indicators and the outcomes of interest. Predictive models reveal those indicators that may have a reinforcing or mitigating influence on other indicators as well as the outcome. While the first wave of the SCORE can be used for identifying directional correlations and benchmarking, second and third waves of the SCORE, where temporal comparisons are possible, can help identify trends and causal relationships. The colour of the connecting lines represents the nature of the relationship - blue symbolizes a positive correlation, and red symbolizes a negative correlation. The thickness of the lines represents the t of the correlation – the thicker the line, the stronger the relationship. This section provides a brief preview of the SCORE methodology. To explore the SCORE methodology in more detail, you can download the SCORE book or explore other SCORE publications.
