Development of an Indicator Framework in Relation to Culture Change Towards Accessibility and Inclusion
of Persons with Disabilities


Final Report

March 23, 2022



Vendor Name:			Institute for Work & Health
Address:			400 University Avenue, Suite 1800
				Toronto, Canada M5G 1S5

      Principal Investigator:		Dr. Emile Tompa
							Health and Labour Economics

			Co-Investigator:		Dr. Amir Mofidi
							Post-doctoral Fellow

			Co-Investigator:		Emma Irvin
							Director of Research Operations

			Co-Investigator:		Dr. Rebecca Gewurtz
							Rehabilitation Sciences

This project was funded by Employment and Social Development Canada.


Contents

EXECUTIVE SUMMARY	5
Culture Change Towards Accessibility and Inclusion of Persons with Disabilities	5
Overarching Methodology	5
Summary of Findings	6
Conclusions and Recommendations	8
Recommendations for Future Research	9
Structure of the Report	10
GLOSSARY OF DEFINITIONS AND ACRONYMS	11
Definitions of Key Terms	11
List of Acronyms	12
INTRODUCTION	14
Background	14
Key Research Question	15
METHODOLOGY	16
Overarching Methodology	16
Scoping Review	16
Step 1. Identify the Research Question	17
Step 2. Conduct the Literature Search to Identify Relevant Studies	17
Step 3. Study Selection	18
Step 4. Additional Sources	20
Environmental Scan	20
Grey Literature Reviews	20
Key Informant Interviews	20
Summarizing, Analysis and Reporting	22
STUDY FINDINGS	23
Overarching	23
What is Culture Change	25
Frameworks to Develop, Monitor, and Evaluate Culture Change Initiatives	29
Application of Frameworks	29
Types of Frameworks	30
Template for Measuring Culture Change Initiatives	34
Initiatives Aimed at Culture Change Among Different Populations	39
Platforms for Culture Change Initiatives	39
Levels of Culture Change Initiatives	41
Examples of Initiatives Aimed at Changing Attitudes	41
Measuring Progress and Achievement of Targets Through Culture Change Initiatives	48
Surveys and Measurement Tools for Accessibility and Inclusion	48
Overview of Indicators	55
Proposed Indicators to Proxy for Culture Change in Relation to the ACA and Beyond	58
Data Gap Analysis	70
CONCLUSIONS AND RECOMMENDATIONS	73
REFERENCES	75
APPENDICES	91
Appendix I. Data Extraction Guidelines	91
Appendix II. Summary of Finding from The Key Informant Interview	94
Indicators Identified by Key Informants	94
Frameworks and Approaches for Culture Change	98
Appendix III. Examples for Application of Frameworks	102
Conceptual Framework for Culture Change Initiatives	102
Results Framework for Improved Quality of Health Services	103
Logical Framework for an HIV Voluntary Counseling and Testing Program	103
Appendix IV. Levels of Culture Change Initiatives	105
Public Health Agency of Canada Framework	105
Cross�s Stigma Intervention Matrix	106
Javed�s Framework for Reducing Stigma of Mental Health Disorders	107
Appendix V. In-depth Review of some Large-scale Culture Change Initiatives	108
Opening Minds (Canada)	108
Time to Change (England)	109
Beyondblue (Australia)	111
Like Minds, Like Mine (New Zealand)	112
Appendix VI. Initiatives Aimed at Changing Attitudes Toward Persons with Disabilities in Canada	113
Appendix VII. Identified Measurement Tools for Attitudes Towards Persons with Disabilities	116
Appendix VIII. Summary of Identified Literature	119
Scoping Review of Publication from 2020-2021	119
Snowballing Search of Peer-Reviewed literature and Environmental Scan	120

List of Tables
Table 1. Overarching research project theme (P.I.C.O. framework)	18
Table 2. Resources identified through different steps and included in the study	23
Table 3. Summary of key informant interviews (N = 17)	24
Table 4. Framework types and applications	30
Table 5. Standard logical framework matrix and definition of terminology	37
Table 6. Example of logical framework matrix for time to change initiative (England)	38
Table 7. Initiatives aimed at changing attitudes toward persons with disabilities	43
Table 8. Studies aimed to measure various cultural domains related to accessibility and inclusion of persons with disabilities	50
Table 9. Differences between monitoring and evaluation indicators	57
Table 10. Indicators to proxy culture change towards accessibility and inclusion of persons with disabilities	59
Table 11. Canadian Data gap analysis	70

List of Figures
Figure 1. Overarching study methodology	16
Figure 2. PRISMA 2020 flow diagram for the scoping review	19
Figure 3. Policy stages for developing culture change strategies	28
Figure 4. Conceptual framework for guiding culture change	31
Figure 5. Example of a results frameworks using the Time to Change initiative in England	32
Figure 6. Example of a logical framework for the �Time to Change� initiative in England	34
Figure 7. Type of indicators monitoring and evaluation	56

Appendices
Appendix I. Data Extraction Guidelines	91
Appendix II. Summary of Finding from The Key Informant Interview	94
Appendix III. Examples for Application of Frameworks	102
Appendix IV. Levels of Culture Change Initiatives	105
Appendix V. In-depth Review of some Large-scale Culture Change Initiatives	108
Appendix VI. Initiatives Aimed at Changing Attitudes Toward Persons with Disabilities in Canada	113
Appendix VII. Identified Measurement Tools for Attitudes Towards Persons with Disabilities	116
Appendix VIII. Summary of Identified Literature	119


EXECUTIVE SUMMARY

Culture Change Towards Accessibility and Inclusion of Persons with Disabilities

The term culture can have various meanings and is used differently in different contexts. On a broad level culture has been defined as a collection of attitudes, values, aspirations, and a sense of self-efficacy of people. In public policy circles it is most commonly used to refer to the set of influences on how individuals, groups and society see the world and react to it. Culture is important because it has a vital guiding effect on the actions people take and the behavior people demonstrate in life. Many policy outcomes depend on how people � as individuals and groups � behave. We know that goals relating to accessibility and inclusion of persons with disabilities (PWDs) depend as much on cultural factors as they do on government actions. Despite the importance of cultural factors in the accessibility and inclusion of PWDs, few studies have been published in this area, particularly in relation to developing, monitoring, and evaluating such initiatives/programs. 

The main objective of this study is to identify knowledge to support the development of frameworks and a reliable set of indicators for monitoring and evaluating culture change in the area of accessibility and inclusion. This is a critical step for assessing progress in the right direction and determining where efforts are more or less successful. This study identifies such a framework and indicators for use in tracking culture change initiatives in relation to accessibility and inclusion of PWDs. The two key research questions addressed by this study are as follows:

1. What is culture change and how can it be operationalized for use in the area of accessibility and inclusion of PWDs in society?
2. What data and measures are available to proxy for culture and culture change in domains relevant to PWDs? 

Overarching Methodology

We undertook two sub-studies to produce insights that were directly integrated into the framework and indicator development process. First, we undertook a scoping review of peer-reviewed literature on culture change initiatives in relation to accessibility and inclusion. In the second sub-study, we undertook an environmental scan of grey literature, existing national and international surveys, and data sources. Under this section, we also interviewed key informants about their knowledge about a framework and indicators for monitoring and evaluating culture change initiatives in relation to accessibility and inclusion. Because Canadian stakeholders are the primary target of this project, the scan is focused on jurisdictions with economies, labour markets, and programs similar to Canada, e.g., United Kingdom, Australia, New Zealand, the United States, the Netherlands, and other Northern European Countries. Overall, this study is comprised of five main components. Findings from 1. scoping review and 2. environmental scan helped us to inform the development of components 3-5 of this project, namely: 3. Identification of a conceptual framework of culture change in relation to accessibility and inclusion of PWDs, 4. Identification of promising initiatives to promote culture change among different populations in relation to accessibility and inclusion, and 5. Identification of indicators to measure progress and achievement of targets spearheaded through culture change initiatives. Lastly, we synthesized identified indicators for different Accessible Canada Act (ACA) priority areas and beyond. 


Summary of Findings

Findings from our scoping review and environmental scan revealed multiple studies from around the world aimed at culture change initiatives in relation to accessibility and inclusion of PWDs. Overall, 238 sources were included in the review, 58 of which were identified through scoping review of peer-reviewed articles published between 2020-2021 in Embase, PsycINFO, and ABI INFORM. Sixty-five studies were identified through a snowballing search of peer-reviewed literature identified through google scholar. 115 sources, including reports, surveys, and governmental and non-governmental organizational websites, etc., were identified through google search and enriched with key informants� interview inputs. We categorized identified studies as described in the following sections:

Key Informant Interviews
Seventeen in-depth interviews were conducted with knowledge leaders from around the world on the topic of culture change towards accessibility and inclusion of persons with disabilities. Key informants were from Canada, USA, UK and Australia. Several themes emerged from the key informants. It was noted that attitude change is difficult to measure, however behavioural changes can be measured and reflects attitude changes. Several tangible and less-tangible indicators of culture change were suggested in the areas of knowledge, attitudes, behaviours and practices that reflect the underlying process and pathways towards culture change, at the individual, institutional, community and other levels. Key informants noted that in some instances culture change happens from the top down, and in others, from the bottom up. Indicators of culture change were ultimately more tangible, clear and objective if they measured outcomes and impacts in relation to various facets of the quality of life of persons with disabilities. There was feedback from disability advocates that what matters most is outcomes, e.g., how many people are employed, the standard of living, experience with violence, access to services, access to jobs, and access to housing.

Several frameworks, approaches and guiding principles were identified by key informants, which we describe in detail below. They include: 1) Contact Based Education, 2) IDEAL Framework, 3) Australia Disability Strategy Outcomes Framework, 4) QuakerLab Inclusion Maturity Model, 5) Framework for a Psychosocially Healthy Workplace, and 6) Addressing Stigma: Towards a More Inclusive Health System. A summary of finding from the key informant interview can be found in Appendix II.

Framework of culture change in relation to accessibility and inclusion of PWDs
We were surprised to find no comprehensive frameworks to develop, monitor, and evaluate culture change Initiatives in relation to accessibility and inclusion of PWDs. Nonetheless, frameworks can be invaluable in helping understand a program�s goals and objectives and delineating the internal and external elements that can affect the program�s success. The most promising study that we are aware of is a framework for monitoring and evaluation of Australia�s disability strategy recently published by the Australia Department of Social Services in 2021. To address the gap in framework development for this area of inquiry, we broaden our research scope on culture change initiatives in other areas. For instance, we explored culture change initiatives aimed at combating violence against women. We also investigate HIV prevention culture change initiatives and counter-trafficking activities. Drawing on these studies, we identified different frameworks that have been used primarily to monitor and evaluate culture change initiatives. Despite some terminology inconsistencies across the literature, we identified three types of frameworks commonly used in culture change areas�conceptual frameworks, results frameworks, and logical frameworks. We provide an overview of each type of framework and illustrate some examples of them. We believe a logical framework, would be a practical tool to develop, monitor, and evaluate culture change Initiatives in areas of accessibility and inclusion of PWDs, which is in line with the Australia Department of Social Services recently published framework.

Initiatives to promote culture change in relation to accessibility and inclusion of PWDs
Our finding has revealed multiple Canadian and international initiatives aimed to enhance culture in relation to accessibility and inclusion of PWDs. These initiatives use multiple platforms (activities) for increasing awareness or changing people's attitudes, values, beliefs, and behavior. Some of the most common platforms are education, contact-based strategies, peer services, protest and advocacy, and social marketing approaches. These initiatives have been implemented by a variety of governmental and non-governmental organizations. Depending on the level of implementation, initiatives are categorized under three groups�personal, organizational, and governmental level. Personal-level Initiatives are directed at changing the attitudes of individuals towards PWDs. They involve the elements, often in combination, such as information, education, training, positive portrayal of PWDs, and supported opportunities for contact between persons with and without disabilities. Organizational-level initiatives address attitudinal barriers in domains such as employment, education, and health services. Policy types include training, complaints mechanisms and targeted information programs. They seek to mitigate the power disadvantages experienced by PWDs by changing the behavior and attitudes of persons without disabilities. Government-level initiatives often focus on legally mandating behavior change. Examples are anti-discrimination legislation and inclusive education. Under this section, we list multiple initiatives aimed to enhance culture in relation to accessibility and inclusion of PWDs. Moreover, we provide an in-depth view of four large-scale, widely known, national-level initiatives�Opening Mind (Canada), Time to Change (England), Beyondblue (Australia) and Like Minds, Like Mine (New Zealand).

Indicators to measure progress and achievement of targets through culture change initiatives
Through our scan, we identified various surveys used to measure different domains of culture in relation to accessibility and inclusion. Most of these surveys have been used by disability units of governments, while some have been used by universities and research centers. The term �culture change� has rarely been used in surveys. Rather, most the surveys measure attitude towards PWDs. Some measure stigma, and some focus on discrimination and discriminatory behaviors, etc. Some surveys estimate the awareness of people in relation to disability-related organizations, legislation, and initiatives. These surveys cover a diverse range of contexts, including attitudes towards PWDs in education, the employment sector, and the general level of comfort with living near PWDs. In terms of the target population, some surveys were designed to collect data from the general population, while others were meant to measure attitudes of specific actors such as employers, customers, health services providers, etc. We also note that multiple questionnaires have been developed and validated to measure various domains related to cultural in terms of accessibility and inclusion of PWDs. Details are provided in the Appendix VI.

We categorized indicators used to measure culture change in relation to accessibility and inclusion of PWDs in various areas where there are barriers to their ability to be confident, independent, and connected with others. To do so, we drew on the seven priority areas of the ACA, which received royal assent in the spring of 2019. We also considered other domains, beyond ACA priority areas, such as education, health services, first responder services and community life. Identified indicators would cover all the nine social life participation domains defined by the International Classification of Functioning, Disability and Health (ICF). The final list of domains considered are as follow: 1. employment, 2. built environment, 3. transportation, 4. information and communication technologies, 5. communication other than information communication technologies, 6. procurement of goods, services and facilities, 7. design and delivery of programs and services, 8. educational sector, 9. healthcare sector, 10. first responder, 11. community life.

Conclusions and Recommendations

Frameworks and indicators for culture change in relation to accessibility and inclusion of PWDs can be extremely useful for governments, community organizations, service providers, businesses, academics, and other stakeholders, as they can be used to identify cultural barriers, move toward solution, monitor and evaluate the impact of efforts, such as legislation, policies, programs, and practices to improve accessibility and inclusiveness in various domains of society, as well as help identify priorities and opportunities for action. Drawing on national and international sources, we identified knowledge to support the development of frameworks and a reliable set of indicators to monitor and evaluate different culture change initiatives for advancing accessibility and inclusion of PWDs in a different context. These frameworks and indicators can be tailored for government, community organization and other stakeholders. The initiatives identified can also serve to inform the development of initiatives in the Canadian context at the local, regional and national level to promote culture change in relation to accessibility and inclusion in different domains and populations. We also identified data gaps that might be filled in future data gathering efforts by stakeholders to help advance the monitoring and evaluation of culture change initiatives.

Recommendations for Future Research

Design and implement a national-level survey to estimate attitudes towards PWDs
Lack of baseline and follow-up data can be addressed by designing and implementing a large-scale, representative population survey on a periodic basis similar to that of other countries such as the public perceptions of disabled people: British Social Attitudes Survey (BSAS) in UK, Survey of Community Attitudes towards People with Disability in Australia, and Survey on Attitudes towards People with Disabilities in the Republic of Ireland. It is important that such a survey is designed and implemented in a way that provides a good understanding of attitude towards PWDs in different systems, such as employment, built environment, transportation, information and communication technologies, communication other than information communication technologies, procurement of goods, services and facilities, design and delivery of programs and services, education, health services, first responder services, and community life. Public attitudes toward PWDs survey can be developed under one of the existing surveys. For instance, Public Opinion Research (POR) studies could develop a module focused on culture change relevant to PWDs.

Develop a national-level result framework for monitoring and evaluation of accessibility and inclusion
A comprehensive accessibility and inclusion result framework should contain both objective and subjective indicators. While objective indicators, such as employment, labour market attainment, earning/income, etc. provide valuable information on the current socio-economic status of PWDs, subjective indicators, which are based on individual�s feelings and their expected life standards, could explain the underlying drivers of such behaviors to varying degrees. Researchers in the area of social indicator increasingly emphasize the value of obtaining both subjective and objective indicators. Interestingly, we are seeing many results frameworks in the area of culture change initiatives benefiting from this integration. For instance, in a recently developed result framework for Australia�s disability strategy, both subjective and objective indicators have been integrated. This can also be observed in culture change initiatives in other areas such as violence against women and girls programming, and HIV prevention programs for most-at-risk populations. The extensive list of subjective indicators identified in this research for different priorities of ACA and beyond, can be used as a complementary piece to objective indicators to monitor and evaluate culture change initiatives in relation to accessibility and inclusion of PWDs in Canada.

Collect and integrate discrimination records
Discrimination complaints, considered lagging indicators of culture in relation to accessibility and inclusion, are limited in current data collection systems. Our study revealed that such data can be found in different places such as federal, provincial, territorial human rights commissions, and administrative data on accessible transportation complaints, as well as in different surveys, including discrimination among PWDs by Canadian Community Health Survey in 2013, and the General Social Survey on Victimization Survey in 2009. Collection and integration of discrimination records are an invaluable resource for monitoring and evaluation of culture change in relation to accessibility and inclusion of PWDs.

Structure of the Report
The first part of the report provides some background information on the importance of accessibility and inclusion and presents the study�s research questions. Methodology section describes how information was gathered via scoping review and environmental scan. Under the finding, we categorize the identified resources under four sections to provide evidence regarding each question of the study. First, we provide an overview of a policy framework for culture change, drawing primarily on a study by the Strategy Unit of the UK Cabinet office. We also reviewed some frameworks that have been used to develop, monitor, and evaluate culture change initiatives in areas of accessibility and inclusion of PWDs and beyond. Under the second sections, we introduced some culture change initiatives around the world related to the accessibility and inclusion of PWDs. In third section, we provide reviewed some studies aimed to measure the progress and achievement of targets through accessibility and inclusion culture change initiatives. And under the fourth section, we list our proposed indicators to proxy culture change in domains relevant to the ACA and beyond. We also provide a data gaps analysis in the Canadian context in this section. In conclusion, we offer some recommendations for future research.

GLOSSARY OF DEFINITIONS AND ACRONYMS

Definitions of Key Terms

Activity � An important task that is seen as an essential step to achieving a result. These are the actions proposed to achieve the result (IOM, 2009).
Aspiration � The desire, aim, or ambition of achieving something (UK Cabinet Office, 2008).
Attitude � The position held toward a fact or action comprising of knowledge, emotional, and motivational elements (UK Cabinet Office, 2008).
Behavior � A specific action or reaction by an individual or group, driven by the interaction of cultural factors with incentives, regulation, legislation, and levels of information and awareness (UK Cabinet Office, 2008).
Conceptual Frameworks � Diagrams that illustrate relationships among relevant organizational, individual, and other factors that may influence a program and the successful achievement of goals and objectives (Frankel et al., 2007).
Cultural Capital � Attitudes, values, aspirations, and sense of self-efficacy � is important because it has an important guiding effect on the actions or behavior that we can choose in life (UK Cabinet Office, 2008).
Culture Change � Refers to 1. interventions to influence underlying attitudes, values and aspirations and how these manifests into behavior, 2. dynamic process by which behavioral patterns become established as part of attitudes and values (UK Cabinet Office, 2008).
Indicators � Quantitative or qualitative benchmarks that provide a simple and reliable basis for assessing achievement, change, or performance. They are a means of analyzing and monitoring the characteristics of operations, services, and processes, and their implementation. In addition, they can also be used to measure, monitor, evaluate and improve performance (IOM, 2009). 
Logical Frameworks � Diagrams that illustrate a linear and logical interpretation of the relationship between inputs, activities, outputs, outcomes and impacts with respect to objectives and goals (Frankel et al., 2007).
Results Frameworks � Diagrams that illustrate a direct causal relationship between the incremental results of the key activities all the way up to the overall objective and goal of the intervention (Frankel et al., 2007).
Stakeholders � These are agencies, organizations, groups, or individuals who have a direct or indirect interest in the issues under investigation, and who affect or are positively or negatively affected, by the implementation of activities. Stakeholders could include governments, donor communities, implementation partners, businesses and project beneficiaries (IOM, 2009).
Stigma � Stigma includes negative beliefs about and hostile perceptions towards others, shame and self-stigma, discriminatory practices in hiring, promotion and recognition of PWDs, and structural and organizational policies and processes that result in inequalities for PWDs (Dobson et al., 2021). 
Theory of Change � A theory of change defines the pieces and steps necessary to bring about a given long-term goal, in other words, it 1. demonstrates the pathway of how to get from here to there, 2. requires underlying assumptions to be detailed out in a way that they can be tested and measured, 3. puts the emphasis first on what the organization wants to achieve rather than on what the organization is doing (UNWomen, 2013).

List of Acronyms

A Measurement Tool for Assessment of Multiple Systematic Reviews (AMSTAR)
Access Awareness Week (AAW)
Accessibility for Ontarians with Disabilities Act (AODA)
Accessible Canada Act (ACA)
Americans with Disabilities Act (ADA)
Association for Behavioral Health and Wellness (ABHW) 
Association Lavalloise du Transport Adapt� (ALTA)
Attention-deficit/hyperactivity disorder (ADHD)
Attitude Toward Disabled Persons (ATDP)
Attitudes Toward Persons with an Intellectual Disability (ATTID)
Attitudes Toward Seeking Professional Psychological Help (ATSPPH) 
Attribution Questionnaire (AQ)
Australian Bureau of Statistics (ABS)
Autism Spectrum Disorder (ASD)
Autism Survey Questionnaire (ASQ)
British Social Attitudes Survey (BSAS)
California Assessment of Stigma Change Scale (CASC)
Canadian Community Health Survey (CCHS)
Canadian Human Rights Commission (CHRC)
Canadian International Development Agency (CIDA)
Canadian Psychiatric Research Foundation (CPRF)
Canadian Radio-television and Telecommunications Commission (CRTC) 
Canadian Survey on Disability (CSD)
Canadian Transportation Agency (CTA)
Canadian Centre for Disability Studies (CCDS)
Classification of Functioning, Disability and Health (ICF)
Community Attitudes Toward the Mentally Ill (CAMI) 
Consumer Experiences of Stigma Questionnaire (CESQ)
Convention on the Rights of Persons with Disabilities (CRPD)
Dementia Knowledge Assessment Scale (DKAS)
Department for International Development (DFID)
Department of Education, Employment and Workplace Relations (DEEWR)
Disability Employment Awareness Month (DEAM)
Disability Social Relationship Scale (DSR)
Disabled Persons Scale (IDP)
Disabled Persons Transport Advisory Committee (DPTAC)
Employees� Views About Persons with Mental Disorders Questionnaire (EVMD)
Employment and Social Development Canada (ESDC)
Employment Fears Scale (EFS)
Employment Value Scale (EVS)
Empowerment (BUES)
Frequency of Contact Scale (FCS) 
German Agency for International Cooperation or (GIZ)
Implicit Association Task (IAT)
Information Technology (IT)
Intended Behavior Scale (RIBS)
Internalized Stigma (ISMI)
International Service for National Agricultural Research (ISNAR)
Joint United Nations Program on HIV/AIDS (UNAIDS)
Logical Framework Approach (LFA) 
Non-Governmental Organizations (NGOs)
Maxwell and Sullivan Attitude Scale (MSAS)
Mental Health Commission of Canada (MHCC)
Mental Health Knowledge Schedule (MAKS)
Mental Retardation Attitude Inventory-Revised (MRAI-R)
Multidimensional Attitudes Scale (MAS)
Multifaceted Life Satisfaction Scale (MLSS)
National Disability Authority (NDA)
Opening Minds (OM) 
Openness Scale (OS)
Organisation for Economic Co-operation and Development (OECD)
Questionnaire on Student Attitudes Toward Schizophrenia (QSAS)
Recovery Attitudes Questionnaire (RAQ)
Social Distance Scale (SDS)
Stigma Scale (SS)
The Working Mind First Responders (TWMFR)
United Nations Development Programme (UNDP)
United States Agency for International Development (USAID)
Working Mind (TWM)


INTRODUCTION

Background

The term culture can have various meanings and is used differently in different contexts. On a broad level culture has been defined as a collection of attitudes, values, aspirations, and a sense of self-efficacy of people. In public policy circles it is most commonly used to refer to the set of influences on how individuals, groups and society see the world and react to it (UK Cabinet Office, 2008). Culture is important because it has a vital guiding effect on the actions people take and the behavior people demonstrate in life. Many policy outcomes depend on how people � as individuals and groups � behave. We know that goals relating to accessibility and inclusion of persons with disabilities (PWDs) depend as much on cultural factors as they do on government actions. 

      Public responses to PWDs influence the likelihood of success or failure of policies aimed at increasing their social inclusion (Scior et al., 2011). Statistics highlighted negative societal responses pose a major barrier for many PWDs in their ability to be confident, independent, and connected with others. Employment and Social Development Canada (ESDC)�s Accessibility Secretariat commissioned a study in 2018, to measure Canadians� awareness and experience with accessibility and disability issues1 (Government of Canada, 2019). In part of this survey, respondents with a disability were asked how often they have experienced attitudinal barriers. Nearly nine in 10 (88%) stated that they had experienced this type of barrier at one point in time, with 11% saying always, 30% saying they experienced it often, 28% sometimes and 17% rarely. In another study, by Canadian Human Rights Commission (CHRC), PWDs continue to be marginalized in Canadian society. CHRC estimated that half of all the discrimination complaints filed in Canada are related to disability. According to a report by CHRC between 2009 and 2013, 41,728 discrimination complaints were filed with human rights commissions and tribunals across Canada. Of those, 49% (20,615 complaints) were filed on the ground of disability (CHRC, 2015). Studies from Australia, the UK, Canada and other countries show a similar result. For instance, these studies indicated that many employers are reluctant to employ people with disabilities (CCDS, 2001; DEEWR 2008; UK Prime Minister�s Strategy Unit 2005), especially those with mental health issues, learning disabilities, or blindness (Zissi et al., 2007).

      To address the issue of cultural barriers, governmental and non-governmental organizations around the world have implemented various programs to change culture towards PWDs (Fisher et al., 2017). These include policies that involve direct contact with PWDs; information and awareness campaigns; education and training about disability; and antidiscrimination enforcement. These programs are targeted at different levels, including the personal level (such as awareness-raising and positive portrayal); the organizational level (such as education, employment, and health), and the government level (such as employment and healthcare sector) to change attitudes towards PWDs (Scior et al., 2011).

      Despite the importance of cultural factors in the accessibility and inclusion of PWDs, few studies have been published in this area, particularly in relation to developing, monitoring, and evaluating such initiatives. Having a framework and a reliable set of indicators for monitoring and evaluating culture change initiatives is critical for assessing progress and determining where efforts are more or less successful. 

      In this study, we aim to identify knowledge to support the development of such a framework and indicators for use in tracking culture change in relation to accessibility and inclusion of PWDs.

Key Research Question

The key research questions addressed by this study is as follows:

1. What is culture change and how can it be operationalized for use in the area of accessibility and inclusion of PWDs in society?
2. What data and measures are available to proxy for culture and culture change in domains relevant to PWDs?

     Given the modest literature base, we realized that there was much new terrain to explore in in terms of frameworks and indicators. To facilitate the process, we took a multi-pronged approach. Specifically, we undertook two sub-studies to produce insights that were directly integrated into the framework and indicator development process. In the first sub-study, we undertook a scoping review of peer-reviewed literature on culture change initiatives in relation to accessibility and inclusion. In the second sub-study, we undertook an environmental scan of grey literature, existing national and international surveys, and data sources. Under this second sub-study, we also interviewed key informants about their thoughts on developing a framework and indicators for monitoring and evaluating culture change initiatives in relation to accessibility and inclusion.

      In the following sections, first, we summarize the methodology of the study and then we summarized identified literature under different sections to provide some evidence regarding each study question. We reviewed some examples of a culture change initiative, framework, indicators, and measures that have been used around the world. This study provides a narrative review of initiatives drawn from different countries to illustrate promising examples or best practices in the field. Although it does not provide an exhaustive or systematic review of these programs, rather selects programs for the elements that best illustrate the points being made. Lastly, we focus on designing culture change interventions monitoring and evaluation framework and related indicators, which could serve as an input for monitoring and evaluation of progress made at the country level over time if it is executed periodically. We also identified data gaps that might be considered in future data gathering efforts by federal and provincial bodies to help better understand accessibility and inclusivity cultural change.
METHODOLOGY

Overarching Methodology

Figure 1 indicates the overarching of the study methodology. Overall, this study is comprised of five main components. The scoping review and environmental scan are the platforms for the identification of the initiatives, frameworks, and indicators from different countries. The latter three components of this project were synthesized based on the evidence from the scoping review and environmental scan. We note that indicators of culture change are often tied to specific initiatives targeted at change within some populations. Hence, we also review initiatives identified in the literature. In the following paragraph, we explain how we draw on a range of academic and professional sources in this regard.

Figure 1. Overarching study methodology


Scoping Review

We follow the scoping review methodology proposed by Arksey et al. (2005) and refined by Levac et al. (2010). According to this framework, there are several steps to undertaking a scoping review: (1) identify the research question(s); (2) identify relevant studies; (3) select studies; (4) Additional sources. In the following sections, we provide more details regarding each step.

Step 1. Identify the Research Question

Following are the research questions and issues that will underpin the work on this project.

1.	What is culture change and how can it be operationalized for use in the area of accessibility and inclusion of PWDs in society?
a.	What theoretical work has been done to conceptualize culture and culture change?
b.	What applied work has been done to measure culture and culture change?
c.	What does a grounded conceptual framework of culture change look like that would address areas relevant to the Accessible Canada Act (ACA) and based on theory and practice?

2.	What data and measures are available to proxy for culture and culture change in domains relevant to PWDs? 
a.	What are the key domains and how are they best defined?
b.	What qualitative and quantitative measures are available to proxy for domains within the conceptual framework?
c.	What are the gaps in the data and measures, and how might they be filled?
d.	How might the conceptual framework and measures be implemented to measure culture and culture change in Canada?
e.	How does the conceptual framework, measures and implementation plan fit with federal government strategy and initiatives?

Step 2. Conduct the Literature Search to Identify Relevant Studies

Databases, Timeline and Language
We consider peer-reviewed literature to identify potentially relevant studies. To access the peer-reviewed literature and also keep within the timeline (2020-2021), we searched the following electronic bibliographic databases for the last year using keywords and database-specific controlled vocabulary terms: 

Embase (Excerpta Medica Database): A biomedical and pharmacological database.
PsycINFO: An electronic bibliographic database providing abstracts and citations to the scholarly literature in the psychological, social, behavioral, and health sciences
ABI/INFORM: A business research database. 

      The searches are limited to English and French. A set of search terms for each category has been developed by our team librarian in consultation with the research team and project authorities and informed by previous review search strategies of relevance to our research question. 

Search Strategy

Table 1 provides a strategy tested by the team and discussed with the project authorities. We use the overarching research theme underlying this project and the P.I.C.O. framework (Population, Intervention, Comparison, Outcome) to guide the search strategy. The controlled vocabulary differs significantly across peer-reviewed databases. Therefore, search terms are customized for each database. Terms within each category are combined with a Boolean OR operator and the two main categories are combined using a Boolean AND operator. 

Table 1. Overarching research project theme (P.I.C.O. framework)
Person/Patient
Intervention
Comparison
Outcome
exp disability/
exp mental disease/
mental health/
policy/
public health campaign/
marketing/
law/
intervention study/
education/
education program/
Program/
Initiative/
Plan/
national.ti,ab.
(organization or organisation).ti,ab.
employer.ti,ab.
community.ti,ab.
individual#.ti,ab.
person#.ti,ab.
public.ti,ab.
Government/ federal/ provincial/ municipal/ sectoral/ industry/ healthcare
stigma/ attitude/ aptitude/ perception/ behavior/ social discrimination/ employment/ social participation/ knowledge/ capital/ cultural/ engagement/ awareness/ prejudice/ recruiting/ barrier/ belonging/ autonomy/ political/ economic outcomes
* Terms in red provided by ESDC 4/06/21. P.I.C.O: Population, Intervention, Comparison, Outcome.

Step 3. Study Selection

We developed a standard set of inclusion/exclusion criteria to be uniformly applied to all references to determine their relevance in answering our research questions. Proposed eligibility criteria are as follows:

Key questions for inclusion and categorization
1. Is the article about culture change and relevant to PWDs or a related topic (e.g., stigma reduction initiatives)? 
NB: Studies may use the terms stigma or culture change but not be about how to change norms within a particular group or social domain. The latter is an important element for a study to be included.
      Answer: yes, uncertain, no
If yes or uncertain then answer the following questions to put the study into the appropriate category (ies) for use in the project.
1. Does the article describe culture change theory or a conceptual framework for culture change?
2. Does the article describe or evaluate a culture change initiative?
3. Does the article identify/describe indicators to measure culture change?
4. Does the article address one of the seven ACA priority areas?

      Figure 2, indicated the PRISMA 2020 flow diagram that we used for our scoping review (Page et al., 2020). We reviewed the titles and abstracts of the one-year search (2020-2021) with no comparison terms undertaken in Embase, PsycINFO, and ABI INFORM. The search resulted in 1,861 unique titles and abstracts. A review of these titles and abstracts identified 58 articles for consideration. Following is a table of the search results.

Figure 2. PRISMA 2020 flow diagram for the scoping review



Step 4. Additional Sources

Beginning with these studies, we used a snowball approach to identify incremental studies and reports, as a complementary search. We looked through these article references. We also searched for peer-reviewed literature from leaders in the field. reference lists of this section included articles that were scanned for references not previously captured.

Environmental Scan

In this module, we conducted an environmental scan of the existing culture change initiative, framework, indicators, and measures around the world. This module consists of two sections: grey literature search and key informant�s interview. Because Canadian stakeholders are the primary target of this project, the scan is focused on jurisdictions with economies, labour markets, and programs similar to Canada, e.g., United Kingdom (UK), Australia, New Zealand, the United States (US), the Netherlands and other Northern European Countries. The latter Nordic countries are included for various reasons�they are advanced social democracies with well-developed policy and program evaluation traditions, and much of their work is published in English. In follow, we provide more detail regarding the methodology of each section.

Grey Literature Reviews

Grey literature consists of materials not published commercially or indexed by major databases. Capturing grey literature is important, as the most common target audiences for grey literature products are often policymakers, reporters, federal and state agencies, foundations, researchers, and grantees. It is sometimes challenging to locate and time-consuming to search. We conducted grey literature research by searching national and international websites of government agencies, research centers, community initiatives, data sources and online portals that gather information on relevant topics. Sites were bookmarked, and relevant documents downloaded (where possible) and organized by jurisdiction (Irvin et al., 2018). Within each jurisdiction, the starting point was the official websites of organizations working in accessibility and inclusion. Where resources appeared germane to the project, the language of the relevant sections were extracted verbatim and recorded in a table.

Key Informant Interviews

Over the period of February 1, 2022, to March 4, 2022, we conducted in-depth interviews with knowledge leaders on the topic of culture change towards accessibility and inclusion of PWDs. Several key informant interviews were completed to identify key domains of relevance for culture change. An initial list of key informants was identified by researchers on the project team, and then snowball sampling was conducted to expand the stakeholder list of potential participants (i.e., recommendations made by initial key informants of other people to interview). Knowledge leaders who participated in the interviews were asked to identify other participants who would be willing to contribute to our environmental scan. In-depth interviews were conducted with key informants in the various jurisdictions where relevant field knowledge has been identified, to further contextualize the information gathered from the websites. 

      Semi-structured interviews were held over the phone or online and lasted between 30 to 60 minutes. We probed participants to elaborate on the impacts of culture change initiatives and how they were measured. Interview questions asked about specific culture change initiatives in relation to the accessibility and inclusion of PWDs. Participants were also asked about framework, measures, or indicators of culture change in relation to accessibility and inclusion within their respective contexts. All interviews were recorded (with participants� consent) for accuracy, and detailed written notes were taken by the interviewer. Key themes were extracted from the interview notes and recordings and directly integrated into our findings. In our interviews, participants were asked a series of questions as indicated in following:

Question 1. Please tell me about the work that you do in relation to PWDs. Has your work focused on culture change in relation to accessibility and Inclusion?

Question 2. Have you been involved in or heard about any culture change initiatives in relation to the accessibility and inclusion of PWDs? It could be a governmental initiative or non-governmental initiative, and focus on organizations, communities, businesses, academia, industry sectors, government, etc.)
Probing Questions
* Please describe the initiative.
* Did it focus on particular kinds of disabilities?
* What populations or groups of people did it focus on?
* What levels of society was it targeting (e.g., individuals, communities, educational sector, government services, health services)?
* Where and when did it take place?
* Do you know of websites, reports, or studies where we can learn more about it?
* Are there particular people you would suggest we speak to about it? 

Question 3. What are some signs, measures, or indicators of culture change in relation to accessibility and inclusion that you have seen used in different areas of society? These could be in relation to initiatives you have mentioned above or others.
	Probing Questions
* Indicator useful and relevant in measuring effectiveness?
* Would you consider revising them if possible?
* What data sources have they drawn on?
* How extensively have they been used?
* Do you know of websites, reports, or studies where we can learn more about it?
* Have you classified the indicators into categories, e.g., as process and outcomes?
* Are there particular people you would suggest we speak to about it? 

Question 4. Do you know of any framework we might draw on that is about culture change in relation to accessibility and inclusion of PWDs? If not, do you know of others we might adapt for this purpose?
      Probing Questions
* Do you have suggestions of other areas of culture change in relation to accessibility and inclusion in society to explore (e.g., education, health care, public services, workplace, emergency services)?
* Do you know of websites, reports, or studies where we can learn more about it?
* Are there particular people you would suggest we speak to about it? 

Summarizing, Analysis and Reporting

Data from relevant studies were extracted to answer the review questions using a standardized data charting form. These forms were prepared, and pilot tested to ensure that the forms are being used consistently. We seek input from our project advisory committee on what information is most relevant to extract. Detailed information was charted include (but not be limited to): author, year, title, country, study type, type of intervention, target audience, context, type of disability, study objective, study outcomes (primary/ secondary), key indicators, project research questions alignment, potential key informant and organizations. Appendix I. provides more detail regarding data extraction guidelines. Due to the time constraints, each article was charted by one reviewer, however, there will be a quality check of 10% as per the AMSTAR tool (A Measurement Tool for Assessment of Multiple Systematic Reviews) (Shea et al., 2007). Reviewers will not chart data on studies they have consulted on, authored, or co-authored. Our analysis included a descriptive analysis of the relevant literature. Guided by input from the project authorities, data related to relevant aspects of the studies was analyzed narratively. 


STUDY FINDINGS

Overarching

Table 2 provides a summary of the variety of literature identified in both the scoping review and environmental scan that have been included in the study. Overall, 238 sources were included in the review, 58 of which were identified through scoping review of peer-reviewed articles published between 2020-2021 in Embase, PsycINFO, and ABI INFORM. Sixty-five studies were identified through a snowballing search of peer-reviewed literature identified through google scholar. Lastly, 115 sources (including 60 grey literature, including reports and surveys, and 55 governmental and non-governmental organizational websites) were identified through google search and enriched with key informant interview input.

Table 2. Resources identified through different steps and included in the study
Search steps
Data sources
Count
Scoping review of publication from 2020-2021 
Embase PsycINFO
ABI/INFORM
58
Snowballing search of peer-reviewed literature
Google Scholar
65
Environmental scan 
Grey literature, including reports and surveys
Governmental and non-governmental websites
Google search, Key informants� interviews
115 
60
55
Overall 

238
Note. Embase (Excerpta Medica Database): A biomedical and pharmacological database, PsycINFO: An electronic bibliographic database providing abstracts and citations to the scholarly literature in the psychological, social, behavioral, and health sciences, ABI/INFORM: A business research database. 

      We identified multiple governmental and non-governmental initiatives around the world aimed culture at promoting culture in the area of accessibility and inclusion of PWDs. We also identified various surveys and measurement tools used to estimate different domains of culture, such as attitude, stigma, discrimination, and behaviors toward PWDs. 

      Over the period of February 1, 2022, to March 4, 2022, seventeen in-depth interviews were conducted with knowledge leaders from around the world on the topic of culture change towards accessibility and inclusion of persons with disabilities. Key informants were from Canada, USA, UK and Australia. See Table 3 for details. Key informants included representatives from the government, community, national organizations, academic institutions, businesses and other stakeholders. Semi-structured interviews were held online and lasted between 30 to 60 minutes. Key themes were extracted from the interview notes and recordings and directly integrated into our findings. 

Table 3. Summary of key informant interviews (N = 17)
Category
Number
Interview Status
17 Completed
Country
9 Canada, 5 United States, 2 Australia, 1 United Kingdom
Occupation
6 Professors, 6 Directors, 4 Managers/Coordinators, 1 Vice President
Organization
7 Universities, 6 Community Groups, 2 United States Government, 1 Australian Government, 1 Crown Corporation
Gender
10 females, 7 males

      Several themes emerged from the key informants. It was noted that attitude change is difficult to measure, however behavioral changes can be measured and reflects attitude changes. Several tangible and less-tangible indicators of culture change were suggested in the areas of knowledge, attitudes, behaviors and practices that reflect the underlying process and pathways towards culture change, at the individual, institutional, community and other levels. Key informants noted that in some instances culture change happens from the top down, and in others, from the bottom up. Indicators of culture change were ultimately more tangible, clear and objective if they measured outcomes and impacts in relation to various facets of the quality of life of persons with disabilities. There was feedback from disability advocates that what matters most is outcomes, e.g., how many people are employed, the standard of living, experience with violence, access to services, access to jobs, and access to housing.

      Several frameworks, approaches and guiding principles were identified by key informants, which we describe in detail below. They include: 1) Contact Based Education, 2) IDEAL Framework, 3) Australia Disability Strategy Outcomes Framework, 4) QuakerLab Inclusion Maturity Model, 5) Framework for a Psychosocially Health Workplace, and 6) Addressing Stigma: Towards a More Inclusive Health System. A summary of finding from the key informant interview can be found in Appendix II.

      In the following sections, we first provide a high-level overview of the culture change process, and then we categorized identified studies into the following areas:

* Frameworks used to develop, monitor, and evaluate culture change initiatives in area of accessibility and inclusion of PWDs and beyond 
o Application of frameworks
o Types of frameworks
o Template for measuring culture change initiatives

* Initiatives aimed at promoting culture change among different populations in relation to accessibility and inclusion, improve attitudes toward PWDs, and reduce stigma and discrimination
o Platforms for culture change initiatives
o Levels of culture change initiatives
o Examples of initiatives aimed at changing attitudes
o In-depth review of some national culture change initiatives

* Studies aimed at measuring progress and achievement of targets through culture change initiatives in the area of accessibility and inclusion 
o Surveys and measurement tools for accessibility and inclusion of PWDs
o Indicators to proxy for culture change in relation to the ACA and beyond
o Canadian data gap analysis related to culture in the area of accessibility and inclusion 

What is Culture Change

Governments have traditionally used a combination of incentives, legislation and regulation in an attempt to encourage and persuade the public to adopt different forms of behavior (UK Cabinet office, 2008).2 In many cases, these have proven to be effective. However, there is increasing recognition that �cultural capital� factors � our attitudes, values, aspirations and sense of self-efficacy � are also important determinants of our behavior. In what follows, we provide a short overview of a policy framework developed to support an understanding of the culture change process. It draws largely on the Strategy Unit of UK Cabinet Office (2008). Given the objective of this study, we do not intend to delve into the theory of culture of change, so readers might refer to the original document for more details.

      Figure 3 presents a seven-phase strategy for implementing and evaluating culture change initiatives (Strategy Unit of UK Cabinet office, 2008). In most instances, it is helpful to follow these steps in the order they are laid out, although the process in not intended to be mechanical. For example, in some instances it may be more appropriate to understand the drivers of attitudes before segmenting target populations. 

1. Identify whether culture change is relevant to the policy area using a set of initial filtering criteria
These provide a handy checklist for identifying issues appropriate for further scoping. For example, a government authority may aim to reduce discrimination again PWDs in education, employment or training.

2. Establish the policy objectives in the area under consideration and assess the rationale for government intervention including any information on likely costs and benefits
The second stage requires clarity about objectives, why they are important, and what the appropriate role for government may be. The Strategy Survival Guide sets out some common questions that should be asked at this stage of any project: 1. What is the problem we are trying to address? 2. Why is it important? 3. Can we clearly articulate what the desired outcome is? 4. Identify and segment relevant population groups and set appropriate goals for each. 5. What has been done to date in this area? 6. How effective has it been? 7. What is the rationale for government intervention? 8. What do we know about the likely costs and benefits (ideally considering direct, indirect and intangible costs and benefits)? 9. How can we measure what we are trying to achieve? 10. What is the mandate for the project? 11. Can we identify a suitable sponsor for the work?

3. Assess the drivers of attitudes and behavior in the area by identifying the relationships between attitudes, behavioral intentions, and behavior
The third stage is relatively more involved and requires being able to segment target populations to tailor the intervention accordingly. Segmenting techniques bring together a range of disciplines to identify the attitudes and motivations of individuals and personalize the policy response accordingly. This often combines traditional methods such as demographic or epidemiological profiling with psychological techniques to develop a detailed map of what motivates people and how they respond to different interventions.

4. Determine suitability of different policy interventions including identifying the efficacy of incentives, legislation, regulation, and information approaches as well as the efficacy of interventions to address the development of attitudes, values, and aspirations
This is a crucial methodological step that requires assessing the relative role cultural capital plays in driving behavior, and the influence of incentives, legislation, regulation and information and engagement in translating this into actual behavior. The purpose of doing so is to identify where any behavior �block� may be coming from. This requires building up a detailed map of the behavioral path for each user group identified in stage 3. Evidence sources can include social surveys, opinion polling, psychological research, self-reporting studies and statistical modelling.

5. Establish how best to monitor effectiveness including the extent to which objectives are being achieved, opinions of stakeholders, and evidence on cost-effectiveness achieved
This stage requires moving from an understanding of what is driving behavior to how policy interventions can effectively influence these. Determining where interventions can be most effective should be based on two factors. First, the relative importance of the different drivers of behavior (based on stage 4). Second, the extent to which interventions have demonstrated effectiveness in influencing these drivers.

6. Roll out and implement preferred interventions, including clarifying delivery responsibility, identifying risks, and creating safe spaces to test, trial, and innovate
Establishing a system to monitor effectiveness of interventions is a critical step to undertake prior to actually rolling out programs. This means: 1. Determining what the appropriate baseline is and, where possible, a control group to test against. 2. Defining how to assess whether objectives have been achieved. 3. Establishing how financial impacts and costs and benefits will be assessed. 4. Setting a plan to systematically monitor views of professionals, public and stakeholders. These should enable continual improvement of each scheme and enable knowledge to be disseminated and deployed elsewhere. Given the likely longer-term nature of culture change initiatives, there may be need to make more use of intermediate targets and assessments for evaluation purposes (UK Cabinet office, 2008). 


Figure 3. Policy stages for developing culture change strategies

Source: UK Cabinet office, 2008.


Frameworks to Develop, Monitor, and Evaluate Culture Change Initiatives

As a part of our scoping review and environmental scan, we looked at different frameworks that have been used for culture change initiative/programs in the area of accessibility and inclusion. Noteworthy is that there are only a few studies in this area. A recent framework developed by the Australian Department of Social Services in 2021, provides a platform for monitoring and evaluation of Australia�s disability strategy (Australia Department of Social Services, 2021). Another study by Dupuis et al., (2016), described a culture change initiative currently underway in Canada, the Partnerships in Dementia Care (PiDC) Alliance, which provides some information on the theoretical and philosophical underpinnings of their initiative. Given the limited number of studies with details about the underlying frameworks for culture change in the area of accessibility and inclusion, we broaden our search for frameworks to other areas. For instance, we explored culture change initiative aimed at combating violence against women (PATH, 2011; Casals et al., 2009; Hagemann?White et al., 2008; European Women�s Lobby, 2002). We also explored an HIV prevention culture change initiatives (UNAIDS, 2008; 2010), and another in the area of counter-trafficking activities (IOM, 2009). 

The purpose of this section is to provide an overview of different frameworks that have been used to develop, monitor and evaluate culture change initiatives. To do so, we first provide an overview of the application of such frameworks. Then we introduce three types of frameworks that have been used in culture change initiatives, and lastly, we illustrate some examples of frameworks that we identified through our study that were used to develop and support culture change initiatives.

Application of Frameworks 

A well-thought-out framework can greatly assist with thinking through programs (initiatives), objectives, and planned activities, as well as help assess whether the programs are indeed the most appropriate ones to implement. Additionally, a framework can help increase understanding of a program�s goals and objectives, define the relationships between factors key to implementation, and delineate the internal and external elements that could affect its success. Overall, frameworks help to: 
* Identify programs to address culture change and related activities
* Articulate program goals and objectives 
* Measurable short, medium and long-term objectives 
* Define relationships among inputs, activities, outputs, outcomes and impacts 
* Clarify the relationship between program activities and external factors 
* Demonstrate how activities will lead to desired outcomes and impacts (Frankel et al., 2007).

Types of Frameworks

Our review revealed that the term �frameworks� has been used in many different ways and for different purposes. Nonetheless, we identified the three most common types of frameworks used to support understanding of culture change initaitives: conceptual frameworks, results frameworks, and logical frameworks (PATH, 2011). More detail about these frameworks and their applications are provided in Table 4 (Frankel et al., 2007).

Table 4. Framework types and applications
Type of framework and brief description
Initiatives /programs management
Basis for monitoring and evaluation
Conceptual 
Interaction of various factors
Determines which factors the program will influence
No: 
can help to explain the result
Result 
Logically linked initiative objectives
Show the causal relationship between program objectives
Yes: 
at the objective level
Logical 
Logically linked inputs, process, output and outcomes
Show the causal relationship between input ad the objectives
Yes:
At all stages of initiatives/ programs from input to process to outputs to outcome/ objectives
Source: Frankel et al., 2007.

Conceptual Frameworks

Conceptual frameworks are often presented in the form of diagrams that illustrate relationships among relevant organizational, individual and other factors that may influence a program and the successful achievement of goals and objectives. They determine which factors (i.e., underlying cultural, economic, and socio-political) will influence the program and identify how each of these factors might relate to and affect the outcomes. They do not form the basis for monitoring and evaluation activities but can help explain program results. Note that different kinds of interventions (policy change, awareness-raising campaigns, community mobilization, improving service delivery and response) will need different kinds of frameworks, tools and indicators.

* Theory of change: Conceptual frameworks illustrate the theory of change and reflect the underlying process and pathways through which the hoped-for change (in knowledge, behavior, attitudes, or practices, at the individual, institutional, community, or other levels) is expected to occur.
* Target population: Who is going to be exposed directly to the initiative? Who will be receiving training? Who will be hearing/seeing/participating in the campaign? Who might be exposed indirectly to the intervention? Who will be exposed to some of the ideas, even if they do not directly participate?
* Timeframe for behavior change: While changes in knowledge may be easy to effect, changes in attitudes and behaviors, especially with community-level norms, requires a much longer process.
* Measuring changes: Conceptual frameworks identify appropriate measures for the kind of change that is expected.

      Figure 4 provides an example of a conceptual framework application for a culture change process, drawing on �PiDC Alliance� Initiative in Canada (Dupuis et al., 2016). In this framework, culture change is seen as a journey that includes re-examining values, beliefs, attitudes, language, practices and policies. In the context of accessibility and inclusion, it includes the full range of efforts needed to transform the culture in a community so that everyone thrives. PiDC Alliance notes the need for culture change initiatives that are grounded in strong theoretical foundations. PiDC Alliance identified key frameworks and models that had interdependent, collaborative relationships at their core.

Figure 4. Conceptual framework for guiding culture change
Source: Dupuis et al., 2016.

Results Frameworks

Results frameworks identify the direct causal relationships between the incremental results of key activities through to the overall objective and goal of the initiative. This type of framework helps determine at which point in an initiative different type of results can be expected so that appropriate measures can be identified for monitoring and evaluation. Results frameworks include an overall goal, a strategic objective, and intermediate results.

      A strategic objective is an outcome that is the most ambitious result that can be achieved and for which the organization is willing to be held responsible. An intermediate result is a discrete result or outcome that is necessary to achieve a strategic objective. Notice that the goal and strategic objective appear at the top of the framework. Before achieving this broader strategic objective, a set of �lower-level� intermediate results must first be reached. Under each intermediate result are subordinate intermediate results that relate directly to the intermediate results (Frankel et al., 2007).

      Figure 5, provides an example of a results frameworks applied to the �Time to Change� Initiative in England. This initiative aimed to change the way people think and act about mental illness problems (Henderson et al., 2016; 2020). We drew on this initiative not only because of the availability of data but also because it aligns with the subject of this study. Notice that intermediate results (IR1 and IR2) and sub-intermediate results need to be measurable. In other words, indicators can be developed for them, and data can be collected to calculate them (Menon, 2009). Appendix III provides more examples to illustrate application frameworks. These examples have been selected in areas other than disability, mainly because we were not able to find disability-related examples. 

Figure 5. Example of a results frameworks using the Time to Change initiative in England


Source: Adapted from Henderson et al., 2016; 2020.

Logical Frameworks

Logical frameworks provide a linear and logical interpretation of the relationship between inputs, activities, outputs, outcomes and impacts with respect to objectives and goals. These frameworks provide a causal relationship. In fact, these frameworks outline the specific inputs needed to carry out the activities/processes to produce specific outputs which will result in specific outcomes and impacts. These frameworks form the basis for monitoring and evaluation activities for all stages of an initiative/program. 

> Inputs: The resources invested in a program�for example, technical assistance, computers, condoms, or training
> Processes: The activities carried out to achieve the program�s objectives
> Outputs: The immediate results achieved at the program level through the execution of activities
> Outcomes: The set of short-term or intermediate results at the population level achieved by the program through the execution of activities
> Impacts: The long-term effects, or end results, of the program�for example, changes in health status. In this context, the term �impact� refers to the health status or conditions that the program is intended ultimately to influence (mortality, morbidity, fertility, etc.), as measured by appropriate indicators.

      Figure 6 provides an example of a logical framework, once again drawing on the �Time to Change� Initiative in England (Henderson et al., 2016; 2020). This framework presents a straightforward view of an initiative designed to improve the way people think and act about mental illness problems. Appendix III provides more examples to illustrate the application of logical frameworks, in areas other than disability.

Figure 6. Example of a logical framework for the �Time to Change� initiative in England

Goal: To change the way people think and act about mental health problems

Source: Adapted from Henderson et al., 2016; 2020.

      Logical frameworks are not intended to account for all of the factors that may influence a program�s operation and results. Rather, they focus on the program�s inputs, activities, and results. This narrow focus assists program managers and planners clarify the direct relationships among elements of particular interest within a specific program. Overall, these frameworks are a valuable tool for:

1. Program planning and development: The logic model structure helps think through a program strategy to help clarify where the program is and where it should be. 
2. Program management: Because it "connects the dots" between resources, activities, and outcomes, a logic model can be the basis for developing a more detailed management plan. Using data collection and an evaluation plan, the logic model helps track and monitor operations to better manage results. It can serve as the foundation for creating budgets and work plans.
3. Communication: A well-built logic model is a powerful communications tool. It can show stakeholders, at a glance, what a program is doing (activities) and what it is achieving (outcomes), emphasizing the link between the two.

Template for Measuring Culture Change Initiatives 

In this section, we introduce an effective analytical and management tool called Logical Framework Approach (LFA) that has been used in cultural initiatives.3 To do so, we first provide a background on LFA, then we explain the format and preparation process. Lastly, we illustrate an application of this framework to a culture change initiative, drawing on the �Time to Change� initiative in England (Henderson et al., 2016; 2020).

      LFA is now used by most multi-lateral and bi-lateral aid agencies, international non-governmental organizations (NGOs) and by many governments to help develop and manage projects (Delevic et al. 2011). LFA was first formally adopted as a planning tool for overseas development activities by United States Agency for International Development (USAID) in the early 1970s. Since then, it has been adopted and adapted by a large number of agencies involved in providing development assistance. They include the British Department for International Development (DFID), Canadian International Development Agency (CIDA), the Organisation for Economic Co-operation and Development (OECD) Expert Group on Aid Evaluation, the International Service for National Agricultural Research (ISNAR), Australia�s AusAID and the German Agency for International Cooperation or (GIZ). The LFA helps to:

* Establish a causal link between inputs, activities, results to the purpose and overall objective,
* Define the assumptions on which the project logic builds,
* Identify the potential risks for achieving the purpose and objective,
* Establish a system for monitoring and evaluation of project performance, and
* Establish a communication and learning process among the stakeholders.

Format and Process of Preparation

Table 5 provides a visualization of the standard logframe and defines the terminology used. The log frame matrix provides a summary of a project down to the activity level. It consists with four columns and four (or more) rows, summarizing the key elements of a project, as follows:

* Project�s hierarchy of objectives (Project Description or Project Intervention Logic),
* How the project�s achievements will be monitored and evaluated (Indicators and Sources of Verification),
* Project environment and key external factors critical to the project�s success (Assumptions).

Define the overall objective to which your project contributes: The overall objective/goal is the higher-order objective that one seeks to achieve through a project, often in combination with others. It usually relates to a program or a sector. Very often a group of projects will share a common overall objective statement.

Define the purpose to be achieved by the project: The specific objective/purpose describes the intended effects of the project (project purpose), the immediate objective for the direct beneficiaries, as a precisely stated future condition. It is to be expressed in terms of the benefits to the target group. Essentially, it summarizes the impact that the project will have. It may describe how the world will be changed as a result of achieving the project�s results. The purpose often describes a change in the behavior of the project beneficiaries.

Define the results from achieving the purpose: Results/outputs are expressed as the targets which the project must achieve. Their combined impact should be sufficient to achieve the immediate purpose. They are to be expressed in terms of a tangible result delivered/produced/ conducted etc.

Define the activities for achieving each result: Activities are expressed as processes, in the present tense starting with an active verb, such as �prepare, design, construct, research, etc.� Avoid detailing activities; indicate the basic structure and strategy of the project. Activities define how the team will carry out the project. Generally, the aim is to provide an indicative list of activities that must be implemented to accomplish each result.

Identifying assumptions related to each level of the logic of intervention: Assumptions are external factors that have the potential to influence (or even determine) the success of a project, but lie outside the direct control of project managers. The assumptions are part of the vertical logic in the log frame. Start from the bottom of the matrix and work upwards.

Table 5. Standard logical framework matrix and definition of terminology

Project Description (Intervention Logic,
Objective Hierarchy)
Performance Indicators
(Objectively Verifiable Indicators, Targets)
Monitoring Mechanisms
(Means of Verification, Source of Verification-information)
Assumptions 
(External Factors)
Overall Objective
What is the overall objective that the project will contribute to?
What indicators are linked to overall objective?
Means of verification indicate where and in what form information on the achievement of objectives and results can be found (e.g., reports of ministries, project reports, laws, statistics, assessments, etc.)
Not required for overall objective
Purpose
What is the purpose of project to be realized by the project?
Indicators showing the extent to which project purpose has been achieved?
Above mentioned
Factors and conditions not under direct project control, but necessary to achieve the overall project objective?
Results
What are concrete visible results to contribute to realization of project purpose? 
What are the indicators showing whether expected results have been achieved?
What are the indicators showing whether expected results have been achieved?
Above mentioned above
Activities
What activities are required and in what order in order to achieve the expected results?
Means
What resources are required for implementation of listed activities?
Costs
What are the costs for each of the means?
Above mentioned above
Source: Delevic et al., 2011.


Table 6. Example of logical framework matrix for time to change initiative (England)

Project Description (Intervention Logic,
Objective hierarchy)
Performance Indicators
(Objectively Verifiable Indicators, Targets)
Monitoring Mechanisms
(Means of Verification, Source of Verification-information)
Assumptions 
(External Factors)
Overall Objective
To change the way people, think and act about mental health problems
Changes in public discriminatory behaviors
Admin Data
Not required for overall objective
Purpose
Enhance mental health-related knowledge among general public in England targeted middle-income groups aged 25�45 between 2009 and 2016
Changes in public: 
* Mental health-related knowledge
* Mental health-related attitudes
* Desire for social distance
Survey of a nationally representative sample using:
* Mental Health Knowledge Schedule (MAKS)
* Community Attitudes towards the Mentally Ill scale (CAMI)
* Reported and Intended Behavior Scale (RIBS)
NA
Results
* Number of events
* Number of content in social media
Public awareness about of Time to Change campaign
* Admin Data
* Survey
NA
Activities
* Social marketing campaign using traditional and social media
* Local community events to bring people with and without mental health problems together
Means
* Human resources
* Financial resources
Costs
* Time to challenge social marketing campaign budget: �8,311,066
* Time to challenge online resource budget: �196,049
NA
Source: Adapted from Delevic et al., 2011, Henderson et al., 2016; 2020.


Initiatives Aimed at Culture Change Among Different Populations

In this section, we reviewed multiple initiatives aimed at changing culture in relation to PWDs and particularly at reducing stigma and discrimination. The details have been taken from the information gathered through the scoping review and supplemented with data from the environmental scan. Although this is by no means an exhaustive list, it seeks to provide an introduction to the relevant literature. In what follows, we first provide a short overview of some of the commonly used platforms aimed at changing culture and attitudes toward PWDs. Then we provide a list of the many Canadian and international initiatives aimed at changing attitudes toward PWDs. Under the last sub-section, we provide an overview of some widely known culture change Initiatives.

Platforms for Culture Change Initiatives

Our review revealed that culture change initiatives have used multiple platforms to increasing awareness or to change values, attitudes, beliefs, behavior, etc. Some platforms are macro in level, like national-level initiatives, while some have are at a smaller scale, like an educational setting. We provide a short explanation for each platform, focusing on the ones previously employed in the area of accessibility and inclusion of PWDs.

> Education: Educational anti-stigma Initiatives present factual information about the stigmatized condition with the goal of correcting misinformation or contradicting negative attitudes and beliefs. They counter inaccurate stereotypes or myths by replacing them with factual information (National Academies of Sciences, Engineering, and Medicine, 2016).

> Contact: Across a wide range of stigmatizing conditions, people without the stigmatized conditions have little meaningful contact with those who have these conditions. Lack of contact fosters discomfort, distrust, and fear. Contact interventions aim to overcome this interpersonal divide and facilitate positive interaction and connection between these groups. In contact-based behavioral health anti-stigma interventions, people with lived experience of mental illness or substance use disorders interact with the public, describing their challenges and stories of success. These strategies are aimed at reducing public stigma on a person-to-person basis but have also been shown to benefit self-stigma by creating a sense of empowerment and boosting self-esteem (National Academies of Sciences, Engineering, and Medicine, 2016).

> Peer Services: Because contact-based strategies can be used to reduce both public and self-stigma, there is a wide range of potential intervention targets. One approach to integrating contact-based interventions into day-to-day activities is through the use of peer services. Peer service providers are people with lived experience who work as team members and foster the provision of nonjudgmental, nondiscriminatory services while openly identifying their own experiences. When integrated into service-provision teams, peers can help others to identify problems and suggest effective coping strategies (National Academies of Sciences, Engineering, and Medicine, 2016).

> Protest and Advocacy: Protest strategies are rooted in advancing civil rights agendas. In the context of this stuy, protest is a formal objection to negative representations of PWDs. Protests are often carried out at the grassroots level by those who have experienced discrimination and by advocates on their behalf. Strategies typically employ letter writing, product boycotts, or public demonstrations. Protest messaging and advocacy can help to engage and activate �fence-sitters��people who have some investment in behavioral stigma change but limited knowledge about how to translate their beliefs into action. A call to action can also energize unengaged stakeholders by raising awareness about the harmful effects of stigma (National Academies of Sciences, Engineering, and Medicine, 2016).

> Social Marketing: Social marketing is an approach commonly used in attempts to influence community attitudes and reduce stigma or to create social change (Randle et al. 2017). Using traditional marketing techniques raises awareness of a given problem or cause and aims to convince an audience to change their behaviors. Social marketing promotes a behavior or lifestyle that benefits society, in order to create the desired change (Huhn et al., 2019). There are few publicly available evaluations of social marketing campaigns that aim to address stigma and negative attitudes toward the inclusion of PWDs. Of those that have been evaluated, some of the more successful are the �Time to Change� campaign in the UK, the �Like Minds, Like Mine� campaign in New Zealand and the �See Me� campaign in Scotland, all of which achieved high penetration of the market to positively influence attitudes. These campaigns received national exposure and utilized multiple communication channels and a range of creative strategies (Randle et al. 2017).

> Simulation: Simulation is a form of alternate clinical experience that exposes students to experiential learning by using technological or nontechnological strategies. In the pre-briefing stage of simulation, students are provided an opportunity to discuss the topic and focus of the simulation. The simulation is an enactment of a life-like scenario followed by a debriefing in which students reflect on their feelings and thoughts about the experience (Burns et al., 2017).

> Virtual simulations: Virtual simulation is an emerging method of illustrating an issue through an actor�s perspective. There are multiple ways to foster perspective-taking. Perhaps the most common method is simply asking people to imagine the thoughts and feelings of another person in a particular scenario (i.e., imagination prompts), which has been shown to increase inclusive intergroup categorization, positive attitudes, and helping behaviors. Other researchers have used films to portray the perspective of a stigmatized target, which has resulted in social acceptance and perceptions of in-group belonging. Virtual simulated experiences allow an individual to become a virtual stigmatized actor and attempt to control actions in real-time. Indeed, an individual�s brain should begin regulating and controlling the stigmatized actor�s behavior in a similar manner as their own body, by creating their own embodied simulation of the relevant environment (virtual or real). Representing and predicting actions, concepts, and emotions in real and virtual environments through the same mechanisms in the brain should encourage perspective taking in virtual simulations (Sarge et al., 2020).

Levels of Culture Change Initiatives

Our review revealed that culture change initiatives often have been categorized in different levels. For instance, Fisher et al., (2017) categorized culture change initiatives under three groups�personal, organizational, and governmental level. This categorization is consistent with many other studies such as the Public Health Agency of Canada (2019b), Cross et al., (2011) and Javed et al., (2021). More detail regarding these studies is presented in Appendix IV.

�	Personal-level initiatives are directed at changing the attitudes of individuals towards PWDs. They involve elements, often in combination, such as information, education, training, positive portrayal of PWDs, and supporting opportunities for contact between persons with and without disabilities.

�	Organizational-level initiatives concern attitudinal barriers in domains such as employment, education, and health. Policy types include training, complaints mechanisms and targeted information programs, which seek to mitigate the power disadvantages experienced by PWDs, by changing the behavior and attitudes of persons without disabilities. At the same time, these policies try to empower PWDs to claim their rights to equal access and participation.

�	Government-level initiatives legally mandating behavior change. These policies are initiated by governments and attempt to influence attitudes by mandating behavior change. This level consists of the policy statements and laws that define actions reflective of positive attitudes, together with the means for implementation and monitoring. Examples are anti-discrimination legislation and inclusive education (Fisher et al., 2017).

Examples of Initiatives Aimed at Changing Attitudes

Table 7 lists multiple initiatives aimed at changing attitudes toward PWDs. Each intervention has been evaluated with reference to the objectives, policy level, target population, type of disability, as well as intervention platform/approach, that is, whether it involved education, and direct contact or contact-based education. This list has been identified through searches of governmental and non-governmental organizations� websites from around the world. Of importance to note is that there are hundreds of initiatives around the world this are in scope. For instance, in Australia alone, there are more than 61 programs/initiatives currently underway that target the reduction of stigma and discrimination (Morgan et al., 2021). Therefore, this list is not meant to be exhaustive. Rather, it is meant to provide information on some larger initiative directed at changing attitudes toward PWDs and to reducing stigma and discrimination. For more detail, readers can refer to appendices. Appendix V provides an in-depth view of some large-scale widely known national-level initiatives aimed at culture change. Appendix VI provides detail on provincial-level initiatives that have been identified through our environmental scan.



Table 7. Initiatives aimed at changing attitudes toward persons with disabilities
Author(s)
Year
Initiatives/Programs
Objective(s)
Country
Target Population
Type of Disability
Platform/approach
Centre for Addiction and Mental Health (CAMH)
NA
Addressing Stigma. Mental Health 101: Understanding Stigma
to reduce stigma associated with mental health
Canada
General population
Mental illness
Education and increasing awareness
Mental health commission of Canada (MHCC)
2017
Working Mind (TWM)
designed to promote mental health and reduce the stigma around mental illness in the workplace
Canada
Workplace employees, managers, and employers)
Mental illness
Evidence-based education program
Mental health commission of Canada (MHCC)
2017
The Working Mind First Responders (TWMFR), formerly known as Road to Mental Readiness (R2MR)
designed to address and promote mental health and reduce the stigma of mental illness in a first-responder setting.
Canada
First-responder setting
Mental illness
Education and increasing awareness
Holland Bloorview Kids Rehabilitation Hospital
2017
Dear Everybody
Campaign aims to reduce stigma, discrimination of kids, youth with disabilities
Canada
Teachers, employers and health-care providers
Kids and youth with disabilities
Mass media advertising (i.e.: Radio and television)
Department of National Defense in Canada
2016
Road to Mental Readiness (R2MR)
is mental health resilience education and training for Canadian Armed Forces members and their families
Canada
Military personal
Mental illness
NA
Bell Canada's 
2015
Let's Talk
to open a public dialogue about mental illnesses
Canada
General population
Mental illness
Uses technology and social media 
Mental health commission of Canada (MHCC)
Since 2007
Opening Minds (OM) anti-stigma initiative
Anti-Stigma Initiative
to improvements day-to-day lives of those living with a mental illness
Canada
1) health care providers, 2) youth, 3) Workforce, 4) media
Mental illness
Contact-based education
Government of Canada
2006
A way with Words and Images: Suggestions for the Portrayal of People with Disabilities
This booklet seeks to promote a fair and accurate portrayal of people with disabilities. It recommends current and appropriate terminology to help you reach this goal
Canada
General population
All PWDs
Publication
Canadian Psychiatric Research Foundation (CPRF)
2004
Imagine Campaign
NA
Canada
General population
Mental illness
Mass media advertising (i.e.: Publication, radio and television)
BC Association of Broadcasters
2003�04
Award-winning campaigns was presented by the BC Schizophrenia Society
Reduce towards schizophrenia
Canada
General population
Mental illness
Mass media advertising (i.e.: Television)
Canadian Mental Health Association�s BC Division
1998-99
Humanity Award campaigns
Change attitude towards mental illness
Canada
General population
Mental illness
Bus shelter posters, radio and television
Government of Canada
Since 1988
National Access Awareness Week (NAAW)
1) to celebrate contributions of PWD, 2) to remove barriers to accessibility and inclusion, 3) ongoing work we all have to do to counter discrimination against PWD
Canada
General population
All PWDs
Varies


Author(s)
Year
Initiatives/Programs
Objective(s)
Country 
Target Population
Type of Disability
Platform/approach
Kim et al., 
(Dementia Australia Ltd.)
2021
Dementia Stigma Reduction (DESeRvE)
reducing the general public dementia-related stigma 
Australia
General population
Mental illness
Education and contact 
Burns et al.
2017
Mental Health First Aid
to teaches members of the public how to provide mental health first aid to others and enhances mental health literacy
Australia
Nursing students
Mental illness
Contact, education, simulation
Beyondblue
2015
Beyondblue
to improve the knowledge and skills of primary care practitioners and other health professionals regarding mental health problems
Australia
youths aged 12-18, health care providers,
employers and the workforce,
news media
Mental illness
contact-based education programs
Koh et al.,
(The Dax Centre)
2014
Exhibition Program
Program of art by people with lived experience open to the general public
Australia
General population
Mental illness
Education
Victorian Office for Disability
2010
Picture this
increasing the cultural participation of PWDs in Victoria Community
Australia
Community
All PWDs
Education
Australian Rotary Health
Since 2000
Mental health awareness forums
to discuss all aspects of mental health
Australia
General population
Mental illness
Contacts, Protest/ Advocacy
Likeminds
2016
Like Minds, Like
Mine
to increase social inclusion and reduce stigma and discrimination against people with experience of mental illness and distress
New Zealand
General population
Mental illness
Social marketing campaign TV, community workshops, internet resources and disability provider training
Henderson et al.
2020
Time to Change program
to reduce mental health-related stigma and discrimination
England
General population
Mental illness
Social marketing campaign using traditional and social media
Walker et al.
2013
Tackling stigma intervention
To tackling stigma associated with intellectual disability among the general public
England
General population
Mental illness
Indirect contact interventions
Myers et al.
2009
See Me campaign (end mental health discrimination)
Against the stigma and discrimination associated with mental health
Scotland
General population
Mental illness
Social marketing campaign
Sarge et al.
2020
Auti-Sim
to reduce public stigma associated with autism spectrum disorder 
US
General population
Mental illness
Virtual simulation engagement; Observation of simulation engagement
DASH
2020
Disability Awareness Starts Here (DASH)
to educate the community about why and how good access makes sense for everyone
US
General population
All PWDs
Education and increasing awareness
Stamp Out stigma 
2021
Association for Behavioral Health and Wellness (ABHW)
reduce the stigma surrounding mental illness and substance use disorders
US
General population
Mental illness
Campaign in order to by encouraging people to talk about mental illness and substance use disorders
NAMI
2021
National Alliance of Mental Illness
A nation�s largest grassroots mental health organization dedicated to building better lives for the millions of Americans affected by mental illness
US
General population
Mental illness
Education and increasing awareness
Alison M.
Since 2003
Active Minds
to reduce the stigma surrounding mental health issues
US
Students in universities
Mental illness
Empowering students to speak openly about their mental health problems through student-run mental health awareness, education, and advocacy
Close G et al.
Since 2010
Bring Change to Mind (BC2M)
to launch a national dialogue about mental health through Public Service Announcements (PSAs), and evidence-based, peer-to-peer high school and undergraduate programs
US
General population
Mental illness
-Leveraged the power of celebrity and media
 -Creating national advocacy and awareness campaigns, 
-Developing a national student-led high school club program, 
-Building a storytelling movement that works to end the stigma in US



Measuring Progress and Achievement of Targets Through Culture Change Initiatives

This section consists of four parts. First, we provide an overview of a handful of surveys from around the world developed to measure various cultural domains related to the accessibility and inclusion of PWDs. Second, we provide some background information on different types of indicators and criteria for selecting appropriate indicators for monitoring and evaluation of a culture change initiative/program. Third, we categorized a battery of indicators that have been used to measure culture change in relation to the ACA priority areas and beyond. These indicators were drawn from knowledge gathered through an extensive scoping review and environmental scan. Forth, we undertake a data gaps analysis, providing a list of data elements for which we struggled to find a source, and which might be considered for inclusion in future data gathering efforts by federal and provincial authorities.

Surveys and Measurement Tools for Accessibility and Inclusion

Table 8 summarizes studies that describe instruments implemented to measure various cultural domains related to the accessibility and inclusion of PWDs. We provide details on region where they were used, department that initiated the survey, objectives, measurement tools, target population, and types of disabilities considered. Several noteworthy items about the identified studies:

* Most describe instruments that have been implemented by a disability unit of governments, while some have been implemented by universities and research centers.

* In terms of objectives, some studies measure attitudes, stigma, discrimination and discriminatory behaviors, etc. Some estimate the awareness of people of disability-related organizations, legislation, and initiatives. Findings of these studies cover a diverse range of contexts, including attitudes towards PWDs in education, the employment sector, or the general level of comfort living near PWDs.

* Many instruments have been designed and validated for use in measuring various cultural domains related to accessibility and inclusion.

* In terms of the target populations, some studies were designed to collect data from the general population, while others were meant to measure attitudes of specific actors such as employers, customers, healthcare, etc.

* In terms of disability types, some studies were generic, considering all types of disabilities, while some focus on certain types, such as persons with mental and intellectual disabilities. We classified studies into two broad categories�those that consider all PWD, and those focused on persons with mental disabilities.

     Appendix VII focuses specifically on measurement tools that have been developed and validated over the past couple of decades to estimate various domains related to attitude towards PWDs. These tools measure different aspects culture and norms such as stigma, attitudes, perceptions, awareness, distance, level of comfort, etc. toward PWDs across different populations. The appendix provides a valuable source of information for the next stage of research in the area of culture change, specifically for the purpose of designing a national survey in Canada to measure various cultural domains related to accessibility and inclusion of PWDs.



Table 8. Studies aimed to measure various cultural domains related to accessibility and inclusion of persons with disabilities
Author(s)
Year
Survey Title 
Country
Department 
Objective(s)
Measurement tools
Target Population
Sample Size
Type of disability
Bollier et al.
2021
National Survey of Community Attitudes towards People with Disability in Australia
Australia
Victorian Government+ and Centre of Research Excellence in Disability and Health
Focused on attitudes that impact the social and economic participation of people with disability of all ages
Questionnaire:
Developed based on 
1. British Social Attitudes Survey, 2. Attitudes to Disability Scale, developed by Power et al. (2009)
General population
2,000
All
Reavley et al.
2012
Stigmatising Attitudes Towards People with Mental Disorders: Changes in Australia over 8 years
Australia
Centre, Centre for Youth Mental Health, University of Melbourne
To investigate whether stigmatising attitudes towards people with mental disorders have changed over 8-year period
Telephone interviews:
1. personal and perceived stigma, 2. social distance, 3. perceived discrimination
General population
4,000
Mental
Australian Government
2011
Employer Perspectives on Recruiting Persons with Disabilities and The Role of Disability Employment
Australia 
Department of Education, Employment and Workforce relation
Measures employers� attitudes toward hiring people with disability and other groups
Questionnaire developed by author supplemented by qualitative data from an online discussion forum
Employers
2,744
All
Woodley et al.
2012
Employer attitudes towards employing disabled people
New Zealand
Ministry of Social Development
To better understand
New Zealand employers� attitudes towards employing disabled people
Questionnaire developed by authors
Employers
106
All
UK Government
2021
UK Disability Survey
England
Disability Unit and Policy Lab
Designed as a vehicle for disabled people, carers of disabled people, and members of the general public with an interest in disability, to share their lived experiences or views with the Disability Unit to inform the development of the National Disability Strategy
Questionnaire developed by authors
General population
14,491
All
Evans-Lacko et al.
2013
Public knowledge, attitudes and behaviour regarding people with mental illness in England 2009-2012
England
The Time to Change evaluation was funded by the Big Lottery Fund, Comic Relief and
Shifting Attitudes to Mental Illness (SHiFT), UK government Department of Health
To investigate whether public knowledge, attitudes and behaviour in relation to people with mental health problems have improved among the English population since the
inception of the Time to Change programme in 2009
Questionnaire: 
1. UK Department of Health Attitudes to Mental Illness, 2. Reported and Intended Behaviour Scale (RIBS), 3. Mental Health Knowledge Schedule (MAKS)
General population
1,700
Mental
Office for Disabilities Issues
2009
Public Perceptions of Disabled People: British Social Attitudes Survey (BSAS)
England
Office for Disability Issues (ODI)
The aim of these questions was to measure public attitudes towards disabled people and disability
Face-to-face survey and Questionnaire developed by authors
General population
3,600
All
Reid et al.
2013
Attitudes to Mental Health in Scotland: Scottish Social Attitudes (SSA) survey
Scotland
Scottish Government Social Research
To provides a detailed picture of public attitudes towards mental health problems. It examines the extent to which individuals with mental health problems are able to live free from stigma, discrimination, injustice and inequality, and also explores attitudes to recovery among those who identified themselves as having or having had a mental health problem
Face-to-face interviews with respondents and a self-completion section developed by authors
General population
1,500
Mental
National Disability Authority (NDA)
2002
Survey on Attitudes towards People with
Disabilities in the Republic of Ireland
Ireland
National Disability Authority of Ireland
The study is the first of its kind in this State and allows
for comparison with attitudes in other countries where such surveys have been a standard
exercise for some time
Face-to-face interview using �The Irish Social Omnibus Survey (ISOS)� which developed by Research and Evaluation Services (RES) (i.e.: an independent research organization)
General population
1,000
All
Frank et al.
2018
Canadian Community Health Survey Cycle 1.2 � Mental Health and Well-being (CCHS-MH Civilian) and its corresponding Canadian Forces Supplement (CCHS-MH)
Canada
Statistics Canada
Aimed to assess the independent contribution of
mental and physical health on both enacted stigma (discriminatory behaviour) and felt stigma (feelings of
embarrassment)
Questionnaire developed by Canadian Community Health
General population
Military (1900)+ civilian (2960)
Mental
Angermeyer et al.
2005
Causal beliefs and attitudes to people with schizophrenia: trend analysis based on data from two population surveys in Germany
Germany
Former Federal Republic of Germany.
To examine how the German
public�s causal attributions of schizophrenia and their desire for social distance from people with schizophrenia developed over 1990s
Structured interview, developed by authors
General population
5,025
Mental
Zissi et al.
2007
Greek employers� attitudes to employing people with disabilities: Effects of the type of disability
Greece
Department of Sociology, University of the Aegean
The aim of this cross-sectional study was to answer two central research questions: (i) what are Greek employers� attitudes to employing people with a range of disabilities; and (ii) do employers� attitudes differ according to the type of disability?
Face to-face interviews, developed by authors
Employers
102
All
Goreczny et al.
2011
Attitudes toward individuals with disabilities: Results of a recent survey and implications of those results
USA
Chatham University
to investigate attitudes of a group of 129 individuals attending a state-wide conference aimed toward improving quality of life of persons with disabilities
Questionnaire developed by authors
General population
129
All
Domzal et al.
2008
Survey of Employer Perspectives on the Employment of People with Disabilities
USA
United States Department of Labor, Office of Disability Employment Policy
The objective of this nationally representative survey was to inform the development and promotion of policy and practice by comparing employer perspectives across various industries and within companies of varying sizes
Telephone interview, questions developed by authors
Employers
3,797
All
Siperstein et al.
2006
A national survey of consumer attitudes towards companies that hire people with disabilities
USA
University of Massachusetts
To measure consumer attitudes toward companies that hire individuals with disabilities were assessed through a national public survey.
Telephone interview, questions developed by authors
Consumers
803
All



Overview of Indicators

What is an Indicator

Indicators are signs of progress � they are used to determine whether an initiative/program is on its way to achieving its objectives and goal. Once the overall objective, project purpose, results and activities have been identified � one needs to determine how to measure progress (IOM, 2009). 

* An indicator is a specific, observable and measurable characteristic that can be used to show changes or progress an initiative/program is making in achieving a specific outcome.
* An indicator should be focused, clear and specific. The change measured by the indicator should represent progress that an initiative is targeted to make.
* An indicator should be defined in precise, unambiguous terms that clearly describe exactly what is being measured. Where practical, the indicator should give a relatively good idea of the data required and the population the indicator targets.
* Indicators should not measure a particular level of achievement, but rather should measure changes of any magnitude, including both positive and negative (UNWomen, 2013).

Criteria for Selecting Indicators

The challenge in selecting indicators is to find measures that can meaningfully capture key changes, combining what is a substantively relevant reflection of the desired result with what is practically realistic in terms of collecting and managing data (Sandhu-Rojon et al., 2003). We listed some important criteria in selecting indicators in follow: 

* Valid: An accurate measure of behavior, practice, task that is the expected output or outcome of the intervention. 
* Reliable: Consistently measurable over time, in the same way by different observers
* Precise. Operationally defined in clear terms.
* Measurable: Quantifiable using available tools and method.,
* Timely: Provides a measurement at time intervals relevant and appropriate in terms of program goals and activities.
* Programmatically Important: Linked to the initiative or achievement of objectives (Gage et al., 2009).

Types of Indicators

Figure 7 illustrates two main types of indicators that are used in monitoring and evaluation: 1) Process indicators and 2) Result indicators. Process indicators are used to monitor the number and types of activities carried out. For instance, the number of people trained, number and type of materials produced, and number of services provided during an educational accessibility initiative. Result indicators aim to evaluate whether an activity achieved the intended objectives. Result indicators consist of three main subcategories�outputs, outcomes, and impact indicators. Output indicators measure change directly related to the activities undertaken within an initiative, for instance, the number of people who completed the training on accessibility culture. Outcome indicators measure change that is realized as a result of an initiative in the medium-to-longer term, for instance, whether and how much knowledge, attitudes, perceptions and practices of participants have changed after an educational accessibility initiative. Impact indicators measure long-term effect of an initiative, for instance, whether and how much the prevalence of discrimination towards PWDs has been reduced after an educational initiative. Comprehensive monitoring and evaluation should incorporate different types of indicators.

Figure 7. Type of indicators monitoring and evaluation

Source: Adapted from Bott et al., 2004.

Indicators for Monitoring and Evaluation

The purpose of monitoring and evaluation is to track implementation and outputs systematically and measure the effectiveness of culture changed initiatives. It helps determine when an initiative is on track and when adjustments may be needed. 

* Monitoring: It can be defined as a continuous function that uses systematic collection of data on specified indicators to provide management and stakeholders with indications on progress and achievement of objectives (IOM, 2009). 
* Evaluation: The systematic and objective assessment of an on-going or completed initiative, project, or program, and its design, implementation and results. The aim is to determine the relevance and fulfillment of objectives, development efficiency, effectiveness, impact and sustainability (IOM, 2009).

      Monitoring and evaluation indicators can be used to demonstrate that an initiative had a measurable impact on expected outcomes and was implemented effectively. It serves to support managers, planners, implementers, policy makers and donors by providing the data and understanding they need to make informed decisions about operations. Monitoring and evaluation provides data needed to guide strategic planning, design and implementation of initiatives, programs, and projects, and to inform resource allocation and re-allocation decisions. 

Table 9. Differences between monitoring and evaluation indicators

Monitoring 
Evaluation
Why?
Check progress,
Inform decision and remedial action,
Update project plans,
Support accountability
Assess progress and worth, 
Identify lessons and recommendations for longer-term planning,
Provide accountability
When?
Ongoing during initiative
Periodic and after an initiative
Who?
Internal, involving initiative implementers
Can be internal or external to an organization
Link to logical hierarchy
Focus on input, activities, outputs and shorter-term outcomes
Focus on outcomes and overall goal
Source: CSNM, 2020. 


Proposed Indicators to Proxy for Culture Change in Relation to the ACA and Beyond

Table 10 summarized indicators that have been used to measure culture change in relation to accessibility and inclusion of PWDs. These indicators were drawn from knowledge gathered through an extensive scoping review and environmental scan. We attempted to categorize indicators based on life situations where barrier to accessibility and inclusion could arise and compromise the ability of PWDs to be confident, independent, and connected with others. We drew on the seven priority areas of the ACA, legislation which received royal assent in the spring of 2019. We have added other domains beyond the ACA priority areas, such as education, health services, first responder services and community life. These additions were distilled from the literature review. An attempt has been made to ensure these indicators cover all nine social life participation domains defined by the International Classification of Functioning, Disability and Health (ICF), namely 1) learning and applying knowledge, 2) general tasks and demands; 3) communication; 4) mobility; 5) self-care; 6) domestic life; 7) interpersonal interactions and relationships; 8) major life areas; and 9) community, social and civic life. The considered domains are as follows:

1. Employment;
2. Built Environment;
3. Transportation; 
4. Information and Communication Technologies;
5. Communication Other than Information Communication Technologies;
6. Procurement of Goods, Services and Facilities;
7. Design and Delivery of Programs and Services; 
8. Education;
9. Health Services;
10. First Responder Services;
11. Community Life.

      We provided a short definition for each indicator and in some cases, examples in footnotes. Identified indicators were labeled as process (P), or result (R) indicators. Result indictors have three categories of output, outcome, and impact. In terms of date sources for indicators, we identified surveys and methodology into separate columns. The last column identifies Canadian data availability for the indicators. Each indicator was categorized into one of three groups: 1) structural- discriminatory social structures, policies, and legislation; 2) Intrapersonal- ignorance/misinformation (problem in knowledge), prejudice (problem in attitude), and discriminatory behavior; and 3) Intrapersonal- self-stigma, and internalization of stigma. References for each indicator are shown in the parenthesis.


Table 10. Indicators to proxy culture change towards accessibility and inclusion of persons with disabilities
a) Employment (ACA)
Indicators
Definition
Type
Existing Surveys
Methodology
CAN Data Availability
Attitudes of employers towards PWDs (Interpersonal)
Employers� attitudes towards
o Work performance of PWDs 
o Social acceptability of PWDs
o Cost of workplace accommodations (Woodley et al., 2012; Randle et al. 2017)4
R/O
o Employer Attitudes Towards the Work Inclusion of PWDs (Nota et al., 2014)
o Pilot Public Opinion Research Survey on Accessibility (Government of Canada, 2019)
o New Zealand Survey on Employer Attitudes Towards Employing PWDs (Woodley et al., 2012)
o British Social Attitudes Survey (Office for Disabilities Issues, 2009)
o Australian National Survey of Employers' Recruitment Experiences (DEEWR, 2010; Australian Government, 2011)
o Survey of Employer Perspectives on the Employment of People with Disabilities (Domzal et al., 2008)
o Questionnaire Developed by Authors
o Questionnaire Developed by Authors
o Questionnaire Developed by Authors
o Attitudes to Disability Scale (ADS)
o Questionnaire developed by Department of Education, Employment and Workplace Relations
o Questionnaire developed by US Department of Labor, Office of Disability Employment Policy
o Pilot Public Opinion Research Survey on Accessibility (Government of Canada, 2019)
Attitudes of co-workers and customers towards PWD
(Interpersonal)
Perception on PWDs
o Productivity (Office for Disabilities Issues, 2009; Bollier et al., 2021)
o Perceived exploitability5 (Bollier et al., 2021)
o Future prospects6 (Bollier et al., 2021)
o Burden stereotypes7 (Bollier et al., 2021)
R/O
o Survey of Employer Perspectives on the Employment of PWDs (Domzal et al., 2008)
o A National Survey of Consumer Attitudes Towards Companies That Hire PWDs (Siperstein et al., 2006)
o Findings from a National Survey of Community Attitudes Toward People with Disability in Australia (Bollier et al., 2021)
o Questionnaire developed by US Department of Labor, Office of Disability Employment Policy
o Questionnaire Developed by Authors
Attitudes to Disability Scale (ADS)
o Data is not available
Self-stigma among PWD
(Intrapersonal)
o Self-stigma and fears of employment among PWDs
R/O
o Self-stigma and Fears of Employment Among Adults with Psychiatric Disabilities Using Employment Fears Scale and an Employment Values Scale (Hielscher et al., 2017)
o Employment Fears Scale (EFS)
o Stigma Scale (SS)
o Consumer Experiences of Stigma Questionnaire (CESQ)
o Employment Value Scale (EVS)
o Data is not available
Reported discrimination towards PWD
o Complaints received by the ground of discrimination towards PWDs 
R/I
o Canadian Human Rights Commission (CHRC, 2015)
o Admin data
o Survey
o Federal, provincial, territorial human right commission
o Labour Force Discrimination (LFD), (CSD, 2017)





b) Built Environment (ACA) 
Indicators
Definition
Type
Existing Surveys
Methodology
CAN Data Availability
Inclusivity culture in design
(strutural)
o Presence of disability perspective in inclusive urban planning and strategies planning
o Presence of trained and inclusion respecting architects and designers in built environment projects (Rebernik et al., 2020; Terashima et al., 2021)

R/I
o Disability Inclusion Evaluation Tool (DIETool) used in two European Cities (Rebernik et al., 2020)
o Questionnaire Developed by Authors
o Data is not available

c) Transportation (ACA)
Indicators
Definition
Type
Existing Surveys
Methodology
CAN Data Availability
Attitudes of transport staff (Interpersonal)
o Attitudes of transport staff towards passengers with disabilities
o Awareness of transport staff about needs of passengers with disability (DPTAC, 2002)
R/O
o Attitudes of Disabled People to Public Transport (DPTAC, 2002)
Questionnaire Developed by Authors
o Data is not available
Fear as a passenger with disabilities (Intrapersonal)
o Proportion of passengers with disabilities afraid:
* being stigmatized
* that services do not meet their needs
* discriminatory behavior (Field, 2007)
o Attitudes of other passengers (Rosenbloom et al., 2007; Bezyak et al., 2017)
R/O
o Travel Fair (SCOPE, 2019)
Questionnaire Developed by Authors
o Data is not available
Reported discrimination towards PWD
o Reported discrimination complaint toward PWDs in public transit
R/I
o NA 
o Admin data
o Questionnaire
o Accessible
Transportation
Complaints (CTA, 2020)

d) Information and Communication Technologies (ICT) (ACA)
Indicators
Definition
Type
Existing Surveys
Methodology
CAN Data Availability
Attitudes of ICT staff (Interpersonal)
o Attitudes of staff towards PWDs
o Awareness of staff about needs of PWD
R/O
o Diversity, Equity, and Inclusion in the IT Workforce (Gierdowski, 2019)
o Questionnaire Developed by Authors
o Data is not available
Inclusive content (strutural)
o Inclusive, non-discriminating contents
o Proportion of anti-stigmatising content
o Awareness-raising campaigns, projects about inclusion and disability
P
o Newspaper coverage of mental illness in England 2008-2011 (Thornicroft et al., 2013)
o Toward a more balanced representation of disability a content analysis (Goethals, 2018)
o Representations of disability in the Canadian news media: a decade of change (Devotta et al., 2013)
o The Place of News Media Analysis within Canadian Disability Studies (Haller, 2012)
o Content Analysis
o Data is not available
Self-stigma (Intrapersonal)
o PWDs perceived stigma of assistive devices (Santos et al., 2020)
o PWDs awareness of what ICTs can do to facilitate their inclusion (ITU, 2013)
R/O
o Aesthetics and the Perceived Stigma of Assistive Technology for Visual Impairment (Santos et al., 2020)
o Questionnaire Developed by Authors
o Data is not available
Reported discrimination towards PWD
o Number of reported complaints on the basis of discrimination toward PWDs 
R/I
o NA 
o NA
o Federal, provincial, territorial human right commission

e) Communication, other than ICT (ACA) 
Indicators
Definition
Type
Existing Surveys
Methodology
CAN Data Availability
Attitudes of Communication staff, other than ICT (Interpersonal)
o Attitudes of staff towards PWDs
o Awareness of staff about needs of PWDs
R/O
o NA
o NA
o Data is not available
Inclusive content (strutural)
o Inclusive, non-discriminating contents
o Proportion of anti-stigmatising content
o Awareness-raising campaigns, projects, articles about inclusion and disability 
P
o Newspaper coverage of mental illness in England 2008-2011 (Thornicroft et al., 2013)
o Toward a more balanced representation of disability a content analysis (Goethals, 2018)
o Representations of disability in the Canadian news media: a decade of change (Devotta et al., 2013)
o The Place of News Media Analysis within Canadian Disability Studies (Haller, 2012)
o Content Analysis 
o Data is not available
Reported discrimination towards PWD
o Number of reported complaints on the basis of discrimination toward PWDs 
R/I
o NA 
o Admin data
o Questionnaire
o Canadian Radio-television and Telecommunications Commission (CRTC, 2020)

f) Procurement of Goods, Services and Facilities (ACA)
Indicators
Definition
Type
Existing Surveys
Methodology
CAN Data Availability
Attitudes of staff (Interpersonal)
o Attitudes of staff towards PWDs
o Awareness of staff about needs of PWDs 
o Available services of assistance for PWDs (Rebernik et al., 2020)
R/O
o Disability Inclusion Evaluation Tool (DIETool) used in two European Cities (Rebernik et al., 2020)
Questionnaire Developed by Authors
o Data is not available
g) Design and Delivery of Programs and Services (ACA)
Indicators
Definition
Type
Existing Surveys
Methodology
CAN Data Availability
Attitudes of staff (Interpersonal)
o Attitudes of staff towards PWDs
o Awareness of staff about needs of PWDS 
o Available services of assistance for PWDs (Rebernik et al., 2020)
R/O
o Disability Inclusion Evaluation Tool (DIETool) used in two European Cities (Rebernik et al., 2020)
Questionnaire Developed by Authors
o Data is not available
Presence and incorporation of PWDs 
(strutural)
o Participation of PWDs in
* strategic planning and design 
* legislation design
* enforcement of disability legislation 
o Presence and incorporation of disability inclusion principles in overall legislation (Rebernik et al., 2020)
P
o Disability Inclusion Evaluation Tool (DIETool) used in two European Cities (Rebernik et al., 2020)
Questionnaire Developed by Authors 
o Data is not available











h) Healthcare Sector
Indicators
Definition
Type
Existing Surveys
Methodology
CAN Data Availability
Attitudes of healthcare professionals towards PWDs (Interpersonal)
o Healthcare professionals� attitudes toward PWD
o Healthcare professionals� awareness about needs of PWDs
R/O
o Stigma and Discrimination Related to Mental Health and Substance Use Issues In Primary Health Care In Toronto, Canada (Murney et al., 2020)
o Reducing Mental Illness Stigma In Healthcare Settings (Kohrt et al., 2020)
o Targeting Stigma of Mental Illness Among Primary Care Providers (Mittal Et A., 2020)
o Physiotherapists' Attitudes Towards Working with People Living with Dementia (Hunter et al., 2020)
o Changing Healthcare Professionals' Attitudes Towards Voice Hearers (Reddyhough et al., 2021)
o Interviews
o  Focus groups
o  Interviews
o Attribution Questionnaire (AQ-9)
o Social Distance Scale (SDS)
o Questionnaire Developed by Authors 
o The Knowledge in Dementia Scale
o Social Distance Scale (SDS)
o Attribution Questionnaire (AQ-20)
o level of Contact Report (LCR)
o Public Health Agency of Canada (2019a; 2019b)
o Stigma and Discrimination related to mental health and substance use issues in primary health care in Toronto, Canada (Murney et al., 2020)
Reported discrimination towards PWD
o Number of discrimination among PWDs in healthcare setting 
R/I
o NA
o Admin data
o Questionnaire
o Public Health Agency of Canada (2019a; 2019b)
o Federal, provincial, territorial human right commission









i) Educational Sector
Indicators
Definition
Type
Existing Surveys
Methodology
CAN Data Availability
Attitudes of students towards student with disabilities (Interpersonal)
o Student's attitude towards school and university students with disabilities
o Student's attitude towards help-seeking 
o Student's knowledge of mental health (Waqas et al., 2020)
R/O
o Nursing Students' Attitudes Toward Seeking Professional Psychological Help Before and After Attending a Mental Health Course (Abuhammad et al., 2020)
o A Comparative Analysis of two Alternative Programs to Counter Stigma and Discrimination Associated with Mental Illness Delivered to Medical Students (Gordon et al., 2021)
o Controlled Study of the Impact of a Virtual Program to Reduce Stigma Among University Students Toward People with Mental Disorders (Rodr�guez-Rivas et al., 2021)
o The Potential Impact of An Anti-Stigma Intervention on Mental Health Help-Seeking Attitudes Among University Students (Shahwan et al., 2020)
o Attitudes Toward Seeking Professional Psychological Help (ATSPPH) scale
o Recovery Attitudes Questionnaire (RAQ)
o Opening Minds Stigma Scale for Health Care Providers (OMS-HC-20)
o Questionnaire on Student Attitudes Toward Schizophrenia (QSAS)
o Attribution Questionnaire (AQ)
o Inventory of Attitudes toward Seeking Mental
o Health Services (IASMHS) scale
o Data is not available
Attitudes of teachers towards student with disabilities (Interpersonal)
o Teachers attitude towards student with disabilities among the school and university students with disabilities
o Teachers' awareness about needs of school and university students with disabilities
R/O
o Field testing an Australian model of practice for teaching young school-age students on the autism spectrum (Beamish et al., 2021)
o Attitudes toward disability: Teacher education for inclusive environments in an Australian university (Tait et al., 2000)
o Advancing research to eliminate mental illness stigma: An interventional study to improve community attitudes towards depression among university students in Singapore (Goh et al., 2021)
o A Review of School-Based Interventions to Reduce Stigma towards Schizophrenia (Flanigan et al., 2020)
o  Interviews
o Disabled Persons Scale (IDP)
o Community Attitudes towards Mental Illness (CAMI) scale
o Questionnaire on Student Attitudes Toward Schizophrenia (QSAS)
o Public Stigma Scale
o Social Distance Scale (SDS)
o Knowledge Test �32-items on factual knowledge of schizophrenia
o Alberta Pilot Site Questionnaire Toolkit
o Data is not available









j) First Responders (police officer, firefighter, paramedic, etc.)
Indicators
Definition
Type
Existing Surveys
Methodology
CAN Data Availability
Attitudes of first responders toward PWDs (Interpersonal)
o First responders� attitudes toward PWDs
o First responders� awareness about needs of PWDs
R/O
o Police Officers' Attitudes Toward and Decisions about Persons with Mental Illness (Watson et al., 2004)
o It's all window dressing: Canadian police officers' perceptions of mental health stigma in their workplace (Bikos et al., 2020)
o Stigma, Pluralistic Ignorance, and Attitudes Toward Seeking Mental Health Services Among Police Officers (Karaffa et al., 2016)
o Recognition of, and attitudes towards, people with depression and psychosis with/without alcohol and other drug problems: results from a national survey of Australian paramedics (McCann et al., 2018)
o  Attribution Questionnaire (AQ)
o Interviews
o Questionnaire Developed by Authors
o Questionnaire Developed by Authors
o Public perceptions of the police in Canada�s provinces (Ibrahim D, 2019)
o It's all window dressing: Canadian police officers' perceptions of mental health stigma in their workplace (Bikos et al., 2020)



k) Community Life
Indicators
Definition
Type
Existing Surveys
Methodology
CAN Data Availability
Attitudes of community on PWDs autonomy (Interpersonal)
Community awareness (Interpersonal)
Community engagement (Interpersonal)
PWDs Belonging, and participation (Interpersonal)
o PWDs autonomy over their lives (Thomson et al., 2010)
o PWDs choice of housemates (Bollier et al., 2021)
o PWDs right to raise children (Bollier et al., 2021)
o Public awareness about PWDs needs, issues, and experiences in community (Thomson et al., 2010)
o Relationships between communities and PWDs (Thomson et al., 2010)
o Acceptance levels in neighborhoods (Rebernik et al., 2020)
o Getting involved in society (Bollier et al., 2021)
o Participation and belonging of PWDs in community (Thomson et al., 2010)
R/O

o Public attitudes towards people with mental illness in England and Scotland (Mehta et al., 2009)
o National survey of community attitudes toward PWDs in Australia (Bollier et al., 2021)
o Public perceptions of disabled people. Evidence from the British Social Attitudes Survey (BSAS) (Office for Disabilities Issues, 2009)
o Stigma doesn�t discriminate: physical and mental health and stigma in Canadian military personnel and Canadian civilians (Frank et al., 2018)
o Attitudes toward people with intellectual disabilities: An Australian perspective (Yazbeck et al. 2004)
o Public attitudes to disability in the Republic of Ireland (National Disability Authority, 2002)
o Causal beliefs and attitudes to people with schizophrenia: trend analysis based on data surveys in Germany (Angermeyer et al. 2005)
o Attitudes toward individuals with disabilities: Results of a recent survey and implications of those results (Goreczny et al., 2011)
o Stigmatizing attitudes towards people with mental disorders: findings from an Australian National Survey of Mental Health Literacy and Stigma (Reavley et al., 2012)
o Community Attitudes Toward the Mentally Ill Scale (CAMI)
o Attitudes to Disability Scale (ADS)
o Attitudes to Disability Scale (ADS)
o Questionnaire Developed by Authors
o Social Distance Scale (SDS)
o Questionnaire Developed by Authors
o Interviews
o Questionnaire Developed by Authors
o Interviews
o Victimization- Cycle 23 (GSS, 2009)
o Mental Health and Well-being (CCHS-MH Civilian) and its corresponding Canadian Forces Supplement (CCHS-MH) (Frank et al., 2018)
o Public Health Agency of Canada (2019a; 2019b)
o Mental health commission of Canada (MHCC, 2015)
Community safety and support (Interpersonal)
o Physical and psychological safety and wellness among PWDs 
* Safe public spaces and public transportation 
* SOS system for PWD
* Safe houses for women and children with disabilities 
o Domestic and non-domestic violence towards PWDs8 (Thomson et al., 2010; UK Government, 2021)
R/O
o Violent victimization of women with disabilities (GSS, 2014) (Cotter, 2018)
o Personal Safety Survey (PSS) in Australia (ABS, 2016)
o Violence against People with Disability in England and Wales: Findings from a National Cross-Sectional Survey, British Crime Survey (Khalifeh et al., 2013)
o US National Survey on Abuse of PWDs (Baladerian et al., 2013)
o Questionnaire Developed by Authors
o Questionnaire Developed by Authors
o Questionnaire Developed by Authors
o Questionnaire Developed by Authors
o Violent victimization of women with disabilities (GSS, 2014) (Cotter, 2018)
o Criminal victimization in Canada (Perreault et al. 2015)
o Trends in self?reported spousal violence in Canada (Burczycka et al., 2016)
o Family violence in Canada: A statistical profile (Burczycka et al., 2018)
o Self?reported sexual assault in Canada (Conroy et al., 2017)
Discrimination towards PWD
o Number of discrimination among PWDs 
o Inappropriate behavior toward PWDs (Bollier et al., 2021)9
o Comfort with expressing prejudice towards PWDs (Office for Disabilities Issues, 2009)
R/I
o British Social Attitudes Survey (BSAS) (Office for Disabilities Issues, 2009)
o Canadian Community Health Survey, Rapid response module (CCHS, 2013)
o A national survey on violence and discrimination among PWD, Danish national survey (Dammeyer et al., 2018)
o UK Disability Survey (UK Government, 2021)
o Discrimination and avoidance due to disability in Australia: evidence from a National Cross-Sectional Survey (Temple et al., 2018)
o Questionnaire Developed by Authors
o Everyday Discrimination Scale
o Questionnaire Developed by Authors
o Questionnaire Developed by Authors
o Survey of Disability, Ageing and Carers in Australia
o Canadian Community Health Survey, rapid response module- everyday discrimination scale (CCHS, 2013) 
o Federal, provincial, territorial human right commission
Note. P: process indicator, R/O: result outcome/output indicator, R/I: result impact indicator. ACA: Accessible Canada Act. Stigma level: structural- discriminatory social structure, policies and legislation, poor and inadequate quality of mental health services. Intrapersonal- ignorance/ misinformation (problem in knowledge), prejudice (problem in attitude), discriminatory behavior. Intrapersonal- self-stigma, internalization of stigma.


Data Gap Analysis

Table 11 provides a list of data elements for measuring culture change for which we struggled to identify a source. The data elements might be considered for inclusion in future data gathering efforts by federal and provincial authorities. Although we were aware that for certain indicators there may not be good quality data available, at times we were surprised by the absence of some key indicator data, e.g., a survey of public attitude toward PWDs. 

Table 11. Canadian Data gap analysis
Priority Areas
Available Database (Surveys)
Data Gaps
Employment
o Pilot Public Opinion Research Survey on Accessibility (Government of Canada, 2019)
o Labour Force Discrimination (LFD) (CSD, 2017)
o Federal, provincial, territorial human right commission
o Employers� attitudes towards work performance of PWDs 
o Employers� attitudes towards social acceptability of PWDs
o Employers� attitudes about cost of workplace accommodations
Built Environment
o Data is not available
o Presence of disability perspective in inclusive urban planning and strategies planning
o Presence of trained and inclusion respecting architects and designers in built environment projects
Transportation
o Accessible Transportation Complaints  (CTA, 2020)
o Attitudes of transport staff towards passengers with disabilities
o Awareness of transport staff about needs of passengers with disability 
o Proportion of passengers with disabilities afraid:
- being stigmatized
- that services do not meet their needs
- discriminatory behavior
o Attitudes of other passengers
Information and Communication Technologies (ICT)
o Discrimination record in Federal, provincial, territorial human right commission
o Attitudes of staff towards PWDs
o Awareness of staff about needs of PWD
o Inclusive, non-discriminating contents
o Proportion of anti-stigmatising content
o Awareness-raising campaigns, projects about inclusion and disability
Communication, other than ICT
o Discrimination record on Canadian Radio-television and Telecommunications Commission (CRTC, 2020)
o Discrimination records in Federal, provincial, territorial human right commission
o Attitudes of staff towards PWDs
o Awareness of staff about needs of PWDs
o Inclusive, non-discriminating contents
o Proportion of anti-stigmatising content
o Awareness-raising campaigns, projects, articles about inclusion and disability 
Procurement of Goods, Services and Facilities
o Data is not available
o Attitudes of staff towards PWDs
o Awareness of staff about needs of PWDs 
o Available services of assistance for PWDs
Design and Delivery of Programs and Services
o Data is not available
o Attitudes of staff towards PWDs
o Awareness of staff about needs of PWDs 
o Available services of assistance for PWDs
o Participation of PWDs in
* strategic planning and design 
* legislation design
* enforcement of disability legislation 
o Presence and incorporation of disability inclusion principles in overall legislation
Health Services
o Public Health Agency of Canada (2019a; 2019b)
o Federal, provincial, territorial human right commission
o No particular data gaps identified
Educational Sector
o Data is not available
o Student's attitude towards school and university students with disabilities
o Student's attitude towards help-seeking 
o Student's knowledge of mental health
o Teachers attitude towards student with disabilities among the school and university students with disabilities
o Teachers' awareness about needs of school and university students with disabilities
First Responders
o Public perceptions of the police in Canada�s provinces (Ibrahim D, 2019)
o It's all window dressing: Canadian police officers' perceptions of mental health stigma in their workplace (Bikos et al., 2020)
o First responders� attitudes toward PWDs
o First responders� awareness about needs of PWDs
Community Life
o Victimization- Cycle 23 (GSS, 2009)
o Mental Health and Well-being (CCHS-MH Civilian) and its corresponding Canadian Forces Supplement (CCHS-MH) (Frank et al., 2018)
o Public Health Agency of Canada (2019a; 2019b)
o Mental health commission of Canada (MHCC, 2015)
o Violent victimization of women with disabilities (GSS, 2014) (Cotter, 2018)
o Criminal victimization in Canada (Perreault et al. 2015)
o Trends in self?reported spousal violence in Canada (Burczycka et al., 2016)
o Family violence in Canada: A statistical profile (Burczycka et al., 2018)
o Self?reported sexual assault in Canada (Conroy et al., 2017)
o Everyday discrimination scale, rapid response module (CCHS, 2013)
o Federal, provincial, territorial human right commission
o Public attitudes towards PWDs
o Public awareness about PWDs needs, issues, and experiences in community
o Relationships between communities and PWDs 
o Acceptance levels in neighborhoods
o Participation and belonging of PWDs in community
o PWDs autonomy over their lives



CONCLUSIONS AND RECOMMENDATIONS

Frameworks and indicators for culture change in relation to accessibility and inclusion of PWDs can be extremely useful for governments, community organizations, service providers, businesses, academics, and other stakeholders, as they can be used to identify cultural barriers, move toward solution, monitor and evaluate the impact of efforts, such as legislation, policies, programs, and practices to improve accessibility and inclusiveness in various domains of society, as well as help identify priorities and opportunities for action. Drawing on national and international sources, we identified frameworks and indicators that can be used to develop, monitor, and evaluate culture change initiatives for advancing accessibility and inclusion of PWDs in a different context. The databases listed in this report yielded both peer-reviewed and non-peer reviewed literature. The initiatives identified can also serve to inform the development of initiatives in the Canadian context at the local, regional and national level to promote culture change in relation to accessibility and inclusion in different domains and populations. We also identified data gaps that might be filled in future data gathering efforts by stakeholders to help advance the monitoring and evaluation of initiatives.

Following are some recommendations for future research:

Design and implement a national-level survey to estimate attitudes towards PWDs
Lack of baseline and follow-up data can be addressed by designing and implementing a large-scale, representative population survey on a periodic basis similar to that of other countries such as the public perceptions of disabled people: British Social Attitudes Survey (BSAS) in UK, Survey of Community Attitudes towards People with Disability in Australia, and Survey on Attitudes towards People with Disabilities in the Republic of Ireland. It is important that such a survey is designed and implemented in a way that provides a good understanding of attitude towards PWDs in different systems, such as employment, built environment, transportation, information and communication technologies, communication other than information communication technologies, procurement of goods, services and facilities, design and delivery of programs and services, education, health services, first responder services, and community life. Public attitudes toward PWDs survey can be developed under one of the existing surveys. For instance, Public Opinion Research (POR) studies could develop a module focused on culture change relevant to PWDs.

Develop a national-level result framework for monitoring and evaluation of accessibility and inclusion
A comprehensive accessibility and inclusion result framework should contain both objective and subjective indicators. While objective indicators, such as employment, labour market attainment, earning/income, etc. provide valuable information on the current socio-economic status of PWDs, subjective indicators, which are based on individual�s feelings and their expected life standards, could explain the underlying drivers of such behaviors to varying degrees. Researchers in the area of social indicator increasingly emphasize the value of obtaining both subjective and objective indicators (Macku et al., 2020). Interestingly, we are seeing many results frameworks in the area of culture change initiatives benefiting from this integration. For instance, in a recently developed result framework for Australia�s disability strategy, both subjective and objective indicators have been integrated. This can also be observed in culture change initiatives in other areas such as violence against women and girls programming, and HIV prevention programs for most-at-risk populations. The extensive list of subjective indicators identified for different priorities of Accessible Canada Act (ACA) and beyond, can be used as a complementary piece to objective indicators to monitor and evaluate culture change initiatives in relation to accessibility and inclusion of PWDs in Canada.

Collect and integrate discrimination records
Discrimination complaints, considered lagging indicators of culture in relation to accessibility and inclusion, is limited in current data collection systems. Our study revealed that such data can be found in different places such as federal, provincial, territorial human rights commissions, and administrative data on accessible transportation complaints, as well as in different surveys, including discrimination among PWDs by Canadian Community Health Survey (CCHS, 2013), and the General Social Survey on Victimization Survey (GSS, 2009). Collection and integration of discrimination data can be an invaluable resource for monitoring and evaluation of culture change in relation to accessibility and inclusion of PWDs.



REFERENCES

ABS (Australian Bureau of Statistics) (2016). Personal Safety, Australia Statistics for family, domestic, sexual violence, physical assault, partner emotional abuse, child abuse, sexual harassment, stalking and safety. Link: https://www.abs.gov.au/statistics/people/crime-and-justice/personal-safety-australia/latest-release#data-download
Abuhammad S, Hamaideh S. Nursing Students' Attitudes Toward Seeking Professional Psychological Help Before and After Attending a Mental Health Course. Nursing Education Perspectives. 2021 Mar 5.
Adams RJ, Smart P, Huff AS. Shades of grey: guidelines for working with the grey literature in systematic reviews for management and organizational studies. International Journal of Management Reviews. 2017 Oct;19(4):432-54.
Akrami N, Ekehammar B, Claesson M, Sonnander K. Classical and modern prejudice: Attitudes toward people with intellectual disabilities. Research in developmental disabilities. 2006 Nov 1;27(6):605-17.
Algahtani H, Shirah B, Alhazmi A, Alshareef A, Bajunaid M, Samman A. Perception and attitude of the general population towards Alzheimer�s disease in Jeddah, Saudi Arabia. Acta neurologica Belgica. 2020 Apr;120(2):313-20.
Alison (2003). Active Minds. Link: https://www.activeminds.org/about-us/our-story/
Angermeyer MC, Matschinger H. Causal beliefs and attitudes to people with schizophrenia: trend analysis based on data from two population surveys in Germany. The British Journal of Psychiatry. 2005 Apr;186(4):331-4.
Antonak, R. F. (1994). Development and psychometric analysis of an indirect measure of attitudes toward persons with mental retardation using the error-choice method. Mental Retardation, 32, 347�355.
Arksey H, O�Malley L. Scoping studies: Towards a Methodological Framework. Int J Soc Res Methodol. 2005;8(1):19-32.
Australia Department of Social Services (2021). Australia�s Disability Strategy 2021-2031. Outcomes Framework. Link: https://apo.org.au/sites/default/files/resource-files/2021-12/apo-nid315397_2.pdf
Australian Government (2011). Employer perspectives on recruiting persons with disabilities and the role of Disability Employment. Link: https://disabilityemployment.org.au/static/items/EmployerServicing2011.pdf
Australian Rotary (2000). Health. Evaluation of the national Mental Health Awareness Campaign
Baladerian, N, Coleman T, Steam, J, Abuse of People with Disabilities, Victims and their Families Speak out, A Report on the 2012 National Survey on Abuse of People with Disabilities, Spectrum Institute, Los Angeles, CA, 2013. Link: https://ncvc.dspacedirect.org/handle/20.500.11990/998
BC Association of Broadcasters. Award-winning campaigns was presented by the Canadian Mental Health Association�s BC Division. Link: https://deareverybody.hollandbloorview.ca/ 
Beamish W, Taylor A, Macdonald L, Hay S, Tucker M, Paynter J. Field testing an Australian model of practice for teaching young school-age students on the autism spectrum. Research in Developmental Disabilities. 2021 Jun 1;113:103942.
Bell Canada's (2015). Let's Talk. Link: http://www.letstalk.bell.ca
Bell S. Suspicious minds: A mixed methods study of police officer and police staff attitudes to mental ill health in England and Wales. Liverpool John Moores University (United Kingdom); 2020.
Beyondblue (2000). beyondblue: the national depression initiative. Link: https://www.beyondblue.org.au/
Bezyak JL, Sabella SA, Gattis RH. Public transportation: an investigation of barriers for people with disabilities. Journal of Disability Policy Studies. 2017 Jun;28(1):52-60.
Bikos LJ. It's all window dressing: Canadian police officers' perceptions of mental health stigma in their workplace. Policing: An International Journal. 2020 Dec 4. Link: https://www.cipsrt-icrtsp.ca/en/publication/its-all-window-dressing-canadian-police-officers-perceptions-of-mental-health-stigma-in-their-workplace
Bollier AM, Sutherland G, Krnjacki L, Kasidis V, Katsikis G, Ozge J & Kavanagh AM. (2021). Attitudes Matter: Findings from a national survey of community attitudes toward people with disability in Australia. Centre of Research Excellence in Disability and Health, The University of Melbourne. Link: https://credh.org.au/projects/community-attitudes-towards-people-with-disability
Borsatto J, Buchanan L, Pineault L. Aphasia friendly Canada: The aphasia friendly business campaign. Aphasiology. 2021 Jan 7:1-20.
Bott S, Guezmes A, Claramunt MC. Improving the health sector response to gender-based violence: a resource manual for health care professionals in developing countries. International Planned Parenthood Federation (IPPF); 2004. Link: http://www.dmeforpeace.org/peacexchange/wp-content/uploads/2016/11/Pages-from-GBV_cdbookletANDmanual_FA_FINAL.pdf
Buckwitz V, Porter PA, Bommes JN, Schomerus G, Hinshaw SP. Continuum beliefs and the stigma of depression: An online investigation. Stigma and Health. 2021 Feb;6(1):113.
Bunch G, Valeo A. Student attitudes toward peers with disabilities in inclusive and special education schools. Disability & Society. 2004 Jan 1;19(1):61-76.
Burczycka, M. 2016. Trends in self?reported spousal violence in Canada, 2014. Family violence in Canada: A statistical profile, 2014. Juristat. Statistics Canada Catalogue no. 85?002?X. Link: https://www150.statcan.gc.ca/n1/pub/85-002-x/2016001/article/14303/01-eng.htm
Burczycka, M. 2018. Stalking in Canada, 2014. Family violence in Canada: A statistical profile, 2017. Juristat. Statistics Canada Catalogue no. 85?002?X. Link: https://www150.statcan.gc.ca/n1/pub/85-002-x/2018001/article/54978-eng.htm
Burns S, Crawford G, Hallett J, Hunt K, Chih HJ, Tilley PJ. What�s wrong with John? a randomised controlled trial of Mental Health First Aid (MHFA) training with nursing students. BMC Psychiatry. 2017;17(1):111.
Canadian Psychiatric Research Foundation (CPRF). �Imagine...� campaign (2004). Link: https://www.heretohelp.bc.ca/visions-stigma-and-discrimination-vol2/canadian-anti-stigma-campaigns
Casals MC. (2009) Proposal of new indicators to measure the effects of gender violence. Link: http://www.surt.org/docs/publicats/2009/gvei_proposta_indicadors_en.pdf
CCDS (Canadian Centre for Disability Studies) (2001). Building Bridges Between the Corporate Sector and the Disability Community: 2000�2001. Manitoba: Canadian Centre on Disability Studies.
CCHS (Canadian Community Health Survey) (2013). Annual Component. Questionnaire: Everyday Discrimination Scale / Component: Rapid Response � EDS. Link: https://www23.statcan.gc.ca/imdb/p3Instr.pl?Function=getInstrumentList&Item_Id=149959&UL=1V
Centre for Addiction and Mental Health (CAMH) (2020). Addressing Stigma. Mental Health 101: Understanding Stigma. Link: https://www.camh.ca/en/driving-change/addressing-stigma
Chadd EH, Pangilinan PH. Disability attitudes in health care: a new scale instrument. American journal of physical medicine & rehabilitation. 2011 Jan 1;90(1):47-54.
CHRC (Canadian Human Rights Commission) (2015). Persons with disabilities continue to be marginalized in Canadian society: CHRC. Link: https://www.canada.ca/en/human-rights-commission/news/2015/12/persons-with-disabilities-continue-to-be-marginalized-in-canadian-society-chrc.html 
Close G et al (2010). Bring Change to Mind (BC2M). Let�s talk about mental health. Link http://bringchange2mind.org
Conroy, S. and Cotter, A. 2017. Self?reported sexual assault in Canada, 2014. Juristat. Statistics Canada Catalogue no. 85?002?X. Link: https://www150.statcan.gc.ca/n1/pub/85-002-x/2017001/article/14842-eng.htm
Cotter A. (2018). Violent victimization of women with disabilities, 2014. General Social Survey (GSS). Link: https://www150.statcan.gc.ca/n1/pub/85-002-x/2018001/article/54910-eng.htm
Cross HA, Heijnders M, Dalal A, Sermrittirong S, Mak S. Interventions for stigma reduction�part 2: practical applications. Disability, CBR & Inclusive Development. 2011;22(3):71-80.
CRTC (Canadian Radio-television and Telecommunications Commission) (2020). How to make a broadcasting complaint. Link: https://crtc.gc.ca/eng/info_sht/g8.htm#filing
CSD (Canadian Survey on Disability) (2017). Canadian Survey on Disability-Definitions, data sources and methods Questionnaires. Link: https://www23.statcan.gc.ca/imdb/p3Instr.pl?Function=assembleInstr&lang=en&Item_Id=348023#qb353525 
CSNM (2020). 2. Project Monitoring & Evaluation. Module 5. M&E in Project Cycle Management. Link: https://csnm.kku.ac.th/learning/course/module/lesson/116-me-project-cycle-management
CTA (Canadian Transportation Agency) (2020). Accessibility complaints about transportation services. Link: https://otc-cta.gc.ca/eng/accessibility-complaints-about-transportation-services
Dammeyer J, Chapman M. A national survey on violence and discrimination among people with disabilities. BMC public health. 2018 Dec;18(1):1-9.
de la Higuera-Romero J, Candelas-Mu�oz A, Jim�nez-Gonz�lez A, Casta�eda-Jim�nez C, Fuica-Pereg P, Zurita-Carrasco M, Mart�nez-Fernandez-Repeto E, Sen�n-Calder�n C. Spanish adaptation and validation of the Peer Mental Health Stigmatization Scale (PMHSS-24). Revista de Psiquiatria y Salud Mental. 2020 Jul 21.
DEEWR (Australian Government Department of Education, Employment and Workplace Relations) (2008). Employer Attitudes to Employing People with Mental Illness. Canberra: Commonwealth of Australia. https://docs.employment.gov.au/system/files/doc/other/employer_attitudes_to_employing_people_with_mental_illness.pdf 
Delevic et al., (2011). Guide To the Logical Framework Approach. A key tool for project cycle management. Second edition. European Integration Office. Link: http://www.evropa.gov.rs/Evropa/ShowDocument.aspx?Type=Home&Id=525#:~:text=The%20Logical%20Framework%20Approach%20(LFA,a%20project%20or%20programme%20idea 
Department of National Defense in Canada (2016). Road to Mental Readiness (R2MR). Link: https://theworkingmind.ca/sites/default/files/workplace_antistigma_programs-part1.pdf
Devotta K, Wilton R, Yiannakoulias N. Representations of disability in the Canadian news media: A decade of change?. Disability and Rehabilitation. 2013 Oct 1;35(22):1859-68.
Disability Awareness Starts Here (DASH). DASH Project � voice for all people with disabilities Link: https://www.dashproject.org
Dobson K, Stuart H. The Stigma of Mental Illness: Models and Methods of Stigma Reduction. Oxford University Press; 2021 Dec 7.
Domzal C, Houtenville A, Sharma R. Survey of employer perspectives on the employment of people with disabilities: Technical report. Office of Disability Employment Policy, Department of Labor; 2008. Link: https://www.dol.gov/sites/dolgov/files/odep/research/surveyemployerperspectivesemploymentpeopledisabilities.pdf
DPTAC (Disabled Persons Transport Advisory Committee) (2002). Attitudes of disabled people to public transport. Link: https://trimis.ec.europa.eu/sites/default/files/project/documents/20060811_110503_45123_UG395_Final_Report.pdf
Driscoll D, Mathew L, Engelhardt D, Moran?Peters J, Eckardt S. Evaluating mental illness�substance abuse stigmatic perceptions through education: A library�nursing initiative. Public Health Nursing. 2021 Jun 10.
Dupuis S, McAiney CA, Fortune D, Ploeg J, Witt LD. Theoretical foundations guiding culture change: The work of the Partnerships in Dementia Care Alliance. Dementia. 2016 Jan;15(1):85-105. Link: https://pubmed.ncbi.nlm.nih.gov/24419355/ 
Eldridge-Smith ED, Loew M, Stepleman LM. The adaptation and validation of a stigma measure for individuals with multiple sclerosis. Disability and rehabilitation. 2021 Jan 16;43(2):262-9.
Engel CS, Sheppard E. Can cartoons which depict autistic characters improve attitudes towards autistic peers?. Journal of autism and developmental disorders. 2020 Mar;50(3):1007-17.
Epps F, Foster K, Alexander K, Brewster G, Chester M, Thornton J, Aycock D. Perceptions and Attitudes Toward Dementia in Predominantly African American Congregants. Journal of Applied Gerontology. 2021 Jan 24:0733464820987350.
European Women�s Lobby (2002). towards a common European framework to monitor progress in combating violence against women. Link: https://www.womenlobby.org/IMG/pdf/ewl_2002_towards_common_european_framework.pdf
Evans-Lacko S, Henderson C, Thornicroft G. Public knowledge, attitudes and behaviour regarding people with mental illness in England 2009-2012. The British Journal of Psychiatry. 2013 Apr;202(s55):s51-7.
Farina N, Hughes LJ, Jones E, Parveen S, Griffiths AW, Galvin K, Banerjee S. Evaluation of a dementia awareness programme in UK schools: A qualitative study. Public health. 2020 Aug 1;185:348-55.
Field MJ, Jette AM, editors. The Future of Disability in America. Institute of Medicine (US) Committee on Disability in America; Washington (DC): National Academies Press (US); 2007. G, Transportation Patterns and Problems of People with Disabilities. Link: https://www.ncbi.nlm.nih.gov/sites/books/NBK11420/ 
Findler L, Vilchinsky N, Werner S. The multidimensional attitudes scale toward persons with disabilities (MAS) construction and validation. Rehabilitation Counseling Bulletin. 2007 Apr;50(3):166-76.
Fisher KR, Purcal C. Policies to change attitudes to persons with disabilities. Scandinavian Journal of Disability Research. 2017 Apr 3;19(2):161-74.
Flanigan LK, Climie EA. A review of school-based interventions to reduce stigma towards schizophrenia. Psychiatric Quarterly. 2020 Dec;91:983-1002.
Frank C, Zamorski MA, Colman I. Stigma doesn�t discriminate: physical and mental health and stigma in Canadian military personnel and Canadian civilians. BMC psychology. 2018 Dec;6(1):1-1.
Frankel N, Gage A. (2007). M&E fundamentals: A self-guided minicourse. United States Agency for International Development (USAID), Washington, DC. Link: https://www.measureevaluation.org/resources/publications/ms-07-20-en/at_download/document
Fung K, Liu JJ, Sin R, Shakya Y, Guruge S, Bender A, Wong JP. Examining different strategies for stigma reduction and mental health promotion in Asian men in Toronto. Community Mental Health Journal. 2021 May;57(4):655-66.
Gaebel W, R�ssler W, Sartorius N, editors. The stigma of mental illness-end of the story?. Cham: Springer International Publishing; 2017.
Gaebel W, Z�ske H, Hesse K, Klingberg S, Ohmann C, Grebe J, Kolbe H, Icks A, Schneider F, Backes V, Wolff-Menzler C. Promoting stigma coping and empowerment in patients with schizophrenia and depression: results of a cluster-RCT. European archives of psychiatry and clinical neuroscience. 2020 Aug;270(5):501-11.
Gage A, Dunn M. (2009). Monitoring and evaluating gender-based violence prevention and mitigation programs. US Agency for International Development, MEASURE Evaluation, Interagency Gender Working Group, Washington DC. Link: https://www.igwg.org/wp-content/uploads/2017/07/gbv-me-handouts.pdf
Gallego J, Cangas AJ, Aguilar JM, Trigueros R, Navarro N, Galv�n B, Smyshnov K, Gregg M. Education students' stigma toward mental health problems: a cross-cultural comparison. Frontiers in psychiatry. 2020 Nov 5;11:1138.
Gandhi S, Jayaraman S, Sivakumar T, John AP, Joseph A, Prathyusha PV. Can employment in a caf� change Clientele Attitude towards the staff when they are Persons with Mental Illness?. International Journal of Social Psychiatry. 2021 Jan 28:0020764021990068.
Gierdowski, Dana C., and D. Christopher Brooks. Diversity, Equity, and Inclusion in the IT Workforce, 2019. Research report. Louisville, CO: ECAR, August 2019. Link: https://library.educause.edu/-/media/files/library/2019/8/hedei19.pdf?la=en&hash=4EDD93EE29647C3AEBE603CB7759534567BD2925
Ginevra MC, Vezzali L, Camussi E, Capozza D, Nota L. Promoting positive attitudes toward peers with disabilities: The role of information and imagined contact. Journal of Educational Psychology. 2021 May 27.
Goethals T, Mortelmans D, Van Hove G. Toward a more balanced representation of disability? A content analysis of disability coverage in the Flemish print media. Journal of Human Development, Disability, and Social Change. 2018;24(1):109-20.
Goh CM, Shahwan S, Lau JH, Ong WJ, Tan GT, Samari E, Kwok KW, Subramaniam M, Chong SA. Advancing research to eliminate mental illness stigma: an interventional study to improve community attitudes towards depression among University students in Singapore. BMC psychiatry. 2021 Dec;21(1):1-2.
Gordon SE, Kininmonth LA, Newton-Howes G, Purdie GL, Gardiner T. A comparative analysis of two alternative programmes to counter stigma and discrimination associated with mental illness delivered to medical students. Australasian Psychiatry. 2020 Nov 24:1039856220971935.
Goreczny AJ, Bender EE, Caruso G, Feinstein CS. Attitudes toward individuals with disabilities: Results of a recent survey and implications of those results. Research in developmental disabilities. 2011 Sep 1;32(5):1596-609.
Government of Canada (1988), Department of the Secretary of State. National Access Ability Week � May 30 to June 5, 2021. Link: https://www.canada.ca/en/employment-social-development/campaigns/national-accessability-week.html 
Government of Canada (2006). A way with Words and Images: Suggestions for the Portrayal of People with Disabilities. Human Resources and Skills Development Canada. www.hrsdc.gc.ca/eng/disability_issues/reports/way_with_words/2007/
Government of Canada (2019). Pilot Public Opinion Research Survey on Accessibility. Link: http://publications.gc.ca/collections/collection_2020/edsc-esdc/Em4-24-2019-1-eng.pdf
GSS (General Social Survey) (2009). Cycle 23: Victimization, Public Use Microdata Files. Link: https://www23.statcan.gc.ca/imdb/p2SV.pl?Function=getSurvey&Id=49195
Hagemann?White C. Measuring progress in addressing violence against women across Europe. International Journal of Comparative and Applied Criminal Justice. 2008 Sep 1;32(2):149-72.
Haller B, Rioux M, Dinca-Panaitescu M, Laing A, Vostermans J, Hearn P. The place of news media analysis within Canadian disability studies. Canadian Journal of Disability Studies. 2012;1(2):43-74.
Hansen A, McGarry D, Johnson A, Roche MA. The impact of an undergraduate students' culture on their learning about mental health: A scoping review. Nursing & Health Sciences. 2021 Apr 1.
Hapuarachchi RB, Herath JC, Ranasinghe PK. Self Stigma Among Caregivers Of People With Schizophrenia. Inaustralian And New Zealand Journal Of Psychiatry 2021 May 1 (Vol. 55, No. 1_ SUPPL, pp. 111-111). 1 OLIVERS YARD, 55 CITY ROAD, LONDON, ENGLAND: SAGE PUBLICATIONS LTD.
Hearst MO, Adelli R, Hepperlen R, Biggs J, DeGracia D, Ngulube E, Maluskiku-Mwewa B, Johnson DE, Rabaey P. Community-based intervention to reduce stigma for children with disabilities in Lusaka, Zambia: A pilot. Disability and Rehabilitation. 2020 Oct 13:1-0.
Heijnders M, Van Der Meij S. The fight against stigma: an overview of stigma-reduction strategies and interventions. Psychology, health & medicine. 2006 Aug 1;11(3):353-63.
Henderson C, Potts L, Robinson EJ. Mental illness stigma after a decade of Time to Change England: inequalities as targets for further improvement. European journal of public health. 2020 Jun 1;30(3):497-503.
Henderson C, Robinson E, Evans?Lacko S, Corker E, Rebollo?Mesa I, Rose D, Thornicroft G. Public knowledge, attitudes, social distance and reported contact regarding people with mental illness 2009�2015. Acta Psychiatrica Scandinavica. 2016 Aug;134:23-33.
Hielscher E, Waghorn G. Self-stigma and fears of employment among adults with psychiatric disabilities. British journal of occupational therapy. 2017 Dec;80(12):699-706.
Holland Bloorview Kids Rehabilitation Hospital. Campaign aims to reduce stigma, discrimination of kids, youth with disabilities. Link: https://www.independentliving.org/docs2/daakit32.html#anchor1298850 
Huhn et al., (2019). What is Social Marketing? (With 7 Stellar Examples). Link: https://www.business2community.com/digital-marketing/what-is-social-marketing-with-7-stellar-examples-02236451 
Hung L, Hudson A, Gregorio M, Jackson L, Mann J, Horne N, Berndt A, Wallsworth C, Wong L, Phinney A. Creating Dementia-Friendly Communities for Social Inclusion: A Scoping Review. Gerontology and Geriatric Medicine. 2021 May;7:23337214211013596.
Hunter SW, Divine A. Understanding the factors influencing physiotherapists� attitudes towards working with people living with dementia. Physiotherapy theory and practice. 2020 Jan 23:1-8.
Ibrahim D. (2019). Public perceptions of the police in Canada�s provinces. Canadian Centre for Justice and Community Safety Statistics. Link: https://www150.statcan.gc.ca/n1/pub/85-002-x/2020001/article/00014-eng.htm 
IOM (International Organization for Migration). (2009). Handbook on Performance Indicators for Counter-Trafficking Projects. Link: https://documentation.lastradainternational.org/doc-center/2174/the-iom-handbook-on-performance-indicators-for-counter-trafficking-projects
ITU (International Telecommunications Union) (2013). The ICT Opportunity for a Disability-Inclusive Development Framework. Link: https://www.itu.int/en/action/accessibility/Documents/The%20ICT%20Opportunity%20for%20a%20Disability_Inclusive%20Development%20Framework.pdf
Janssens KM, van Weeghel J, Dewa C, Henderson C, Mathijssen JJ, Joosen MC, Brouwers EP. Line managers� hiring intentions regarding people with mental health problems: a cross-sectional study on workplace stigma. Occupational and Environmental Medicine. 2021 Feb 4.
Javed A, Lee C, Zakaria H, Buenaventura RD, Cetkovich-Bakmas M, Duailibi K, Ng B, Ramy H, Saha G, Arifeen S, Elorza PM. Reducing the stigma of mental health disorders with a focus on low-and middle-income countries. Asian journal of psychiatry. 2021 Feb 13:102601.
Jones SC, Akram M, Gordon CS, Murphy N, Sharkie F. Autism in Australia: Community Knowledge and Autistic People�s Experiences. Journal of autism and developmental disorders. 2021 Jan 3:1-3.
Kalargyrou V, Pettinico W, Chen PJ. Attitudes toward people with physical disabilities: An examination of social context, discipline, disability type, and demographics. Journal of Vocational Rehabilitation. 2021(Preprint):1-7.
Karaffa KM, Koch JM. Stigma, pluralistic ignorance, and attitudes toward seeking mental health services among police officers. Criminal Justice and Behavior. 2016 Jun;43(6):759-77.
Khalifa G, Sharif Z, Sultan M, Di Rezze B. Workplace accommodations for adults with autism spectrum disorder: a scoping review. Disability and rehabilitation. 2020 Apr 23;42(9):1316-31.
Khalifeh H, Howard LM, Osborn D, Moran P, Johnson S. Violence against people with disability in England and Wales: findings from a national cross-sectional survey. PloS one. 2013 Feb 20;8(2):e55952.
Kim S, Richardson A, Werner P, Anstey KJ. Dementia stigma reduction (DESeRvE) through education and virtual contact in the general public: A multi-arm factorial randomised controlled trial. Dementia. 2021 Jan 31:1471301220987374.
Koh E. The Cunningham Dax Collection: a unique mental health resource. Austr Psychiatry. 2014;22(1):41�3.
Kohrt BA, Turner EL, Rai S, Bhardwaj A, Sikkema KJ, Adelekun A, Dhakal M, Luitel NP, Lund C, Patel V, Jordans MJ. Reducing mental illness stigma in healthcare settings: proof of concept for a social contact intervention to address what matters most for primary care providers. Social Science & Medicine. 2020 Apr 1;250:112852.
Krah� B, Altwasser C. Changing negative attitudes towards persons with physical disabilities: An experimental intervention. Journal of Community & Applied Social Psychology. 2006 Jan;16(1):59-69.
Kumar P, Patel VK, Kanabar BR, Vasavada DA, Bhatt RB, Tiwari DS. Changing Attitude and Stigma towards Mental Illness through Education among the Rural School Students. Journal of Indian Association for Child & Adolescent Mental Health. 2021 Apr 1;17(2).
Levac D, Colquhoun H, O'Brien KK. Scoping studies: advancing the methodology. Implement Sci. 2010;5:69.
Liang J, Jang Y, Aranda MP. Stigmatising beliefs about Alzheimer�s disease: Findings from the Asian American Quality of Life Survey. Health & Social Care in the Community. 2020 Oct 22.
Likeminds (2016). Like Mine programme. New Zealand. Link: https://www.likeminds.org.nz/ 
Link BG, Phelan JC. Conceptualizing stigma. Annual review of Sociology. 2001 Aug;27(1):363-85. 
Luelmo P, Kasari C, Fiesta Educativa, Inc. Randomized pilot study of a special education advocacy program for Latinx/minority parents of children with autism spectrum disorder. Autism. 2021 Apr 16:1362361321998561.
Mac C�rthaigh S, L�pez B. Factually based autism awareness campaigns may not always be effective in changing attitudes towards autism: Evidence from British and South Korean nursing students. Autism. 2020 Jul;24(5):1177-90.
Macku K, Caha J, P�szto V, Tu?ek P. Subjective or objective? How objective measures relate to subjective life satisfaction in Europe. ISPRS International Journal of Geo-Information. 2020 May;9(5):320.
Magliano L, Affuso G, Prisco N, Arpino A, Romano L. Improving employees� views about persons with mental disorders as potential workmates: A 2-year partially controlled study. American Journal of Orthopsychiatry. 2021;91(1):66.
Maki Y, Takao M, Hattori H, Suzuki T. Promoting dementia?friendly communities to improve the well?being of individuals with and without dementia. Geriatrics & gerontology international. 2020 Jun;20(6):511-9.
Marsh, David. 1999. Results Frameworks & Performance Monitoring: A Refresher (ppt). Link: http://www.childsurvival.com/tools/Marsh/sld001.htm 
Mascayano F, Toso-Salman J, Ho YC, Dev S, Tapia T, Thornicroft G, Cabassa LJ, Khenti A, Sapag J, Bobbili SJ, Alvarado R. Including culture in programs to reduce stigma toward people with mental disorders in low-and middle-income countries. Transcultural psychiatry. 2020 Feb;57(1):140-60.
McCann TV, Savic M, Ferguson N, Cheetham A, Witt K, Emond K, Bosley E, Smith K, Roberts L, Lubman DI. Recognition of, and attitudes towards, people with depression and psychosis with/without alcohol and other drug problems: results from a national survey of Australian paramedics. BMJ open. 2018 Dec 1;8(12):e023860.
McLeroy KR, Bibeau D, Steckler A, Glanz K. An ecological perspective on health promotion programs. Health education quarterly. 1988 Dec;15(4):351-77.
Mehta N, Kassam A, Leese M, Butler G, Thornicroft G. Public attitudes towards people with mental illness in England and Scotland, 1994�2003. The British Journal of Psychiatry. 2009 Mar;194(3):278-84.
Menon S, Karl J, Wignaraja K. Handbook on planning, monitoring and evaluating for development results. UNDP Evaluation Office, New York, NY. 2009;68(3):10. Link: http://web.undp.org/evaluation/handbook/documents/english/pme-handbook.pdf
Mental Health Commission of Canada (MHCC) (2009). Opening Minds (OM). Link: https://www.mentalhealthcommission.ca/English/opening-minds
Mental Health Commission of Canada (MHCC) (2015). Informing the Future: Mental Health Indicators for Canada. Link: https://www.mentalhealthcommission.ca/wp-content/uploads/drupal/Informing%252520the%252520Future%252520-252520Mental%252520Health%252520Indicators%252520for%252520Canada_0.pdf 
Mental Health Commission of Canada (MHCC). (2017). The Working Mind First Responders (TWMFR), formerly known as Road to Mental Readiness (R2MR). Link: https://theworkingmind.ca/working-mind-first-responders 
Mental Health Commission of Canada (MHCC). (2017). Working Mind (TWM). Link: https://theworkingmind.ca/working-mind 
Mittal D, Owen RR, Ounpraseuth S, Chekuri L, Drummond KL, Jennings MB, Smith JL, Sullivan JG, Corrigan PW. Targeting stigma of mental illness among primary care providers: Findings from a pilot feasibility study. Psychiatry research. 2020 Feb 1;284:112641.
Morgan AJ, Wright J, Reavley NJ. Review of Australian initiatives to reduce stigma towards people with complex mental illness: what exists and what works?. International journal of mental health systems. 2021 Dec;15(1):1-51. 
Morin D, Crocker AG, Beaulieu?Bergeron R, Caron J. Validation of the attitudes toward intellectual disability�ATTID questionnaire. Journal of Intellectual Disability Research. 2013 Mar;57(3):268-78.
Murney MA, Sapag JC, Bobbili SJ, Khenti A. Stigma and discrimination related to mental health and substance use issues in primary health care in Toronto, Canada: A Qualitative Study. International journal of qualitative studies on health and well-being. 2020 Jan 1;15(1):1744926.
Myers F. Evaluation of'see Me'-the National Scottish Campaign Against the Stigma and Discrimination Associated with Mental Health. Social Research; 2009. Link: https://silo.tips/downloadFile/health-and-community-care-5
NAMI (2021). National Alliance of Mental Illness. Link: https://www.nami.org/About-NAMI
National Academies of Sciences, Engineering, and Medicine (2016). Ending discrimination against people with mental and substance use disorders: The evidence for stigma change. National Academies Press; 2016 Sep 3. Link: https://www.ncbi.nlm.nih.gov/books/NBK384914/
National Disability Authority (NDA) (2002). Public attitudes to disability in the Republic of Ireland. Link: http://nda.ie/ndasitefiles/attitude.pdf
Newton-Howes G, Senior J, Beaglehole B, Purdie GL, Gordon SE. Does a comprehensive service user-led education programme effect more positive attitudes towards recovery and less stigmatising attitudes towards people with lived experience of mental distress in medical students? A comparative cohort study. Australian & New Zealand Journal of Psychiatry. 2021 Jan 17:0004867420987886.
Niewohner J, Pierson S, Meyers SJ. �Leave no one behind�? The exclusion of persons with disabilities by development NGOs. Disability & Society. 2020 Aug 8;35(7):1171-6.
Nota L, Santilli S, Ginevra MC, Soresi S. Employer attitudes towards the work inclusion of persons with disabilities. Journal of Applied Research in Intellectual Disabilities. 2014 Nov;27(6):511-20.
Nowicki, EA. (2005). Understanding children�s perceptions of intellectual and physical disabilities: Attitudes, knowledge, and social cognition. Exceptionality Education Canada, 15(3), 21�39.
Office for Disabilities Issues (2009). Public perceptions of disabled people. Evidence from the British Social Attitudes Survey (BSAS) in 2009. Link: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/325989/ppdp.pdf
Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021; 372:n71. doi: 10.1136/bmj.n71
Parker LR, Monteith MJ, South SC. Dehumanization, prejudice, and social policy beliefs concerning people with developmental disabilities. Group Processes & Intergroup Relations. 2020 Feb;23(2):262-84.
PATH (2011). Monitoring and evaluation of initiatives on violence against women and girls. Link: https://www.endvawnow.org/uploads/browser/files/me_asset_english_nov_2011__final.pdf
Perreault S. (2015). Criminal victimization in Canada, 2014. Juristat. Statistics Canada Catalogue no. 85?002?X. Link: https://www150.statcan.gc.ca/n1/pub/85-002-x/2015001/article/14241-eng.pdf
Pescosolido BA, Perry BL, Krendl AC. Empowering the next generation to end stigma by starting the conversation: bring change to mind and the college toolbox project. Journal of the American Academy of Child & Adolescent Psychiatry. 2020 Apr 1;59(4):519-30.
Peter LJ, Schindler S, Sander C, Schmidt S, Muehlan H, McLaren T, Tomczyk S, Speerforck S, Schomerus G. Continuum beliefs and mental illness stigma: a systematic review and meta-analysis of correlation and intervention studies. Psychological medicine. 2021 Apr 8:1-1.
Power MJ, Green AM, WHOQOL?DIS Group. The Attitudes to Disability Scale (ADS): development and psychometric properties. Journal of Intellectual Disability Research. 2009 Sep;54(9):860-74.
Public Health Agency of Canada (2019a). Addressing Stigma: Towards a More Inclusive Health System. Link: https://www.canada.ca/en/public-health/corporate/publications/chief-public-health-officer-reports-state-public-health-canada/addressing-stigma-toward-more-inclusive-health-system.html
Public Health Agency of Canada (2019b). Addressing Stigma: Towards a More Inclusive Health System � What We Heard Report. Link: https://www.canada.ca/en/public-health/corporate/publications/chief-public-health-officer-reports-state-public-health-canada/addressing-stigma-what-we-heard.html
Ran MS, Hall BJ, Su TT, Prawira B, Breth-Petersen M, Li XH, Zhang TM. Stigma of mental illness and cultural factors in Pacific Rim region: a systematic review. BMC psychiatry. 2021 Dec;21(1):1-6.
Randle M, and Reis D. (2017) Changing community attitudes toward greater inclusion of persons with disabilities. A Rapid Literature Review. Link: https://www.facs.nsw.gov.au/__data/assets/file/0008/372608/Rapid-Review-V3-interactive.pdf
Reavley NJ, Jorm AF. Stigmatising attitudes towards people with mental disorders: Changes in Australia over 8 years. Psychiatry research. 2012 May 30;197(3):302-6. 
Rebernik N, Szajczyk M, Bahillo A, Goli?nik Maru?i? B. Measuring disability inclusion performance in cities using Disability Inclusion Evaluation Tool (DIETool). Sustainability. 2020 Jan;12(4):1378.
Reddyhough C, Locke V, Paulik G. Changing healthcare professionals� attitudes towards voice hearers: An education intervention. Community Mental Health Journal. 2021 Jul;57(5):960-4.
Reid S, Hinchliffe S, Waterton J. Attitudes to Mental Health in Scotland: Scottish Social Attitudes Survey 2013. Scottish Government Social Research; 2014. Link: https://www.gov.scot/publications/attitudes-mental-health-scotland-scottish-social-attitudes-survey-2013/
Rillotta F, Nettelbeck TE. Effects of an awareness program on attitudes of students without an intellectual disability towards persons with an intellectual disability. Journal of intellectual and developmental disability. 2007 Mar 1;32(1):19-27.
Rodr�guez-Rivas ME, Cangas AJ, Fuentes-Olavarr�a D. Controlled study of the impact of a virtual program to reduce stigma among university students toward people with mental disorders. Frontiers in psychiatry. 2021;12.
Rosenbloom S (2007). Transportation Patterns and Problems of People with Disabilities. Link: https://www.ncbi.nlm.nih.gov/books/NBK11420/
Rossetto A, Potts LC, Reavley NJ, Henderson C. Perceptions of positive treatment and discrimination toward people with mental health problems: Findings from the 2017 and 2019 attitudes to mental illness surveys. Stigma and Health. 2020 Feb 13.
Sandhu-Rojon R. Selecting Indicators for impact evaluation. United Nations Development Programme (UNDP). 2003 Mar;7:2004.
Santos AD, Ferrari AL, Medola FO, Sandnes FE. Aesthetics and the perceived stigma of assistive technology for visual impairment. Disability and Rehabilitation: Assistive Technology. 2020 Jun 2:1-7.
Sarge MA, Kim HS, Velez JA. An Auti-Sim Intervention: The Role of Perspective Taking in Combating Public Stigma with Virtual Simulations. Cyberpsychology, Behavior, and Social Networking. 2020 Jan 1;23(1):41-51.
Sartorius N. Fighting stigma: theory and practice. World Psychiatry. 2002 Feb;1(1):26.
Schnitzler M. The political economy of disability in South Africa, between social grants and job-creation programmes. Review of African Political Economy. 2020 Jul 2;47(165):432-48.
Scior K. Public awareness, attitudes and beliefs regarding intellectual disability: A systematic review. Research in developmental disabilities. 2011 Nov 1;32(6):2164-82.
SCOPE (2019). Equity for disabled people. Travel Fair. Link: https://www.scope.org.uk/scope/media/files/campaigns/travel-fair-report.pdf
Shahin S, Reitzel M, Di Rezze B, Ahmed S, Anaby D. Environmental factors that impact the workplace participation of transition-aged young adults with brain-based disabilities: a scoping review. International journal of environmental research and public health. 2020 Jan;17(7):2378.
Shahwan S, Lau JH, Goh CM, Ong WJ, Tan GT, Kwok KW, Samari E, Lee YY, Teh WL, Seet V, Chang S. The potential impact of an anti-stigma intervention on mental health help-seeking attitudes among university students. BMC psychiatry. 2020 Dec;20(1):1-4.
Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, Porter AC, Tugwell P, Moher D, Bouter LM. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC medical research methodology. 2007 Dec;7(1):1-7. Link: https://amstar.ca/docs/AMSTARguideline.pdf
Siperstein GN, Romano N, Mohler A, Parker R. A national survey of consumer attitudes towards companies that hire people with disabilities. Journal of Vocational Rehabilitation. 2006 Jan 1;24(1):3-9.
Stamp Out stigma (2021). Association for Behavioral Health and Wellness (ABHW). Link: http://www.stampoutstigma.com/contact-us/
Suaidi MT, Wong PK, Mohd Tahir NA, Chua EW. Community Pharmacists� Knowledge, Attitude, and Practice in Providing Self-Care Recommendations for the Management of Premenstrual Syndrome. Medicina. 2020 Apr;56(4):181.
Tachibana, T., & Watanabe, K. (2004). Attitudes of Japanese adults toward persons with intellectual disability: Relationship between attitudes and demographic variables. Education and Training in Developmental Disabilities, 39, 109�126.
Tait K, Purdie N. Attitudes toward disability: Teacher education for inclusive environments in an Australian university. International Journal of Disability, Development and Education. 2000 Mar 1;47(1):25-38.
Temple JB, Kelaher M, Williams R. Discrimination and avoidance due to disability in Australia: evidence from a National Cross-Sectional Survey. BMC public health. 2018 Dec;18(1):1-3
Terashima M, Clark K. The precarious absence of disability perspectives in planning research. Urban Planning. 2021 Feb 24;6(1):120-32.
Tervo RC, Azuma S, Palmer G, Redinius P. Medical students' attitudes toward persons with disability: a comparative study. Archives of physical medicine and rehabilitation. 2002 Nov 1;83(11):1537-42.
Thomson M, Planning S. (2010). Literature review: Outcomes and indicators of welcoming and inclusive communities and workplaces initiatives. Prepared for Welcome BC Immigrant Integration Brance of the Ministry of Advanced Education and Labour Market Development. Vancouver: Social Planning and Research Council of BC. Link: http://www.amssa.org/wp-content/uploads/2015/05/WelcomeBC_report.pdf
Thornicroft A, Goulden R, Shefer G, Rhydderch D, Rose D, Williams P, Thornicroft G, Henderson C. Newspaper coverage of mental illness in England 2008-2011. The British journal of psychiatry. 2013 Apr;202(s55):s64-9.
UK Cabinet Office (2008). Achieving Culture Change: A Policy Framework. Link: https://crawford.anu.edu.au/sparc/pdf/2010/achieving_culture_change.pdf
UK Government (2021). Research and analysis UK Disability Survey research report. Link: https://www.gov.uk/government/publications/uk-disability-survey-research-report-june-2021/uk-disability-survey-research-report-june-2021 
UK Prime Minister�s Strategy Unit (2005). Improving the Life Chances of Disabled People. London: Department for Work and Pensions, the Department of Health, the Department for Education and Skills, and the Office of the Deputy Prime Minister. A Joint Report.
UNAIDS (Joint United Nations Programme on HIV/AIDS). (2008). A framework for monitoring and evaluating HIV prevention programmes for most-at-risk populations. Link: https://www.unaids.org/sites/default/files/sub_landing/17_Framework_ME_Prevention_Prog_MARP_E.pdf 
UNAIDS (Joint United Nations Programme on HIV/AIDS). (2010). Indicator Standards: Operational Guidelines for Selecting Indicators for the HIV Response. Link: https://www.unaids.org/sites/default/files/sub_landing/files/4_3_MERG_Indicator_Standards_2.pdf
UNWomen (2013). Ending Violence against Women and Girls Programming Essentials 102. Link: https://www.endvawnow.org/en/articles/336-indicators.html
Victorian Office for Disability (2010). Picture this: increasing the cultural participation of people with a disability in Victoria�literature review and analysis. Link: https://www.vgls.vic.gov.au/client/en_AU/search/asset/1161971/0
Waqas A, Malik S, Fida A, Abbas N, Mian N, Miryala S, Amray AN, Shah Z, Naveed S. Interventions to reduce stigma related to mental illnesses in educational institutes: a systematic review. Psychiatric Quarterly. 2020 Sep;91(3):887-903.
Walker J, Scior K. Tackling stigma associated with intellectual disability among the general public: A study of two indirect contact interventions. Research in developmental disabilities. 2013 Jul 1;34(7):2200-10.
Watson AC, Corrigan PW, Ottati V. Police officers' attitudes toward and decisions about persons with mental illness. Psychiatric services. 2004 Jan;55(1):49-53.
Werner S, Corrigan P, Ditchman N, Sokol K. Stigma and intellectual disability: A review of related measures and future directions. Research in developmental disabilities. 2012 Mar 1;33(2):748-65.
Wilson MC, Scior K. Attitudes towards individuals with disabilities as measured by the Implicit Association Test: A literature review. Research in developmental disabilities. 2014 Feb 1;35(2):294-321.
Wilson, Erin, Nick Hagiliassis, Kelli Nicola-Richmond, Anne Mackay and Robert Campain. 2008. Defining and measuring outcomes of inclusive communities. Presentation for National Disability Services, Australia. Link: http://www.nds.org.au/wa/Archive/2008/LW08-Wilson1.pdf
Woodley A, Metzge N (2012) Employer attitudes towards employing disabled people. Ministry of Social Development. New Zealand. Link:https://www.odi.govt.nz/assets/Guidance-and-Resources-files/Employers-Research.pdf
Yazbeck M, McVilly K, Parmenter TR. Attitudes toward people with intellectual disabilities: An Australian perspective. Journal of Disability Policy Studies. 2004 Sep;15(2):97-111.
Zissi A, Rontos C, Papageorgiou D, Pierrakou C, Chtouris S. Greek employers� attitudes to employing people with disabilities: Effects of the type of disability. Scandinavian Journal of Disability Research. 2007 Jan 1;9(1):14-25.

APPENDICES

Appendix I. Data Extraction Guidelines

Table AI.1. Data Extraction Guidelines
Column Title
Cell Content
Explanation
Examples/Details
Author
Free text
First author name, no punctuation
Beamish
Year 
Free text
Year of publication
2021
Title
Free text
Full title of reference
The adaptation and validation of a stigma measure for individuals with multiple sclerosis
Country / Region
Free text
Specify as much as possible the specific study area.
United Kingdom (England, Wales, Scotland, Northern Ireland)
Peer-reviewed
Drop down Category:
Yes/No
Specify if the study is peer-reviewed
Yes, No
Study Type

Drop down category, check the one category only that best fits the value of this study for our project
Descriptive study of a beliefs, values, attitudes and behaviours, etc. toward PWDs among different populations
Study can draw on secondary data sources or primary data collection using surveys, questionnaires, interviews, observation, focus groups.


Study of initiative, program, or intervention
Studies that describe and/or evaluate an initiative, program, or intervention. It may also include an environmental scan of programs, initiatives or interventions


Literature review/evidence synthesis studies
Literature review, evidence synthesis studies such as systematic review, rapid review, scoping review, meta-analysis, etc. (Adams et al., 2017).


Measurement protocol development and/or evaluation
Include all studies that develop and/or evaluate measurement protocols (tools, scale, assessment framework, and guidelines, etc.)


Best practice guidance
Best-practices guidance related to culture change, framework for development, monitoring, and evaluation, etc.
Type of Intervention
Free text
If study type is an intervention, specify type of intervention as much as possible
Educational intervention, Contact based intervention, etc.
Target Audience
Free text
Specify as much as possible the specific study area.
University students, General Public, Healthcare workers, etc.
Context
Free text
Which context intervention has been implemented
Healthcare, University, School, workplace, public, etc.
Note: for review studies, selected varies if it is multiple. 
Type of Disability
Drop down Category, check all that are specified in the study
1. Sensory
2. Pain
3. Mobility
4. Learning
5. Intellectual
6. Mental Health
7. Autism Spectrum Disorder (ASD)
8. ADHD
9. Other, specify below
10. Not Specified
Sensory � Blindness /deafness
Pain � Chronic or episodic pain
Mobility � e.g. Wheelchair users, amputees, etc.
Learning � Difficulty acquiring a specific skill/competency (Dyslexia, dysgraphia, etc.)
Intellectual � related to cognitive function, often determined via IQ assessment
Mental Health � mood and psychotic disorders such as depression, anxiety, PTSD, schizophrenia, or BPD.
Autism Spectrum Disorder (ASD)
ADHD
Other, specify below � See next row
Not Specified � The study does not specify what disabilities they are referring to.
Study Objective
Free text
One sentence explaining the key objectives of the study. Cut and past from PDF if possible.
�to determine the public perception and attitude towards Alzheimer disease�
Study Outcomes (primary/ secondary)
Free text
One sentence explaining the key objectives of the study. Cut and past from PDF if possible.
�Change perception of the general public�
Key Indicators 
Free text
Name of measures developed/used to determine outcomes
�Mental Health
Knowledge Schedule (MAKS)�
Study Results/ findings
Free text
One sentence explaining the key objectives of the study. Cut and past from PDF if possible.
�A significant difference was found between pretest and post-test means for three of the four subscales�
Sample Size
Free text
Study sample size
Number
Abstract including Findings and Outcomes
Free text
Cut and past from PDF if possible.
Use abstract if available, if not, extract the most relevant part from the introduction or executive summary.
Project Research Questions Alignment
Drop down Category, check all that apply (Q1-Q5)
Q1: Is the article about culture change and relevant to persons with disabilities or a related topic (e.g., stigma reduction initiatives)?
Q2: Does the article describe culture change theory or a conceptual framework for culture change
Q3: Does the article describe or evaluate a culture change initiative
Q4: Does the article identify/describe indicators to measure culture change
Q5: Do the article address one of the seven ACA priority areas
Related to Q5: ACA priority areas are: 
* Employment
* Built Environment
* Transportation
* Information and Communication Technologies
* Communication, Other than Information and Communication Technologies
* Procurement of Goods, Services and Facilities
* Design And Delivery of Programs and Services
Potential Key Informant & Organizations
Free text
Cut and past from document if possible
Individuals, organizations, companies


Appendix II. Summary of Finding from The Key Informant Interview

Indicators Identified by Key Informants

Several themes emerged from the key informants. It was noted that attitude change is difficult to measure, however behavioral changes can be measured and reflects attitude changes. Several tangible and less-tangible indicators of culture change were suggested in the areas of knowledge, attitudes, behaviors and practices that reflect the underlying process and pathways towards culture change, at the individual, institutional, community and other levels. Key informants noted that in some instances culture change happens from the top down, and in others, from the bottom up. Indicators of culture change were ultimately more tangible, clear and objective if they measured outcomes and impacts in relation to various facets of the quality of life of persons with disabilities. There was feedback from disability advocates that what matters most is outcomes, e.g., how many people are employed, the standard of living, experience with violence, access to services, access to jobs, and access to housing.

Employment Rates

Several key informants suggested that a clear indicator of culture change was an increase in the employment rate of persons with disabilities in meaningful jobs, rather than attitudinal changes in workplaces. An increase in the hiring of persons with disabilities in meaningful jobs and internships, and an improvement in the employment rate of persons with disabilities, particularly in meaningful jobs, is reflective of a culture change. The Australia Disability Strategy (ADS) (Australia  Government, 2021a) developed an Outcomes Framework (Australia  Government, 2021b), which is currently tracking outcomes on employment. They indicated that the best measure of culture change is the number of persons with disabilities in workplace and in meaningful jobs. A key informant noted that if there are more persons with disabilities in that workplace in meaningful jobs, it is because their management�s attitudes have changes, and furthermore, that coworkers� attitudes have also change.

Sustained Employment

Sustained employment is an indicator used by the Employment Works program (Worktopia, 2022). Retaining work that was meaningful to persons with disabilities was considered a measure of culture change (6). The host organization and researchers in 12 to 15 communities across Canada administered surveys as part of the program. This was an effective way to measure how employer perspectives were changing. Measures included the number of participants still working after 6, 12, 18 months post-program and where they were at in their employment situation. A year after program ended, program administrators canvassed participants about persons with disabilities that were hired to see if they were still working. 

Internships for Persons with Disabilities

A good indicator of culture change in the workplace is the practice of hiring persons with disabilities in internship programs, as it increases the probably of a person being hired for longer-term employment. One key informant spoked about how their organization encourages companies to have internships for persons with disabilities. There was mention of a study entitled Disability-Inclusive Employer Practices and Hiring of Individuals with Disabilities that found that companies who have internships for persons with disabilities are six times more likely to hire persons with disabilities for longer-term employment (Erickson et al., 2014). 

Built Environment

The development and improvements of the built environment was suggested as an indicator that an organization had changed it norms in relation to accessibility and inclusion. Of note were changes such as building wheelchair ramps for persons with mobility impairments. Other examples of improvement in the built environment noted by key informants include crosswalks, and braille on elevator buttons. 

Implementation of Legislation and Policies

Application of legislation and policy changes across services within large-scale organizations and the public sector emerged a common theme across key informants. One example is the Equality Act 2010 in UK in the health care services (15). Applying equality legislation across services is a good indicator of culture change. In general, legislation that includes persons with disabilities in day-to-day life in broad terms was considered important. 

Workplace policies, practices and educational efforts can also serve as a marker of change and can be evaluated and tested. Implementation of legislation on training for managers about the duty to accommodate was also noted. Changing policies and practices is a means to changing systemic behavior and culture. 

Reduction in Disability Claims

One key informant noted that disability-confident champions in corporate culture did their own training and assessment of mental health status and return to work, and in time, cases of disability claims and health care leaves decreased, stigma decreased, and after two years a significant economic benefit was observed. Their return on investment in training became a significant advantage. 

Disability-Confident Workplaces

Organizations becoming disability confidant was noted by key informants as an important indicator of culture change. Proxy measures include improvement to the built environment and use of universal and inclusive design principles, top management involvement culture change initiatives, and improvements in access to employment opportunities for job candidates with disabilities. Some of these proxy measures are detailed below.

Employment Process: Website, Application and Interview

Improving accessibility of websites for job candidates and accessibility of the application and interview process were identified as proxies for organizational culture change. The accessibility of the employment process should be part of the conversation on inclusion of persons with disabilities at many levels. Indicators of website accessibility include improving accessibility of overlays (a piece of code that creates a skin on a website) that can prevents the use of assistive technology (e.g., use of a screen reader that is interrupted when downloading something, so that a person must turn one off to use the other). 

Top Management

It was noted that without the support and involvement of top management, changing the culture at an organizational can be very difficult. Thus, involvement and commitment by top management was a key indicator. If the senior management of organizations sent out statements showing their commitment to equity, diversity and inclusion, this would suggest leadership was promoting accessibility and inclusions. A diverse board of directors or senior management that included the presence of persons with disabilities was also noted as an indicator of an inclusive culture.

Bottom-up Culture Change

Culture change can also happen from the bottom up by persons of influence, or culture carriers. Culture carriers are persons who have accrued influence, a nexus of attitude and behavior, ideally influencing positive change. The person does not have to be senior in the organization to be an influencer.

Change in Language

Changes in the use of language and use of labels was noted as a sign of attitudinal and behavioral change. Language use is supportive if persons feel included, not labeled, and vocabulary used to describe them is not stigmatizing. Changes in language can be observed in recruitment advertisements, calls for applications for research grants, and in the interactions with colleagues who have been on sick or disability leave. In terms of the latter, support language would be to say, �It is great to see you again�, �I�m so glad you�re back�, rather than asking if they feel better. Small shifts in language can have a huge impact on workplace culture. Language used by the media has an impact on public opinion. Not perpetuating stereotypes can lead to reductions in isolation and discrimination of marginalized populations such as persons with mental illness.

Medical Documentation for Accommodations or Sick Leave

Eliminating or reducing the need for doctors� notes as evidence for illness and disability can reduce barriers to accommodations. For example, not requiring medical evidence from students requesting accommodations for exams, was suggested as an indicator of inclusive culture. Doctors� notes can be costly for students and workers and are not needed for all accommodation requests. 

Workplace Supports

A good indicator of culture change is increasing access to supports such as counseling, professional support, Employee/Family Assistance Program, community supports, and accommodations. Having supports available promotes change in help-seeking behaviors. Indicators can be collected through engagement surveys that have an option to self-identify as a person with a disability. Engagement surveys provide a sense of how people feel about an organization and, through repeated cycles, can be used for monitoring and evaluation of progress over time. 

Promoting Accommodation Dialogue

Making accommodations part of conversations between managers and workers was seen as a way to create comfort with disclosure. Questions about accommodation needs could be part of annual reviews, so that workers understand that it is acceptable to need accommodations. It should also be made part of the dialogue in the hiring and onboarding processes. Identify comfort levels with accommodation conversations can be assessed through anonymous health and well-being surveys.

Training and Education

Mental health training for managers and other workplace parties was noted as important for promoting inclusion. Mental health training and promoting support seeking were noted as critically important for first responders, as they are at substantially higher risk for post-traumatic stress injury. Many first responders now having specialized teams who deal with mental health issues. Training improves mental health literacy and results in greater awareness of mental health issues at all levels with and beyond the organization (e.g., with family members and in the community).

Sense of Belonging and Relational Indicators

A sense of belonging and feeling connected were noted as an important outcomes. Belonging provides a feeling of security and being valued. It can protect against stigma, discrimination and ableism experience every day by persons with disabilities. Indicators of a sense of belonging included people knowing a person�s name, not because it is the right thing to do, but because the person is welcome. Belonging means that if a person is not there, somebody wonders where they are. It can be difficult to measure a sense of belonging, however some survey with questions such as �I feel part of community�, �I feel sense of belonging�, �I feel supported�, �I feel have friends� are used in some surveys. Additional indicators of belonging and connectedness include questions such as �Would you be friends with someone with depression?,� �Would you date someone with depression?,� �Would you introduce someone with depression to your family?� and �Would you share an apartment with someone with depression?�

Universal/Inclusive Design in the Educational Sector

In higher education, proactive accommodations for visual barriers, auditory barriers, sensory barriers, mobility barriers, etc. are indicators of universal design. Embedding broader principles of inclusive design in the overall curriculum is important, e.g., exam accommodations such as options to record responses to questions that are same length as written answers, getting extra time as needed for assignments. Progress can be measured within faculties on the sort of accessibility initiatives that have been embedded within the course by looking at the course outline, e.g., �Has the instructor integrated and included accessibility principles without knowing they have students with disabilities?� Another indicator would be to have mandatory training for instructors on universal/inclusive design and accommodating persons with disabilities. An audit tool could include questions such as �How many professors are involved in EDI principles?,� �What training is received as part of onboarding at university to provide inclusive practices?,� and �Do instructors have an open door policy for students who require accommodations?� Even with a universal design approach, it is important to realize that one size does not fits all. 

Frameworks and Approaches for Culture Change

Several frameworks, approaches and guiding principles were identified by key informants, which we describe in detail below.

Contact-Based Education

A running theme that emerged was the concept of contact-based education and exposure. The concept of contact-based education was noted by several people, where able bodies persons are exposed to persons with lived experience, in the workplace, school or community settings, to increase compassion and understanding of those with disabilities and to normalize the presence of diverse persons in different social roles. Exposure to persons with disabilities helps to normalize having diverse persons in the community and normalizes having accommodations. Contact based education is a key tool for de-stigmatization, providing opportunities for telling stories of lived experiences and narrative of recovery. Storytelling by persons with lived experiences also helps others to understand those with episodic disabilities, an approach used in a training workshop spearheaded by Realize, and specifically for mental health in the Opening Minds anti-stigma initiative for youths called Inquiring Minds, influences culture change.

Theater and plays were used to create awareness in dementia care environments for persons with dementia. Role models and contact-based education increases exposure, social contact and empathy with persons with lived experiences and breaks down social distance. Role models reduce stigma. A program called HeadStrong, part of the Opening Minds initiative, uses contact-based education to reduce stigma of mental illness among young persons by exposing them to powerful stories, sometimes by celebrities, to show support and how to get help they need. It was a successful program across all demographic groups, including Indigenous communities and immigrant communities. The youth then take home what they learn and talk about it with their parents. Contact-based education teaches people how to seek help and also helps provide them with the tools and training to know how to support their peers, i.e., if a young person is having suicidal ideations, even if they promised not to tell anybody or go to a parent or teacher, they are confident in knowing how to help a person.

Realize�s IDEAL Framework

The IDEAL (Inclusion, Diversity, Equity, Accessibility, Leading to Belonging) framework was developed by Realize to address accessibility and inclusion. The framework goes beyond Equity, Diversity and Inclusion (EDI). The framework has guided Realize, informing the way they approach all their projects and activities. It defines their identity as an organization and their work going forward.

Guiding questions for the IDEAL Framework are as follows:

1.	Who is at the table? Who is not at the table, when being considered in things?
2.	How do we center, appreciate and learn from different voices and lived experience in our process of developing this framework?
3.	How do we go beyond the current norms in the sector, and recognize that broad range of lived experiences and voices in all of the aspects of our own work?
4.	What kind of barriers are being faced by people? 
5.	What impactful measures can be implemented by our organization to address some of the barriers that individuals from under-represented groups are facing? 
6.	What physical, architectural, technological, information, communication, and attitudinal barriers, or anything that may be a result of a policy or practice, that might hinder the full and equal participation of people with any kind of impairment, i.e., physical, mental, intellectual, cognitive, learning, communication or sensory, to involving themselves and engage with the work that we do?
7.	What can we do to remove the barriers and better facilitate that full and equitable participation? 
8.	Belonging piece: Do individuals involved with the organization feel empowered, and feel that they are invited to reach their full potential in their involvement with our organization? 

Measures used include:
�	Ensuring the right people are at the table;
�	The approach taken with projects and activities; and 
�	Ensuring there follow up on commitments.

For all new products, online courses, workshops, etc. Realize has a consultant who works in the context of disability, discrimination and race, that ensures they are following through on the commitments made. The success of their framework is partly seen in the diverse makeup of their board and staff.

Australia Disability Strategy (ADS) Outcomes Framework

The Australia Disability Strategy (ADS) includes an Outcomes Framework which tracks, reports and measures the outcomes for persons with disability across all seven Outcome Areas of the Strategy, in the hopes of driving change (Australia  Government, 2021a; 2021b). The Outcomes Framework will be used to track progress of the ADS. Outcomes range from employment and financial security; inclusive homes and communities; safety, rights and justice; personal and community support; education and learning; health and wellbeing, and community attitudes. In addition, a longitudinal survey on attitude change is being designed by Australian National University.

QuakeLab Inclusion Maturity Model

The QuakeLab Inclusion Maturity Model is a framework for use by an organization to assess and measure their progress in relation to EDI culture change (described as the diversity, equity, and inclusion (DEI) journey) (QuakeLab, 2020). An organization�s maturity is categorized into four levels:

* Level 1: Internal assessment and action 
* Level 2: Embedded action and looking externally
* Level 3: Innovation inclusion
* Level 4: Dismantle the system

The model is designed to go beyond the focus of diversity with the workplace to encapsulating efforts to rigorously investigate historical systems of oppression.

Mental Health Commission of Canada - Framework for a Psychosocially Health Workplace

The Mental Health Commission of Canada, in partnership with Ottawa Public Health, adapted content from The Mindful Employer (2020), to develop a series of short, animated videos that aim to raise awareness about the 13 factors that can impact the mental health of employees in the workplace. It is intended to encourage conversations on methods for promoting psychologically safe places to work. Indicators of a psychosocially health workplace, can be found at the following link: 13 Factors: Addressing Mental Health in the Workplace (MHCC,  2022). 

Addressing Stigma: Towards a More Inclusive Health System

A key informant, who was not available for an interview, sent along the Chief Public Health Officer�s Report on the State of Public Health in Canada 2019, which included an Action Framework for Building an Inclusive Health System (Table 2, p. 41) (Public Health Agency of Canada, 2019). Emphasis is given to the need for training front-line health care providers (i.e., nurses, doctors, allied health care workers, pharmacists, receptionists, etc.) on how to interact with and support persons with disabilities. 

Additional References Introduced by Key Informants

Australia  Government (2021a). Australia Disability Strategy (ADS). Link: https://www.disabilitygateway.gov.au/ads/strategy  
Australia  Government (2021b). Outcomes Framework. Link: https://www.disabilitygateway.gov.au/document/3121
Erickson, William A.; von Schrader, Sarah; Bruy�re, M.; VanLooy, Sara A.; Matteson, S. (2014). Disability-Inclusive Employer Practices and Hiring of Individuals with Disabilities. Link: https://www.ingentaconnect.com/content/springer/rrpe/2014/00000028/00000004/art00007
MHCC (Mental Health Commission of Canada) (2022). 13 Factors: Addressing Mental Health in the Workplace. Link: https://mentalhealthcommission.ca/13-factors-addressing-mental-health-in-the-workplace/
Public Health Agency of Canada (2019). Action Framework for Building an Inclusive Health System. Link: https://www.canada.ca/content/dam/phac-aspc/documents/corporate/publications/chief-public-health-officer-reports-state-public-health-canada/addressing-stigma-what-we-heard/stigma-eng.pdf
QuakeLab (2020). QuakeLab Inclusion Maturity Model. Link: https://quakelab.ca/blog/quakelab-inclusion-maturity-model
The Mindful Employer Canada (2020). The Mindful Employer Canada: Working Towards a Healthy Future. Link: https://www.mindfulemployer.ca/  
Worktopia (2022). Employment Works program. Link: https://worktopia.ca/


Appendix III. Examples for Application of Frameworks

Conceptual Framework for Culture Change Initiatives

Figure AII.1. illustrates the cycle of culture change. This has been published by the UK Cabinet Office back in 2008 to sets out the state of knowledge about �culture change� and a framework to show how this can be practically used to inform policy development. The authors mentioned in understanding behavior we should start by looking at the development of our cultural capital. This arises from our interaction with both the very immediate world around us and broader society-wide forces. Cultural capital forms the behavioral intentions we have in regard to the specific decisions and choices we can make. For example, an individual may have the attitude that it is sensible to remain in education or training after the age of 16; this would form a specific behavioral intention to seek opportunities to do so. In some cases, cultural capital will have a very strong influence on our behavioral intentions, in other cases less so. It is then how these behavioral intentions interact with the incentives, legislation, regulation, and level of information and engagement we face in any given situation that determines our actual behavior. Over time our behavioral patterns regularize into our new default. Over the long term, this feeds back into our cultural capital � for example in forming new attitudes or values about the choices we have experienced.

Figure AII.1. Conceptual framework for culture change initiative/programs

Sources: UK Cabinet Office, 2008.


Results Framework for Improved Quality of Health Services

Figure AII.2. provides an example of a results framework showing results factors that lead to improved quality of health services. As you can see under intermediate results 2 (IR2), the information system, training and supervision of clinicians, and provider performance are factors that lead to improved quality of health services. Notice that intermediate results and sub- intermediate results need to be measurable; in other words, indicators can be developed for them, and data can be collected to calculate them (Menon, 2009).

Figure AII.2. Example of results framework showing results factors that lead to improved quality of health services

Source: Marsh et al., 1999; Frankel et al., 2007.

Logical Framework for an HIV Voluntary Counseling and Testing Program

Figure AII.3 provides another example, is a small portion of a logic model for an HIV voluntary counseling and testing (VCT) program. It is important to remember that, within a program, several activities can have their own inputs and outputs. Collectively, the outputs of the activities contribute to the program�s outcomes and impacts. In some cases, the output of one program activity could be an input for another activity. For example, if an activity is to develop guidelines, the output of that activity is the guidelines, which can then be an input (VCT protocols) in this overall logic model for VCT service delivery. Using frameworks is one way to develop a clearer understanding of the goals and objectives of a project, with an emphasis on identifying measurable objectives, both short-term and long-term (Frankel et al., 2007).
Figure AII.3. Example of a logical framework for a HIV voluntary counseling and testing program


Source: Frankel et al., 2007.


Appendix IV. Levels of Culture Change Initiatives

Public Health Agency of Canada Framework

Figure AIII.1. represents a framework designed by Public Health Agency of Canada for building an inclusive health system. They provide a high-level summary of types of interventions that can be considered when addressing stigma in the health system. This framework provides direction on developing multi-level and comprehensive approaches that respect differences across stigma experiences while offering the potential for lasting effects across stigmas. 

Figure AIII.1. Framework for building an inclusive health system

Source: Public Health Agency of Canada, 2019b.
Cross�s Stigma Intervention Matrix

Figure AIII.2. presents a tool for the planning and management of stigma interventions (Cross et al., 2011). It is a matrix that provides an approach for cross-checking intervention plans with greater precision. The Intervention Matrix incorporates levels of stigma as suggested by McLeroy and colleagues, and the components of stigma as suggested specifically by Link and Phelan, but also indirectly by Sartorius and others (McLeroy et al, 1988; Link et al., 2001; Sartorius, 2002)

Figure AIII.2. Stigma intervention matrix

*Consider at which level of stigma the intervention is intended to have an impact at 
**Consider how the intervention addresses each component of stigma at the level of stigma targeted 

Source: Cross et al., 2011.
Javed�s Framework for Reducing Stigma of Mental Health Disorders 

Figure AIII.3. present stigmatization of mental health could manifests at the level of the individual (intrapersonal), society (interpersonal) and health systems (structural). This is the framework presented by Javed et al., (2021). They explore various aspects of stigma towards mental health with a focus on Low- and middle-income countries (LMICs) and assesses measures to increase help-seeking and access to and uptake of mental health services. Stigma impacts persons living with mental illness, their families and caregivers and healthcare professionals (mental health professionals, non-psychiatric specialists and general practitioners) imparting mental health care.

Figure AIII.3. Multiple levels of stigmatization of mental health

Sources: Javed et al., 2021.


Appendix V. In-depth Review of some Large-scale Culture Change Initiatives

In this section, drawing on previous literature, we provide an in-depth view of some large-scale widely known national-level initiatives aimed to change the culture toward PWDs and to reduce stigma and discrimination. These examples were selected mainly because of data availability regarding the Initiative�s input, activities, output, target population. Following information, we mainly drew on the National Academies of Sciences, Engineering, and Medicine, (2016) and Gaebel et al. (2017).

Opening Minds (Canada)

Overview
� Seeks to change behaviors and attitudes toward people living with mental illness to ensure they are treated fairly and as full citizens
� Established by the Mental Health Commission of Canada (MHCC) 
� Started in 2009 (ongoing) 	
� Grassroots contact-based education programs aimed at:
o youths aged 12-18
o health care providers
o employers and the workforce
o news media
o research and evaluation (Mental Health Commission of Canada, 2009)10

What Did They Do: The commission began Opening Minds with a small, public education media campaign designed to communicate positive messages about people with mental illness. The results were disappointing, and the commission decided against a costly, long-term social media campaign. Instead, the project team issued a request for interest. It was distributed to a wide network of government agencies, universities, stakeholders, and existing grassroots anti-stigma programs across Canada. These initiatives shared one thing in common: they all used some form of contact-based education. The project team linked them with Opening Minds researchers for evaluation and scale-up of effective programs. Work focused on four target groups: youths aged 12-18, health care providers, the workforce or employers, and the news media. Principal investigators were recruited from leading Canadian universities for each target group. The project teams used similar evaluation strategies so that researchers could compare outcomes across settings to help determine which program activities would yield the greatest effects. The goal was to develop effective, evidence-based models that could be replicated and disseminated to other communities and stakeholders who want to begin anti-stigma efforts (National Academies of Sciences, Engineering, and Medicine, 2016).

Examples of Activities: One activity is �HEADSTRONG,� a program targeting youth. This activity brings together youth from local high schools to a regional summit where they participate in exercises, learn about the problems created by stigma, and hear stories from people with lived experiences of mental health problems or mental illnesses. Equipped with toolkits and examples of anti-stigma activities, these students go back to their schools and lead anti-stigma efforts bringing mental health awareness along with messages of hope and recovery. The youth champions are also supported by a coordinator who links them with a coalition of community groups. Another activity was �Understanding Stigma�, an anti-stigma program aimed at health care professionals that emerged as one of Opening Mind's most effective anti-stigma programs. The program comprises a 2-hour workshop that includes six key ingredients such as a PowerPoint show of famous people with mental illness that also functions as an introduction to stigma; a group exercise comparing earaches with depression to illustrate the need for timely treatment and social support; a short discussion of the definition of stigma as a form of prejudice and discrimination; along with locally made films, myth-busting (countering myths about mental illness), and a keynote speech by a person with mental illness that engenders discussion among participants. Workshops were originally developed for use by emergency room staff, but they were later adapted for other groups. The program objectives are to raise awareness among health professionals of their own attitudes; to provide them with an opportunity to hear personal stories of mental illness, hope, and recovery from people with mental illness; and to demonstrate that health care providers can make a positive difference. The program also includes pre- and posttests as well as take-home resources and the opportunity to sign an anti-stigma commitment. Opening Minds has also produced a guide for media reporting on mental health. Mindset: Reporting on Mental Health includes sections that help journalists distinguish among various mental disorders (stressing that mental illness is a broad category and reporting should specify diagnoses), and guidelines for interviewing people with and about mental illness, and appropriate language to use when reporting on mental illness, suicide, and addiction (National Academies of Sciences, Engineering, and Medicine, 2016).

Time to Change (England)

Overview
� A social movement to change the way people think and act about mental illness
� Co-funded by Comic Relief, the Department of Health and the Big Lottery Fund
� Started in 2007 and ended in March 2021
� Intervention components:
o Social marketing and mass media activity
o Local community events to bring people with and without mental illness together
o A grant scheme to fund grassroots projects led by people with mental illness
o A program to empower a network of people experiencing mental illness to challenge discrimination
o Targeted work with stakeholders to improve practice and policy (Henderson et al., 2016; 2020)11

What Did They Do: Time to Change is England's largest-ever program to reduce stigma and discrimination against people with mental health problems. The project began in 2008 and is ongoing. funding covered the development and implementation of the anti-stigma activities, as well as evaluation activities, including the collection of nationally representative baseline data and follow-on surveys of the English population from which progress could be measured in the future. Between 2008 and 2015, the project received �40 million ($60 million US) to design and deliver a multiphase, multifaceted campaign that included:
* social marketing and mass media activity at the national level to raise awareness of mental health issues;
* local community events to bring people with and without mental health problems together;
* a grant program to fund grassroots projects led by people with mental health problems,
* a program to empower a network of people with experience of mental health problems to challenge discrimination,
* targeted work with stakeholders, for example, medical students, teachers in training, employers, and young people.
* Funding also allowed the campaign to do formative research during the first year involving more than 4,000 people with direct experience of mental health problems to provide input on stigma and discrimination and specific targets for change, which then guided the campaign (National Academies of Sciences, Engineering, and Medicine, 2016).

Examples of Activities: Based on insight from the developmental phase, the mass media campaign (including national television, print, radio, and outdoor and online advertisement and social media as well as cinema) targeted specific groups of individuals. The film Schizo,9 one component of the national-level campaign, was shown in movie theatres across the country, and later adapted for use in the United States. Nationally representative surveys of the general public concerning knowledge, attitudes, and behavior in relation to people with mental health problems were used to assess change over time. At the community and grassroots levels, the project included varied activities based on the theme �start a conversation.� Community-level social contact included �Living Libraries� where, instead of borrowing only books, library visitors could borrow a person and hear about firsthand experiences of stigma discrimination from those with lived experience of mental illness. Data were collected at the community level during these social contact events in different cities across England to assess the relationship between the quality of the social contact and intended stigmatizing behavior and campaign engagement. Grassroots-level components also included volunteer-led activities (contact-based and peer-service programs) at college campuses and other public places that provided data on the impact of disclosure of mental or substance use disorders on self-stigma and the sense of well-being and empowerment, again through the use of validated tools (National Academies of Sciences, Engineering, and Medicine, 2016).

Beyondblue (Australia)

Overview
* Improve the knowledge and skills of primary care practitioners and other health professionals regarding mental health problems 
* Largely funded by the Australian national government and some of the territorial (state) governments, with some financial and in-kind support from nongovernmental source
* Started in 1990s. The organization began its work in 2000 as a 5-year initiative yet it continues.
* Intervention components:
o Increasing awareness of depression and anxiety
o Reducing stigma and discrimination
o Improving help-seeking for people with depression and anxiety (and more
o recently, for those at risk of suicide)
o Reducing the impact and disability associated with these conditions (Beyondblue, 2000)12

What Did They Do: beyondblue is an Australian not-for-profit organization that began as �beyondblue: the national depression initiative� but now addresses both depression and anxiety. The initiative aimed to improve the knowledge and skills of primary care practitioners to address mental health problems. The goal of the mental health literacy campaign was to raise awareness of the importance of the public's knowledge, beliefs, and skills related to mental disorders, including prevention and treatment. There were five priority areas: community awareness and de-stigmatization, consumer and caregiver support, prevention and early intervention, primary care training and support, and applied research. Information was disseminated and messages conveyed over multiple media platforms, including television, radio, the internet, and print media. beyondblue partnered with an organization called Schools Television to raise awareness and provide information about mental illness and engaged well-known actors to talk openly about their personal experiences with mental illness. The activities are largely funded by the Australian national government and some of the territorial (state) governments, with some financial and in-kind support from nongovernmental sources (National Academies of Sciences, Engineering, and Medicine, 2016).

Examples of Activities: There were many varied activities including mass-media advertising, sponsorship of events, community education programs, training of prominent people as champions, and web and print information. Mental Health First Aid training was developed in Australia in 2000 by Betty Kitchener starting as a small volunteer effort that has now been replicated in many other countries. Other prominent interventions included Mind Matters, programs in high school that are incorporated into regular lessons; RUOK Day�people ask others about their mental well-being �Are you OK?�; Rotary community forums on mental illness across the country that involve elected officials and average citizens; and Mindframe, a national media initiative that includes training programs and guidelines for responsible reporting about suicide. The campaign also provides funding to initiate and continue research on depression and anxiety, and over the course of the campaign, the funded research activities have grown in number and been more aligned with stakeholder-identified priorities (National Academies of Sciences, Engineering, and Medicine, 2016).

Like Minds, Like Mine (New Zealand)

Overview
� One of the first comprehensive national campaigns in the world to counter stigma and discrimination associated with mental illness
� Funded by the Ministry of Health and Health Promotion Agency
� Started in 1997 (ongoing), with multiple National Plans including First (1999�2001), Second (2001�2003), Third (2003�2005), Fourth (2007�2013), Fifth (2014�2019)
� Intervention components:
o Changing public attitudes through advertising campaigns, radio shows, creative art initiatives
o Changing behaviors through educational workshops
o Changing policy and Law
o Empowering people with lived experience to challenge stigma and discrimination (Likeminds, 2016)13

What Did They Do: Like Minds, New Zealand�s program to combat stigma and discrimination, has now been in existence for 17 years. It has not been a straightforward journey but one full of twists and turns, changes in focus and direction. The underlying philosophy of the program has drawn on health promotion and public health, on human rights and the disability movement, and the voice of lived experience. Like Minds has been a program of social change, and social change evolves over time and requires commitment. Social change does not always sit easily with being a government funded program sitting within the health sector whose core business is the provision of health services. However, Like Minds has been responsible for changes in the way people with experience of mental illness are viewed, and changed the New Zealand social environment for the better. The work of social change is not complete, and there remains a place for concerted and coordinated efforts focused specifically on addressing and reducing discrimination associated with mental illness (Gaebel et al., 2017).

Examples of Activities: Like Minds has aligned its work to the strategies of contact, protest and education, with a strong focus on contact as the central and most effective strategy for change. These strategies have been used to work towards four key outcomes: changing public attitudes towards people with mental illness, changing the behavior of those in most contact with people with experience of mental illness, changing policy and law to combat discrimination and empowering people with experience of mental illness to challenge discrimination and self-stigma (Gaebel et al., 2017).
Appendix VI. Initiatives Aimed at Changing Attitudes Toward Persons with Disabilities in Canada

Figure AV.1. Initiatives aimed at changing attitudes toward persons with disabilities in different provinces across Canada
Province
Initiative/ program
Organizer(s)
Link 
British Columbia
Disability Employment Awareness Month (DEAM)
Provincial government
https://news.gov.bc.ca/releases/2020SDPR0043-001615 
British Columbia
Diversity Includes
Inclusion BC
https://inclusionbc.org/our-campaigns/diversity-includes/ 
British Columbia
COVID Disability
Inclusion BC
https://inclusionbc.org/our-campaigns/covid19disability/ 
British Columbia
Kids Can�t Wait 
Inclusion BC
https://inclusionbc.org/our-campaigns/kids-cant-wait/
British Columbia
All Students Belong 
Inclusion BC
https://inclusionbc.org/our-campaigns/inclusive-education/
British Columbia
Inclusive Housing for All
Inclusion BC
https://inclusionbc.org/our-campaigns/inclusive-housing-for-all/
British Columbia
Ending Poverty
Inclusion BC
https://inclusionbc.org/our-campaigns/ending-poverty/ 
British Columbia
campaigns to raise awareness, including 
- ACCESS AWARENESS DAY
- Say Yes to Access
SPARC BC
https://www.sparc.bc.ca/areas-of-focus/accessibility-and-inclusion/campaigns/ 
British Columbia
Different activity including events
Disability Alliance BC
https://www.facebook.com/DisabilityAllianceBC/ 
British Columbia
GET LOUD Campaign
Canadian Mental Health Association
https://www.getloudbc.ca/

Ontario
Disability Employment Awareness Month (DEAM)
Community Living Toronto
https://cltoronto.ca/october-is-disability-employment-awareness-month-deam/ 
Ontario
NA (New Campaign)
Government 
https://www.ontario.ca/page/ministry-community-safety-and-correctional-services-2016-accessibility-report 
Ontario
Learning Disability (LD) Awareness month
Learning Disabilities Associations of Ontario
https://www.ldao.ca/ld-awareness-month-october-2021-press-kit/ 
Alberta
Disability Employment Awareness Month (DEAM)
Government 
https://www.deamalberta.com/

Alberta
Disability Employment Awareness Month (DEAM)
City of Calgary
https://www.calgaryemploymentfirst.ca/deam/ 
Alberta
ACCDs Campaign for Barrier Free Health and Medical

http://www.ccdonline.ca/en/socialpolicy/access-inclusion/ACCD-Video
Saskatchewan
Disability Employment Awareness Month (DEAM)
Provincial government
https://www.saskatchewan.ca/government/news-and-media/2016/october/01/province-declares-disability-employment-awareness-month-in-october 
Quebec
Quebec's disability awareness
Association Lavalloise du Transport Adapt� (ALTA) and Soci�t� de transport de Laval (STL)
https://www.newswire.ca/news-releases/quebec-week-for-disabled-persons-stl-to-provide-accessible-transportation-to-disabled-women-fleeing-domestic-violence-865585423.html 
Quebec
Qu�bec Week for Disabled Persons
Office des personnes handicap�es du Qu�bec�s
https://www.curateur.gouv.qc.ca/cura/en/outils/nouvelles/20190603b.html
Quebec
Quebec Intellectual Disability Week
Quebec
https://www.sqdi.ca/en/quebec-intellectual-disability-week/#:~:text=The%20next%20Quebec%20Intellectual%20Disability, March%2020%20to%2026%2C%202022. 
Quebec
Skills Link program
FNHRDCQ
https://www.cdrhpnq-fnhrdcq.ca/inclusion?lang=en 
Manitoba
Disability Matters Vote (DMVote)
DMVote
https://www.dmvote.ca/ 
Manitoba
initiative of Abilities Manitoba
Abilities Manitoba
https://abilitiesmanitoba.org/ 
Manitoba
Barrier Free Manitoba
Barrier Free Manitoba
https://www.barrierfreemb.com/home 
Manitoba
MarketAbilities Campaign
Provincial government 
https://news.gov.mb.ca/news/print,index.html?archive=&item=4896 
Nova Scotia
Disability Employment Awareness Month (DEAM)
Provincial government 
https://www.halifax.ca/about-halifax/diversity-inclusion/national-employer-disability-awareness-month 
Nova Scotia
Access Awareness Week (AAW) in Nova Scotia
Partnership for Access Awareness Nova Scotia (PAANS)
https://www.halifaxpubliclibraries.ca/blogs/post/access-awareness-week-2021-a-virtual-celebration/ 
Nova Scotia
Paper Egg Campaign
More than able 
https://www.easterseals.ns.ca/2020/03/papereggs2020/ 
Newfoundland and Labrador
Learning Disabilities Awareness Month
Provincial government
https://www.gov.nl.ca/releases/2021/education/1013n03/ 
Newfoundland and Labrador
I Can Too!
Eastersealsnl
https://eastersealsnl.ca/what-we-do/career-and-educational-services/i-can-too/ 
Newfoundland and Labrador
'Be Kind' campaign

https://www.saltwire.com/nova-scotia/news/be-kind-campaign-raises-awareness-about-disabled-people-in-newfoundland-and-labrador-who-cant-wear-a-mask-506960/ 
New Brunswick
Disability Awareness Week
Provincial government 
https://www2.gnb.ca/content/gnb/en/departments/pcsdp/daw2021.html 
New Brunswick
Summer Awareness Campaign
Learning Disabilities Association of New Brunswick
https://www.ldanb-taanb.ca/ 
New Brunswick
HIRE for TALENT Awareness Campaign
Hirefortalent
https://hirefortalent.ca/main/toolkit/customized-employment/2-general 
Prince Edward Island
Disability Employment Awareness Month (DEAM)
Provincial government 
NA
Prince Edward Island
Easter Seals Society of PEI
Rotary Club of Charlottetown in partnership with the Rotary Club of Montague and the Rotary Club of Summerside.
https://eastersealspei.org/about-us/#:~:text=Rotary%20Club%20of%20Charlottetown%20in%20partnership%20with%20the%20Rotary%20Club%20of%20Montague%20and%20the%20Rotary%20Club%20of%20Summerside. 


Appendix VII. Identified Measurement Tools for Attitudes Towards Persons with Disabilities

Table AVII.1. Measurement tools for attitudes towards persons with disabilities
Author(s)
Year
Measurement tools title
Objective
Country
Type of disability
Nota et al.
2014
Work for people with disability questionnaire
Employer attitudes towards work inclusion of persons with disabilities
Italy
All
Krah� et al.
2006
Questionnaire about Attitudes towards the Physically Disabled14
Attitudes towards persons with physical disabilities
Germany
Physical
Bunch et al.
2004
Guide questions framed interviews
Student attitudes toward peers with disabilities in inclusive and special education schools
Canada
All
Chadd et al.
2011
University of California, Los Angeles Geriatrics Scale
Attitudes towards PWDs in healthcare sector 
USA
All
Chadd et al.
2011
Maxwell and Sullivan Attitude Scale (MSAS)
Attitudes towards PWDs in healthcare sector
USA
All
Tait et al.
2000
Disabled Persons Scale (IDP)
Attitudes toward disability
Australia
All
Yazbeck et al.
2004
The Scale of Attitude Toward Mental Retardation,
The Mental Retardation Attitude Inventory,
The Community Living Attitudes Scale Mental Retardation
Attitudes toward person with intellectual disabilities
Australia
Mental
Tervo et al.
2002
Attitude Toward Disabled Persons (ATDP) Scale
Attitudes toward PWDs 
USA
All
Tervo et al.
2002
Scale of Attitudes Toward Disabled Persons (SADP)
Attitudes toward PWDs
USA
All
Tervo et al.
2002
Rehabilitation Situations Inventory (RSI)
Perception of the difficulty of typical rehabilitation situations
USA
All
Power et al.
2009
Attitudes To Disability Scale (ADS)
Assess the personal attitudes of individuals with either physical or intellectual disabilities 
UK
Mental
Rillotta et al.
2007
Attitudes Toward Persons with an Intellectual Disability Questionnaire (ATTID)
Unidimensional scale measuring student's attitudes toward persons with intellectual disabilities
Australia
Mental
Antonak et al.
1994
Mental Retardation Attitude Inventory-Revised (MRAI-R)
Multidimensional scale composed of 4 subscales to measure attitudes toward individuals with mental retardation
USA
Mental
Tachibana et al.
2004
Tachibana and Watanabe's scale
Attitudes of Japanese adults toward persons with intellectual disability: Relationship between
Japan
Mental
Nowicki et al. 
2005
Pictographic Scale
Measures affective aspects of attitudes by tapping into children's feelings about a target child
Canada
Mental
Akrami et al. 
2006
Modern and Classical Attitudes Scales Toward People with intellectual disabilities
Attitudes toward people with intellectual disabilities 
Sweden
Mental
Findler et al.
2007
Multidimensional Attitudes Scale Toward Persons with Disabilities (MAS)
Develop scale toward persons with disabilities
Israel
Mental
Morin et al.
2013
Attitudes Toward Intellectual Disability Questionnaire (ATTID)
New questionnaire for tapping into the general population's attitudes toward individuals with intellectual disability
Canada
Mental
Henderson et al.
2020
Mental Health Knowledge Schedule (MAKS)
An instrument to assess stigma-related mental health knowledge among the general public
England
Mental
Kohrt et al.
2020
Social Distance Scale (SDS)
To measure people's willingness to participate in social contacts of varying degrees of closeness to PWDs 
Nepal
All
Rossetto et al.
2020
Mental Health Knowledge Schedule (MAKS), 
Community Attitudes Toward the Mentally Ill (CAMI) scale, 
Intended Behavior Scale (RIBS)
Perceptions of positive treatment and discrimination toward people with mental health problems
England
Mental
Sarge et al.
2020
Cohen�s Identification scale, 
Attribution Questionnaire (AQ), 
Societal Attitudes Towards Autism (SATA),
Helping Behavioral Intention scale
Public stigma with virtual simulations
US
Mental
Mittal et al. 
2020
Attribution Questionnaire (AQ)
Stigma of mental illness among primary care providers
US
Mental
Higuera-Romero et al.
2020
Mental Health Stigmatization Scale (PMHSS-24)
Stigmatizing attitudes towards mental health
Spain
Mental
Mac Carthaigh et al.
2020
Autism Survey Questionnaire (ASQ),
Openness Scale (OS),
Frequency of Contact Scale (FCS),
Adult Autism-spectrum Quotient
Attitudes towards autism
England
Mental
Shahwan et al.
2020
Attitudes towards Seeking Mental Health Services (IASMHS)
Stigma toward people with mental disability
Singapore
Mental
Abuhammad et al.
2021
Attitudes Toward Seeking Professional Psychological Help (ATSPPH) Scale
To measure general attitudes toward seeking professional psychological help
Jordan
Mental
Goh et al.
2021
Community Attitudes towards Mental Illness (CAMI) scale
To evaluate attitudes towards depression
Singapore
Mental
Gordon et al.
2021
Opening Minds Stigma Scale for Health Care Providers (OMS-HC-20)
To measure stigma in health care provider populations
New Zealand
Mental
Kalargyrou et al.
2021
Disability Social Relationship Scale (DSR)
To measure attitudes toward PWD
Austria
All
Kim et al.
2021
Dementia knowledge assessment scale (DKAS)
To measure of dementia knowledge for diverse populations 
Australia
Mental
Kumar et al.
2021
California Assessment of Stigma Change Scale (CASC)
attitude and stigma towards mental illness
India
Mental
Magliano et al.
2021
Employees� Views About Persons with Mental Disorders Questionnaire (EVMD)
employees' views about persons with mental disorders as potential workmates
Italy
Mental
Newton-Howes et al.
2021
Recovery Attitudes Questionnaire (RAQ), Opening Minds (OM) Scale
Attitudes towards recovery and less stigmatizing attitudes towards people with mental distress
New Zealand
Mental
Reddyhough et al.
2021
Implicit Association Task (IAT), 
Social Distance Scale (SDS)
To measure Implicit and Explicit Stigma
Australia
Mental
Rodr�guez-Rivas et al.
2021
Questionnaire on Student Attitudes, Toward Schizophrenia (QSAS),
Attribution Questionnaire (AQ-27)
Stigma toward mental illness
Spain
Mental
Appendix VIII. Summary of Identified Literature

Scoping Review of Publication from 2020-2021

Table AVIII.1. provides a high-level summary of the 58 articles identified through the journal database in scoping review. Specifically, data on region, study type, policy type, policy level, target population and type of disability are presented. Studies identified provide insights into areas of relevance in terms of culture change in relation to inclusion and accessibility. Following are key summary points from the identified studies:
> 	Among the 58 articles, most are primary data collection and analysis studies (45), several are literature reviews, specifically simple reviews (3), scoping reviews (5), systematic reviews (3), and a couple are about the development of a measurement tools for estimating stigma toward PWDs (2).

> 	The articles report on studies that have been implemented in different countries, including US (14), Australia (6), Canada (8), England (7), Germany (2), India (2), Italy (2), Japan (1), Austria (1), Jordan (1), Malaysia (1), Nepal (1), Netherlands (1), New Zealand (2), Pakistan (1), Saudi Arabia (1), Singapore (2), South Africa (1), Spain (2), Sri Lanka (1), Zambia (1).

> 	Most studies are about persons with mental health conditions and cognitive/learning disabilities (52) (i.e., general mental health (30), autism (6), dementia (6), aphasia (1), autism (1), depression (1), developmental disabilities (1), schizophrenia (3), Alzheimer (2), and multiple sclerosis (1)�5 studies address other types of disabilities and only 1 study specifically address physical disabilities.

> The articles report on studies that target a wide variety of populations, including the general public (11), university students (11), PWDs (7), healthcare professionals (6), students (5), children (2), Caffe staff (1), caregivers (1), community pharmacists (1), employees (3), library staff (1), low- and middle-income countries (2), police staff (1), managers (1), parents (1), Asian Torontonians (1), African Americans (1), Asian Americans (1), and international NGOs (1).

> 	Some studies (either primary analysis or review studies) evaluated the impact of interventions (e.g., interventions to reduce stigma, improve attitudes) (37), whereas others were descriptive, reporting on the results from surveys about attitudes towards PWDs (21).

> Studies that evaluated impacts of interventions considered a wide variety of interventions, including awareness campaigns (1), combining information with imagined contacts (1), continuum interventions (1), educational interventions (17), environmental factors (1), models of practice (1), school-based schizophrenia stigma interventions (1) workplace accommodations (1), cartoons which depict autistic characters (1), contact and educational interventions (1), face-to-face programs, online resources, awareness campaigns, and advocacy work (1), exposure interventions (1), active involvement of people with dementia, environmental design, public education, raising awareness and customized strategies (1), virtual simulation engagement and observation of simulation engagement (1), job creation programs (1), contact interventions (1), culture in programs (1), dementia friendly communities (1), acceptance and commitment therapy, contact-based empowerment education and mixed interventions (1), multi-component programs (1), inclusive mental health policies and legislations, awareness programs, capacity building of mental health workforce and improved access to mental health services (1).

> Several measurement tools, such as questionnaires and interviews, have been employed in order to evaluate the target outcome of studies. Evaluations of anti-stigma interventions typically assess knowledge and attitudes and rarely measure behavior toward PWDs. Although some studies assess social distance, prejudice, stereotyping, ignorance and misunderstanding toward PWDs. Most of the studies employed pre standardized questionnaires, while some has developed their own measurement tools.

> 	Overall, identified articles provide invaluable insights into areas of relevance for culture change theory, frameworks, initiatives and indicators.

Snowballing Search of Peer-Reviewed literature and Environmental Scan

A high-level summary of all other identified resources through the snowballing search of peer-reviewed literature and environmental scan are available in attach excel file.


Table AVIII.1. High-level summary of the 58 articles identified through the journal database search for 2020-2021
Country (Region)
Study Type
Policies (includes program and various types of intervention)
Target Population
Type of Disability
US (14), Australia (6), Canada (8), England (7), Germany (2), India (2), Italy (2), Japan (1), Austria (1), Jordan (1), Malaysia (1), Nepal (1), Netherlands (1), New Zealand (2), Pakistan (1), Saudi Arabia (1), Singapore (2), South Africa (1), Spain (2), SriLanka (1), Zambia (1)
Primary data collection (45), Review (3), Scoping review (5), Systematic review (3), measurement tools development (2)
Awareness campaigns (1), Combining information with imagined contact (1), Continuum intervention (1), Educational intervention (17), Environmental factors (1), Model of Practice (1), School-based schizophrenia stigma interventions (1) Workplace accommodations (1), Cartoons which depict autistic characters (1), Contact Intervention, Educational intervention (1), Face-to-face programs, Online resources, Awareness campaigns, Advocacy work (1), Exposure intervention (1), Active involvement of people with dementia; Environmental design, Public education; Raise awareness; Customized strategies (1), Virtual simulation engagement; Observation of simulation engagement (1), Job creation programs (1), Contact intervention (1), Culture in programs (1), Dementia friendly community (1), Acceptance and Commitment Therapy; Contact-based Empowerment Education; combined (1), Multi-component program (1), Inclusive mental health policies and legislations; Awareness programs; capacity building of mental health workforce; improved access to mental health services (1), NA (21)
General Public (11), University Students (11), PWDs (7), Healthcare Professionals (6), Students (5), Children (2), Caffe Staff (1), Caregivers (1), Community Pharmacists (1), Employees (3), Library Staff (1), Low- and Middle-Income Countries (2), Police Staff (1), Managers (1), Parents (1), Asian Torontonians (1), African American (1), Asian American (1), International NGOs (1)
Health (30), Autism (6), Dementia (6), Aphasia (1), Autistic (1), Depression (1), Developmental Disabilities (1), Schizophrenia (3), Alzheimer (2), Multiple Sclerosis (1), Physical Disabilities (1), (2), All Types (5)

Table AVIII.2. Identified literature from journal database search for 2020-2021
No
Authors
Year
Country 
Study Type
Type of Initiative
Context
Target population
Type of Disability
Objective
Target outcomes 
Key indicators15
1
Abuhammad S.
2021
Jordan
Study of initiative, program, or intervention
Educational intervention
University
University Students
Mental Health
to compare the attitudes of nursing students in Jordan toward seeking professional psychological help, before and after taking part in a mental health course.
Attitudes Toward Seeking
Professional Psychological Help subscales
Attitudes Toward Seeking Professional Psychological Help
(ATSPPH) Scale
2
Algahtani.
2020
Saudi Arabia
Descriptive study of a beliefs, values, attitudes and behaviours, etc. toward PWD among different populations
NA
NA
General population
Alzheimer
to determine the public perception and attitude towards Alzheimer�s disease
Perception of the general public
Self-administered questionnaire with 40 items
3
Beamish W.
2021
Australia
Study of initiative, program, or intervention
Model of Practice
Schools
Students 
Autism
investigated the experiences and perceptions of Australian mainstream teachers who field-tested a validated Model of Practice designed to support their daily work with young schoolaged
students on the spectrum.
teacher perceptions of knowledge, confidence, and self-efficacy
Semi-structured interviews
and surveys
4
Borsatto J.
2021
Canada
Study of initiative, program, or intervention
Educational intervention
Workplace
Employees
Aphasia
to increase awareness of aphasia in businesses and to test the efficacy of our approach.
employees� declarative knowledge regarding
aphasia and their perceived self-efficacy in the workplace
Occupational Self-Efficacy Scale
5
Buckwitz V.
2021
US
Study of initiative, program, or intervention
Continuum intervention
NA
General population
Depression
We examined the impact of the continuum perspective on mental illness stigma (social distance, negative stereotypes) regarding MDD, hypothesizing that an increased endorsement of the continuum perspective would be associated with lower levels of stigma
stigma toward individuals with depression, continuum beliefs
Continuum-beliefs measure and stigma-related
questionnaires
6
Eldridge-Smith ED.
2021
US
Descriptive study of a beliefs, values, attitudes and behaviours, etc. toward PWD among different populations
NA
NA
Person with disabilities
Multiple Sclerosis
to adapt the Reece Stigma Scale for use with this specific population and examine its reliability
and validity
adherence, depression, anxiety, quality of life, self-efficacy, and post-traumatic
growth
Reece Stigma Scale
7
Engel CS.
2020
England
Study of initiative, program, or intervention
Cartoons which depict autistic characters
NA
Children
Autistic
to assess the efficacy of two cartoons which depict autistic characters in improving attitudes towards autistic
peers in two separate studies
attitudes and knowledge about autism
Chedoke-McMaster Attitudes Towards Children
with Handicaps
8
Epps F.
2021
US
Study of initiative, program, or intervention
Educational intervention
NA
African American
Dementia
to identify the perceptions and attitudes of African American congregants toward dementia
before and after attending a dementia-focused workshop
perceptions and attitudes
toward dementia
Free association assessment exercises
9
Farina N.
2020
England
Study of initiative, program, or intervention
Educational intervention
Schools
Students 
Dementia
to explore adolescents' reaction to of a Dementia Friends information session and how it affected their perceptions of dementia.
raise awareness of dementia
Dementia scale
10
Flanigan LK.
2020
Canada
Literature review/evidence synthesis studies
School-based schizophrenia stigma interventions
Schools
Students 
Schizophrenia
to review and critically analyze the most recent (2003-present) school-based schizophrenia stigma interventions, with seven studies identified
review
review
11
Fung K.
2021
Canada
Study of initiative, program, or intervention
Acceptance and Commitment Therapy; Contact-based Empowerment Education;  combination of these two
Population
Asian Toronto
Mental Health
evaluates three intervention approaches to reduce stigma of mental illness among Asian men
in Toronto, Canada
reduce stigma of mental illness
Self-report measures on stigma (CAMI, ISMI) and social change (SJS)
12
Gaebel W.
2020
Germany
Study of initiative, program, or intervention
Educational intervention
Healthcare
Person with disabilities
Schizophrenia
to promote stigma coping and empowerment (STEM) was developed and tested for efficacy in patients with schizophrenia or depression
Primary outcome was improvement in quality of life (QoL), secondary outcome measures (symptoms, functioning, compliance, internalized stigma, self-esteem, empowerment)
quality of life (WHO-QOL-BREF), internalized stigma (ISMI), empowerment (BUES), selfesteem
(Rosenberg SES), and general symptoms
(SCL-27)
13
Gallego J.
2020
Canada
Descriptive study of a beliefs, values, attitudes and behaviours, etc. toward PWD among different populations
NA
NA
Students 
Mental Health
analyzes the existing stigma among University Education students in three
countries with different education systems and cultures, namely Spain, Russia, and
Canada
stigma
Questionnaire on Students� Attitudes toward Schizophrenia
(QSAS)
14
Ginevra MC.
2021
Italy
Study of initiative, program, or intervention
Combining information with imagined contact
Public
Children
All Types
examined the effectiveness of an intervention providing information about peers with sensory disability, intellectual disability, and behavioral difficulties (cognitive intervention), an intervention using imagined contact with peers with these disabilities 
attitudes; stereotypes; feelings toward and intentions to engage in contact with peers with disabilities; Behavioral Intentions
Questionnaire developed by Nota et al.,
15
Goh CMJ.
2021
Singapore
Study of initiative, program, or intervention
Educational intervention
University
University Students
Mental Health
to evaluate the effects of an anti-stigma intervention on attitudes towards depression in university students
attitudes
Community Attitudes towards Mental Illness (CAMI) scale 
16
Gordon SE.
2021
New Zealand
Study of initiative, program, or intervention
Educational intervention
University
University Students
Mental Health
To assess and compare attitudes of medical students in response to two service-user-led anti-stigma and discrimination education programmes
Attitudes
Opening Minds Stigma Scale for Health Care Providers (OMS-HC-20) 
17
Hapuarachchi RB.
2021
Sri Lanka
Descriptive study of a beliefs, values, attitudes and behaviours, etc. toward PWD among different populations
NA
NA
Caregivers
Schizophrenia
to describe perceived stigma among care givers of patients with schizophrenia attending outpatient clinics of NIMH
stigma
Self-administered 10-item questionnaire
18
Hearst MO.
2020
Zambia
Study of initiative, program, or intervention
Educational intervention
Schools
General population
All Types
to reduce stigma towards children with disabilities living in family-based care
reduce community-level stigma towards children with disabilities
Kusamala?was a feasible, acceptable and broad reach with limited program impact.
19
Henderson C.
2020
England
Study of initiative, program, or intervention
Educational intervention
Public
General population
Mental Health
This study investigates changes in stigma-related public knowledge, attitudes and desire for social distance in England since Time to Change began in 2008G��19 and for 2017G��19
measure of attitudes; measures of knowledge and desire for social distance 
Mental Health
Knowledge Schedule (MAKS), Community Attitudes towards the Mentally ill scale (CAMI), Star Social Distance Scale
20
Hunter SW.
2020
England
Descriptive study of a beliefs, values, attitudes and behaviours, etc. toward PWD among different populations
NA
NA
Healthcare Professionals
Dementia
 to assess personal, educational, and clinical experiences on physiotherapists'  attitudes toward working with people with dementia
dementia knowledge, confidence, and attitudes
Questionnaire covered seven domains
21
Janssens KME.
2021
Netherlands
Descriptive study of a beliefs, values, attitudes and behaviours, etc. toward PWD among different populations
NA
NA
Managers
Mental Health
to evaluate line managers' knowledge and attitudes concerning job applicants with mental health problems (MHP), and to assess which factors are associated with the intention (not) to hire an applicant with MHP
knowledge, attitudes and experiences  concerning applicants/employees with MHP
Questionnaire  developed fitting the purpose of this study
22
Javed A.
2021
Pakistan
Literature review/evidence synthesis studies
Inclusive mental
health policies and legislations; Awareness programs; capacity building of
mental health workforce;  improved access to mental
health services
Public
Low- and middle-income countries
Mental Health
This paper explores various aspects of stigma towards mental health with a focus on LMICs and assesses measures to increase help-seeking and access to and uptake of mental health services.
review
review
23
Jones SC.
2021
Australia
Descriptive study of a beliefs, values, attitudes and behaviours, etc. toward PWD among different populations
NA
NA
General population
Autism
explored community attitudes to autism in Australia; and autistic people and their families' perspectives of community attitudes
community attitudes to autism
Questionnaire  developed fitting the purpose of this study
24
Kalargyrou V.
2021
Austria
Descriptive study of a beliefs, values, attitudes and behaviours, etc. toward PWD among different populations
NA
NA
University Students
Physical disabilities
First, the study seeks to examine how social context (i.e., work, dating, marriage) influences attitudes toward people with physical disabilities (PWPDs). Second, it examines potential influence of participant characteristics (i.e., gender, major, disability education, and frequency of contact). Third, it explores whether attitudes vary across different disability types.
attitudes toward people with physical disabilities
Disability Social Relationship Scale (DSR)
25
Kim S.
2021
Australia
Study of initiative, program, or intervention
Educational intervention
Public
General population
Dementia
To examine the efficacy of the Dementia Stigma Reduction (DESeRvE) programme, aimed at reducing the general public dementia-related stigma utilising 'education' and 'contact' approaches.
dementia-related stigma
Dementia Knowledge Assessment Scale 
26
Kohrt BA.
2020
Nepal
Study of initiative, program, or intervention
Contact intervention
Healthcare
Healthcare Professionals
Mental Health
We employed the �what matters most� conceptual framework and findings from social psychology to design an intervention for reducing
primary care providers stigma against patients with mental illness. We conducted a proof-of-concept testing in rural Nepal including qualitative and quantitative evaluations.
stigma against persons with mental illness
Social Distance Scale (SDS), knowledge assessment, attitudes assessment, ENhancing Assessment of Common Therapeutic factors (ENACT)
27
Kumar P.
2021
India
Study of initiative, program, or intervention
Educational intervention
School
Students 
Mental Health
aimed at a change in attitude and reducing stigma related to mental illness through education
attitude towards mental illness
California Assessment of Stigma Change Scale
28
Liang J.
2020
US
Descriptive study of a beliefs, values, attitudes and behaviours, etc. toward PWD among different populations
NA
NA
Asian American
Alzheimer
study examined the prevalence and predictors of three stigmatising beliefs about Alzheimer's Disease (AD) (beliefs that associate AD with a normal process of ageing, family embarrassment and social avoidance) using a sample of Asian Americans
Stigmatising beliefs about Alzheimer's Disease (AD)
Self made questions
29
Magliano L.
2021
Italy
Study of initiative, program, or intervention
Educational intervention
Workplace
Employees
Mental Health
study explored whether the participation of people with MDs in a job-training course would positively influence employees' opinions about workmates with these disorders
employees' opinions about workmates with these disorders
Employees� Views About Persons with Mental Disorders Questionnaire (EVMD)
30
Mascayano F.
2020
US
Literature review/evidence synthesis studies
Culture in programs
Healthcare
Low- and middle-income countries
Mental Health
how much stigma reduction interventions are effective and are culturally sensitive
review
review
31
Mittal D.
2020
US
Study of initiative, program, or intervention
Contact Intervention,  Educational intervention
Healthcare
Healthcare Professionals
Mental Health
our primary aim is to test the feasibility, and preliminary implementation of two interventions (contact and education) to help change provider attitudes and behavior.
Stigmatizing attitudes among primary care
Opening Minds Scale, Attribution Questionnaire, Social Distance Scale
32
Morgan AJ.
2021
Australia
Literature review/evidence synthesis studies
Face-to-face programs, Online resources, Awareness campaigns, Advocacy work 
Varies
Healthcare Professionals
Mental Health
 To inform implementation of this policy, this review aimed to identify and examine the effectiveness of existing Australian programs or initiatives that aim to reduce stigma and discrimination.
review
review
33
Murney MA.
2020
Canada
Descriptive study of a beliefs, values, attitudes and behaviours, etc. toward PWD among different populations
NA
Healthcare
Healthcare Professionals
Mental Health
 examines health providers' understandings and experiences regarding stigma towards mental health and substance use (MHSU) issues, as well as their ideas for an effective intervention to address stigma and discrimination, in three CHCs in Toronto, Ontario
experiences regarding stigma towards mental health and substance use
interviews
34
Beaulieu M.
2020
US
Literature review/evidence synthesis studies
NA
NA
University Students
All Types
This study provides a comprehensive review of the published literature on this topic, which summarizes the clinical and practice relevance of the attitudes held by preprofessional students in the United States. 
review
review
35
Bell S.
2021
England
Descriptive study of a beliefs, values, attitudes and behaviours, etc. toward PWD among different populations
NA
NA
Police staff
Mental Health
 to quantifiably measure police attitudes to mental ill health and benchmark them against the public alongside qualitative data gleaned from survey responses and interviews. 
�
�
36
Higuera-Romero J.
2020
Spain
Measurement protocol development and/or evaluation
NA
NA
Person with disabilities
Mental Health
aims to adapt and validate the scale of stigmatizing attitudes towards mental health among peers (Peer Mental Health Stigmatization Scale PMHSS-24)
stigmatizing attitudes towards mental health
Mental Health Stigmatization Scale (PMHSS-24)
37
Driscoll D.
2021
US
Study of initiative, program, or intervention
Educational intervention
Public libraries
Library staff
Mental Health
 to identify any changes in stigmatic perceptions among public library staff regarding mental health and substance abuse, post 3-day educational sessions
stigmatic perceptions
Self designed survey intended to measure general stigmatic perceptions
related to mental health and substance abuse collectively
38
Gandhi S.
2021
India
Study of initiative, program, or intervention
Exposure intervention
Workplace
Caffe staff
Mental Health
to explore and understand how and whether perception about Mental Illness (PwMI)  changes when they are seen working like persons without mental illness among those availing services of ROSes cafe at NIMHANS, Bengaluru
clientele attitude towards staff with mental illness
Community Attitude toward Mentally Ill (CAMI)
39
Hansen A.
2021
Australia
Literature review/evidence synthesis studies
NA
NA
University Students
Mental Health
 addressed the question: 'What is the impact of an undergraduate student's culture on their learning about mental health?
review
review
40
Hung L.
2021
Canada
Literature review/evidence synthesis studies
Active involvement of people with dementia; Environmental design;
Public education; Raise awareness; Customized strategies 
Public
Person with disabilities
Dementia
explores key strategies of creating inclusive dementia-friendly communities that support people with dementia and their informal caregiver. Background(s): Social exclusion is commonly reported by people with dementia
review
review
41
Khalifa G.
2020
Canada
Literature review/evidence synthesis studies
Workplace accommodations
Varies
Employees
Autism
To identify workplace accommodations that can contribute to obtaining or maintaining employment for adults with autism spectrum disorder in the peer- reviewed literature
review
review
42
Luelmo P.
2021
US
Study of initiative, program, or intervention
Educational intervention
Public
Parents
Autism
a randomized controlled pilot intervention study examining the effectiveness and feasibility of a low-intensity (i.e. three sessions), low-cost, parent advocacy intervention
review
review
43
Mac Carthaigh S.
2020
England
Study of initiative, program, or intervention
Awareness campaigns
University
University Students
Autism
explored the relationship between autism knowledge, autistic traits, frequency of contact with autistic people and attitudes towards these individuals in British and South Korean student nurses and whether these relationships were affected by the presence of autistic traits
attitudes towards  autistic people 
Autism Survey Questionnaire (ASQ), Openness Scale (OS), Frequency of Contact Scale (FCS), Adult Autism-spectrum Quotient (AQ)
44
Maki Y.
2020
Japan
Literature review/evidence synthesis studies
Dementia friendly community
Public
Person with disabilities
Dementia
This article discusses promoting dementia-friendly community from the perspective of human relationships within a community
review
review
45
Suaidi MT.
2020
Malaysia
Descriptive study of a beliefs, values, attitudes and behaviours, etc. toward PWD among different populations
NA
NA
Community 
Mental Health
to understand the level of knowledge and attitudes of community pharmacists regarding mental health as well as to barriers to delivering effective counselling to patients with the mental illness
role, knowledge, perception
and barrier
Questionnaire from Watkins, et al. (2017)
and Owusu-Daaku et al. (2010)
46
Newton-Howes G.
2021
New Zealand
Study of initiative, program, or intervention
Educational intervention
University
University Students
Mental Health
to investigate the impact of a service user-led anti-stigma and discrimination education programme, encompassing numerous interventions focused on facilitating multiple forms of social contact, the promotion of recovery, and respect for human rights, on medical student attitudes
attitudes
Recovery Attitudes Questionnaire, Opening Minds Scale for Healthcare Providers
47
Niewohner J.
2020
US
Descriptive study of a beliefs, values, attitudes and behaviours, etc. toward PWD among different populations
NA
NA
International NGOs
All Types
idendify reasons that many private international non-governmental organizations (NGOs) have failed to develop disability inclusive policies
Reasons for exclusion of PWD
NA
48
Parker LR.
2020
US
Descriptive study of a beliefs, values, attitudes and behaviours, etc. toward PWD among different populations
NA
NA
General population
Developmental disabilities
investigated the nature of prejudice toward people with developmental disabilities, its potential roots in dehumanization, its implications for social policy beliefs affecting this target group, and strategies for reducing prejudice toward people with developmental disabilities
nature of prejudice toward people with developmental disabilities
prejudice took a hostile or ambivalent (both hostile and benevolent components), dehumanization
49
Pescosolido BA.
2020
US
Descriptive study of a beliefs, values, attitudes and behaviours, etc. toward PWD among different populations
NA
NA
University Students
Mental Health
To examine outcomes in a 4-year college pilot program built on stigma change research
attitudes, behavioral predispositions emerged
College-Specific Social Distance,General and College-Specific Prejudice, College-Specific Social Distance, Perceptions of Campus Mental Health (MH)
50
Peter LJ.
2021
Germany
Literature review/evidence synthesis studies
NA
Varies
General population
Mental Health
examines whether continuum beliefs are associated with lower stigma, and whether continuum interventions reduce stigma
review
review
51
Ran MS.
2021
US
Literature review/evidence synthesis studies
NA
NA
General population
Mental Health
to investigate: 1) the cultural factors related to stigmatizing beliefs about mental illness in Pacific Rim region, and 2) culture-specific measures and interventions on stigma of mental illness
review
review
52
Reddyhough C.
2021
Australia
Study of initiative, program, or intervention
Educational intervention
Healthcare
Healthcare Professionals
Mental Health
examined the efficacy of an education intervention in changing the explicit and implicit stigma held by healthcare professionals towards voice hearers
explicit and implicit stigma
Implicit Stigma: Implicit Association Task (IAT), Explicit Stigma: Social Distance Scale (SDS)
53
Rodr�guez-Rivas ME.
2021
Spain
Study of initiative, program, or intervention
Multi-component 
program
University
University Students
Mental Health
study focused on an online multi-component program on reducing stigma toward mental illness that included project-based learning, clinical simulations with standardized patients and E-Contact with real patients
stigma toward mental illness 
Questionnaire on Student Attitudes Toward Schizophrenia (QSAS), Attribution Questionnaire (AQ-27)
54
Rossetto A.
2020
England
Descriptive study of a beliefs, values, attitudes and behaviours, etc. toward PWD among different populations
NA
NA
General population
Mental Health
findings on perceived and self-reported avoidance, discrimination, and positive treatment from the 2017 and 2019 Attitudes to Mental Illness surveys, which evaluate England�s Time to Change program.
perceived and self-reported avoidance, discrimination
Mental Health Knowledge Schedule (MAKS), Community Attitudes Toward the Mentally Ill (CAMI) scale, Intended Behavior Scale (RIBS)
55
Sarge MA.
2020
US
Study of initiative, program, or intervention
Virtual
simulation engagement; Observation of simulation engagement
public
General population
Autism
examines the short-term effectiveness of engagement with a virtual simulation, Auti-Sim, to combat stigma by giving lay people a first-person experience of sensory overload
perspective taking, 
emotional concern, helping intentions,  willingness to volunteer, Positive attitudes
Cohen�s Identification scale, Attribution Questionnaire (AQ), Societal Attitudes Towards Autism
(SATA) scale, Helping Behavioral
Intention scale
56
Schnitzler M.
2020
South Africa
Study of initiative, program, or intervention
Job creation programmes
NA
Person with disabilities
All Types
examines the interface between disability, social welfare and the
labour market in the post-apartheid society
employment
NA
57
Shahin S.
2020
Canada
Literature review/evidence synthesis studies
Environmental factors
Workplace
Person with disabilities
Mental Health
to investigate environmental facilitators and barriers relevant to workplace participation for transition-aged young adults aged 18-35 with brain-based disabilities
review
review
58
Shahwan S.
2020
Singapore
Study of initiative, program, or intervention
Educational intervention
University
University Students
Mental Health
 to examine the potential impact of an anti-stigma intervention that includes education about depression, information about help-seeking as well as contact with a person with lived experience, on help seeking attitudes.
impact of an anti-stigma intervention
Attitudes towards Seeking Mental Health Services (IASMHS) 

1 Pilot Public Opinion Research Survey on Accessibility
2 This chapter adapted from UK Cabinet Office (2008)
3 This section adapted from Delevic et al. (2011)
4 Some common employer�s stigma: Employing PWDs is a hassle, PWDs take more time off work, it costs more to employ PWDs, Employing PWDs require big, disruptive or expensive changes to workplace, PWDs are less productive (Woodley et al., 2012). Persons with mental illnesses are unpredictable and potentially dangerous, persons with an intellectual disability are less capable than they actually are due to lack of knowledge (Randle et al. 2017).
5 Perceived exploitability: It�s easier to take advantage of PWD
6 PWDs less look forward than others
7 PWDs are burden on families/society
8 e.g.: over half of PWDs worry about being insulted or harassed in the street, and 45% worry about being physically attacked by strangers (UK Government, 2021)
9 e.g.: Impatience with PWD, making fun of disabilities, ignoring PWD, treating PWDs like they have no feelings (Bollier et al., 2021). 
10 https://mentalhealthcommission.ca/opening-minds/
11 https://www.time-to-change.org.uk/about-us
12 https://www.beyondblue.org.au/
13 https://www.health.govt.nz/publication/minds-mine-national-plan-2014-2019
14 Fragebogen zur Einstellung gegenu�ber Ko�rperbehinderten� In German
15 measures developed/used to determine outcomes
---------------

------------------------------------------------------------

---------------

------------------------------------------------------------

Page 2 of 2


Page 2 of 2