The Raoul Wallenberg Institute of Human Rights and Humanitarian Law recently carried out an impact assessment of QuizRR Rights and Responsibilities, a digital educational tool which aims to strengthen knowledge of and respect for labour rights in factories in China.
The assessment was commissioned by Axfoundation, which since 2015 has supported use of QuizRR among companies in the Axel Johnson Group, all of them with suppliers in China.
The impact assessment was conducted by Malin Oud, Director of RWI’s China Office and Team Leader of Economic Globalisation and Human Rights together with Liang Xiaohui, Chief Researcher at Office for Social Responsibility of China National Textile and Apparel Council and Adjunct Professor at Peking University Law School
The Executive Summary of the evaluation report is provided below. For a summary in Swedish and comments by Axfoundation and QuizRR, please visit Axfoundation’s website.
EXECUTIVE SUMMARY
This Executive Summary highlights the conclusions and recommendations of an external evaluation of QuizRR Rights and Responsibilities, a digital educational tool for global buyers and suppliers which aims to strengthen knowledge on rights and responsibilities at processing factories in Asia through a series of customized educational short films, followed by quiz questions. The evaluation assessed the relevance and effectiveness of QuizRR in select factories in China that have used the training module ‘Rights and Responsibilities’ (hereinafter QuizRR RR) during the period 2015-2017. The evaluation was carried out from August 2017 to February 2018, and included interviews with six factories in China, eight Swedish brands/buyers that use QuizRR RR, as well as other stakeholders representing QuizRR’s partners, major international brands/companies with production in China, industry associations, labour rights and related organisations, and trade unions.
The evaluation report provides an external and independent assessment of the general logic, relevance, strengths and weaknesses of QuizRR’s strategy and approach in China, based on qualitative interviews, stakeholder perceptions and the evaluators’ experience and expertise in this field in China. Due to evaluation limitations and methodological challenges, the findings of the evaluation are preliminary and not necessarily representative of QuizRR results more broadly in factories in China (or elsewhere). A fully-fledged impact assessment of QuizRR with regard to strengthened knowledge, changed behaviours, better working conditions, better workplace dialogue, improved staff retention, increased productivity etc. would require more in-depth and long-term research.
Relevance and effectiveness
• The concept and idea behind QuizRR clearly meets a growing demand for tools that go “beyond audit” to build capacity in factories. Furthermore, as the political space and possibilities to support independent trade unions and local labour rights organisations have become more and more limited in recent years in China, QuizRR offers a pragmatic and feasible training alternative to companies that want to strengthen awareness and respect for labour rights among their Chinese suppliers. There is furthermore a need for interactive, scalable and continuous training, with training content adapted to the local legal context and delivered in different languages. As such, QuizRR offers an attractive solution for many buyers — in particular middle-sized companies with limited in-house training capacity and leverage;
• The QuizRR training methodology was very well received among the interviewees, and examples of positive effects of QuizRR concerning increased awareness regarding maternity leave, labour contracts, social insurance and personal protective equipment (PPE) were identified. These findings indicate that QuizRR RR trainings have generated or contributed to raised awareness and changed procedures, particularly on health & safety issues;
• However, despite what appears in QuizRR statistics as a large number of training sessions and outputs, the relevance of these numbers and the effectiveness of the training sessions are limited and curtailed by the realities in the factories and in relation to the needs and interests of the different target groups. One such reality and weakness of the QuizRR system is the risk for cheating and consequently unreliable data and measurability;
• A key finding of the evaluation is that no factory included in the evaluation had institutionalised QuizRR training or used it in a systematic way. Furthermore, while the interviews indicate positive results with regard to health & safety issues, it is more questionable if QuizRR trainings have generated or contributed to results on more complex issues concerning human rights beyond compliance with Chinese laws and regulations. The evaluators found no indications that QuizRR trainings had generated the strong and collective awareness and pressure required to translate into such changes. Rather, factory interviews seemed to indicate that QuizRR trainings had resulted in a better understanding, acceptance and appreciation of existing factory rules, management and standards. The evaluators see several different reasons for this, having to do both with QuizRR training content, methodology and the Chinese context;
• The evaluators thus find a significant gap between, on the one hand, QuizRR’s objectives and claims, and, on the other hand, what QuizRR can realistically deliver and achieve in the Chinese context. Relatedly, the evaluators see a risk that QuizRR could function more as a useful tool for companies to ‘measure and share’ progress based on numbers of trainings conducted, and less as a useful tool for workers in terms of strengthened human rights knowledge, empowerment and improved working conditions. In this context, it should be noted that the six factories included in the evaluation have only conducted QuizRR RR trainings for a short period of time (1-3 years) and that none of them had conducted trainings with the QuizRR Worker Engagement module (which is more focused on workplace dialogue and related rights);
• Many of the brands expect there to be buy-in and ownership of QuizRR in the factories, and that they would be willing to shoulder or share the costs for QuizRR within a near future. However, a majority of factories interviewed for this evaluation stated that they would only be willing to do so if buyers offer better business benefits1. This poses a dilemma for QuizRR and many of the small and medium-sized Swedish companies in China. With very limited influence or leverage over the factories, it is difficult for them to place demands on the factories, in particular if it involves additional processes and costs. This dilemma also raises questions about the broader “business case” for QuizRR;
• In conclusion, it is suggested that the different stakeholders should have more realistic expectations on what QuizRR can achieve in a complex environment like China, and that QuizRR should be more careful and realistic about what it promises to deliver.
Recommendations
To further strengthen the relevance and effectiveness of QuizRR in China, the evaluation team makes the following recommendations:
• Formulate a realistic and human rights-based Theory of Change and results framework, including a comprehensive description of how and why the desired change is expected to happen in the Chinese context;
• Put in place training baselines and training plans in all factories. QuizRR has recently introduced an improved process for training implementation and follow-up, with baseline questions, training plans and technical support. These were however not in place in any of the factories included in this evaluation. The importance of such training plans and baselines is a major lesson to be drawn from the piloting phase of QuizRR in China;
• Develop more tailored and accessible training and incentives that meet the particular needs and interests of the different target groups, including women, migrant workers and different levels of staff at the supplier factory. QuizRR’s recently developed partnership with CSR CCR is a good example of such more tailored trainings, meeting specific needs and delivered in partnership with a relevant local organisation. QuizRR should furthermore be made into an application or other real online version, and concrete incentives put in place both for workers and factory owners to use QuizRR, including compensation for their time and input;
• Finally, the next generation of “beyond audit” capacity building tools and initiatives furthermore need to consider how to go from awareness-raising to “actualising rights”. In many factories in China, the fundamental problem is not that workers lack awareness of their rights, but rather that they are not able to enjoy them or put them into practice. The module QuizRR WE aims to address this, but was not within the scope of this assessment.