CONTENTS
1.1 Introduction
1.2 Risk Assessment
1.2.1 Matrix-Based Approach
1.2.2 Reliability
1.3 Objectives of the Study
1.4 Methodology
1.4.1 Stages of the Study and Procedures
1.4.2 Sample: Analyzed Tasks and Identified Risks
1.4.3 Data Collection
1.4.3.1 Documental Research
1.4.3.2 Tools
1.4.3.3 Equipment
1.4.4 Participants
1.4.5 Used Matrices
1.4.5.1 3 Ć 3 Simple Matrix Method or MMS 3 Ć 3
1.4.5.2 BS8800 Simple Matrix Method or BS8800
1.4.5.3 P: Complex Matrix Method or MMCP
1.4.5.4 William T. Fine Method or WTF
1.4.6 Statistical Analysis
1.5 Results and Discussion
1.6 Conclusions
References
1.1 INTRODUCTION
Every year, millions of people in the European Union are injured at work or have their health seriously harmed in the workplace. With this in mind, we can understand why risk assessment is so important and is considered the key to healthy workplaces. Risk assessment is a dynamic process that allows enterprises and organizations to put in place a proactive policy of managing workplace risks. Risk assessment is the cornerstone of the European approach to prevent occupational accidents and ill health (EU-OSHA, 2009).
The European Agency for Safety and Health at Work (EU-OSHA, 2008) states that risk assessment is the basis for effective management of occupational safety and health (OSH) and the key to reduce both accidents at work and occupational diseases. When properly performed, it can improve safety and health at work and, in general, the performance of companies.
The risk matrix method is probably the most common approach to evaluation in risk analysis. It is a semiquantitative method where probabilities and consequences are categorized, instead of using numerical values (Harms-Ringdahl, 2013).
According to Carvalho and Melo (2015), risk matrices present several advantages, including being generalist, user-friendly, and easy to apply. Nevertheless, they emphasize that we cannot disregard the existing gap in terms of reliability of these applications. In other words, to be useful, these methods must prove to be reliable. To highlight the relevance of reliability, we can recall Kaplan and Goldsenās (Krippendorff, 2004, p. 211): āThe importance of reliability rests on the assurance it provides that data are obtained independent of the measuring event, instrument or person. Reliable data, by definition, are data that remain constant throughout variations in the measuring process.ā
There are very few studies reflecting a concern about risk assessmentās outputs when different methods are used, particularly methods relying on risk matrices. In Portugal, the few known studies reinforce the need for further scientific research in this area to ensure the reliability of risk assessments (Branco et al., 2007; Carvalho, 2007).
This chapter reports the results of a study on the reliability of matrix-based methods. This study involved a comparative analysis of four matrices, which were used to estimate and assess six risks identified in two tasks accomplished to produce car airbags.
1.2 RISK ASSESSMENT
Workplaces, work practices, and processes comprise different types of hazards to which many workers are exposed on a daily basis. This exposure frequently results either in workplace accidents or occupational diseases that necessarily must be prevented. There is absolutely no doubt that risk assessment is a fundamental step in the occupational risk management process contributing to the reduction of both types of consequences.
Although the process of risk assessment has no strict established rules to be followed, it should be carried out in a logical and structured manner. The following steps should be observed: identification of the hazards, identification of the possible consequences, estimation of the likelihood of possible consequences, estimation of the possible consequencesā severity, estimation of the risk magnitude (i.e., how big is it?), evaluation of the significance of the risk (e.g., is it acceptable?), and recording of the findings. The results will inform on the level and relevance of risks, as well as if the existing control measures are adequate or additional preventive and protective actions are needed.
Risk assessment techniques range from a simple qualitative approach to a detailed quantitative assessment, and each of them presents advantages and inconveniences. While the former approach lacks of objectivity and does not allow costābenefit analysis, the latter is rather complex and time-consuming and requires well-trained analysts to be applied.
1.2.1 MATRIX-BASED APPROACH
The use of matrices is probably the most common approach to evaluation in risk analysis. It is a semiquantitative method where probabilities and consequences are categorized (Harms-Ringdahl, 2013). The basis for the risk estimate is usually qualitative, although numbers can be used for labeling either the consequences or the frequencies, or both, expressing the hierarchy in both scales. Therefore, these categories are defined either numerically or by a description.
The simplest matrices interpret risk as the combination of consequence (severity) and likelihood (frequency). Therefore, both variables must be coded according to a scale. In this approach, risk level is obtained by either combining the used variables in a preestablished manner or multiplying the attributed values. Once risk level is computed, it is compared to the risk index scale to prioritize actions in terms of preventive or protective countermeasures. This kind of approach is considered very important in occupational risk assessments, because it allies the advantages of both the quantitative and qualitative approaches and overcomes some of their limitations. Plus, it is very effective at promoting audience participation during risk management programs.
Carvalho and Melo (2007) state that this kind of approach has proven to be, in most cases, the only available technique and the most suited to carry out this task. These last evidences assume particular relevance when we think of small and medium enterprises. Most of them do not have the resources to assess risk quantitatively: there is not a permanent OSH practitioner available to perform a risk assessment on a regular basis, and in some cases, it is the employer himself who makes this first approach to risk management, even without experience or adequate knowledge to do so.
Despite the benefits referred, the validity and reliability of risk matrices and other evaluation techniques have not been studied enough (Harms-Ringdahl, 2013).
1.2.2 RELIABILITY
In general, reliability refers to the extent to which a test, experiment, or measuring procedure gives the same results on repeated trials or applications (Olsen, 2013). Matrix-based risk assessment methods rely on coding categories, which are considered to be reliable if separate coding attempts end up with the content coded in a similar way.
Reliability and agreement are still generally broad terms and require further definition to ensure their correct application to measurements within the OSH risk assessment domain. Reliability refers to a proportional consistency of variance among coders and is correlational in nature, while agreement refers to the interchangeability among coders, and addresses the extent to which coders make essentially the same coding (Olsen, 2013). Therefore, coders can be reliable in the coding process when the range of codes assigned by one coder is consistent with the range of codes assigned by another coder, even if the codes assigned to each individual event do not meet with consensus. There is agreement among coders when codes assigned to each individual event are the same between coders (i.e., consensus is attained on the codes assigned to each individual event).
Then, high reliability can be obtained even when there is low agreement and the opposite is true.
According to Krippendorff (2004), there are three types of reliability: stability, reproducibility, and accuracy. These are distinguished not by how agreement is measured, but by the way the reliability data are obtained (Table 1.1). Without information about the circumstances under which the data for reliability assessments have been generated, agreement measures remain uninterpretable.
Stability is the degree to which a process is unchanging over time. It is measured as the extent to which a measuring or coding procedure yields the same results on repeated trials. The data for such assessments are created under testāretest conditions; that is, one observer rereads, recategorizes, or reanalyzes the same text, usually after some time has elapsed, or the same measuring device is repeatedly applied to one set of objects. Under testāretest conditions, unreliability is manifest in variations in the performance of an observer or measuring device. There is stability when one person is consistent with himself or herself; for example, coding categories present intracoder reliability if a coder can categorize the same content, on a later occasion, similarly to how he or she coded it previously.
Reproducibility is the degree to which a process can be replicated by different analysts working under varying conditions, at different locations, or using different but functionally equivalent measuring instruments. Demonstrating reproducibility requires reliability data that are obtained under testātest conditions; ...