Using Observation Data to Keep the Process Alive and Well

Grainne A. Matthews, Ph.D.

Design

Data analysis begins, of course, during the design phase with decisions about what data to gather and how. One of the most important features of a Behavioral Safety process is its simplicity so the team carefully targets data collection and analysis activities. Collect only the information needed and only analyze it as much as necessary to make planning decisions. Complexity bogs down the process and burns out your team. The team also weighs the need for data with the need for confidentiality and anonymity. For example, it might be nice to know the time of the observation, but employees may be concerned that observation data will be used against them and time of observation would allow the observed employee to be identified in an anonymous process.

The principle in deciding what to measure is to start with the end in mind and keep it very focused and very simple.

Review

Data is reviewed as frequently as possible and at least every other week. One of the defining principles of the behavioral approach is that it is data driven, that means that decisions are constantly evaluated and adjusted based on the data. The team chooses a strategy to improve safety or the process, implements it, and observes the result as it is revealed in the data. If their plan is effective, they add that approach to their tool kit for future use. If the results are not what they hoped, then they adjust as soon as possible rather than wasting resources on an ineffective strategy. The process will also lose credibility with the workforce and management if unsuccessful tactics are allowed to continue.

Process Goal and Measures

Measurement is driven by the goals of the process. A safety process usually has one ultimate goal – to decrease the number of injuries. Although the process may accomplish many other desirable changes, such as decreasing environmental incidents, increasing productivity, increasing communication, or improving relationships, it is not successful if injuries do not decrease. Identifying the goal and defining success before investing in any new process will increase the probability of an effective and efficient effort.

Number of Injuries

If the company identifies injury reduction as the ultimate goal of the process, then one of the design issues will be a definition, baseline, and intermediary targets. The company designs, implements, and monitors a simple tracking system. This is frequently the area where those involved in Behavioral Safety have the least input but, if the company is to decide the success or the failure of the process based on these data, then the these measures must be carefully determined. For example, if behavioral safety is to be implemented in one area within a plant, injury data for only that area is relevant. Define “injury” so that you get general agreement. In general, steer clear of using the OSHA definitions because they have little face validity.

In addition, this is the point at which to install a near miss reporting system that works. A near miss reporting system works if there are 100 reports for every medical attention incident. That usually means simple and anonymous. The same starting point applies here – define carefully what is meant by “near-miss” and design a reporting system that will gather just the information you need with the least amount of effort.

In-Process Objectives and Measures

Behavioral Safety is a process (a series of activities that occur sequentially) designed to achieve the goal of injury reduction. Since it takes time to determine if the goal will be achieved, the team uses in-process measures to ensure that they are on track.

The process is built upon the logic that incidents will decrease if both the frequency with which employees are working safely and the number of safe work conditions increases. This basic logic gives us our first two in-process measures. The first step is to agree upon definitions, baseline levels, target levels, and timelines for achieving each target. Then a simple tracking system is designed, implemented, and monitored. Each organization answers these questions for their situation during process design.

Employees Working Safely

Some examples I have seen include:

1. Number of employees using the target safety practice per observation
   Example: In the welding area of a machine shop, an employee observes all the welders for 15 minutes to count the number of them who are wearing protective eyewear and welding face masks throughout each welding task or the length of the observation.
2. Number of observations where no safety concerns were observed
   Example: In a batch chemical plant, the process involves an employee observing her partner engaging in a routine task. The observer records whether the employee used all applicable safety practices as listed on the checklist. If not, she indicates which practice caused her concern.
3. Number of opportunities where all steps of the safety practice were completed
   Example: Field service technicians who drive to customers' locations have a self-observation checklist that includes many possible safety precautions. After completing each service call, they check whether they completed all applicable precautions.
4. Number of times employees engage in the target safety practices
   Example: Unloading boxes on a loading dock in a warehouse, the observer counts the number of times the employee uses ideal lifting techniques out of all lifts performed during the observation.

Other presenters have spoken about ways to choose your target safety practices, some people refer to them as “pinpoints”; the essential point is to select only those behaviors that are related to preventing injuries. Some people refer to them as “critical” safety behaviors. They are your leverage points to achieving your goal.

Safe Conditions

Some options include:

1. Defining "safe environment" for a specific work area and counting the number of safe conditions observed.
   Example: In a laboratory, an observer records whether chemicals have been stored safely.
   This is an example of a condition that is the result of the behavior of the target people. It is actually the behavior of storing the chemicals appropriately that will be targeted but it is not necessary for the observer to actually watch the lab tech put the chemical in the correctly labeled container to know that it was done.
2. Observing one employee performing a task and counting the number of unsafe conditions that affect that employee and task
   Example: In a refinery, an observer watching an employee catching a sample on a line counts the number of hazardous conditions, such as an unsteady platform or unmarked pipes, etc.
   This is an example of conditions that the operator has little control over but has chosen to work with rather than fix or get fixed. The behavior the process is designed to improve is reporting or fixing unsafe conditions before doing the task.

Process Measures

Neither the ultimate process goal, nor the in-process objectives will be achieved if the process is not running as planned. Therefore, the team also measures the health of the process itself. How does a healthy Behavioral Safety process work?

• Individual coaching (individual and immediate feedback based on work observation)
• Group goal setting, feedback, and celebration
• Removing barriers to safe work practices.

Therefore, we will need measures for each of these activities. Depending upon individual design, options include:

Coaching

1. Number of employees trained as coaches
2. Number/percentage of trained employees conducting a coaching session as scheduled
3. Number/percentage of employees submitting a quality observation form
4. Number of coaching sessions or Number/percentage of quality coaching sessions
5. Number/percentage of employees allowing or inviting a coaching session

Group Goals and Celebration

Natural work groups set their own goals for improving target safe practices. If employees are coaches, then the group sets goals for coaching, too. These are all based on the measures decided upon for Working Safely, Safe Conditions, and coaching. Goals are based on the current level of performance, are attainable, and may either have a specific time-period or be open-ended. Graphs are posted publicly and, when goals are met, the group celebrates.

Removing Barriers

Graphing the data for natural work group goals and celebrations allows the team to monitor the health of the process. If safety and coaching goals are being met, the process is working. If attainable goals are not being met, the team analyzes the data to develop a targeted intervention. If coaching and goal setting, feedback, and celebrations are failing to produce improvements in safe work and conditions, or if participation is not meeting targets, the team uses the data and other information to discover the barriers and make plans to decrease them.

How does the team decide what to target and how? For safety practices and conditions, they select the most important practice or condition that is not improving. “Most important” could mean the one with the most exposure, such as lifting or noise, or the one that could prevent the most serious injury, such as using fall protection or unguarded presses.

Behavior Analysis

The team conducts a behavioral analysis of safety or participation as necessary. Do employees have the necessary skills and knowledge? Tools and equipment? Time and place? to do this safety practice, to attend to this condition, or to participate as coaches? Are antecedents available that prompt the behavior at the right time and place? If antecedents are adequate, are there meaningful, positive consequences that reinforce the behavior? Are there competing consequences, such as positive consequences for a conflicting behavior or negative consequences for the behavior?

Since a key element of the process is the observation and feedback of coaching, the team also ensures that this is occurring as designed. Are coaches conducting reliable observations or have they drifted since training? Are they providing feedback that is specific enough to allow the employee to know what they should do again? Is the positive feedback positive and the ratio to corrective feedback at least 3:1?

Action Plans

Finally, the team decides on a plan to remove the identified barrier – education, training, job aids, signs and symbols, tool replacement, equipment repair, facility engineering, workflow redesign, supervisor training, etc. Sometimes adjusting the process solves the problem, for example, targeting coaching to the times or areas where the practice should occur. Obviously many of these are not quick fixes and they involve more than just the team. This is where the team needs to develop skill in working with people other than the employees who are the core of the process. This might include managers, engineers, schedulers and planners, supervisors, trainers, and safety professionals, etc. The same principles apply but that is another presentation.

If the team determines that both antecedents and consequences that affect individual behavior are in place and the goal is still not achieved, then the problem may be that too much, too soon is expected. The safety practice can be further broken down so that coaches can reinforce progress towards accomplishing the entire practice. For example, “driving forklift safely” may be defined as wearing seatbelt, staying under 5 miles per hour, and using horn at corners. The goal of 100% safe for the week has not been achieved so the work group decides to pay particular attention to coaching “wearing seatbelt” and the Behavioral Safety team provides additional group incentives for achieving the goal of 100% safe on “wearing seatbelt”. Once this is achieved, using horn is added, and finally driving the speed limit is added. This is called shaping.

Another option is to lower the goal so that the work group experiences the recognition and celebration for achieving a small step and may be motivated to continue to move forward.

It is important to track all action plans resulting from the process and communicate them to the employees. This lets employees know that the process is working and it increases accountability and recognition of the team and for others who support the team in implementing the interventions, such as managers, engineers, trainers, and safety professionals. A simple report that lists all the action plans that the team has initiated and their status can be posted. Everyone will see that a certain number of plans have been completed successfully, some are underway and producing results, some are under revision, and some are on hold for a stated reason.

This is the text which accompanies the Powerpoint presentation of Dr. Matthews BSN 2001 talk.

Dr. Matthews is a Project Manager with Quality Safety Edge. She currently specializes in the application of behavioral technology to employee driven safety and quality improvement efforts. She has sixteen years experience designing, implementing, and evaluating successful behavioral programs in a wide variety of organizational settings. Programs developed include innovative training, feedback, and reward systems for performance improvement. She is the outgoing President of the Behavioral Safety Network and an active member of the Organizational Behavior Management Network. Her articles have been published in the Journal of Organizational Behavior Management, Industrial Safety and Hygiene News, Occupational Health and Safety, and Professional Safety magazines. Companies with whom Dr. Matthews has recently worked include: Bayer, Chevron Canada Resources, Citgo, Exxon/Mobil Chemical, ICI, Nalco/Exxon Energy Chemicals, National Starch and Chemical, and Stewart and Stevenson.