Development and Pilot Testing of a Food Safety Curriculum for Managers and Staff of Residential Childcare Institutions (RCCIs)

Authors


  • Previous affiliations at time work was completed: Nyachuba, Dept. of Nutrition, Univ. of Massachusetts, Amherst, MA 01003-9282, U.S.A. Couto, Nutrition and Food Sciences Dept., Univ. of Rhode Island, 530 Liberty Lane, West Kingston, RI 02892, U.S.A. Hagan, Food Science Dept., Univ. of Massachusetts, Chenoweth Laboratory, Room 236, Amherst, MA 01003, U.S.A. Breau, Kids First, Providence, RI 02906, U.S.A.

Abstract

Food safety training materials, targeted for residential childcare institution (RCCI) staff of facilities of 20 residents or less, were developed, piloted, and evaluated. The goal was to assist in the implementation of a Hazard Analysis Critical Control Points (HACCP)-based food safety plan as required by Food and Nutrition Service/United States Department of Agriculture. RCCI staff and instructor training manuals, Food Safety Basics: Food Safety Education for RCCIs was developed and pilot tested in RCCIs in both Rhode Island and Massachusetts. The program was evaluated by participants and pre-/posttraining onsite inspector and microbial assessments. The core curriculum contained 3 modules and corresponding activities: Module 1: cause, prevention, and strategies to reduce foodborne illness; Module 2: components of a HACCP-based food safety plan and standard operating procedures; and Module 3: actual food safety plan development, including instructions for implementation and corresponding monitoring procedures required. There were 50 to 82 RCCI staff members that attended 2 or 3 modules. Participants rated the modules 4.5 ± 0.4, 4.3 ± 0.4, and 4.4 ± 0.4 out of 5-point Likert scale (1 = strongly disagree, 5 = strongly agree) for teaching modules 1, 2, and 3, respectively. The evaluations assessed their understanding of key topic areas taught. Onsite inspections showed improvement in many critical food safety issues. Some deficiencies still appeared to be problematic, such as refrigerator temperature monitoring and issues with storage and cross-contamination.

Introduction

The National School Lunch Program (NSLP) is a federally assisted meal program that operates in public and nonprofit private schools as well as residential child care institutions (RCCIs). The Food and Nutrition Service (FNS) branch of the United States Department of Agriculture (USDA) administers the program at the federal level. State education agencies administer the program through agreements with local or district level school food authorities (USDA 2006). In 2004, the Child Nutrition and WIC Reauthorization Act amended the National School Lunch Act to require that state school food authorities implement a Hazard Analysis Critical Control Points (HACCP)-based food safety program by July 1, 2005 (Public Law 108–265) for all institutions that participate in the NSLP or School Breakfast Programs (SBP; USDA 2005). NSLP and SBP recipients must comply with the food safety requirements; not only the large school-based foodservice operations but also large and small RCCIs. Each site must implement a food safety plan based on a HACCP approach and develop standard operation procedures, including sanitation (USDA 2005).

RCCIs can include a number of different institutions that provide short- or long-term residential and care services to children. These institutions meet a broad definition of “school” for purposes of providing all child nutrition programs. RCCIs can include homes for the mentally, emotionally, or physically impaired, unmarried mothers and their infants, group homes, halfway houses, orphanages, temporary shelters for abused and/or runaway children, long-term care facilities (hospitals) for the chronically ill, and juvenile detention centers (7 CFR part 210; Office Federal Register 2011). The RCCI definition allows for group homes or shelters with as few as 4 children to qualify for participation in the federally funded meal program. Some RCCIs have staff that are highly trained and already following HACCP principles (for example, hospitals, training schools). Many others institutions function as a “group home” environment or transitional living program. These RCCIs often have staff or foodservice professionals who prepare meals, but may have other duties as well (Brennan-Olson 2006; Carey 2006).

As early as 1997, Walter and others (1997) reported the need for training in community-based homes. These researchers found staff knowledge lacking in several areas and limited awareness of food preparation techniques and safe food-handling practices. Between 1990 and 2004, 43 foodborne illnesses were documented at childcare facilities (Riggins and Barrett 2008). Evidence of lack of implementation of food safety training was found in licensed childcare facilities providing childcare services to preschool-aged children (Riggins and Barrett 2008). Roberts and others (2008) found employee training was the top barrier to implementation of food safety programs at foodservice. In a more recent effort to evaluate the training needs of RCCIs, particularly facilities of 20 residents or less, Pivarnik and others (2012) conducted a needs assessment of food safety knowledge of managers/foodservice directors of RCCIs in the Northeast. Respondents answered 44 food safety questions on content areas of cook/prepare, chill/cold storage, clean/hygiene, receiving/general storage, and food handling. Using 80% as the standard for subject mastery, only 27 of the 44 survey questions (55%) met the proficiency standard. The content areas of cook/prepare (66%) and chill/cold storage (67%) had the lowest mean correct scores. Facilities with ≤20 residents scored significantly lower (P < 0.05) than larger institutions. Smaller facilities were less likely to have food safety plans (67% compared with 83% for larger) or recipes based on HACCP implementation (28% compared with 64% for larger). Of those surveyed, 60% indicated that they would like to have onsite food safety training for their staff. The results showed an overwhelming need for programming and training that would target RCCIs of 20 residents or less. Therefore, the goal of this research project was to develop, pilot test, and evaluate a food safety training curriculum for RCCI staff that could be delivered by food safety professionals.

Materials and Methods

Curriculum development

The results of the needs assessment for managers/foodservice directors of RCCIs in the Northeast (Pivarnik and others 2012) directed both curriculum development and outreach initiative. A training program consisted of a student notebook (Food Safety Basics: Food Safety Education for RCCIs) with presentations, fact sheets, activities, log/recording templates, and module/lesson evaluations. Initial curriculum development was reviewed by representatives from the Rhode Island (RI) and Massachusetts (MA) Departments of Education. Based on input from instructors charged with the outreach initiative, an instructor's manual was developed with presentations/notes, instructions for hands-on activities, and a teaching outline with suggestions for training implementation strategies and evaluations. Also included were appendices that contained all fact sheets and data collection tools (audits, log sheets). The core curriculum contained 3 modules and corresponding activities: Module 1 (Food Safety Basics)—cause, prevention, and strategies to reduce foodborne illness; Module 2 (Developing a Written Food Safety Plan)—components of a HACCP-based food safety plan and standard operating procedures (SOPs); and Module 3—food safety plan development, including instructions for implementation and corresponding monitoring procedures required. Therefore, the core curriculum consisted of 3 training sessions, one for each module, and each took 1.5 to 2 h to complete. RCCI facility representatives in attendance at workshops were provided with a flash drive that contained presentations, and “blank” sheets useful in preparing and implementing the food safety plan (for example, the log sheet templates for cleaning/sanitation, cooked/cooled food temperature, hot holding temperature, refrigeration temperature, and thermometer accuracy/calibrations) as well as a monthly audit checklist. While a “food safety kit,” was provided to training participants and contained suggested food safety supplies (for example, color cutting boards, digital thermometer, sanitizer test strips); an informational pamphlet is now provided as a resource.

Training pilot program

Volunteer participation

Using the respondents from the original mail survey administered to RCCI facilities in the Northeast (Pivarnik and others 2012), volunteer RCCI sites, of 20 residents or less, from RI and MA that were interested in participating in the pilot training were asked to provide their name and contact information on a form attached to the mail-in survey. Other potential sites were identified through the Departments of Education in RI and MA. Individual RCCI sites/facilities are normally under the direction of a “sponsor” organization (for example, 3 separate sites, one sponsor organization). Since all RCCI sites must have their own food safety plan, the project directors allowed sites under the same umbrella organization to participate. Ten RCCIs (5 in each state) originally volunteered to be part of the pilot instructional program, which included instruction and evaluation. One MA site dropped out of the pilot after completing training and pre-inspections. Two different outreach strategies were implemented for posttraining: direct contact by RI instructor (e-mail or phone contact) or instructions to facility participants to contact the MA instructor if needed.

Training evaluation

Evaluations were used to revise the curriculum as needed. There were 4 training evaluation strategies used for the RI and MA pilot: (1) pre-/posttraining onsite inspections; (2) pre-/posttraining onsite microbiological assessment; (3) participant evaluation of each instruction module; and (4) assessment of outreach initiative by the MA/RI pilot instructors.

The onsite inspections, in RI and MA, were conducted by an inspector that had been trained by and had worked for the RI Division of Food Protection. The inspection looked at the presence/absence of a food safety plan, issues relating to personal hygiene/employee health, food preparation (for example, cooking, reheating, thawing), cold and dry storage of food, cleaning and sanitizing, pest control, and physical attributes of the facility (for example, maintenance, garbage disposal). In addition, the inspector also evaluated the presence or absence of SOPs. The rating scale was yes, no, not applicable, or not observed. For example, if the inspector was assessing whether the facility personnel were reheating leftovers to proper temperature and recorded, he would list the practice as done correctly (yes), not done (no), not observed, or not applicable to the facility. Inspections were done 2 to 3 mo following training, with on exception where it was done after 1.5 mo.

Microbiological assessments of facility kitchens were also completed for each pilot facility before and after completion of training. Aerobic plate, coliform, Escherichia coli, and yeast/mold counts were evaluated using Simplate (most probable number, MPN) media kits and plates from Biocontrol, Bellevue, WA, USA. Listeria spp. and Salmonella spp. were delineated as present or absent using SDI RapidChek Listeria and Salmonella test kits and media from Strategic Diagnostics (SDIX) from Newark, DE, USA. Any additional media required was purchased through Fisher Scientific, Suwannee, GA, USA. Sterile sponges, used for environmental swabbing, were obtained from Fisher Scientific. Twenty-four hours prior to onsite assessments, 10 mL of neutralizing broth was added to prelabeled, sterile Whirl-Pak bags containing sponges and held refrigerated until used. Predetermined food and nonfood contact kitchen surfaces were identified for sampling. Each site was swabbed, in duplicate, sampling a 10 × 10 cm area using the method outlined by Venkitanarayanan and others (1999). A 100 cm2 (4 inch × 4 inch) surface was aseptically swabbed twice, with 2 separate swabs from top to bottom and from right to left for 2 different locations on the same surface. Samples were stored upright and transported back to the laboratory on ice and held, refrigerated until testing no more than 4 h later. The lower limit of detection for this protocol was <0.5 cfu/cm2 (<1.0 cfu/2 cm2). Simplate total plate counts (TPC-CIa), and Simplate Coliform and E. coli counts (CEc-CI) and yeast/molds were based on detection limit for protocol. There are known difficulties associated with quantitative microbiological assessment of surfaces, since, viable organisms recovered from a surface could be affected by numerous variables such as agitation, sample size, sample temperature, swab pressure, and organism release from swab (Eifert and Kang 2005). Kitchen areas were also evaluated for hygiene/sanitation using FLASH rapid test strips (BioControl, Bellevue, WA, USA) to identify surfaces contaminated with protein as positive or negative. Areas monitored for microbial and protein contamination were refrigerator shelves/drawers/handles, cutting boards and/or preparation areas, sink handles, microwave key pads, drying racks, stove tops, and sink drains. A total of 15 to 16 sites were monitored at each facility, both pre- and posttraining sampling.

Training evaluation was completed by participants using a 5-point Likert scale (1 = strongly disagree, 5 = strongly agree). Questions included their perceived understanding of principles (for example, As a result of this training I.…(1) understand the components/principles of an effective SOP; (2) understand the critical control point concept and its application to the HACCP-based food safety plan of my menu items; (3) understand what records are needed; or (4) understand the daily requirements of a food safety plan). Questions also reflected the participant's thoughts concerning implementation of the food safety program. In addition, students were asked about the overall effectiveness and usefulness of the training and handouts, as well as class organization. In addition, the RI/MA instructors provided suggested revisions based on their experiences with the program.

A final evaluation strategy was used following the model of Byrd-Bredbenner and Schaffner (2003) that was used for their food safety social marketing campaign for college students. Land Grant institutions were invited to submit proposals for mini-grants describing a plan to implement the training program developed by this project for RCCIs in their states. Once proposals were selected, the subawardees will be required to evaluate the training using the participant assessments developed for the pilot project. Awardees were also required to provide their overall appraisal of the program. The request for proposals were announced using Food and Nutritional Professionals who Educate the Public and National Institute of Food and Agriculture state food safety contact list serves. Tennessee State (TS) and Kansas State (KS) universities received funding. All resource materials developed for this project were sent to the principal investigators of these institutions.

Data Analysis

Data analysis was carried out using the SPSS statistical program, version 20, and Microsoft Office Excel, version 2003. Descriptive analyses (that is, frequencies, distributions, ranges, standard deviations) were computed for all variables. One-way ANOVA followed by the F and t-tests statistics were run to determine statistical differences between means.

Results and Discussion

Table 1 to 3 show the results of the participant evaluations for modules 1, 2, and 3 during the pilot program. There were over 50 RCCI staff members that attended all 3 training modules and 82 that attended the first 2 only. This was probably due to the fact that the last module, food safety plan development and monitoring, was more targeted to those RCCI staff that had some managerial oversight for the kitchens or facility. Overall, the participants rated modules 4.5 ± 0.4, 4.3 ± 0.4, and 4.4 ± 0.4, for teaching modules 1, 2, and 3, respectively. The participants indicated that they understood the principles and their applications and appeared confident that they could complete the requirements of a food safety plan. Most notably, although participants appeared to feel comfortable implementing the program (4.2 ± 0.8), they were still not sure if they would need help (3.6 ± 0.9). Evaluation trends for each module were similar for RI and MA. However, participants in the MA program had scores that were lower then RI participants—with modules 1 and 3 being significantly lower (P < 0.01). This could be due to the different outreach protocols: the RI training was site-specific, MA training was more centralized. Overall, it appeared, from the evaluations that the participants felt they understood the concepts and could begin to implement the program.

Table 1. Rhode Island (RI) and Massachusetts (MA) participant RCCI Module 1 evaluation: Cause, prevention, and strategies to reduce foodborne illness
 Scorea (Ave. ± Std. Dev.)
 RIb (n = 50)MAc (n = 34)Total (n = 84)
  1. a

    Score based on 5-point scale (1 = strongly disagree, 2 = disagree, 3 = undecided, 4 = agree, 5 = strongly agree).

  2. b

    Represents 5 training sessions, one session at each RCCI site.

  3. c

    Represents 3 training sessions, reflecting 5 RCCI sites.

  4. d

    MA evaluation scores significantly different (P < 0.01) than RI scores.

Information was presented in a clear, organized manner4.7 ± 0.54.2 ± 0.64.5 ± 0.6
Presenters were organized and well prepared4.7 ± 0.54.4 ± 0.54.6 ± 0.5
Hand-outs and materials are relevant to the information presented4.8 ± 0.54.5 ± 0.54.7 ± 0.5
Important points were reviewed and emphasized4.7 ± 0.54.4 ± 0.54.6 ± 0.5
Group participation and questions were encouraged4.7 ± 0.54.3 ± 0.54.5 ± 0.5
Questions were answered sufficiently4.7 ± 0.54.3 ± 0.44.5 ± 0.5
Handouts will be useful4.7 ± 0.44.2 ± 0.64.5 ± 0.6
Food safety principles were understood4.9 ± 0.34.3 ± 0.54.5 ± 0.5
Total score4.7 ± 0.4d4.3 ± 0.4d4.5 ± 0.4
Table 2. Rhode Island (RI) and Massachusetts (MA) participant RCCI Module 2 evaluation: Components of a HACCP-based food safety plan and standard operating procedures
 SCOREa (Ave. ± Std. Dev.)
As a result of this training, I…RIb (n = 57)MAc (n = 25)Total (n = 82)
  1. a

    Score based on 5-point scale (1 = strongly disagree, 2 = disagree, 3 = undecided, 4 = agree, 5 = strongly agree).

  2. b

    Represents 5 training sessions, one session at each RCCI site.

  3. c

    Represents 3 training sessions, reflecting 5 RCCI sites.

understand the components/principles of an effective SOP4.3 ± 0.54.2 ± 0.44.3 ± 0.5
understand the components/principles of an effective food safety plan4.4 ± 0.54.2 ± 0.44.3 ± 0.5
understand the HACCP principles4.3 ± 0.64.2 ± 0.44.3 ± 0.5
am familiar with the format needed for SOPs4.2 ± 0.74.3 ± 0.54.2 ± 0.6
understand how to categorize menu items as to “process”.4.4 ± 0.64.4 ± 0.54.4 ± 0.6
understand how to prepare a “flow chart” for menu items4.3 ± 0.64.2 ± 0.64.3 ± 0.6
understand about major operational steps in foodservice4.4 ± 0.64.2 ± 0.54.3 ± 0.5
can determine control measures for recipe process4.3 ± 0.74.2 ± 0.64.3 ± 0.6
understand the CCP concept and its application to the HACCP-based food safety plan of my menu items4.3 ± 0.64.3 ± 0.54.3 ± 0.5
understand the CL concept and its application to the HACCP-based food safety plan of my menu items4.4 ± 0.64.3 ± 0.54.3 ± 0.5
understand the monitoring concept and its application to the HACCP-based food safety plan of my menu items4.3 ± 0.64.2 ± 0.64.3 ± 0.6
understand the corrective action concept and its application to the HACCP-based food safety plan of my menu items4.3 ± 0.64.2 ± 0.54.3 ± 0.6
understand what validation is and its importance4.4 ± 0.54.3 ± 0.64.3 ± 0.5
understand why it is important to review records4.4 ± 0.64.4 ± 0.64.4 ± 0.6
think the hand-outs and materials are relevant to the information presented4.5 ± 0.54.4 ± 0.54.5 ± 0.5
think that important points were reviewed and emphasized4.5 ± 0.54.4 ± 0.54.4 ± 0.5
think group participation and questions were encouraged4.5 ± 0.54.2 ± 0.64.4 ± 0.5
think questions were answered sufficiently4.4 ± 0.54.4 ± 0.54.4 ± 0.5
think hand-outs will be useful4.5 ± 0.54.4 ± 0.54.4 ± 0.5
Total score4.4 ± 0.44.3 ± 0.44.3 ± 0.4
Table 3. Rhode Island (RI) and Massachusetts (MA) participant RCCI Module 3 evaluation: Safety plan development, implementation, and monitoring
 Scorea (Ave. ± Std. Dev.)
As a result of this training, I…RIb (n = 30)MAc (n = 23)Total (n = 53)
  1. a

    Score based on 5-point scale (1 = strongly disagree, 5 = strongly agree).

  2. b

    Represents 5 training sessions, one session at each RCCI site.

  3. c

    Represents 3 training sessions, reflecting 5 RCCI sites.

  4. d

    MA evaluation scores significantly different (P < 0.01) than RI scores.

understand how to write my SOPs4.7 ± 0.54.0 ± 0.84.4 ± 0.7
understand how to write my HACCP-based plan4.7 ± 0.53.8 ± 0.74.3 ± 0.7
understand what records I need to complete4.8 ± 0.44.4 ± 0.64.6 ± 0.5
am familiar with the format needed for SOPs4.8 ± 0.44.2 ± 0.54.6 ± 0.5
understand how to group menu items as to “process”4.5 ± 0.64.2 ± 0.64.4 ± 0.6
know how to integrate CCPs and critical limits into my recipes4.4 ± 0.74.0 ± 0.94.3 ± 0.8
understand both the importance of and use for records4.8 ± 0.44.3 ± 0.64.6 ± 0.5
understand the daily requirements of my food safety plan4.8 ± 0.44.4 ± 0.64.6 ± 0.5
feel I can complete the USDA requirements for a food safety plan at my facility4.5 ± 0.54.1 ± 0.94.3 ± 0.7
feel I can carry out the daily requirements of my food safety plan4.6 ± 0.64.3 ± 0.64.5 ± 0.6
understand the daily requirements of my facility's SOPs (ex. hygiene, cleaning/sanitation)4.8 ± 0.44.4 ± 0.54.6 ± 0.5
feel comfortable with implementing this program.4.3 ± 0.74.0 ± 0.94.2 ± 0.8
feel that I will need help with implementing this program3.6 ± 0.73.6 ± 1.03.6 ± 0.9
think the hand-outs and materials are relevant to the information presented4.5 ± 0.64.4 ± 0.54.5 ± 0.6
think the hand-outs are relevant to the operations of our facility4.7 ± 0.54.1 ± 0.84.4 ± 0.7
think that the important points were reviewed and emphasized4.7 ± 0.54.3 ± 0.54.5 ± 0.5
think group participation and questions were encouraged4.6 ± 0.54.3 ± 0.64.5 ± 0.5
think questions were answered sufficiently4.7 ± 0.54.2 ± 0.64.5 ± 0.6
think hand-outs will be useful4.8 ± 0.44.2 ± 0.74.5 ± 0.6
Total score4.6 ± 0.3d4.2 ± 0.5d4.4 ± 0.4

Onsite inspection audits showed improvement (Table 4). Only 4 sites in MA participated in the pre-/posttraining evaluation. There was an increase in food safety plans, SOPs, better temperature control during food preparation, increase in wrapping, dating and inventory control (FIFO) for refrigerated storage, increased use of sanitizer test strips, increased use of records and logs, and increased availability of material and safety data sheets. Some deficiencies still appeared to be problematic such as refrigerator temperature monitoring and issues with storage and cross-contamination. There did appear to be greater improvement in RI sites compared with MA sites. Two different follow-up strategies were used by the outreach specialists. In MA, after the last module, participants were simply told to contact the specialist/instructor if they needed help. This was followed by e-mail contact. There were no requests. In RI, the efforts of the specialist for posttraining follow-up included both e-mails and direct contact by telephone. As a result, RI participants seemed to respond to this direct contact and asked for and received more site-specific aid and guidance. The latter approach appeared, in addition to the higher number of RI workshop participants, to result in a greater overall improvement in the RI sites when the onsite assessment audits were evaluated. For example, the preinspection audit showed that none of the RI sites and 1 of 4 sites in MA had food safety plans. Postinspection showed 4 of 5 sites in RI had food safety plans and only 1 of 4 sites in MA had plans. Preinspection showed no sites in RI and MA had SOPs for employee health. Postinspection revealed all RI sites had SOPs and only 1 of 5 sites in MA had these SOPs. While there was improvement in many areas of implementation, written SOPs for food preparation, cold storage, and cleaning/sanitizing were still deficient. In studying food safety programs in childcare centers, Riggins and Barrett (2008) also found written operational procedures for kitchen practices, equipment cleaning, equipment maintenance, and food safety training, were implemented the least often of the HACCP-based programs. Furthermore, staff turnover was also a problem in these facilities and could have contributed to lack of improvement. However, overall inspections showed improvement in many critical food safety issues in RCCIs for both states.

Table 4. Summary of pretraining and posttraining inspection of pilot RCCI facilities in Rhode Island (N = 5) and Massachusetts (N = 4)
 PreinspectionPostinspection
 YesNoYesNo

Note

  1. Where total of facilities does not equal 9, the rest were not observed or not applicable during inspection.

I. Food safety plan
A. There is a food safety plan in place1854
B. Documentation is collected as required by the food safety plan1863
II. Personal hygiene/employee health
A. SOPs relating to personal hygiene and employee health are in place and adhered to 954
B. Staff/residents wear clean clothing when preparing and serving food8 8 
C. Hair is covered/restrained2535
D. Fingernails are trimmed with no polish619 
E. Jewelry is limited to plain ring, watch619 
F. Hands are washed properly, frequently and at appropriate times6161
G. Chewing gum, smoking, eating, or drinking while preparing and serving food is not allowed316 
H. Sinks used for handwashing are stocked with single-use paper towels and soap6372
I. Water used for handwashing is at least 100 F9 9 
J. People who are sick are not allowed in the kitchen to prepare and/or serve food1 4 
K. Cuts and other wounds are properly bandaged and covered with a waterproof bandage    
III. Food preparation
A. SOPs relating to all aspects of food preparation are in place and adhered to1854
B. There is no bare hand contact with ready-to-eat foods618 
C. Single-use gloves are used properly6 8 
D. Frozen food is properly thawed3 41
E. Fresh fruits and vegetables are washed before preparing and/or serving  1 
F. Food is tasted using the proper procedure    
G. Food is cooked to the required internal temperature and is tested with a calibrated1863
food thermometer and final cooking temperature is recorded in food temperature logs    
H. Leftovers and food prepared for reheating are properly cooled1121
I. Leftovers and other foods are reheated to the proper temperature and final 522
reheat temperature is recorded in food temperature logs    
IV. Food storage—cold
A. SOPs relating to all aspects of storage of cold food are in place and adhered to1854
B. Raw meat, fish, and poultry is stored below ready-to-eat- foods6135
C. All food is wrapped and dated1872
D. Refrigerator/freezer thermometers are used and temperatures are recorded 936
twice a day in the early morning and mid-afternoon    
F. All units are kept clean and orderly639 
G. There is some method of inventory control (FIFO)1827
V. Food storage—dry
A. SOPs relating to all aspects of storage of cold food are in place and adhered to1854
B. All foods and paper supplies are stored 6 inches off the floor and can639 
be easily cleaned under and around    
C. Open bags are stored in containers with tight-fitting lids and labeled4563
D. There are no bulging or leaking canned goods9 9 
E. Cleaning supplies are clearly labeled and stored away from food and food-related supplies9 9 
F. The storage area is kept clean and orderly639 
G. There is some method of inventory control (FIFO)1827
VI. Cleaning and sanitizing
A. SOPs relating to all aspects of cleaning and sanitizing are in place and adhered to2754
B. MSDS are readily available for all cleaning and sanitizing supplies2763
C. Water temperature is correct for wash and rinse cycles7 8 
D. Food preparation equipment including cooking pans and utensils are properly sanitized639 
E. If using a chemical sanitizer, it is mixed properly and test strips are used to test the concentration2763
F. Cutting boards and other food preparation equipment are properly washed and sanitized818 
G. All areas where food is prepared and served are kept clean.6381
VII. Pest control
A. Food preparation areas are kept free of pests9 9 
B. Openings to the outside are tight fitting to prevent pests from entering9 9 
C. There is a pest control program in place9 9 
VIII. Physical facilities
A. Physical facility is maintained and clean8181
B. There is adequate ventilation and lighting9 9 
C. Refuse is properly stored and disposed of9 9 

Microbiological assessment did not detect pathogens; pathogen indicator, Listeria spp, was found once, in one refrigerator drawer pretraining. The facility was notified and was not found posttraining assessment. Table 5 shows the results of the microbiological assessment. From a microbiological perspective, there appeared little improvement between pre- and posttraining and borderline sanitation. Of course, some facilities did better then others, as indicated by the range of contamination. Common areas of microbiological contamination areas in the kitchens were refrigerators (shelves, drawers), preparation areas/cutting boards, and handles (sinks, refrigerators). Based on 9 RCCI facilities and 15 to 16 sites tested per foodservice area, the level of detection was approximately 46% and 39% of all kitchen areas swabbed for microbial indicators of contamination for pre- and posttraining, respectively. Similar results, regarding lack of pathogenic indicators but appearance of total viable bacterial counts, were found when the microbial profile was assessed to validate the food safety management of a foodservice operation (Lahou and others 2012). Even though there were total coliforms present, there were no E. coli detected. Childcare centers showed low-frequency E.coli detection in foodservice areas even when aerobic plate counts (APC) were high (Cosby and others 2008). FLASH tests (data not shown), also showed evidence of protein and reflecting cleaning validation, and was positive for many areas tested—with what appeared to be little overall improvement pre- and posttraining. While onsite inspections indicated that cleaning and sanitizing efforts had improved, it appeared that actual implementation or application still needed assistance. This could possibly be explained by the presence of an inspector. In a study investigating food safety knowledge and behaviors of cooks in Texas childcare centers, Saskel and others (2007) hypothesized that the differences in observed health inspection and researcher food safety behavior scores could have been due to better performance by staff when an inspector was present. A study of food contact surfaces in childcare centers (Cosby and others 2008) found that the extent of bacterial contamination was dependent on the center, time of day, and area sampled. Overall, if the limits of acceptability suggested by Sneed and others (2004) is set as the standard for this study, APC was above the limits of <1.3 log APC/cm2. However, many of these foodservice areas were similar to home kitchens with potential access by clientele whose handling practices were not or assessed. A training CD, targeting students, was provided to staff/managers to encourage outreach to their clientele but this there was no evaluation by this project as to its use.

Table 5. Microbiological assessment of foodservice/kitchen areas of pilot RI (N = 5) and MA (N = 4) RCCI facilties pre- and posttraining
 PretrainingPosttraining
Areas testedSites positiveTACaTCCaYeast/moldbSites positivesTACTCCyeast/mold
  1. a

    log10 MPN.

  2. b

    log10 CFU.

  3. c

    Mean and standard deviation.

  4. d

    Number positive for contaminant.

  5. e

    Range.

  6. TAC, total aerobic plate count; TCC, total coliform count; Lower limit of detection, <0.50 CFU/cm2.

  7. Note: Positive sites may not be positive for all microbiological contaminants. Positive site reflects minimum of one contaminant and is not counted multiple times.

Sink area
Drains142.74 ± 0.56c2.80 ± 0.392.25112.33 ± 0.422.29 ± 0.311.04 ± 0.27
  N = 14dN = 12N = 2 N = 8N = 7N = 3
  1.29–3.34e2.34–2.572.18–2.32 1.32–2.571.81–2.570.82–1.34
Handles91.79 ± 0.701.94 ± 0.692.01 ± 0.4282.05 ± 0.452.05 ± 0.571.49 ± 0.61
  N = 7N = 3N = 5 N = 8N = 5N = 5
  0.85–2.571.15–2.571.60–2.57 1.18–2.571.15–2.571.08–2.57
Faucet41.24 ± 0.652.151.9142.57 ± 0.01.50 ± 0.090.85
  N = 4N = 1N = 2 N = 3N = 3N = 1
  0.70–2.37 1.89–1.93  1.45–1.60 
Refrigerator area
Shelves91.71 ± 0.741.432.10 ± 0.9491.98 ± 0.601.97 ± 0.641.76 ± 0.76
  N = 9N = 2N = 4 N = 8N = 3N = 8
  0.74–2.570.85–2.00.70–2.57 1.04–2.571.23–2.340.85–2.57
Drawers72.03 ± 0.261.95 ± 0.561.64 ± 0.8561.83 ± 0.721.12ND
  N = 6N = 4N = 3 N = 6N = 2 
  1.68–2.341.15–2.340.70–2.34 0.92–2.571.04–1.20 
Handles71.20 ± 0.650.71.12 ± 0.161.70 ± 0.841.891.44 ± 1.0
  N = 7N = 1N = 3 N = 5N = 1N = 3
  0.70–1.99 1.04–1.25 0.70–2.57 0.70–2.57
Preparation area/cutting boards
 101.46 ± 0.661.58 ± 0.701.49 ± 0.8171.61 ± 0.811.95 ± 0.881.34 ± 0.87
  N = 9N = 4N = 4 N = 5N = 3N = 3
  0.70–2.340.70–2.340.70–2.34 0.70–2.570.95–2.570.78–2.34
Microwave keyboard
 41.13 ± 0.44ND2.0131.35 ± 0.06NDND
  N = 4 N = 1 N = 3  
  0.70–1.56   1.28–1.40  

The final evaluation strategy, ease of implementation, and adaptation of the curriculum by other Land Grant programs, was completed. The “mini-grants” awarded provided valuable information that resulted in a final, revised curriculum and instructor's guide. KS completed all modules but Tennessee could only complete the first 2. Using the same participant evaluation protocol, the 13 participants KS program, rated the modules 4.7 ± 0.1 (N = 12), 4.3 ± 0.1 (N = 13), and 4.3 ± 0.3 (N = 12) teaching modules 1, 2, and 3, respectively. The project director/trainers at KS also developed pre-/postknowledge evaluations for their clientele. Knowledge was clearly impacted with correct pre-/postscores of 52% to 87% (N = 12), 48% to 69% (N = 12), and 81% to 86% (N = 11) for modules 1, 2, and 3, respectively. In the TS program, 18 participants evaluated the program and rated the modules 5.0 ± 0.0 (N = 18) and 4.6 ± 0.1 (N = 15) for modules 1 and 2, respectively. Unfortunately, during the implementation of the program, the state of TN had closings of RCCI group homes due to budgetary constraints and module 3 was never taught or evaluated. Overall, both KS and TS project directors found the training materials were well organized and informative and the training achieved its overall goal. Revision suggestions, mostly targeting manual organization and instruction clarifications, by the KS/TS project directors were incorporated into the final curriculum.

Finally, there was expansion of a revised outreach program in RI during this project: 2 workshops with a total of 28 participants reflecting 16 RCCI facilities. Overall, the participants rated the modules 4.8 ± 0.1 (N = 31), 4.4 ± 0.1 (N = 19), and 4.3 ± 0.3 (N = 20) of 5 for teaching modules 1, 2, and 3, respectively (Table 6). Outreach efforts are continuing in RI through a partnership with RI Dept. of Education.

Table 6. RI training evaluation scores for modules 1, 2, and 3 for expanded program
 Scorea
  1. a

    Score based on 5-point scale (1 = strongly disagree, 5 = strongly agree).

  2. b

    Represents 23 facilities and 4 trainings.

Module 1 (n = 31)b 
Information was presented in an organized manner4.7 ± 0.5
Presenters were organized and well prepared4.6 ± 0.5
Hand-outs and materials are relevant to the information presented4.9 ± 0.3
Important points were reviewed and emphasized4.8 ± 0.4
Group participation and questions were encouraged4.7 ± 0.4
Questions were answered sufficiently4.8 ± 0.4
Handouts will be useful4.8 ± 0.4
Food safety principles are understood4.7 ± 0.4
Total score4.8 ± 0.1
Module 2 (n = 27)b 
As a result of this training, I… 
understand the components/principles of an effective SOP4.3 ± 0.6
understand the components/principles of an effective food safety plan4.4 ± 0.5
understand the HACCP principles4.3 ± 0.6
am familiar with the format needed for SOPs4.3 ± 0.5
understand how to categorize menu items as to “process”4.5 ± 0.5
understand how to prepare a “flow chart” of menu items4.4 ± 0.6
understand about major operational steps in foodservice4.5 ± 0.6
can determine control measures for recipe process4.4 ± 0.5
understand the critical control point concept and its application to the HACCP-based food safety plan of my menu items4.3 ± 0.6
understand the critical limit concept and its application to the HACCP-based food safety plan of my menu items4.3 ± 0.6
understand the monitoring concept and its application to the HACCP-based food safety plan of my menu items4.4 ± 0.6
understand the corrective action concept and its application to the HACCP-based food safety plan of my menu items4.4 ± 0.6
understand what validation is and its importance4.3 ± 0.7
understand why it is important to review records4.4 ± 0.6
think the hand-outs and materials are relevant to the information presented4.6 ± 0.5
think that important points were reviewed and emphasized4.6 ± 0.5
think group participation and questions were encouraged4.6 ± 0.5
think questions were answered sufficiently4.6 ± 0.5
think hand-outs were useful4.7 ± 0.5
Total score4.4 ± 0.1
Module 3 (n = 28)b 
As a result of this training, I… 
understand how to write my SOPs4.4 ± 0.5
understand how to write my HACCP-based plan4.4 ± 0.6
understand what records I need to complete4.5 ± 0.5
am familiar with the format needed for SOPs4.4 ± 0.5
understand how to group menu items as to “process”4.5 ± 0.5
know how to integrate CCPs and critical limits into my recipes4.4 ± 0.5
understand both the importance of and use for records4.5 ± 0.6
understand the daily requirements of my food safety plan4.5 ± 0.5
feel I can complete the USDA requirements for a food safety plan at my facility4.2 ± 0.6
feel I can carry out the daily requirements of my food safety plan4.3 ± 0.5
understand the daily requirements of my facility SOPs (ex. Hygiene, cleaning/sanitation)4.4 ± 0.5
feel comfortable with implementing this program4.2 ± 0.6
feel that I still need help with implementing this program3.1 ± 1.1
think the hand-outs and materials are relevant to the information presented4.5 ± 0.5
think that the had-outs are relevant to the operations of our facility4.3 ± 0.6
think that important points were viewed and emphasized4.5 ± 0.5
think that group participation and questions were encouraged4.4 ± 0.5
think questions were answered sufficiently4.4 ± 0.5
think hand-outs will be useful4.5 ± 0.5
Total score4.3 ± 0.3

Conclusion

A food safety training curriculum for RCCI staff to aid in the development of a USDA-mandated food safety plan was successfully developed, implemented, and evaluated. However, while participant evaluations of the training modules and resources were high and onsite pre- and posttraining inspections revealed improvement in food safety protocols, it appears that follow-up instructions or aid would still be necessary to ensure compliance. Trainers may not have the time or resources to do the onsite follow-up required for complete success. While a food safety plan and SOP development and implementation were discussed in detail, accompanied by easy-to-use templates and interactive training, a more hands-on approach would be more effective. Alternate teaching strategies, being considered for implementation in the RI program, could include development of at least a portion of a site-specific food safety plan and a SOP during the workshop. This would require that participants bring menu and recipe items and that the training has access to computers. Another possible strategy would be a follow-up workshop where all prior participants could come with questions. The final curriculum version is posted on the URI food safety Web site: http://web.uri.edu/foodsafety.

Acknowledgments

This study was funded by a grant from the USDA Integrated Research, Education and Competitive Program under agency award Nr 2007–51110- 03816. This study has been assigned Contribution Nr 5326 by the US Dept. of Agriculture at the Univ. of Rhode Island, Agricultural Experiment Station. The authors would also like to acknowledge Steven Carey, Rhode Island Dept. of Education, Rita Brennan-Olson, Massachusetts Dept. of Education, Dr. Junehee Kwon, Kansas State Univ., and Rita Fleming, Tennessee State Univ.

Ancillary