1 Food Environment Introducing the Measures Registry User Guide:Teaching Slide Deck

2 Measures Registry User GuidesThe User Guides are designed to: Provide an overview of measurement Describe general principles of measurement selection Present case studies that walk users through the process of using the Measures Registry to select appropriate measures Direct researchers and practitioners to additional resources and sources of useful information The User Guides cover the four domains for the Measures Registry: Individual Diet Food Environment Individual Physical Activity Physical Activity Environment The User Guides aim to help move the field forward by fostering more consistent use of measures, which will allow for standardization, meta-analyses, and synthesis.

3 In this Presentation Introduction to NCCOR and the Measures RegistryMeasuring Food Environment Measurement Characteristics Measurement in Action

4 Introduction to NCCOR and the Measures Registry

5 NCCOR Mission and GoalsThe mission of NCCOR is to accelerate progress in reducing childhood obesity. NCCOR is focused on: Collaborating to turn the tide on childhood obesity Supporting researchers with tools that help build the capacity for research and surveillance Maximizing outcomes from research and supporting evaluations Using innovation to stimulate broad thinking to generate fresh, synergistic ideas Increasing knowledge to find solutions by promoting new research funding mechanisms and translating and disseminating research findings Looking to the road ahead and building new partnerships to solve problems

6 Collaboration Building on each other’s strengths, CDC, NIH, RWJF, and USDA formed a unique public private partnership – NCCOR

7 Strategic Alliance In 2015, NCCOR formed a strategic partnership with The JPB Foundation to develop User Guides to strengthen the Measures Registry, a landmark tool to help researchers find and select measures for childhood obesity research

8 Measures Registry

9 Measuring Food Environment

10 Why Study the Food Environment?Influences health-related outcomes: Weight status/obesity Dietary patterns Chronic disease Target of public health interventions Robust measures are needed for epidemiologic and intervention studies Over the past decades, public health researchers and practitioners have become increasingly interested in the influence of the food environment on health-related outcomes. Many studies have sought to elucidate the relationship between the food environment and obesity, dietary patterns, chronic disease, and other health-related factors. Furthermore, numerous agencies including the World Health Organization, the Institute of Medicine, and the Centers for Disease Control and Prevention have identified interventions targeting changes in the food environment as strategies for creating population-wide improvements in dietary patterns and weight status. The food environment is recognized as an important piece of the puzzle in understanding population-level obesity risk for both obesity researchers as well as public health practitioners working toward effective community-level approaches to reduce the incidence and prevalence of obesity. Because of this recognized importance, considerations of how we assess the food environment are critically important. Robust measures of the food environment are needed if we want to (1) differentiate the obesity risk between and within population groups and (2) evaluate the ability of interventions (either natural experiments or planned environmental change) in impacting change. The NCCOR registry and User Guides are meant to help evaluate dissemination and implementation of food environmental approaches for reducing obesity risk by providing information and sources of robust measures of the food environment.

11 Dietary-related Disease RiskConceptual Model of Environmental Factors Related to Dietary Disease Risk Dietary-related Disease Risk Dietary Consumption Food Choices PHYSICAL ENVIRONMENT SOCIAL PERSON-CENTERED The food environment includes the physical, social, and person-centered environments that play a role in what people choose to eat. This figure is a conceptual model that includes these three major domains of the food environment arranged in a causal model suggesting how each of these domains are related to each other and to health outcomes of interest (food choice behaviors, dietary consumption, and diet-related disease risk). Measures of the food environment can be conceptualized as fitting into one of these three broad categories. This model suggests that the physical and social environments are directly related to food choices of individuals but that some of their effect is indirect, mediated by factors at the person-centered environment. This model also stresses that the individual is the most proximal actor involved in what foods are chosen. While the individual’s choices are influenced by foods available in their physical environment and elements of the social environment impacting choice, most individuals are left with a great deal of autonomy in what they choose to eat. When food choices are less restricted by economics, accessibility, or availability imposed by the physical environment, the social and person-centered environment may be more predictive in understanding food choices and, ultimately, the impact that the food environment will have on health.

12 1. Physical How many and what types of food venues are present?PHYSICAL ENVIRONMENT 1. Physical Home Childcare, preschool, school, and community venues Stores and restaurants How many and what types of food venues are present? What foods are available? What foods are accessible? What health-related information is present?

13 2. Social Support for healthy food choicesSOCIAL ENVIRONMENT 2. Social Social referents: Other Youth Parents Teachers Other adults Support for healthy food choices Role modeling or social expectation of food choice, eating behavior Food choice incentives or rewards Policies, practices, or rules about eating behavior

14 PERSON-CENTERED ENVIRONMENTPerceptions of the physical environment Availability, access Affordability Acceptability of product Perceptions of the social environment Social norms Social support Perceptions of policies, rules Perceptions of cultural appropriateness

15 Measuring Food EnvironmentPERSON-CENTERED ENVIRONMENT SOCIAL ENVIRONMENT PHYSICAL ENVIRONMENT We’ll now turn to thinking about how we measure these three aspects of the environment.

16 1. Physical PHYSICAL ENVIRONMENT Geo-spatial analyses or GISNumber, location, density of food stores or restaurants Proximity of food stores or restaurants to home, schools or community venues, and each other Observational scans or assessments AKA “log,” “record,” “audit,” “environmental scan” Product availability Pricing Placement/merchandising Advertising and information Measures Registry: GIS; environmental observation; record or log

17 Here is one type of data display possible with GISHere is one type of data display possible with GIS. This figure shows the number of fast food and convenience stores per 1,000 residents in this geographically defined area. From this figure you can see that Berkeley, Piedmont, Castro Valley, and Hayward have the greatest density of stores with between 3.5 and 8.7 outlets per 1,000 residents.

18 Audit Tools, Environmental ScansIn addition to using GIS to collect information on the physical environment, other types of environmental scans or assessments can be used. This picture shows simple, paper-pencil audit forms and use of direct digital entry of data.

19 2. Social Interviews or questionnaires with parents and childrenSOCIAL ENVIRONMENT 2. Social Interviews or questionnaires with parents and children Self-administered/self-report questionnaire Questionnaires of stakeholders/policy decision makers (e.g., food service staff or school principal) Measures Registry: questionnaires; records or logs

20 CDC: School Health Policy and Practice Surveyhttps://www.cdc.gov/healthyyouth/data/shpps/questionnaires.htm https://www.cdc.gov/healthyyouth/data/shpps/questionnaires.htm

21 PERSON-CENTERED ENVIRONMENTPhone interview Self-administered questionnaire Questionnaire read to children Measures Registry: questionnaire, record or log

22 CATCH: Health Behavior Questionnaire

23 Measurement Characteristics

24 Using the Measures Registry: How Does the Conceptual Model Help?What domain or domains do I need to study? Is my purpose to examine predictors of food choice in a population or to examine one aspect of the environment already linked to a health outcome? If I’m interested in studying the physical environment, what venues to I want to assess? If I’m interested in studying the social environment, what aspects and referents should I focus on? Is it important to assess people’s perceptions of their environment? Is my question etiologic or am I testing an intervention?

25 Two Critical Features of Measures: Important in Environmental Measures as Well!RELIABILITY Does the measure used seem to assess the factor of interest? (face validity) Is the measure used related to a gold standard measure of the environment? (criterion) Do the items used to assess the environment include all of the relevant aspects of the environment? (content) Is the environmental measure related to other factors in expected directions? (construct) VALIDITY Do two independent observers record data on the environment in a similar way? (inter-rater) Is there consistency over time in how the environment is assessed? (test-retest) Are items designed to measure the same aspect of the environment correlated? (internal consistency) The “psychometric properties” of a measure are considered as indicators of overall measure quality and generally fall into two categories: reliability and validity. In general terms, reliability is the extent to which a measure is consistent or stable over time. Reliability helps to assess the quality of your questions and instructions in your measurement tool as well as the stability of the abstract concepts that your measurement tool is trying to assess. Validity refers to the ability of a measure to assess what it intends to assess. We often think of four kinds of validity that include face validity, criterion validity, content validity, and construct validity.

26 Measurement in Action

27 List of Included Case StudiesCase Study 1: Study to Evaluate a School-based Intervention on its Ability to Positively Influence the School Food Environment Case Study 2: Study to Evaluate a Family-based Intervention on Its Ability to Reduce BMI-z Scores in Obese Children Case Study 3: Intervention to Improve Healthy Eating Behaviors in Independent Neighborhood Restaurants Case Study 4: Study on Implementing a Farmers Market- based Obesity Treatment Program to Change Purchase and Eating Behaviors for Women and Children Enrolled in WIC/SNAP

28 Case Study 1: Study to Evaluate a School-based Intervention on Its Ability to Positively Influence the School Food Environment

29 Case Study 1: BackgroundSCHOOL Case Study 1: Background Children spend > 1,200 hours in school each year School’s physical food environment influences availability and choice School provides a strong social environment for role modeling and setting expectations regarding eating Early learning at school influences attitudes and beliefs around food and health .

30 Case Study 1: BackgroundSCHOOL Case Study 1: Background School food environment includes: Foods available in the cafeteria as part of federally reimbursed meals Competitive foods available throughout the school including vending and à la carte in the cafeteria, other locations Role modeling of eating behaviors from peers and other adults Policies and practices regarding food used as incentives and reinforcements The school food environment includes: Foods available in the school cafeteria as part of federally supported reimbursable meals Competitive foods available throughout the school including vending and à la carte in the cafeteria as well as vending in other locations Role modeling that occurs around eating from peers and other adults Policies and practices about using food for incentives and reinforcements

31 Case Study 1: BackgroundSCHOOL Case Study 1: Background Project = improving à la carte food healthfulness with a focus on: Calories Added sugars 24 schools in a metropolitan area Intervention partners with School food service staff to increase the availability of healthy foods on the line (the physical environment) Students to increase willingness to purchase healthier options (social and person-centered environments) In this case study, the project team has chosen to focus on improving the healthfulness (based on calories and added sugars) of foods served on à la carte lines in cafeterias. They have successfully recruited 24 schools in a metropolitan area to be involved in a study evaluating the effectiveness of a school-based intervention. The intervention will focus on working with school food service staff to increase the availability of healthy foods on the line (the physical environment) as well as on increasing students’ willingness to purchase healthier options (social and person-centered environments).

32 Case Study 1: BackgroundSCHOOL Case Study 1: Background Group-randomized, pre-post design Primary outcome: Change in foods sold in the à la carte line Primary hypothesis: There will be an increase in the proportion of healthier foods and beverages sold on the à la carte line Secondary outcome: Change in the foods available to students Secondary hypothesis: There will be more healthy items available to students on the à la carte line The team is planning a group-randomized, pre-post design where their primary outcome is examining change in foods sold on the à la carte line (an individual-level outcome). They also want to know if the intervention was successful in positively influencing the the foods available to students (an environmental outcome). Their primary hypothesis is that following the intervention, there will be an increase in the proportion of healthier foods and beverages sold on the à la carte line. Their secondary hypothesis is that following the intervention, there will be more healthy items available to students on the à la carte line.

33 Case Study 1: ConsiderationsSCHOOL Case Study 1: Considerations How will they document what is sold on the à la carte lines before and after the intervention? How will they document what is available on the à la carte lines before and after the intervention? How often and when will data collection occur? What resources are available for data collection, cleaning, and analysis?

34 Case Study 1: ConsiderationsSCHOOL Case Study 1: Considerations Will they collect sales data directly, or will they need to interview or observe students as they purchase food? What is the relationship with the school staff? Are they happy collaborators or have little say or interest in the change to à la carte? How much detail on the foods available and sold is necessary to evaluate the success of their intervention? Selection considerations: For their primary outcome (sales data) the team has already verified that the schools will be able and willing to provide sales data that can detail daily food item purchases. The investigators also want to assess foods and beverages available in the schools before and after the intervention period as process data. They want to be able to verify that the intervention was delivered as planned, and there was an increase in healthy foods being offered on à la carte in the intervention schools. The choices they consider for documenting what is available on the à la carte line are: 1) ask the cook manager for purchase orders from vendors; 2) ask the cook manager to list all of the à la carte items available; 3) have evaluation staff document all of the items available on à la carte using an inventory; or 4) have evaluation staff document what is available using a checklist of types of foods.

35 Case Study 1: Measures SelectionSCHOOL Case Study 1: Measures Selection Decisions: Items on à la carte change quickly, so multiple data collection days need to be scheduled for both the pre and post phases of data collection They can use direct sales data from cash registers to collect data on sales Study staff will need to collect the data on foods available on the line, but resources are limited As they make these decisions they need to consider the following: How many data collection periods will there be? What resources are available for data collection, cleaning and analyzing the data? What is their relationship with the school staff? Are school staff at all of the schools involved willing and able to provide the data required? What level of information is needed for the study? Is it satisfactory to know more basic information such as the proportion of healthy foods to less healthy food types available in à la carte or is there a desire to know average calories or grams of fat available from foods on à la carte? As part of their formative assessment, they discover that purchase orders from vendors were not available and that not all schools involved were willing and able to follow a study measurement protocol. Therefore, study staff will need to collect the data. Formative assessment also showed that some of the schools have dozens of à la carte items and that the items change frequently. Therefore, the researchers decided to look for a valid and reliable data collection tool to be completed by the measurement team to document the foods available on à la carte.

36 Case Study 1: Measures SelectionSCHOOL Case Study 1: Measures Selection Select the “Food Environment” domain, enter “School” as measure type, find >100 matches One study (TACOs) was found that used sales data from cash register information to collect data on sales. TACOs also had a food inventory measure, but the protocol was deemed to be too labor intensive and beyond the study’s resources. The team selects the “Food Environment” domain, and then enters “School” as the measure type and finds more than 100 matches. They identify several measures that focus on assessing à la carte sales in schools. One study (TACOs) was found that used sales data from cash register information to collect data on sales. TACOs also had a food inventory measure, but the protocol was deemed to be too labor intensive and beyond the study’s resources.

37 Case Study 1: Measures SelectionSCHOOL Case Study 1: Measures Selection One study (IDEA/ECHO) was found that used an inventory approach with information on à la carte items collected by study staff À la carte items classified as more or less healthful using a set criteria Criterion validity of the scale had been established The inventory form could be used, or slightly adapted, to meet their study needs and budget One study (IDEA/ECHO) was found that used an inventory approach with information on à la carte items collected by study staff. À la carte items were classified as meeting or not meeting pre-determined criteria for total calories and added sugars. Criterion validity of the scale had been established. The inventory form could be used, or slightly adapted, to meet their study needs and budget.

38 Case Study 1: Summary TableTitle: Study to Evaluate a School-based Intervention on Its Ability to Positively Influence the School Food Environment Type of Case Study Intervention Background School-based obesity prevention intervention to change à la carte offerings in 24 metro area middle school cafeterias Primary outcome is foods sold on à la carte using sales data from cash register receipts Secondary outcome is foods available on à la carte Considerations Schools are willing/able to provide daily detailed sales data Study staff will need to assess food and beverage availability Measures Registry > Food Environment > Schools Measure Selection TACOs for primary outcome IDEA/ECHO checklist for documentation of available food Case study 1 (research based): Research study to evaluate a school-based intervention on its ability to positively influence the school food environment Project purpose: A research team is planning a school-based obesity prevention intervention attempting to change a la carte offerings in middle school cafeterias. The primary outcome of the study is foods sold in the cafeteria using sales data from cash register receipts. Twenty-four schools in a metropolitan area have been recruited to participate in the study. Selection considerations: For their primary outcome (sales data) the team has already verified that the schools will be able and willing to provide sales data that can detail daily food item purchases. The investigators also want to assess foods and beverages available in the schools before and after the intervention period as process data. They want to be able to verify that the intervention was delivered as planned, and there was an increase in healthy foods being offered on à la carte in the intervention schools. The choices they consider for documenting what is available on the à la carte line are: 1) ask the cook manager for purchase orders from vendors; 2) ask the cook manager to list all of the à la carte items available; 3) have evaluation staff document all of the items available on à la carte using an inventory; or 4) have evaluation staff document what is available using a checklist of types of foods. As they make these decisions they need to consider the following: How many data collection periods will there be? What resources are available for data collection, cleaning and analyzing the data? What is their relationship with the school staff? Are school staff at all of the schools involved willing and able to provide the data required? What level of information is needed for the study? Is it satisfactory to know more basic information such as the proportion of healthy foods to less healthy food types available in à la carte or is there a desire to know average calories or grams of fat available from foods on à la carte? As part of their formative assessment, they discover that purchase orders from vendors were not available and that not all schools involved were willing and able to follow a study measurement protocol. Therefore, study staff will need to collect the data. Formative assessment also showed that some of the schools have dozens of à la carte items and that the items change frequently. Therefore, the researchers decided to look for a valid and reliable data collection tool to be completed by the measurement team to document the foods available on à la carte. Selected measures: The researchers used the registry entering “School” as a search tool and found more than 100 matches. Within that search, they found reference to the TACOS study (French et al 2004) that was an intervention attempting to positively affect change in the foods offered on à la carte lines in 24 secondary schools. Their dependent variable was the use of sales data using cash register information. This study design and primary outcome matched the researchers’ objectives well so they chose to use the protocol from the TACOS study to collect data for the primary outcome. TACOs used a comprehensive inventory to document all of the foods available in the à la carte line. Trained study staff went into the 24 schools before randomization and documented for each à la carte item available. Information collected on all foods available for sale in à la carte areas included brand name, package size, serving size, and grams of fat per serving. Teams of two or three TACOS staff members met with kitchen managers at each school to review and verify the à la carte food list; however, TACOS staff made return visits to the school and also follow-up telephone calls to food service staff and food manufacturer representatives to gather details about foods offered and their nutritional information. The list of individual food products was grouped under several broader categories during the data collection process based on foods similar in fat or other nutrients of interest or foods that composed a large share of à la carte sales (e.g., pizza, bagels, soft pretzels; each of these had its own category due to large sales volume for these items). Foods were organized into 24 categories such as chips and crackers; candy/candy bars; and fruits and vegetables. The foods available were entered into a nutrition software package (Nutrition Data System) and details on macro and micro nutrient content of the foods were available. The researchers considered this option, liking the detail available on both the foods and nutrients available in the school à la carte lines. But they were concerned that they did not have the staff, time, or other resources to collect the data at the schools or to do the required data entry, cleaning and analysis of the dietary data. They continued looking on the registry and found an article that used a similar inventory system to collect data on à la carte items in middle schools as part of the TREC IDEA study. This group had found the complete inventory approach too burdensome to collect in the high schools in their study since the high schools had a great number of products available. They developed a simple checklist that included a list of 20 categories of foods based on CDC’s School Health Policy and Practice Survey and conducted a validation study in 38 schools to determine if the healthfulness rating between the inventory approach and the checklist approach would similarly rank schools. To determine the healthfulness of à la carte offerings in the middle schools using the inventory method, each food and beverage item on the inventory was classified as not meeting (score = 0) or meeting (score = 1) IOM12 criteria. IOM criteria include (1) food servings with less than 35% of calories from fat, (2) food servings equal to or less than 200 calories, (3) a serving size of less than 4 ounces of 100% fruit juice for middle school students, and (4) water without additives or carbonation. A total score representing the proportion of foods and beverages offered that met the IOM criteria was created for each school. Based on this information and the consideration for the resources available in their study, the group decided to use the TACOs data collection method for their primary outcome and the IDEA/ECHO checklist for their documentation of foods available on the à la carte lines.

39 Case Study 2: Study to Evaluate a Family-based Intervention on Its Ability to Reduce BMI-z Scores in Obese Children

40 Case Study 2: BackgroundFAMILY Case Study 2: Background Youth obesity risk related to family and home environment Eating, activity, and sedentary behaviors all influenced by home/family physical and social environments Home/family either protects or creates risk Obesity treatment must include home/family Obesity risk for youth is strongly related to the family and home environment. A youth’s eating, activity, and sedentary behaviors are all influenced by both the physical and social environments of the home and family. The home and family can serve as either a protective or risk factor for youth obesity risk. Obesity treatment for youth must have strong family and home intervention components to be successful.

41 Case Study 2: BackgroundFAMILY Case Study 2: Background Home-based factors for obesity risk: Physical – foods and beverages, amount of media and physical activity equipment available Social – parental role modeling of eating, activity, and sedentary behavior; parental parenting practices around weight-related behavior Person-centered – youth assimilate attitudes, beliefs,and values around health and health behaviors from their families Many factors can influence obesity risk in the home including elements of the: Physical environment including foods and beverages available and the amount of media and physical activity equipment available Social environment including parental role modeling of eating, activity, and sedentary behavior and parental parenting practices around weight-related behaviors Person-centered environment as youth assimilate attitudes, beliefs, and values around health and health behaviors from their families

42 Case Study 2: BackgroundFAMILY Case Study 2: Background Project = evaluate a family-based intervention to reduce obesity risk in obese 8- to 10-year-olds 100 families recruited from area clinics Randomized controlled trial Intervention = Foods available in the home (physical environment) Foster positive parenting practices (social environment) Foster positive attitudes around child eating behavior (person-centered) In this case study, the project team has chosen to focus on evaluating a family-based intervention to reduce obesity risk in 8-10 year olds that have already been diagnosed with obesity. They have recruited 100 families from area clinics to participate in a randomized control trial. The intervention will focus on foods available in the home (physical environment) and fostering positive parenting practices (social environment) and positive attitudes around child eating behavior (person-centered).

43 Case Study 2: BackgroundFAMILY Case Study 2: Background Primary outcome: Body mass index (BMI) Primary hypothesis: BMI trajectory of those exposed to the intervention will show slower gain as compared to students in the control condition Secondary outcomes: Foods available in the home; parenting practices and attitudes around children’s eating behaviors Secondary hypotheses: The intervention will result in healthier foods in the home, greater use of positive parenting practices, and more positive parental attitudes around their child’s eating behavior The team is planning a individual randomized control trial where their primary outcome is body mass index (BMI) measured by trained data collectors. They also want to know if the intervention was successful in positively influencing foods available in the home and on parenting practices and attitudes around children’s eating behaviors. Their primary hypothesis is that following the intervention, BMI trajectory of those exposed to the intervention will show slower gain as compared to students in the control condition. Secondary hypotheses are that the intervention will result in a healthier foods in the home, greater use of positive parenting practices, and more positive parental attitudes around their child’s eating behavior.

44 Case Study 2: ConsiderationsFAMILY Case Study 2: Considerations The team has experience in measuring BMI and assessing change in the child’s diet, but they are looking for ways to assess: Foods available in the home Parenting practices Parental attitudes about their child’s eating behaviors

45 Case Study 2: ConsiderationsFAMILY Case Study 2: Considerations What specific aspects of the home food environment will the intervention target? Will the intervention target a wide range of foods available or just a few types of foods? Is there a valid and reliable instrument available? Are there a valid and reliable tools for assessing parenting practices and parents’ attitudes related to obesity?

46 Case Study 2: ConsiderationsFAMILY Case Study 2: Considerations How do we minimize response burden and response bias? How many times do we need to assess the home food environment? How much do foods in the home change day-by-day? As part of parents’ consent to be in the study, access to their homes and their willingness to complete study-related surveys has been agreed to.

47 Case Study 2: Measures SelectionFAMILY Case Study 2: Measures Selection Select “Food Environment” and enter “Home food inventory” as a search option Decides: Assess the larger home food environment, not just specific foods Eliminate measures that are not appropriate Age (i.e., preschool age children) Population characteristics (WIC participants) The team begins their search by selecting “Food Environment” and entering “Home food inventory” as a search option. The team decides that they want to assess the larger food environment so they can pick up potential increases in healthy foods available as well a potential decreases in less healthy foods. Some selections are immediately and easily eliminated because they did not match this study’s target age group (i.e., preschool age children) or population characteristics (WIC participants).

48 Case Study 2: Measures SelectionFAMILY Case Study 2: Measures Selection Home Food Inventory found that: Assesses a wide variety of foods Can be completed by parents in a short amount of time Can be used to construct an obesogenic food index from the data Has good reliability and validity, including construct validity between obesogenic score and energy intake of parents and youth Home Food Inventory found: Assesses a wide variety of foods. Can be completed by parents in a short amount of time. Creates information on how to construct an obesogenic food index from the data. Was evaluated and found to have good reliability and validity, including construct validity between obesogenic score and energy intake of parents and youth.

49 Case Study 2: Measures SelectionFAMILY Case Study 2: Measures Selection Choose Child Feed Questionnaire (CFQ) because: It included seven scales related to child eating behavior and the family-food environment including parenting practices related to parents pressuring their child to eat, parental monitoring of child behavior, and parents’ concern about their child’s eating It had been tested in three different, diverse samples of parents and found to be valid and reliable Construct validity showed a significant relationship between scores on the CFQ and child weight status The team finds three options for an instrument to assess parenting practices and attitudes related to child’s eating behavior. They choose the Child Feed Questionnaire (CFQ) because: It included seven scales related to child eating behavior and the family-food environment including parenting practices related to parents pressuring their child to eat, parental monitoring of child behavior, and parents’ concern about their child’s eating It had been tested in three different, diverse samples of parents and found to be valid and reliable Construct validity showed a significant relationship between scores on the CFQ and child weight status

50 Case Study 2: Summary TableTitle: Study to Evaluate a Family-based Intervention on Its Ability to Reduce BMI-z Scores in Obese Children Type of Case Study Intervention Background Evaluating effectiveness of family-based obesity treatment intervention for children ages 8–10 (RCT; n=100 families) Need to characterize obesogenic nature of the homes and change in-home and family social environment related to the intervention Considerations Need to assess foods available in the home Need to assess parenting practices and attitudes related to children’s eating behavior Measures Registry > Food Environment > Home Food Inventory Measure Selection Home Food Inventory Child Feeding Questionnaire Case study 2 (research based): Research study to evaluate a family-based intervention on its ability to reduce BMI-z scores in obese children Project purpose: A research team is planning to evaluate the effectiveness of a family-based obesity treatment intervention for children ages 8–10 using a randomized controlled trial with 100 families. Their primary outcome is change in child’s BMI-z score and the secondary outcomes are child-level caloric intake and Healthy Eating Index (based on four 24-hour recalls). Interventionists will work with families randomized to the intervention condition to help change the foods available in the home and on fostering positive parenting practices and attitudes around child eating behavior. Selection considerations: The research team has experience in collecting anthropometric data to assess BMI-z score and they also have experience in collecting and analyzing 24-hour recall data. They are looking to the Measures Registry for resources to assess foods available in the home and to assess parenting practices and attitudes related to children’s eating behavior. These data will be used to characterize the obesogenic nature of the homes as well as to assess change in the home environment related to the intervention. What are the specific environmental targets that the intervention is attempting to change? Will the intervention target all foods in the home or just selected foods (for example, increasing fruits and vegetables or eliminating soft drinks in the home environment)? Are there valid and reliable existing parenting practice and attitude scales for children ages 8–10? Who will collect the data? What resources are available for cleaning and analyzing the data? Selected measures: They began the search by selecting “food environment” and entering “home food inventory” as a search option. Of the options available, several were immediately eliminated because they were for the wrong age group (i.e., infants, preschool) or population (i.e., WIC participants or Spanish/Somali populations). The investigators decided they were interested in a more comprehensive picture of the home food environment and therefore eliminated the inventories that were designed to assess only fruits and vegetables or only packaged foods using UPC codes. Of the options left, one (Home Food Inventory (HFI), Fulkerson et al, 2008) met some important criteria: (1) a wide range of foods in the home were assessed; (2) the instrument could be completed by parents; (3) information on how to construct an obesogenic index from the HFI was delineated; (4) inter-rater reliability had been tested and shown to be good; (5) criterion validity had been tested using research staff as the criterion and shown to be very good; and (6) construct validity had been tested and shown to be acceptable, including a significant association between the constructed obesogenic home food availability score and parent and child’s energy intake. In addition, the instrument was available as a download on the NCCOR Measures Registry site. Next they turned to looking for an appropriate instrument to assess parental practices and attitudes related to child’s eating behavior. They included the 6–11 age group and entered “parenting practices” as a search term. Several good options emerged. Larios et al (2008) reported on a bilingual (Spanish and English) survey administered to Latino mothers about parenting strategies for eating and activity. The constructs assessed included limit-setting, monitoring, discipline, control, and reinforcement. Reliability related to internal consistency was assessed and found to be good and predictive, and concurrent validity was also assessed and found to be good. In particular, correlations with child’s BMI-z score were in the expected direction for all five of the constructs. The questions used were available in the article as published. The Gattshall et al article (2008) included two scales assessing parental role modeling around healthy eating and parental policy on healthy eating that were developed for and tested with overweight children between the ages of 8–12. These scales showed good reliability as well as construct validity; parental role modeling and parental policies were related to child and parent eating habits. The Child Feeding Questionnaire (CFS) (Birch et al, 2001) was another good option and included seven scales assessing the following constructs related to child eating behavior and the family food-related environment: perceived responsibility of parents; parents’ perceptions of their own weight during the life course; parents’ perceptions of their child’s weight; parents’ concern about their child’s weight; food restriction in the home; parent practices related to pressuring their child to eat; and parental monitoring of their child’s eating behavior. The scales were tested in three different samples of parents, including Hispanic mothers and fathers. Internal consistency of the scales was shown to be good, and construct validity linking scores on the CFS and child weight status was confirmed. Since the CFS included the broadest interpretation of parental food practices and attitudes, had been developed, tested, and used in similar age samples, and was found to have good psychometric properties, the investigators opted to use the CFS.

51 Case Study 3: Intervention to Improve Healthy Eating Behaviors in Independent Neighborhood Restaurants

52 Case Study 3: BackgroundNEIGHBORHOOD Case Study 3: Background Neighborhood environment includes foods available in restaurants, pricing and promotion of those foods Small, independent restaurants may be open to working with community change agents to provide healthier options at competitive prices The neighborhood food environment includes foods that are available in restaurants and the pricing and promotion of those foods. While it is difficult for communities to promote change in menus and pricing of foods for large, national chain restaurants, smaller independent restaurants may be more open to working with community change agents to provide healthier options at competitive prices.

53 Case Study 3: BackgroundNEIGHBORHOOD Case Study 3: Background Large city health department is working with the local restaurant association to improve residents’ access to healthy foods in locally owned and independently operated restaurants (physical environment) 2-year project Baseline data collection Follow-up data collection at 2 years Project goal: Identify change in availability and pricing and changes in menu sales over 2 years In this case study, a large city health department is working with the local restaurant association to improve residents’ access to healthy foods in locally owned and independently operated restaurants (physical environment). The project will occur over 2 years with baseline data collection and follow-up data collection at 2 years. The goal of the project is to identify change in availability and pricing and changes in menu sales over 2 years. This case study represents a community-based program evaluation rather than a randomized controlled intervention trial.

54 Case Study 3: ConsiderationsNEIGHBORHOOD Case Study 3: Considerations Recruit independent restaurant owners Training health department and restaurant association staff to collect data on: Menus in each restaurant (including types of foods offered, serving size, and price per serving) Contextual factors that may influence patrons’ decision making about menu selection (such as menu labeling or nutrition information) Sales records from before and after the program begins Independent restaurant owners in the area served by the city health department will need to be recruited to participate in the program evaluation. The project will require training health department and restaurant association staff to collect data on: Menus in each restaurant (including types of foods offered, serving size, and price per serving) Contextual factors that may influence patrons’ decision making about menu selection (such as menu labeling or nutrition information) Sales records from before and after the program begins

55 Case Study 3: Measures SelectionNEIGHBORHOOD Case Study 3: Measures Selection Select the “Food environment” domain, the “Environmental Observation” measure type and the “Metro/Urban” context Nearly 100 matches show up Eight matches look like they are relevant to the project and meet the resource scope

56 Case Study 3: Measures SelectionNEIGHBORHOOD Case Study 3: Measures Selection The team leader clicks on “Compare” for the eight most relevant measures that emerged from the original search All eight measures report on the validity and reliability of their tool Only four of the candidates make the complete measurement tool available on the Measures Registry

57 Case Study 3: Measures SelectionNEIGHBORHOOD Case Study 3: Measures Selection Of the four measures that include the complete tool on the Measures Registry, the NEMS-R tool has some advantages including: Has been widely used Offers free training in how to use and analyze the data Has demonstrated reliability

58 Case Study 3: Measures SelectionNEIGHBORHOOD Case Study 3: Measures Selection Choose NEMS-R – but it has limited evidence of construct validity Team designs ways to evaluate the construct validity as part of the pilot testing of the tool Examine relationships between constructs: Prices of healthier entrees versus less healthful entrees Sales of healthier entrees versus less healthful entrees

59 Case Study 3: Summary TableTitle: Intervention to Improve Healthy Eating Behaviors in Independent Neighborhood Restaurants Type of Case Study Program evaluation Background Large city health department with goal to prevent obesity and chronic disease and promote economic development Collect baseline and follow-up data on foods and prices in locally owned and operated restaurants Considerations Data collectors are health department and restaurant association staff Measures Registry > Food Environment > Environmental Observation > Metro/Urban > Compare Measure Selection NEMS-R: Instrument is freely available, has been widely used, includes a free online training, and has demonstrated reliability Case study 3 (practice-based): Practice-based intervention to improve healthy eating behaviors in independent neighborhood restaurants Project purpose: A large city health department is working with the local restaurant association to improve healthy eating behaviors within independent neighborhood restaurants. Their goal is to prevent obesity and chronic disease among city residents and promote economic development. The project involves baseline data collection of the availability and prices of healthy options, an intervention to support restaurant owners as they revise their menus, and repeated data collection at the end of the 2-year project. Their goal is to identify change in availability and pricing over time and changes in menu item sales. Selection considerations. The project partners are interested in working with restaurants to increase healthy food offerings at prices that encourage consumption. After recruiting independent restaurants who serve populations most affected by diet-related chronic diseases, they must train health department and restaurant association staff to collect data about menu offerings (e.g., types of food, serving size, price per serving), contextual factors in the restaurant that may influence decision making (e.g., presence of menu labeling), and analysis of a sample of sales records from before and after the intervention. A team leader navigates to the NCCOR Measures Registry to identify existing measures that can be used verbatim or adapted for the study. To narrow the choices, the team leader selects the “Food Environment” domain, the “Environmental Observation” measure type, and the “Metro/Urban” context. Selected measures: The team leader scans the measure names on the list of nearly 100 matches for words that are most relevant to the study purpose (e.g., restaurant, menu, and price). Based on these additional criteria, the team leader clicks “Compare” on the eight most relevant measures. They consider the Food Price Comparison (FPC) (Reid, 1997), Food Price Surveys (FPS) (Latham, 2007), Healthy Food Availability and Pricing Checklist (HFAPC) (Guy, 2004), Marketing and Availability of Healthy Options in Restaurants (MAHOR) (Lewis, 2005), Menu Checklist on Healthy Choice Cues (MCHCC) (Cassady, 2004), Nutrition Environment Measures Survey – Restaurant (NEMS-R) (Saelens, 2007), Price and Availability Indices of Healthy Food (PAIHF) (Donkin, 2000), and Restaurant Physical Environment Profile (RPEP) (Lewis, 2005). Although all eight candidate measures have known validity and reliability, which is important to every project, only four of the candidate measures make the complete instrument available. Therefore, measures without available instruments are ruled out (FPC, FPS, MCHCC, and PAIHF). The team leader reviews the four remaining options with project partners (HFAPC, MAHOR, NEMS-R and RPEP). Given that the NEMS-R has been widely used, offers a free training, and has demonstrated reliability, it is chosen for this project. However, given limitations in established construct validity, project leaders decide to structure their work so that they can contribute to the field by testing for evidence of construct validity in the relationships between, for example, the sum (price) of individual items compared to a combo meal, prices of healthy entrées compared to regular ones, presence of charge for a shared entrée, or price for smaller portion compared to regular portion and hypothesized sales of “healthy” versus “unhealthy” options.

60 Case Study 4: Study on Implementing A Farmers Market-based Obesity Treatment Program to Change Purchase and Eating Behaviors for Women and Children Enrolled in WIC/SNAP

61 Case Study 4: BackgroundFARMERS MARKETS Case Study 4: Background Program initiated by a state department of health and human services Community-based program Change purchasing and eating behaviors for women and children receiving WIC or SNAP benefits Goal: Improve overall dietary intake Reduce obesity risk A state department of health and human services is working with other state and county stakeholders to implement a statewide program to change purchasing and eating behaviors for women and children receiving WIC or SNAP benefits. This program is designed to improve overall dietary intake as well as reduce obesity risk. This case study represents a community-based program evaluation.

62 Case Study 4: BackgroundFARMERS MARKETS Case Study 4: Background Program involves nutrition educators employed through WIC/SNAP Evaluating progress over 1 year Intervention components: Group classes on preparing fruit and vegetable dishes with an emphasis on canned and frozen foods Marketing and promotion of fruits and vegetables in WIC/SNAP stores and farmers markets Home visits to provide skill building Quarterly check-ins with nutritionists This program will involve nutrition educators employed through WIC and SNAP statewide and will be evaluated over the course of 1 year. The state-wide program will consist of: Group classes on preparing fruits and vegetables dishes with an emphasis on canned and frozen foods Marketing and promotion of fruits and vegetables in stores and farmers markets honoring WIC and SNAP benefits Home visits to provide skill building Check-ins with a nutritionists

63 Case Study 4: ConsiderationsFARMERS MARKETS Case Study 4: Considerations Nutritionists will collect BMI data at home visits and conduct surveys with parents to assess changes in attitudes, skills, and eating behavior The team is looking for ways to assess foods, particularly fruits and vegetables, available in the home

64 Case Study 4: Measures SelectionFARMERS MARKETS Case Study 4: Measures Selection Select “Food Environment,” “Environmental observation” measure type, add “home” to the search box More than a dozen measures are found The team leader scans the list to look for the options that appear most relevant

65 Case Study 4: Measures SelectionFARMERS MARKETS Case Study 4: Measures Selection Many of the potential options are easily eliminated since they don’t fit the population or the focus of this study One measure, Exhaustive Home Food Inventory for WIC Participant Households, looks promising

66 Case Study 4: Measures SelectionFARMERS MARKETS Case Study 4: Measures Selection Advantages of the measure: Scanned Universal Product Codes to input objective data on all canned and frozen fruit and vegetable products found in the home Quickly administered in the home by data collectors who do not need extensive training Data linked to a reference database providing excellent profiles of all foods found in the home

67 Case Study 4: Measures SelectionFARMERS MARKETS Case Study 4: Measures Selection Disadvantages: Reliability and validity data are limited Must build that in to the project Collects processed fruit and vegetable data only, not fresh Consider adding items Collect purchase occasion data to know how long frozen/canned items have been in the home Must build that in to the project:

68 Case Study 4: Summary TableTitle: Study on Implementing a Farmers Market-based Obesity Treatment Program to Change Purchase and Eating Behaviors for Women and Children Enrolled in WIC/SNAP Type of Case Study Program evaluation Background State health department and Farmers Market Coalition partnership to change purchase and eating behaviors Outcomes are parent BMI, child BMI, changes in home food environment, fruit and vegetable consumption Considerations Home food environment will be measured by both participants and health department staff Measures Registry > Food Environment > Environmental Observation > Search: “Home” Measure Selection Exhaustive Home Food Inventory for WIC Participant Households Case study 4 (practice-based): Study on implementing a farmers market-based obesity treatment program to change purchase and eating behaviors for women and children enrolled in WIC/SNAP benefits Project purpose: A state department of health and human services is partnering with the statewide farmers market coalition to implement an obesity treatment intervention to change purchase and eating behaviors for women and children receiving WIC/SNAP benefits. The intervention includes classroom-based training including preparing fruits and vegetable dishes from canned and frozen foods, marketing and promotion of fruits and vegetables in stores, home visits, and check-in calls with a nutritionist. The primary outcome for the year-long project is parent and child BMI. Secondary outcomes are changes in home food environment and fruit and vegetable consumption. Selection considerations: One important component of the project is to alter the home food environment changes as a result of the intervention. The home food environment will be measured both by the project participants (the WIC/SNAP benefit recipient who lives in the home) and the project administrators. A team leader navigates to the NCCOR Measures Registry to identify existing measures that can be used verbatim or adapted for the study. To narrow the choices, the team leader selects the “Food Environment” domain, the “Environmental Observation” measure type, and adds the word “home” to the Search box to yield more than a dozen choices. The team leaders scan the measure names on the list for words and phrases in the titles that appear most relevant to the study. Selected measures: Most of the measures that are listed in the results from this search do not seem relevant to the home environment given their titles: Assessment of Worksite Canteen Lunches (Lassen A, 2007); Availability of Nutrition Information from Chain Restaurants (Wootan MG, 2006); Food Desert Identification (Guy CM, 2004); Food Price Comparison (Meid M, 1997); Healthy Food Availability and Pricing Checklist (Guy CM, 2004); Healthy Food Pricing for 5 to 16 Year Olds (Banks J, 2012); Marketing and Availability of Health Options in Restaurants (Lewis LB, 2005); Price, Availability, and Variety of Fruit and Vegetables (Winkler E, 2006); Restaurant Physical Environment Profile (Lewis LB, 2005). However, one measure looks promising, Exhaustive Home Food Inventory for WIC Participant Households (Stevens J, 2012). According to the registry, this food inventory has objective measures of food quality for all foods in the home. It needs to be administered by project or research staff through direct in-person observation. Training is required to complete the measures; however, the time to training and time to administer the measure is not reported. The “How to Use” tab for the Exhaustive Home Food Inventory includes information about the data collection protocol. It is administered by collecting information through Scanned Universal Product Codes (UPC), which are then transferred to a laptop computer, and linked to a reference database. Given the close match to this project, the team opts to use this measure.

69 Conclusions Finding the appropriate measurement tool is an essential step in any research project or program evaluation. Be certain the tool you choose meets the specific needs of your project and is appropriate for your population of interest. Look for one that has some demonstrated reliability and validity, and try to contribute to reliability and validity in your study. Choose a tool that will provide the most rigorous measure possible given your project resources. There is no PERFECT tool! Do the best you can.

70 Questions? For More Information: