Childhood obesity is a health problem that cannot be overlooked because it is increasing worldwide [1]. In 2013, almost a quarter of children in developed countries and more than 10% in developing countries were overweight or obese based on the body mass index (BMI) [2]. Children with obesity are likely to remain obese in adulthood [3, 4]. They are more at risk of developing adult health problems, such as type 2 diabetes, heart disease, stroke, various types of cancer, and osteoarthritis [5, 6, 7]. The dietary and physical activity behaviors of children are affected by many social factors, and schools play a vital role by establishing a secure and cooperative environment with policies and practices to support healthy behaviors [8].

To carry out an intervention program effectively, there is a need for a systematic and scientific method to evaluate the effects of the program objectively on various levels [9, 10]. The RE-AIM framework developed by Glasgow et al. has been used to assess the overall health impact of health promotion programs [11, 12]. The RE-AIM framework aims to assess the effectiveness of the health promotion program and seek more efficient resource allocation in the future with reference to five dimensions: reach, efficacy, adoption, implementation, and maintenance [11]. According to a systematic review [13] of interventions focused on applying the RE-AIM framework, 26 out of the 71 articles published from 1999 to December 2010 pertained to physical activity and obesity; only two were conducted in the school setting. Thus, few studies on physical activity and nutrition intervention in a school setting have been evaluated using the RE-AIM framework.

Mobile phones and the transtheoretical model (TTM) allow for more systematic and effective implementation of intervention programs. Intervention programs using mobile phones have the advantage that service users can easily receive tailored information anytime, anywhere [14]. Mobile-based approaches are beneficial for implementing successful interventions on eating habits and physical activity [15]. The TTM, developed by Prochaska et al. [16], has been used in intervention studies to assess various behavioral changes, enabling tailored education at the stage of behavioral change [17]. Mobile technology or TTM have been applied to various interventions in adults, but limited studies have used the same with children [18]. In addition, few studies have applied new technology and the TTM simultaneously [19].

This study developed feasible evaluation criteria of the RE-AIM framework for application to an elementary-school-based health promotion program. First, potential evaluation criteria were proposed with an analysis of previous studies that evaluated the elementary-school-based health promotion program using the RE-AIM framework. Second, the feasibility of the RE-AIM framework was evaluated using the authors’ elementary-school-based pilot health promotion program.

This study aimed to identify evaluation criteria used in the RE-AIM framework using a systematic review and evaluated the feasibility of the RE-AIM framework using an elementary-school-based pilot health promotion program. The systematic review revealed no studies in Korea evaluating the effectiveness of health promotion programs for elementary-school students using the RE-AIM framework. Thus, four international studies were selected [22, 23, 24, 25]. In the four studies selected, “Reach” was assessed by the rate of participation in the program out of the eligible population. “Efficacy” was assessed based on the pre- and post-test outcomes for the intervention group and control groups. “Adoption” was assessed by the usage rate of the program and the participation rate in the next stage of the program; “Implementation” was assessed by the progress of the program components and lessons. “Maintenance” was assessed by the rate of participants and teachers who responded positively to the program and intended to continue to use it. The feasibility of the RE-AIM framework was examined by applying it to evaluate a pilot study based on the evaluation components identified from the review. Students (n = 989) from Grades 5 and 6 from four elementary schools in Gimpo City and Incheon City were intervened for 12 weeks using the “Happy Me” application developed as a customized mobile service for the prevention and management of childhood obesity. The results of the intervention were evaluated using the RE-AIM framework. Reach and Efficacy could be evaluated, but Adoption, Implementation, and Maintenance could not be assessed in the pilot study. Accordingly, assessment indicators, such as future participation rate, knowledge improvement rate, and availability of or satisfaction with the application, are suggested for future studies.

Previous studies used the RE-AIM framework in the evaluation phase, planning the intervention, and reviewing the literature on health intervention [9, 26, 27]. On the other hand, only one Korean study that used the RE-AIM framework to evaluate an alcohol education program for university students was found [28]. In this study, the RE-AIM framework was used to develop an evaluation plan in the development stage. Reach was assessed with reference to the rate of students who had received alcohol education. Efficacy was evaluated by the changes in awareness, knowledge, attitude, and skills pertaining to moderate drinking. Adoption was analyzed by the rate of schools that adopted the alcohol education program. Implementation was examined by the degree of change in the educated students’ knowledge and utilization of skills and whether the staff followed the instructional guidelines. Maintenance was determined by whether the students’ moderate drinking practice lasted longer than six months and whether alcohol education was included as a regular component of the curriculum.

In previous studies [22, 23, 24, 25], the RE-AIM framework was used to evaluate the effects of the nutrition and physical activity intervention in elementary-school students in the United States and the Netherlands. On the other hand, in the present study, the pilot study involved sending messages using a mobile application as an intervention based on the TTM. Mobile intervention is highly accessible and cost-effective and can provide personalized information [29, 30, 31, 32, 33, 34]. Additionally, the TTM is the most promising model for promoting behavioral changes associated with healthy lifestyle habits [35].

Similar to the intervention method used in this study, three studies used a mobile application and the TTM simultaneously [36, 37, 38]. One of these, in a study conducted in Australia, messages developed using the TTM were sent to participants for 12 weeks via their mobile phones [38]. The study was similar to the present study in the contents and intervention period. The intervention group received eight mobile messages, one email, five personal coaching calls, a dietary brochure, and other materials through the mobile application and website every week. By contrast, the control group received four mobile messages and a printed dietary and physical activity guideline only. The pre- and post-online surveys showed that after 12 weeks, the intervention group had a lower weight, higher daily intake of vegetables, and lower intake of sweet beverages and instant food than the control group. Furthermore, the total amount of physical activity in the intervention group increased by 252.2 MET-minutes per week, and the frequency increased by 1.3 days per week. Patridge et al.[38] reported a greater magnitude of changes in the intake frequency of vegetables and instant food than those observed in the present study. These differences are because the target population and the intervention factors in the two studies were different. The target population of the present study was 5^th and 6^th-grade elementary school students, while that by Patridge et al.[38] was 18- to 35-year-old young adults with a high risk of weight gain and overweight. In addition to providing messages using the mobile application, Patridge et al. also provided information through personal coaching calls and a dietary brochure [38].

The limitations of the present study are as follows. First, the mobile intervention had several advantages, but it was difficult to assess the Maintenance aspect because a longer intervention period indicated a lower participation rate [39]. Second, although the 12-week period was sufficient for obesity intervention, further research is required because several school-based prevention and management programs demand long-term intervention [40]. Third, because this study evaluated the results of the pilot study using the RE-AIM framework, it is necessary to reevaluate the same in an expanded study in the future. Fourth, the pilot study did not include a dietary intake measurement tool in the pre- and post-test. To evaluate the efficacy in future studies, the dietary intake needs to be assessed in detail using methods such as the 24-hour recall. Nevertheless, this study was meaningful because it was the first to evaluate the feasibility of the RE-AIM framework through a systematic review of elementary-school-based intervention studies. In addition, the effects of intervention with a mobile application and TTM were similar to those of previous studies, suggesting that these methods have the potential for effective intervention.

In this study, only some of the five dimensions of the RE-AIM framework were evaluated because of the limitations of the measurement tools. According to a systemic review of the RE-AIM framework, most studies using the RE-AIM framework indicated Reach and Efficacy. On the other hand, only a few addressed Adoption, Implementation, and Maintenance. In the future, more effective program evaluation and dissemination will be possible if the evaluation criteria are selected by considering all five dimensions of the RE-AIM framework at the planning stage of the intervention program.