A great big thank you to all who participated by filling out a survey or attending the classes in the Community Med School pilot study. We are headed to Harvard Medical School this week to make our presentation and hopefully make a strong argument for a larger study. Special thanks to Barbe Ratcliffe with the University of Denver’s Osher Lifelong Learning Institute and Mark Manton with the Colorado School of Traditional Chinese Medicine.
Community Med School: A Pilot Study in Lifelong Learning, Health Promotion, and Integrative Medicine
Modern research has documented the potential contribution of complementary and integrative medicine practices to promote and support health literacy and empowerment in health promotion. Authors conducted a pilot study aimed to determine the feasibility and acceptability of a lifelong learning program using a series of public education lectures with a complementary and integrative medicine approach to improving critical health literacy.
To meet the needs of a variety of stakeholders of older adults, clinicians, practitioners, and/or policymakers, the primary outcome measures used were from the RE-AIM (Reach, Efficacy, Adoption, Implementation, and Maintenance) health promotion evaluation framework. Older adults were recruited to attend a series of public lectures featuring content from the preventive medicine approaches of traditional Chinese medicine.
Using an “open-label” design, data for the RE-AIM evaluation was collected using a mixed methods approach including questionnaires, evaluations, focus groups and interviews with stakeholders. This is a pilot study, therefore, no behavioral change or program maintenance data was collected.
Overall, the program was effective for all four of the RE-AIM criteria measured. The ability to reach the target audience was evidenced by Emails sent to 5600 to older learners in the Denver area. Recruiting a representative sample of 434 older adults (26% male, 74% female. Ages ranged from 29-94. Average age was 71). Effectiveness: 42 participants signed up for classes. Group A, 20 participants, Group B 22 participants compared to an average of 18 students per class in non-related classes. Adoption of the program was shown when sponsors were in favor of continuing with a larger study in the future. Seminar content was considered acceptable by the stakeholder’s groups. Logistical concerns in the implementation of the program indicated a few areas where improvements can be made.
Preliminary evaluation of the program suggests that the program is feasible and acceptable to implement and that it can provide credible, evidence-based information on complementary and integrative medicine to an interested public. A larger controlled trial is warranted.
Quality health education programs have the potential to improve the health and quality of life for millions of people. Health promotion and preventive medicine have been fundamental features of traditional Chinese medicine and modern research has documented the potential contribution of these complementary and integrative medicine approaches to promote and support health literacy and empowerment in health promotion. We conducted a pilot study aimed to determine the feasibility and acceptability of a lifelong learning program using a series of public education lectures with a complementary and integrative medicine approach to improving critical health literacy. The evaluation is based on practical considerations including participation, budgetary constraints, and the number of participants needed to reasonably evaluate feasibility goals. This pilot tested the methods and procedures to be used in a larger scale efficacy trial. As part of this process, investigators have spent time refining their intervention through an iterative approach to testing the feasibility of the final approach.
The Centers for Disease Control and Prevention (CDC) Framework for Program Evaluation in Public Health, was used throughout the planning process which recommends six steps for effective program planning: 1) engaging stakeholders, 2) describing the program, 3) focusing the evaluation design, 4) gathering credible evidence, 5) justifying conclusions, and 6) ensuring use and sharing lessons learned.
During the planning process, we convened various stakeholder groups consisting of participant groups, clinicians, physicians, practitioners, and policymakers. The project used conferences, interviews, surveys, and debates to determine:
- What type of difference would be meaningful in the determination of a clinically meaningful effect?
- What types of seminar content would be acceptable to both participants and western physicians?
- What is important about this program?
- What are the critical evaluation questions?
- Effect size calculation of how many participants = success?
- How easy/hard is it to recruit that many people?
- How much does it cost to fill a class?
- What would you like this program to accomplish?
- How will you use the results of this evaluation?
- What resources (e.g., time, evaluation experience, funding) can you contribute to this evaluation?
Physicians, Healthcare Policy Stakeholders
Oxford International Roundtable
University of Denver Osher Lifelong Learning Institute
Colorado School of Traditional Chinese Medicine
Harvard Medical School Osher Institute for Integrative Medicine Conference
Chinese Medicine Community Organization
Participants, Individuals Target Population and Local Stakeholder Groups
Healthy Aging Roundtable focus group or town meeting
University of Denver’s Osher Lifelong Learning Institute participants
Program Operations Stakeholders
Living Younger Longer Institute
University of Denver Osher Lifelong Learning Institute
Colorado School of Traditional Chinese Medicine
Chinese Medicine Community Organization
Resources for Evaluation Outcome measures
National Center for Complementary and Integrative Health. Research design and outcome measures.
Stakeholders took the following steps:
- Set performance indicators
- Prioritized evaluation questions.
- Ensure the future use of evaluation results.
- Recommendations for evaluation resources
Statement of Problem
“Health illiteracy causes more deaths than cancer, diabetes or any other major killer”
Walter Bortz M.D.
Media messages about health can be overwhelming and confusing, and this is especially true for complementary and integrative medicine. Research indicates that adults in the US have difficulty using the everyday health information that is available in healthcare, media, and in the community., According to the 2003 National Assessment of Adult Literacy Study, nearly 90% adults may lack the skills needed to manage their health and prevent disease. Without a clear understanding of prevention and self-care, people have a hard time avoiding or managing chronic diseases. This is especially true for older adults, lower income and minority groups. Quality health education programs have the potential to improve the health and quality of life for millions of people in the United States.
For 5000 years traditional Chinese medicine has emphasized preventive medicine in the community. Combining this ancient wisdom with the latest scientific findings in integrative medicine, this pilot study is designed to test the feasibility of a community-based health education program focused upon the preventive medicine aspects of traditional Chinese medicine. One advantage of integrative medicine is that it makes a wider array of appropriate treatment options available to people, ultimately blurring the boundaries between conventional and complementary medicine. Both disciplines should be subject to rigorous scientific inquiry so that interventions that are efficacious and effective are systematically distinguished from those that are not.
One-half of aging baby boomers (50-70 yrs.) currently use some form of complementary and alternative medicine (CAM).
According to the National Center for Complementary and Alternative Medicine (NCCAM) and the Center for Disease Control’s National Center for Health Statistics, in 2007, adults in the United States spent $33.9 billion out-of-pocket on visits to complementary and alternative medicine (CAM) practitioners and purchases of CAM products, classes, and materials for self-care. Nearly one-third of this out-of-pocket spending was used to pay for CAM practitioner costs. Further, according to this government survey, the number of visits to acupuncturists rose 32% between 2002 and 2007. Also, nearly 12% of the out-of-pocket money was specifically spent on yoga, Tai chi, and Qigong for self-care.19 However, most of the information they receive about CAM lacks a basis in research. With skyrocketing health care costs, the need to increase public awareness of the evidence base of CAM and to its utilization where appropriate is imperative.
- 50% of the US public, use alternative Medicine
- MD’S refer for Chinese Medicine more than any other alternative medicine
- 51% of MD’S understand the efficacy and value of Tai Chi
The need exists for better public education about the safe and effective use of complementary and integrative medicine approaches to preventive medicine and community-based health promotion. Stakeholders input was used to create a logic model for the intervention.
The Community Med School uses informational, behavioral and social approaches proven to improve healthy lifestyles in Baby Boom generation (50-70 years) in the Denver Metropolitan Area. To meet the needs of a variety of stakeholders of older adults, clinicians, practitioners, and/or policymakers, the primary outcome measures used were from the RE-AIM (Reach, Efficacy, Adoption, Implementation and Maintenance) health promotion evaluation framework., Older adults were recruited to attend a series of public lectures featuring content from the preventive medicine approaches of traditional Chinese medicine.
Using an “open-label” design data for the RE-AIM evaluation were collected using a mixed methods approach including questionnaires, evaluations, focus groups and interviews with stakeholders. As this is a pilot study, no behavioral change or program maintenance data was collected. We studied the interventions ability to reach the target audience and their acceptance of the seminar content, the effectiveness of the intervention in as far as attendance, the willingness of stakeholders to aid in the adoption of the intervention, and logistical concerns in the implementation of the program.
Methods of Evaluation and Preliminary Data:
The results of the evaluation will guide us in maintaining or modifying any aspects of the program and indicate if the program is worth expanding. The program used the RE-AIM model of program evaluation If you would like to learn more about the RE-AIM framework, please see www.re-aim.org or “RE-AIM for Program Planning: Overview and Applications,” produced by the Center for Healthy Aging at the National Council on Aging, at www.healthyagingprograms.org.
- Reach – To determine best practices in reaching the target population using social marketing methods. Reach the target audience and their acceptance of the seminar content, Participation rates Sign-in sheets, Number of participants, “open-label” design, where participants know what they are signing up for
- Effectiveness – Effectiveness of the intervention in as far as attendance, What seem to be the most popular activities? Focus group of potential participants, Did awareness of opportunities increase? Observation, Interview, Opinions of the target population
- Adoption – Identify potential partnerships across sectors and recruit organizations that share the mission of improving health promotion to help sponsor and promote the program to their members.
- Implementation – Tracking the execution and quality of the presentations through student evaluations, surveys, and focus groups. Logistical concerns in the implementation of the program. Are the proposed activities being carried out? Presence of classes? If not, why not?
- Maintenance – Long term follow up study planned to increase basic research into best practices with a two year follow up on participants to track participation in ongoing community programs.
- Report on results: Share best practices across sectors – Oxford Roundtable, Healthy Aging Roundtable
Overall, the program was effective for all four of the RE-AIM criteria measured. The ability to reach the target audience: Emails went to 5600 to older learners in the Denver area. A representative sample of 434 older adults (26% male, 74% female, ages ranged from 29-94 and average age 71). The effectiveness of the intervention: 42 participants signed up for classes. One group of 20 and a comparison group of 22. Average participation in non-related classes was 18 students/class. Evaluations of the content, 67% ranked the content as excellent and would recommend the class to a friend. Adoption: Sponsors were in favor of continuing with a larger study in future. Seminar content was considered acceptable by stakeholders groups. Logistical concerns in the implementation of the program indicated a few areas where we can make improvements.
Reaching the Target Population
The project is targeting older adults age 50+ in the Denver Metro area interested in lifelong learning, complementary and integrative medicine. Roughly half of the older adults in Denver already use complementary and integrative medicine, and in interviews, subjects said the main reason they signed up for the class was to learn more about alternatives in medicine. In the last decade, the use of Email and other social media to disseminate health messages has grown significantly and continues to increase. The internet has become an effective way to increase knowledge and awareness of credible, evidence‐based health messages. Email news can be used to spread health messages where and how people choose to receive them.
Subjects were recruited by Emails going to 5600 to older learners in the Denver area. A representative sample of 434 older adults interested in alternatives in medicine responded by survey with feedback and all reported some experience with alternatives in medicine and a desire to learn more about them. Subjects were 26% male and 74% female and ages ranged from 29-94 and average age 71, and 42 participants signed up for classes. One group of 20 with the University of Denver’s Osher Lifelong Learning Institute and a comparison group of 22 at the Colorado School of Traditional Chinese Medicine. The intervention Email recruitment rate was 9%. The attrition rate was 5% in Group A where sign-ups were closed and an increase of 40% in attendance in Group B over the course of the program due to word of mouth.
Group A: University of Denver
Osher Lifelong Learning Institute Members
Folks Interested in Lifelong Learning
Email to 3000 OLLI members
333 responded to survey average age = 72 Range 44-94 years of age
20 individuals signed up for class
1 never Showed up
Slightly higher than average OLLI signups
Avg. = 18 people/class
Group B: Colorado School of Traditional Chinese Medicine
Barefoot Doctors Journal Subscribers
Folks already interested in Complementary and Integrative Medicine
Email to 2600 Subscribers
111 responded to survey Avg. age 67 Range 29 – 87
22 initially signed up for class
13 Participated First Class
19 Participated 2nd Class
22 Participated 3rd class = 40% increase due to word of mouth
9 never showed up
Based upon the permission-based Email newsletter list of the target market (#2500 50-70 years old interested in health and complementary medicine). The initial e-mail list of 2500 was found to be representative of the general population, well within the confidence interval for the percentage of sedentary older adults. List of 2500 were surveyed as part of the Oxford study in 2013. Subjects, N=305, Average age, mean 56.8 years median 69, Range 21-87 years, 90% between ages 40-70. A stage of change model was used to gauge subjects physical activity levels.
Survey of 300 out of the 2500 currently on the Membership List
77.6% were already somewhat physically active
22.4% were sedentary
Pre-contemplation phase 0%
Contemplation phase 2.6%
Preparation phase 15%
Action phase < 6 months 11%
Maintenance phase > 6 months 61%
71% eat a Healthy Diet, 29% do not.
CDC Statistics Denver, Aurora (Latest as of 2015)
No Leisure-Time Physical Activity Within Past Month target ≤ 32.6%
50-64 years 19.0% Confidence Interval 17.5-20.4 2010
50-54 years 16.4% 14.0-18.8
55-59 years 19.6% 17.1-22.2
60-64 years 21.9% 19.2-24.6
65 years or older 24.8% 23.1-26.6
Eating ≥ 2 Fruits Daily 65 years or older 45.9% 43.7-48.0 2009
Eating ≥3 Vegetables Daily 65 years or older 29.9% 27.9-32.0
Disability 65 years or older 38.7% 36.7-40.6 Evaluation used the RE-AIM (Reach, Efficacy, Adoption, Implementation, and Maintenance) health promotion evaluation framework to examine the factors that influenced program implementation.
73 (16.4%) did not give permission to Email
42 (9.7 %) unsubscribed
10 Never Showed Up
1 fell ill and dropped before the first class
Potential Reach of the Program
• Baby Boomers in East Denver/Northwest Aurora= approximately 68,928
Assuming 50 percent are interested in Complementary and Integrative approaches = 34,464
• Assuming 50 percent of those are interested in health education = 17,232
• 200 participants recruited to date (survey)
• 200/17,232 = approximately 1 percent REACH
This REACH may seem modest, but the Community Med School is a new program that is just beginning to collaborate on dissemination. If the program can continue to reach similar numbers over time, it will ultimately reach a much more significant percentage of older adults
Efficacy or Effectiveness
A measure of primary outcome relative to public health goal
Use national guidelines from Healthy People 2020.
Educational and Community-Based Programs:
Goal – Increase the quality, availability, and effectiveness of educational and community-based programs designed to prevent disease and injury, improve health, and enhance the quality of life.
Intervention – This pilot study aimed to determine the feasibility and acceptability of a lifelong learning program using a series of public education lectures with a complementary and integrative medicine approach.
42 signed up for classes
20 in group A
22 in Group B
DU lifelong Learning Program got an average of 18 sign-ups/class in a wide variety of classes so interest in complementary and alternative medicine is slightly higher than average. Interesting that group B achieved a 40% increase in attendance over the course of the classes from word of mouth. People brought family members with them.
As a pilot program, only two sponsors were approached about offering classes.
Group A: University of Denver Osher Lifelong Learning Institute (OLLI) sponsored a series of 4 classes. The OLLI program is a lifelong learning program for older adults.
Feedback from the Executive Director.
The most useful aspect of the Community Med School program:
Attendance – “Offering course which promotes health and preventative medicine and therefore the quality of life for the lifelong learners in the Denver area is an integral part of the OLLI at DU program. Our ‘WellAware’ program and the offering from the Community Med School program are in exact alignment. Our members, who selected the program, desired to learn more about alternatives to their health care. Providing OLLI at DU members with this information allows them to be more proactive in their own healthcare process. And my opinion on the feasibility – Continue without modifications — feasible as is.”
Group B: Colorado School of Traditional Chinese Medicine Sponsored a series of 4 classes
Feedback from CEO.
The most useful aspect of the program: Community engaged in learning about health and AOM
Feasibility: Feasible as is and suggest to continue and monitor participant feedback so the program can be adapted and improved from feedback.
In any larger study we would target a larger number of settings or organizations in the local population that we hope to target. (e.g., lifelong learning programs, libraries, health food stores)* We will conduct a community-wide survey of organizations that offer health promotion programs, have classroom configuration to support study)*
Conduct formative evaluations to identify what intervention features potential program adoptees would like.
Implementation of the Community Med School intervention was 100%. All 4 CSTCM classes and all 4 DU OLLI classes were completed as planned.
Cost of intervention was minimal in that classroom space was donated by the DU OLLI program and the Colorado School of traditional Chinese medicine. The teacher also donated time however in future we need to be able to pay something for the teachers time. The costs moving forward should be self-sustaining in that the program took in $470 in student fees leaving $100/class available for a teachers honorarium and some extra money for AV equips. , etc.
Finally, qualitative feedback from participants indicated that they found the program engaging and informative and were enthusiastic about having more classes on similar topics.
Primary barriers to participation in the program
Adaptations made to intervention during the study (not fidelity) changes that made the intervention easier to delivery or to fit into real-world settings and suggestions to improve the implementation of the program.
- We can address these barriers by offering more class choices in convenient community locations in order to enhance participation.
- We can also seek sponsorships to be able to offer the program free to the public.
- Portable AV equipment to facilitate classes in low tech environments.
Being a pilot study no behavior change or long-term effects were measured. In a larger study, we would like to pursue a long-term follow-up of at least 6 months to 1 year following your last intervention contact. We would also like to follow up on participating organizations or settings who wish to continue the intervention after your study has ended.
Complementary and Integrative medicine approaches to health promotion have an important role to play in supporting individuals to take control of their self-care and requires accurate and credible information to be available to the public. Overall, the study was well-received but the results raise a number of considerations. A systematic approach to the analysis of feasibility revealed issues with recruitment and retention that would need to be addressed for future studies or clinical implementation of this program. However, for the subset of subjects who did complete the intervention, adherence was excellent, and satisfaction with the program was confirmed by course evaluations and interview comments. Preliminary evaluation of the program suggests that it is feasible and acceptable to implement and that it can provide credible, evidence-based information on complementary and integrative medicine to an interested public. A larger controlled trial is warranted.
A larger study is needed and should include the following modifications:
Reach can be improved by larger community-wide recruiting efforts.
Effectiveness can be improved by comparing our primary outcome relative to public health goals such as Healthy People 2020 Goal for Educational and Community Based Programs – Increase the quality, availability, and effectiveness of educational and community-based programs designed to prevent disease and injury, improve health, and enhance the quality of life.
Three emerging public health issues in the area of educational and community-based programs have been identified, pain, depression, and anxiety. Any larger longitudinal study should utilize the Patient-Reported Outcomes Measurement Information System (PROMIS). These tools are well suited for use as endpoints in clinical studies on the effectiveness of treatment. Further refinement and testing of PROMIS and other patient-reported outcome tools will lead to improved ways to assess pain, depression, anxiety, and other symptoms not easily measured with other diagnostics.
Adoption can be improved by using a whole community approach to increase the capacity for community health promotion through alternative and integrative medicine. We would seek to identify a network of stakeholders connected by existing community organizations, consumer-based networks, and academia. The Community Med School shall support training and capacity building to ensure that the public and community-based alternative and integrative medicine practitioners have the knowledge, skills, and tools necessary to implement community health promotion approaches.
Implementation can be improved by paying teachers something for their time. Perhaps switching to a once a month format to reduce the time and travel for subjects. Establishing an evidence base for community health and education increase the number of educators capable of delivering the program. Promote modern communications to share expertise and knowledge about community health promotion and provide participating individuals and organizations with the most recent and cutting-edge information about community health improvement. The Web offers a platform for creating a virtual center for this expertise, which would facilitate the sharing of knowledge, evidence-based programs, and promising practices, and promote dialogue between communities, CDC, and other community health experts. Key elements of this Web-based platform would include data, communication tools, evidence-based research, training, and a forum for practitioners to contribute and share knowledge.
Maintenance can be measured in a more long-term study by using a broader and stronger outcome measure of effectiveness in improving global health measures such as the Promis – Global Health Measures with emphasis on relieving pain, depression, and anxiety. At ≥ 6 mo and ≥ 2-year follow-up.
Sharing Data Across Sectors
Results will be presented at relevant conferences and prepared for publication in peer-reviewed journals. Various products, such as presentations, briefing reports, and webinars, will be developed to inform key stakeholders of the findings. Recommendations and reports will be shared with stakeholders and other audiences (communication), and follow up to promote maximum use.
Harvard Medical School, MA, Medicine Network Forum: (Nov. 16, 2018)
Oxford International Roundtable
DU OLLI Conference
University of Denver Osher Lifelong Learning Institute
Colorado School of Traditional Chinese medicine
Healthy Aging Round Table, Health Sciences Center, CCD Lowry, February 6th, 2016.
Oxford International Roundtable, Harris Manchester College in the University of Oxford 2013.
National Association for Health and Fitness in Partnership with the American College Of Sports Medicine, Health and Fitness Summit, Las Vegas, Nevada. Exercise is Medicine and the Art of Fitness, 2012
Oxford International Roundtable, Oxford Union Debating Hall University, Oxford, England, 2006
Joseph Brady M.S.T.C.M, L Ac., Dipl. O.M. Adjunct Professor,
Colorado School of Traditional Chinese Medicine
Jacqueline Shumway M.A.
Lifelong Learning Site Coordinator,
University of Denver, Osher Lifelong Learning Institute
Stacey Fowler M.S. Colorado State Coordinator for the Presidents’ Challenge
Andrew Freeman M.D. National Jewish Hospital, director Walk With A Doc
Grace Alfonsi M.D. Denver Health Medical Center, Associate Professor of Medicine, CU Medical School
H. Solow, Shifu: Pres., Shaolin Hung Mei Kung Fu; R&D Consultant for Diagnostic, Performance and Rehabilitation
Enid Cox Phd. Former Director University of Denver Institute of Gerontology (Retired)
Sue Bozinovski, Manager, Specialty Programs at Adams County Dept. of Human Services
Nora Lyster M.D. Geriatrician Kaiser Permanente (Retired)
Zachary Taylor, M.D., M.S.Assistant Surgeon General for Region 8 (Retired)
Lizzy M. Sinatra, MPH Community-Campus Partnership, CU Anschutz Medical Campus
Shellie Phol Executive Director Presidents Council for Physical Fitness, Health, and Nutrition
Mark Manton MS President CEO Colorado School Of Traditional Chinese Medicine
This document has been prepared for the purpose of inviting comments and suggestions on
The proposals contained therein, which will then be considered by the Oxford International Roundtable. Comments MUST be received by 14 June 2019 and should be addressed to the coordinator and principal investigator Joseph Brady at email@example.com