508 Compliant

Section 508 requires Federal departments and agencies that develop, procure, maintain, or use electronic and information technology to ensure that Federal employees and members of the public with disabilities have access to and use of information and data, comparable to that of the employees and members of the public without disabilities-unless it is an undue burden to do so (http://www.access-board.gov/sec508/brochure.htm).

7Ps of Stakeholders

The framework that puts all potential stakeholders into seven broad categories. The 7Ps are: patients and the public, providers, purchasers, payers, policy makers, product makers, and principal investigators. This parallels a similar framework proposed by the Agency for Healthcare Research and Quality. Agency for Healthcare Research and Quality (AHRQ) is a federal agency focused on research on health care quality. Its Effective Healthcare Program focuses on comparing outcomes and effectiveness of different treatments and clinical approaches, as well as on the dissemination of those findings. Its site provides a definition of CER and provides a significant amount of information on conducting systematic reviews. http://www.effectivehealthcare.ahrq/gov/index/cfm/what-is-comparative-effectiveness-research

Application

Also known as implementation, this is the process by which research findings are used to improve health and healthcare. The engagement of stakeholders can accelerate the adoption research evidence into practice. By sustaining bi-directional relationships from the outset of a research enterprise, researchers may find that stakeholders become ambassadors for high-quality evidence. This is important, because most researchers do not have the time or resources to carry their message into every health care practice setting. Having stakeholders conduct messaging in partnership with researchers can improve the application of evidence in practice.

Bias

A systematic error introduced into sampling or testing by selecting or encouraging one outcome or answer over others.

Clinical Audience

A group of individuals and/or organizations with expertise in clinical practice or clinical research. A clinical audience is likely to have more scientific and medical knowledge than a non-clinical audience (for example, the general public).

Cochrane Collaborative

In 1985, the US Public Health Service and English Department of Health co-sponsored work in Baltimore and Oxford to identify unpublished controlled trials in perinatal medicine and to investigate publication bias. In 1992, the British government approved funding for Cochrane Centre to "facilitate the preparation of systematic reviews of randomized controlled trials of health care." This Center was followed by Centers in the U.S., Norway, and other countries. These were linked together as the Cochrane Collaborative in 1993 which focused primarily on synthesizing available evidence. A number of industrialized nations - including England, Germany, Canada, and Australia - developed national CER initiatives to deliberately synthesize, disseminate, and prioritize future research and drive healthcare decision-making regarding medications, medical technologies and devices, and treatment strategies.

Comparative Effectiveness Research (CER)

The generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care. The purpose of CER is to assist consumers, clinicians, purchasers, and policy makers to make informed decisions that will improve health care at both the individual and population levels. (IOM, "Initial National Priorities for Comparative Effectiveness Research", 2009)

Comparator (Comparator of interest)

The control arm intervention such as the current standard of care, sham procedure or placebo.

CONCEPT AND PROCESS MAPPING

A visual representation of workflow or processes that assesses potential conflicts and solutions, and shows "what if" scenarios.

Condition

A defective state of health, for example "a serious heart condition.

Consistency

The degree to which the summarized studies have a common direction or effect.

Cost-Effectiveness Analysis

A form of economic analysis that examines separately both the costs (resource utilization or economic costs) and the effectiveness (the resulting outcomes) of two or more courses of action.

Data Monitoring

The process by which clinical data are examined for completeness, consistency, and accuracy. (http://www.appliedclinicaltrialsonline.com/appliedclinicaltrials/article/articleDetail.jsp?id=392455&sk=&date=&pageID=4)

Decision analysis

An explicit, prescriptive, normative approach to decision making in the face of uncertainty. Decision analysis specifies the alternatives, information, and preferences of the decision maker and then finds the logically implied decision.

Delphi technique

A series of consecutive anonymous questionnaires to determine the perceptions of a group of individuals.

Delphi technique (modified)

Similar to the Delphi Technique, the Modified Delphi Technique is a series of questionnaires, the first of which is used to elicit research topics. In the Modified Delphi Technique, the first questionnaire includes research topics that are preselected.

Directness

The level to which interventions successfully link to research outcomes. Directness shows the impact of the intervention on the outcome of interest.

Disclosure of Interest

A statement acknowledging an individual's personal, political, professional or financial interests that might impede his or her objectivity.

Dissemination

The distribution of information and intervention materials to a targeted public health or clinical audience.

Dissemination and Application

The active distribution of research findings to real-world settings that may include adoption into clinical practice.

Engagement

A bi-directional relationship between stakeholder and researcher that results in informed decision-making about the selection, conduct and use of research. (Concannon, JGIM, 2012)

Implementation science actively engages community settings in research on the adoption of evidence-based practices into real-world settings.

Evaluating

Determining the significance, worth, or condition through careful appraisal and study.

Evidence Generation

The creation of information that can help patients, providers and other health decision-makers compare the benefits and harms of various interventions and strategies for preventing, diagnosing, treating, and monitoring health conditions in real-world settings.

Evidence Interpretation and Integration

Evidence interpretation and integration are the processes of conducting cost-effective analysis and decision analysis of research evidence to specify the options, identify important outcomes, determine the chances of harm and benefit, evaluate the quality of studies, integrate the evidence and report on clinically meaningful results.

Evidence Prioritization

A systematic process that uses pre-specified criteria to prioritize or determine the most important evidence needed by patients, providers and other decision-makers in order to make informed choices about health interventions and strategies.

Evidence Prioritization Activities

A series of steps in the evidence prioritization process that may include the assistance of stakeholders.

Evidence Prioritization Protocol

A thorough description of the scope, objectives, and proposed methods by which researchers hope to arrive at a mission for research, a prioritized list of research topics, or a prioritized list of research questions. This protocol could be used to assure that every team member and stakeholder understands the process.

Evidence Synthesis

The process of systematically reviewing, evaluating and integrating research evidence (related terms include systematic review, meta-analysis, research synthesis, or pooling).

EXTERNAL COMMUNITY

People not directly involved with research, such as the media and the public.

FORMAL NETWORK

An interconnected group of people or organizations with an established structure, formed for the purpose of meeting a specific goal.

GUIDELINES

A statement or other indication of policy or procedure by which to determine a course of action. Clinical practice guidelines are statements that include recommendations intended to optimize patient care. (http://www.iom.edu/Reports/2011/Clinical-Practice-Guidelines-We-Can-Trust.aspx)

GUIDELINE DEVELOPMENT PROCESS

A systematic review of evidence and an assessment of the benefits and harms of alternative care options (http://www.iom.edu/Reports/2011/Clinical-Practice-Guidelines-We-Can-Trust.aspx), based on formal standards (http://www.iom.edu/~/media/Files/Report%20Files/2011/Clinical-Practice-Guidelines-We-Can-Trust/Clinical%20Practice%20Guidelines%202011%20Insert.pdf).

HEALTH INTERVENTION

Any type of treatment, preventive care, or test that a person could take or undergo to improve health or to help with a particular problem. Health care interventions include drugs (either prescription drugs or drugs that can be bought without a prescription), foods, supplements (such as vitamins), vaccinations, screening tests (to rule out a certain disease), exercises (to improve fitness), hospital treatment, and certain kinds of care (such as physical therapy). (http://effectivehealthcare.ahrq.gov/index.cfm/glossary-of-terms/?pageaction=showterm&termid=95)

HEALTHCARE STRATEGY

The designing, managing, and implementation of plans to affect positive changes in the delivery of health- and medicine-related services.

HEALTH CARE PROVIDER

One who directly or indirectly administers interventions that are designed to improve the physical or emotional status of patients. 2. A person licensed, certified or otherwise authorized or permitted by law to administer health care in the ordinary course of business or practice of a profession, including a health care facility. (http://www.appliedclinicaltrialsonline.com/appliedclinicaltrials/article/articleDetail.jsp?id=392455&sk=&date=&pageID=6)

IMPLEMENTATION

The utilization of strategies or approaches to introduce or modify evidence-based interventions within specific settings. This involves the identification of and assistance in overcoming barriers to, the application of new knowledge obtained from a disseminated message or program.

IMPLEMENTATION SCIENCE

The study of methods to promote the integration of research findings and evidence into healthcare policy and practice.

INFORMAL NETWORK

An interconnected group of people or organizations with no established structure, formed naturally around a common interest or issue.

INFORMANTS

Active participants in the research process who provide relevant information and expertise.

INTEGRATING

The process of combining different and disparate sources of evidence into a decision or cost-effectiveness analysis.

INTENSITY OF ENGAGEMENT

The level to which stakeholders are immersed in, and committed to, the research process.

INTERNAL COMMUNITY

People directly involved in research, such as the Principal Investigator, research team, and other investigators.

INTERVIEWS

A method for direct contact with stakeholders that promotes engagement and gathers individual, detailed data and feedback.

INTERVENTION (INTERVENTION OF INTEREST)

In research, the variable that is evaluated or measured in comparison to other variables or a placebo.

META-ANALYSIS

Methods to combine results from different studies and to explore their differences with the goal of identifying patterns among study results, sources of disagreement among those results, or other interesting relationships that may come to light in the context of multiple studies.

MULTISOURCE ASSESSMENT

A tool that captures perspectives from superiors, staff and peers; identifies positives and negatives, and keeps workers motivated to excel in all aspects of responsibilities.

Nominal Group Technique

A structured problem-solving or idea-generating activity in which individuals' ideas are gathered and combined in a face-to-face, nonthreatening group environment. The process is intended to promote creative participation in group problem-solving. Each member of the group is invited to express their opinions, which are used to generate a list of priorities. Members may be asked to vote or rank priorities from the list. The voting process may occur multiple times.

OUTCOMES (OUTCOME OF INTEREST)

Dependent variables are used to evaluate differences between treatments. The outcome of interest is the variable that the intervention seeks to improve. There are several types of outcomes:

Clinical outcome: a change in the health of an individual, group of people or population which is attributable to an intervention or series of interventions.

Patient-reported outcome: information provided by patients, typically in the form of questionnaire responses. Surrogate outcome: a variable that is a substitute for a "true outcome", used because it is easier, quicker or cheaper to measure. Surrogate outcome: a variable that is a substitute for a "true outcome", used because it is easier, quicker or cheaper to measure.

PATIENTS

Individuals awaiting or under medical care and treatment.

Patient-centered outcomes research (PCOR)

Helps people and their caregivers communicate and make informed health care decisions, allowing their voices to be heard in assessing the value of health care options. This research answers patient-centered questions such as:

1. Given my personal characteristics, conditions and preferences, what should I expect will happen to me?
2. What are my options and what are the potential benefits and harms of those options?
3. What can I do to improve the outcomes that are most important to me?
4. How can clinicians and the care delivery systems they work in help me make the best decisions about my health and healthcare?

To answer these questions, PCOR:

• Assesses the benefits and harms of preventive, diagnostic, therapeutic, palliative, or health delivery system interventions to inform decision making, highlighting comparisons and outcomes that matter to people
• Is inclusive of an individuals preferences, autonomy and needs, focusing on outcomes that people notice and care about, such as survival, function, symptoms, and health-related quality of life
• Incorporates a wide variety of settings and diversity of participants to address individual differences and barriers to implementation and dissemination
• Investigates (or may investigate) optimizing outcomes while addressing burden to individuals, resource availability, and other stakeholder perspectives.

Patient Population

The demographics and other particulars of a group of people being serviced, e.g. ethnicity, socioeconomic status, geographical location, etc.

Peer-reviewed literature

Papers published after they have been reviewed by other scientists, knowledgeable in the field of inquiry, to determine whether the studies they describe are of reasonable quality and the conclusions reported are supported by the evidence. Reputable scientific journals use a process of peer review before accepting a paper for publication.

Physicians / Health care Providers

A person licensed to practice medicine.

PICO Framework

To specify an analytic framework and formulate an answerable research question, use the acronym PICO: P=patient, population: identify the relevant patient population of interest; I=intervention: identify the intervention being proposed (test, treatment); C=comparator, comparison or control intervention used to compare the intervention to other treatment options; O=outcome: identify the outcome of interest (e.g., mortality, morbidity, cost, benefit, harm).

Policy Makers

A person who sets the plan pursued by a government, business, or other entity (http://www.thefreedictionary.com/policy+maker).

Pooling

Methods to combine results from different studies and to explore their differences with the goal of identifying patterns among study results, sources of disagreement among those results, or other interesting relationships that may come to light in the context of multiple studies. (see also: Meta-analysis)

Precision

The degree of certainty surrounding an effect estimate with respect to a given outcome.

PROCESS EVALUATION

The method by which an intervention is implemented. Process evaluation answers questions about what has happened as a result of an intervention's implementation, such as: to whom, what, when, where, how much intervention has been delivered/received by participants? How have participants reacted to the intervention? (www.hhs.gov/opa/familylife/.../sound_process_evaluation.ppt)

PRODUCT ASSESSMENT

The process of measuring whether a study's goals were met and the impact of the research that was conducted.

Public health audience

A group of people or organizations who are interested in public health policy and procedures, but may not have scientific or medical expertise.

QUALITATIVE ELICITATION

The act of drawing out a person or population's beliefs or impressions around a particular question, topic, or experience.

QUALITY MONITORING

The process of gathering data to determine the effectiveness of a research study or intervention.

Random Sampling

Using an unpredictable procedure to choose a set of members of a population.

Relevance

A primary purpose of stakeholder engagement is to ensure that research questions and findings are relevant to patients, providers and other stakeholders. This is important because stakeholders are by definition also key decision makers in selection of health care interventions and strategies. If decision makers believe that research is relevant to the decisions they make, they are more likely to use evidence-based research to support their decisions.

REPORT CARDS

A written record of performance that identifies strengths and opportunities.

Researcher

A person who studies, tests, and discovers new treatments to improve the public's health and quality of life.

RESEARCH PRIORITIZATION

A systematic process to determine importance of research topics in relation to each other and the order by which studies should be conducted.

Research Synthesis

A research method in which

1. A systematic and reproducible search strategy is used to find as many studies as possible that address a given topic
2. Clear criterion are presented for inclusion/exclusion of individual studies into a larger analysis.

Research syntheses, defined in this way, represent a level of evidence below that of meta-analytical reviews in which results of included studies are then statistically combined to determine an overall effect (effect size) of one variable on another.

Review

A review may be a self-contained unit - an end in itself - or a preface to and rationale for engaging in primary research. A review is a required part of grant and research proposals and often a chapter in theses and dissertations. Generally, the purpose of a review is to analyze critically a segment of a published body of knowledge through summary, classification, and comparison of prior research studies, reviews of literature, and theoretical articles.

Risk of bias

The degree to which the research outcome is protected against biases.

SURVEY

Tools used to obtain feedback from study participants and/or stakeholders. These can be administered via email, hardcopy, over the phone, or in-person.

Stakeholder

An individual or group who is responsible for, or affected by, health- and healthcare-related decisions that can be informed by research evidence. (Concannon, JGIM, 2012)

STAKEHOLDER ENGAGEMENT

The process by which an organization involves individuals or groups who may be affected by the decisions it makes or can influence the implementation of its decisions. They may support or oppose the decisions, be influential in the organization or within the community in which it operates, holds relevant official positions or be affected in the long term.

STAKEHOLDER ENGAGED EVIDENCE

Data, information or findings resulting from the involvement in research activities of certain individuals or groups.

Snowball Sampling

A technique where existing study subjects recruit future subjects from among their acquaintances. Thus the sample group appears to grow like a rolling snowball.

Study Design

The formulation of trials and experiments that comprise the procedures for a research project.

Systematic Review

A summary of the medical literature that uses explicit methods to perform a comprehensive literature search and critical appraisal of individual studies and that uses appropriate statistical techniques to combine these valid studies.

Topic Generation

The act of brainstorming ideas for research projects or studies.

Transparency

Bi-directional engagement between researchers and research stakeholders throughout the six stages of CER and PCOR can help to improve the transparency of the research enterprise. This is important because stakeholders who have an appreciation of the process by which research is produced may have a better understanding of its strengths and limitations. With an appreciation for the strengths and limitations of a research enterprise, healthcare decisions makers may find it easier to know when and where the research evidence can be applied and when and where it cannot. Stakeholders may also grow to have more trust in research findings if they know more about how research is conducted.

Tufts Clinical and Translational Science Institute CER Spectrum Model

• Evidence prioritization: a systematic process that uses pre-specified criteria to prioritize or determine the most important evidence needed by patients, providers and other decision-makers in order to make informed choices about health interventions and strategies.
• Evidence generation: the creation of information that can help patients, providers and other health decision-makers compare the benefits and harms of various interventions and strategies for preventing, diagnosing, treating, and monitoring health conditions in real-world settings.
• Evidence synthesis: the process of systematically reviewing, evaluating and integrating research evidence (related terms include systematic review, meta-analysis, research synthesis, or pooling).
• Evidence interpretation and integration: are the processes of conducting cost-effective analysis and decision analysis of research evidence to specify the options, identify important outcomes, determine the chances of harm and benefit, evaluate the quality of studies, integrate the evidence and report on clinically meaningful results.
• Evidence dissemination and application: the active distribution of research findings to real world settings that may include adopting into clinical practice, informing practice guidelines, influencing health policy, and teaching research subjects and the general public.
• Feedback and assessment: feedback is formative evaluation that provides information for improvement directly from the source. Evaluation is summative: it outlines how well a goal was met, and provides judgment after the fact.