Publication

Oct 19, 2016
Authors:
Elizabeth Jordan,
Mae Cooper

Introduction

Academic School Children Learning Elementary Concept

Policymakers play an important role in supporting children, youth, and family well-being through policy legislation and funding of programs. Healthy discussion and debate often center on what evidence proves the effectiveness of these supports, how much they should cost, who should administer them, and whether they are the best use of limited public resources.

Just as rigorous research informs our medical care so too should knowledge about children and the supports they need for healthy development shape policy decisions. Thankfully, there is a growing consensus that research and data should inform policy. Local, state, and federal policymakers often incentivize or require the use of evidence-based programs or practices. For example, with the federal Every Student Succeeds Act (ESSA), signed into law on December 10, 2015, states are encouraged and, in certain cases, required to adopt practices that are supported by some level of evidence. The new law also creates the potential for states to build an even stronger evidence base by encouraging the evaluation of innovative strategies to support student success.

Policymakers are increasingly reliant on research to inform or even guide investments[/inlinetweet] in programs and practices that are most likely to meet the needs of particular populations. Researchers, and the advocates and intermediaries who share research with policymakers, therefore need to understand the ways that policymakers use research in their decisions. Child Trends reviewed the available literature to explore the conditions under which policymakers are most likely to use research, including the presentation formats that best facilitate their use. This brief provides an overview of that literature. We hope that these lessons inform the work of the research community, and those who regularly bring research to policymakers, in order to improve effective services and supports for children and families.

Who are our 7,383 state legislators?

Why and how do policymakers use research?

Legislators are not simply passive consumers of research. They gather and use research in conjunction with their own values and experiences. They also use it to understand problems they encounter in their capacity as lawmakers, and to substantiate their ideas and positions.1 The extent to which they use research varies widely, based on their organizational culture,2 as well as their ability to acquire research that is relevant and rigorous, adapt it to local conditions, and apply it to current problems.3

Generally, the literature identifies five ways that policymakers use research:

  1. Instrumental use – Research helps a policymaker learn about a particular problem, and provides motivation to address it. Alternatively, research helps a policymaker learn about a particular program that may address a relevant issue.
  2. Conceptual use – Research either generally informs the policymaker on a topic or changes his or her attitudes toward it.
  3. Strategic or tactical use – Research is used to further the policymaker’s political goals.4 For instance, a study of school board debates found that when a board was told “the research says X,” this was perceived as a nearly irrefutable source of credibility; the validity of the claim was almost never questioned, much less the source or the methodology of the research.5
  4. Process use – Participation in research changes the way that policymakers think and act. For instance, a grant may require a local school board to use implementation studies to evaluate new programs. Doing this may change how the board thinks about the process of education.
  5. Imposed use – Policymakers use research to fulfill a requirement,6 such as one written into a contract or piece of legislation. For instance, a state may receive a grant to implement a pilot program, with the caveat that they evaluate it through a rigorously designed evaluation.

Researchers continue to study these concepts, further examining whether and how research is used by policymakers to change and shape their ideas.7

Although it is rarely a linear process, policies go through stages of development before they are finalized, and research is used in different ways at each stage. First, legislators define a problem that needs a solution, often in a way that suggests a particular type of policy. At this stage, policymakers will combine research on the extent and cause of the problem, combined with their own ideas and values, which may themselves be informed by research. In other words, conceptual usage of research dominates. After the problem has been defined and a general policy direction has been chosen, the specific details of a plan are hammered out. This is when instrumental use of research is most common, but, since time is short, predesigned programs may take precedence. In the next stage, legislators push for the policy to be enacted, primarily using research in a tactical way: to persuade allies and discredit detractors.8 Legislators are often only peripherally involved in the final stage, implementation.

using-research-from-problem-to-policy

Whether and how research is used depends on both the political and research environments surrounding a specific policy issue. For example, policymakers are much more likely to use research strategically when discussing politically charged issues, such as gun violence. If there is little relevant research on a topic, or if the existing research is not accessible to policymakers, its use in this context will be less likely.9

Examples of how research has informed public policy

Services and supports for young children and their families: Tremendous growth and development occur during the early years. Young children’s experiences, both positive and negative, play a significant role in their later success. With this knowledge, policymakers have invested in services and supports for young children and their families. See The Research Base for a Birth through Age Eight State Policy Framework at https://www.childtrends.org/wp-content/uploads/2013/10/2013-42AllianceBirthto8.pdf.

Extension of foster care beyond age 18: Children’s brains and bodies continue to grow and develop beyond age 18. They need positive experiences and supportive relationships to grow into healthy and productive adults. With this in mind, policymakers made federal funds available to states interested in extending foster care to youth beyond age 18, to provide them with continuity and supports as they transition to adult roles and responsibilities. See The Adolescent Brain: New Research and Its Implications for Young People Transitioning From Foster Care at http://www.jimcaseyyouth.org/sites/default/files/documents/The%20 Adolescent%20Brain_prepress_proof%5B1%5D.pdf.

Evidence-based policy can be a two-way street: Rigorous evidence of the short- and long-term positive outcomes for children and families who participated in early childhood home visiting led the Obama Administration to create a new federal home visiting program. The federal program reserves the majority of funds for home visiting programs that have evidence they work, while keeping a smaller part of the money for testing innovative ideas. In addition to money for services, the program statute includes an ongoing research portfolio to keep testing what works, for whom and under what circumstances. See Overview of the Federal Home Visiting Program at http://pediatrics.aappublications.org/content/132/Supplement_2/S59.

The value of personal connections between researchers and policymakers

Research tells us that most policymakers prefer a personal connection or conversation to a written report. There are several reasons for this. Reports are undigested information, meaning they require some expertise to pull out the information that is most relevant to the situation at hand. Those less familiar with the research may also require assistance identifying or interpreting how the research can inform specific policy issues. If a policymaker has a question about what the research means, or whether it is useful in their local context, there is no way to ask the report.10

Policymakers may also prefer personal connections because they allow them to express their own preferences for detail. The policymaker can be more confident that the information is current, since he or she can simply ask the researcher about new developments in the field. Moreover, as elected officials who participate in the political process, legislators tend to value personal connections, more so than written reports. Policymakers are accustomed to reading people and assessing their credibility in person, but may lack the background to evaluate the quality of research in a report. Especially in politically-charged policy debates, or when there are people who stand to gain or lose significant resources, it is important that policymakers trust in their sources.11

A study in Australia found that policymakers who had personal connections with researchers used them in four different ways:

  1. Galvanizing ideas – A researcher and policymaker might have a conversation on a general topic, the researcher would bring up a particular problem or promising program, and the policymaker would be inspired to work on it.
  2. Clarification and advice – Policymakers may rely upon researchers for clarification and advice while crafting policy.
  3. Persuasion – Policymakers also use researchers and research findings to help persuade their legislative colleagues. Researchers could provide arguments to use in political debates, or act directly as advocates for particular policies.
  4. Defense – Policymakers use their researcher connections as a defense against attacks alleging bias. University researchers were seen as above the fray, and their support granted legitimacy to arguments the policymakers could use to counter claims of bias.12

Because personal connections are important to policymakers, we looked into what characteristics of the relationships they particularly value. Several rose to the top:

  • Trust and absence of bias – Every day on every issue, legislators hear many voices offering opinions and ideas.13 It is important that researchers build trusting relationships with policymakers, and that their research is as unbiased as possible. Such trust can only be built up over time. The absence of bias can be used to defend the validity of a policy choice.14
  • Responsiveness to legislators’ needs – Researchers must be able to answer questions quickly and coherently, in language that the legislator can understand.15
  • Good communication – Researchers should be able to summarize disparate and sometimes conflicting literature. Researchers whose focus is too narrow, or who cannot give a short summary, will be of limited use to the policymaker.
  • Practicality – Research should be useful to policymakers and relevant to their work and responsibilities. Ideally, in a long-term relationship, legislators’ needs will help shape the topics the researcher pursues, ultimately making the research more useful to the world outside of academe.16

It should also be noted that research about children and youth is often summarized and presented to policymakers and their staffs by advocacy groups, business leaders, educators, faith leaders, and other individuals. The way information is presented by any one of these groups also varies. The setting may be a formal legislative hearing involving expert testimony, an informal briefing, a conversation, a policy fact sheet, or even an email note to a legislator by a constituent.

Strategies for developing written materials for policymakers

what-to-include-in-a-publication-for-policymakersWhile personal connections are usually best, no legislator can build and maintain relationships with experts in every field. Usually, it is legislative staffers who fill this gap. Reports that summarize findings from a body of research are particularly useful to staffers, as they cover a variety of topics at one time. This type of summary allows staffers to quickly become “experts” on a topic, when necessary. However, because of both the volume of available information and the technical language often used in research reports, it is important that research be presented in a usable way.17 One study found that, on average, state and local policymakers read in detail 27 percent of health information that they received, skimmed 53 percent for general content, and never read the remaining 35 percent at all.18

For research to be useful to policymakers and their staff, it must be relevant. The information must relate to current policy debates, show an impact on “real people,” present information that is useful across states or localities, and be easy to read.19 It can be difficult for researchers to keep up with constantly shifting policy priorities, and comprehensive, unbiased summaries are invaluable to legislators trying to get a handle on a topic, even if the results are only preliminary.20 Moreover, legislators need to be able to connect the research to the issues they are currently facing. Written materials should clearly explain why an issue is salient to legislators from a variety of districts. Ideally, policy implications are spelled out in the text.21

There are some formatting decisions that can help improve a written report’s accessibility. Bulleted lists, highlighted text, charts, and graphs can help a policymaker or staffer quickly absorb the main points of the research. Stories of “real people” often make more sense to policymakers than a lot of numbers, and are easier for them to use to persuade peers.22 Pyramid formats that include a high-level overview, a longer summary, and a full technical report, have been well-received by policymakers. This allows each reader to appreciate the information according their level of engagement and expertise. A one-page summary of the main points can be read by a busy legislator, a three-page summary can be read by a busy staffer, and a 25-page report that can be used by the staffer if they need to go deeper into the topic. Including all three will be the most useful.23 You can discuss these summaries during a personal visit as well. Other recommendations include using large fonts,24 non-technical language (eighth- or ninth -grade reading level), and graphics that look clear in grayscale, for legislators who want hard copy.25

For written products aimed at a policymaker audience, a couple of elements should be included whenever possible. First, they should survey the range of research findings, and point out where there is agreement and disagreement.26 Summaries of what works should include information about benefits, risks, and costs. Be clear about the uncertainty of estimates and, when available, provide information about effects for different populations.27 Whenever possible, include easy-to-use tools, or detailed protocols, so that programs can be more easily adapted to local conditions.28 Finally, always include the researchers’ advice or guidance on the policy implications of their work. As one policymaker put it, “I may not follow the researcher’s advice, but I want to know what they think.”29

Conclusion

Research can play a pivotal role in the development of policies. Researchers can leverage the growing emphasis on using evidence to make the practical and fiscal case for policies concerning children and families. By knowing how best to meet policymakers’ needs, the research community can inform the development of effective policies that work for all children and families.

Child Trends gratefully acknowledges the generous support of the Doris Duke Charitable Foundation for this research and its dissemination.

We’d love to hear your thoughts on this publication. Has it helped you or your organization? Email us at feedback@childtrends.org.

Copyright 2016 by Child Trends, Inc.

Child Trends is a nonprofit, nonpartisan research center that studies children at all stages of development. Our mission is to improve outcomes for children by providing research, data, and analysis to the people and institutions whose decisions and actions affect children. For additional information, including publications available to download, visit our website at childtrends.org.

References

¹ Tseng, V. & Nutley, S. (2014) Building the infrastructure to improve the use and usefulness of research in education. In K. S. Finnigan and A. J. Daly (eds.), Using Research Evidence in Education: From the Schoolhouse Door to Capitol Hill, Policy Implications of Research in Education 2 (163-175). Switzerland: Springer International Publishing.

² Jewell, C. J. & Bero, L. A. (2008). “Developing good taste in evidence”: Facilitators of and hindrances to evidence-informed health policymaking in state government. The Milbank Quarterly, 86(2), 177-208.

³ Ouimet, M., Landry, R., Ziam, S., & Bedard, P. (2009) The absorption of research knowledge by public civil servants. Evidence & Policy, 5(4), 331-350.

⁴ Maciolek, S. (2015) Use of research evidence: Social services portfolio. New York: William T. Grant Foundation.

5 Asen, R., Gurke, D., Solomon, R., Conners, P., & Gumm, E. (2011) “The research says”: Definitions and uses of a key policy term in federal law and local school-board deliberations. Argumentation and Advocacy, 47(4), 195-213.

6 Maciolek, S. (2015) Use of research evidence: Social services portfolio. New York: William T. Grant Foundation.

7 Lubienski, C., Debray, E., Scott, J. (2016) The Push and Pull of Research: Lessons from a Multi-site Study of Research Use in Education Policy. William T. Grant Foundation. http://wtgrantfoundation.org/push-pull-researchlessons-multi-site-study-research-use-education-policy.

8 McDonnell, L. M. & Weatherford, M. S. (2013) Evidence use and the Common Core state standards movement: From problem definition to policy adoption. American Journal of Education, 120(1), 1-25.

9 Jennings, E. T. Jr. & Hall, J. L. (2012) Evidence-based practice and the use of information in state agency decision making. Journal of Public Administration Research and Theory, 22(2), 245-266.

10 Feldman, P. H., Nadash, P., & Gursen, M. (2001) Improving communication between researchers and policymakers in long-term care: Or, researchers are from Mars; policymakers are from Venus. The Gerantologist, 41(3), 312-321.

11 Brownson, R. C., Royer, C., Ewing, R., & McBride, T. D. (2006) Researchers and policymakers: Travelers in parallel universes. American Journal of Preventive Medicine, 30(2), 164-172.

12 Hanes, A. S., Gillespie, J. A. Derrick, G. E. et al. (2011) Galvanizers, guides, champions, and shields: The many ways that policymakers use public health researchers. The Milbank Quarterly, 89(4), 564-598.

13 Jennings, E. T. Jr. & Hall, J. L. (2012) Evidence-based practice and the use of information in state agency decision making. Journal of Public Administration Research and Theory, 22(2), 245-266.

14 Tseng, V. & Nutley, S. (2014) Building the infrastructure to improve the use and usefulness of research in education. In K. S. Finnigan and A. J. Daly (eds.), Using Research Evidence in Education: From the Schoolhouse Door to Capitol Hill, Policy Implications of Research in Education 2 (163-175). Switzerland: Springer International Publishing.

15 Jennings, E. T. Jr. & Hall, J. L. (2012) Evidence-based practice and the use of information in state agency decision making. Journal of Public Administration Research and Theory, 22(2), 245-266.

16 Tseng, V. & Nutley, S. (2014) Building the infrastructure to improve the use and usefulness of research in education. In K. S. Finnigan and A. J. Daly (eds.), Using Research Evidence in Education: From the Schoolhouse Door to Capitol Hill, Policy Implications of Research in Education 2 (163-175). Switzerland: Springer International Publishing.

17 Jennings, E. T. Jr. & Hall, J. L. (2012) Evidence-based practice and the use of information in state agency decision making. Journal of Public Administration Research and Theory, 22(2), 245-266.

18 Sorian, R. & Baugh, T. (2002). Power of information: Closing the gap between research and policy. Health Affairs, 21(2), 264-273. DOI: 10.1377/hlthaff.21.2.264

19 Sorian, R. & Baugh, T. (2002). Power of information: Closing the gap between research and policy. Health Affairs, 21(2), 264-273. DOI: 10.1377/hlthaff.21.2.264

20 Lavis, J., Davies, H., Oxman, A., et al. (2005). Towards systematic reviews that inform health care management and policy-making. Journal of Health Services Research & Policy, 10(Suppl 1), S35-S48.

21 Feldman, P. H., Nadash, P., & Gursen, M. (2001) Improving communication between researchers and policymakers in long-term care: Or, researchers are from Mars; policymakers are from Venus. The Gerantologist, 41(3), 312-321.

22 Brownson, R. C., Royer, C., Ewing, R., & McBride, T. D. (2006) Researchers and policymakers: Travelers in parallel universes. American Journal of Preventive Medicine, 30(2), 164-172.

23 Lavis, J., Davies, H., Oxman, A., et al. (2005). Towards systematic reviews that inform health care management and policy-making. Journal of Health Services Research & Policy, 10(Suppl 1), S35-S48.

24 Nelson, S. R., Leffler, J. C. & Hansen, B. A. (2009). Toward a research agenda for understanding and improving the use of research evidence. Portland, OR: Northwest Regional Educational Laboratory.

25 Stamatakis, K. A., McBride, T. D., & Brownson, R. C. (2010). Communicating prevention messages to policy makers: The role of stories in promoting physical activity. Journal of Physical Activity and Health, 7(Suppl 1), S99-S107.

26 Feldman, P. H., Nadash, P., & Gursen, M. (2001) Improving communication between researchers and policymakers in long-term care: Or, researchers are from Mars; policymakers are from Venus. The Gerantologist, 41(3), 312-321.

27 Lavis, J., Davies, H., Oxman, A., et al. (2005). Towards systematic reviews that inform health care management and policy-making. Journal of Health Services Research & Policy, 10(Suppl 1), S35-S48.

28 DuMont, K. (2015) Leveraging knowledge: Taking stock of the William T. Grant Foundation’s use of research evidence grants portfolio. New York: William T. Grant Foundation.

29 Sorian, R. & Baugh, T. (2002). Power of information: Closing the gap between research and policy. Health Affairs, 21(2), 264-273. DOI: 10.1377/hlthaff.21.2.264

 

Authors