A Proposal for PolicyPilots.gov

We all want policies that work. But too often academics seeking opportunities to study important problems and policymakers seeking analytical muscle to evaluate policies cannot find one another. We propose a way to connect them.

Researchers’ quests for knowledge and academic publications have recently sparked a “causal revolution” focused on uncovering the determinants of observed outcomes. The open data revolution has similarly kindled a strong demand for knowledge about how policies are working. Such knowledge can be generated by pairing academic researchers, with the requisite skills, interests, and training, with government agencies looking to enact evidence-based policies. These partnerships often yield important insights: For example, some of the most significant recent discoveries about intergenerational mobility have come from academic-government collaborations.

But while the potential for collaboration between government and academia is great, so too are the obstacles. Academics often lack knowledge of political timing, program specifics, and the constraints of working within a government bureaucracy. They are also unaware of which government agencies are interested in rigorous policy testing and how that might overlap with their research. Moreover, concerns about the ability to publish research that is based on internal government data and a lack of clarity about data access and publishing can deter academics from pursuing such projects at the outset.

Likewise, government officials might not know the capabilities of academic researchers who study issues that could have direct policy application. More fundamentally, agency officials often face difficulties finding interested and able empirical researchers, who are typically geographically dispersed.

Indeed, looking for collaborators requires a heavy investment of time, with an uncertain payoff. Programs like the Presidential Innovation Fellowship—which permits talented individuals to pursue short “tours of duty” solving problems in government—are inherently limited in number and are not currently set up to support part-time collaboration.

The good news is that a few existing models have achieved some success in connecting policy problems and resources to policy solvers, to the benefit of both. Challenge.gov is an online platform for all prize competitions and challenges issued by the federal government. Data.gov hosts open datasets released by the federal government. And ClinicalTrials.gov posts information on ongoing and completed clinical trials.

Although these platforms are not specifically geared toward academics, a variety of government-sponsored research opportunities—for example, those provided by the Institute for Educational Sciences—allow researchers to apply for and complete multi-year projects, keeping in mind evaluation at the outset. Other networks like the Latif Jameel Poverty Action Lab, which focuses on randomized impact evaluations for poverty-related issues, as well as state policy labs like those in California and Rhode Island, connect and support public officials and researchers.

Another model is to bring academics into agencies on a temporary basis. One successful version of this is the Edison Visiting Scholar Program within the Office of the Chief Economist at the U.S. Patent and Trademark Office, a visiting scholar position that one of us currently holds. The Office of Evaluation Sciences within the U.S. General Services Administration also embeds researchers in government agencies to work on projects that leverage behavioral insights.

Building on these models, we propose the creation of an open, online networking platform for rigorous policy pilots, which could be hosted at, for example, PolicyPilots.gov or PolicyPilots.org. At the site, government agencies could publish requests for proposals on rigorous policy pilots (prospective or retrospective) in particular subject matter areas. Academics could post their own research skills and interests, and respond to agency requests that interest them the most. Likewise, agencies could reach out to relevant researchers who might be a good fit for projects they are considering.

Listings on the platform could be required to provide much of the information that potential partners need to determine fit and speed the match process. Agencies could indicate the parameters of the evaluation opportunity—for example, the question at its heart and why it matters, the authority to carry it out, the hypothesis and theory of change, testing methods that support causal inference, relevant evidence, and resources—consistent with the “MATTER” framework one of us has developed for proposing a rigorous pilot. Researchers could also post their areas of interest, current projects, willingness to travel, and resource needs. The high quality and ready availability of state and local data in many jurisdictions, as well as the skills of academic researchers, could make collaborations especially desirable.

Even if a government agency is not seeking a partner, it could still list planned pilots online to facilitate peer and third-party evaluations of methodologies and protocols before interventions are implemented, putting projects on the right track from the start. Research suggests that agency piloting is widespread but that the conditions needed for rigorous evaluation are not always present at the beginning.

A PolicyPilots.gov platform could also facilitate the sharing of enabling documents, policies, and resources for supporting rigorous evaluation in ways that are consistent with good science while also respecting privacy and addressing ethical, legal, and fairness concerns.

An obvious challenge with this sort of endeavor is getting a critical mass of participants. We propose a few strategies that could reduce this start-up problem.

First, the proposed networking platform would benefit from a credible, neutral third-party hosting organization, such as the U.S. General Services Administration or the Partnership for Public Service. This would enable both federal and state government agencies and academic researchers to feel confident engaging with the site, posting opportunities, and accepting projects from it. Ideally, the third-party intermediary would also have built-in connections with both government and academia, which could help it boost the program at the start by connecting potential collaborators in its network.

Second, to populate the site, the sponsoring organization should encourage agency officials to come forward and post short descriptions of existing and planned pilots on which they invite comment or collaborations. Recruiting a few government agencies—state and federal—to post their existing projects, visiting scholar opportunities, and requests for proposals on the website would encourage academic researchers to interact with the site.

A platform, by itself, is not a panacea. The incentives need to be aligned, and even the value of failed experiments should be recognized. But even within existing parameters, our proposed platform has the potential to deliver broad benefits. Researchers would receive help finding agency partners and overcoming common barriers to rigorous piloting. Public servants would get access to expertise for advancing their agencies’ missions and learning agendas. And all of this would be in service of policies that work, to improve the lives of the American people.

Colleen V. Chien is a professor at the Santa Clara University School of Law.

Neel U. Sukhatme is a professor at Georgetown University Law Center and a visiting scholar at the U.S. Patent and Trademark Office.