Methodological Expertise

BPA's areas of Methodological Expertise include:

  • Extraction and analysis of large-scale databases. BPA has considerable experience in collecting and managing large data sets, as well as the ability to access and process government databases for quantitative analysis. We have created and analyzed data from hundreds of thousands of program participants, and have extensive expertise in merging data from different administrative sources, assuring the quality and consistency of data across multiple sources, and protecting the integrity and confidentiality of these data. BPA has extensive computer resources to store, secure, and analyze very large databases.
     
  • Random assignment and experimental research. BPA is one of the few small businesses in the field of policy research with expertise in the implementation of random assignment in policy studies and the proper use of experimental research methods in impact analysis. When random assignment is not a viable design option, BPA staff use their statistical and methodological expertise to develop appropriate quasi-experimental alternatives to evaluate social programs.
     
  • Development of performance indicators and standards. In a variety of fields, including vocational rehabilitation for people with disabilities and employment training programs, BPA has played an important role in developing performance measures, setting national numerical performance goals, and establishing local performance targets. We have also studied the effects of performance standards on the service systems affected, including affects on caseworker behavior and impacts of standards on outcomes.
     
  • Participatory evaluation. We have designed numerous participatory process evaluations that allow stakeholders to participate in the study design and the collection and analysis of data. We have done innovative work with participating community-based organizations to evaluate a wide range of child welfare, poverty, and disability programs.
     
  • Surveys. We offer a complete range of services—including random sampling, questionnaire development, data collection, and data management—for obtaining telephone, mail, in-person, and on-line survey information that is reliable and accurate. BPA is especially effective in managing and conducting surveys of hard-to-survey populations, such as former welfare recipients, individuals with a wide range of disabilities, migrant and seasonal farm workers, small business owners, physicians and other health care providers, and corporate executives.
     
  • Focus groups. We use focus groups to explore people’s needs, wants, and motivations in depth. We have experience not only with focus groups designed to learn more about the problems that people face what their unmet needs are, but also to systematically discuss what types of programs or supports are working well, and the likelihood of success of system or policy changes.
     
  • Statistical analysis and econometric modeling. BPA staff have considerable expertise in the use of statistical methods for program evaluation and public policy research. Our emphasis is on developing models that are not needlessly complex and that yield interpretable results. We also conduct extensive sensitivity analysis on our findings to ensure that they are robust to the use of particular methods and models.
     
  • Collection and analysis of longitudinal data. Utilization of multiple observations over time of program participant outcomes or other program performance indicators is a cornerstone of evaluation research. BPA staff have extensive experience working with longitudinal program data and with the statistical methods associated with the analysis of such data.
     
  • Key informant interviews. Many of our studies involve gathering information, perspectives and insights of key informants with expertise in the issues being studied. BPA staff are expert in identifying and securing participation of key informed experts, developing informal topic guides to ensure the most important information is obtained, gathering valuable information through an investigative reporting approach, following leads to other key informants, and extracting the most important findings from the interviews. Key informant interviews are used to guide research design efforts, lay the groundwork for other data collection approaches, or as the central source of data for some studies.
     
  • Delphi and Consensus Panels. A carefully designed Delphi process can be an extremely valuable tool for gathering the best information from the field, developing consensus and prioritizing issues. BPA has experience with Delphi groups as large as 200, using at least two rounds of data collection to develop and refine priorities and strategies for addressing key social issues. BPA also has experience with pulling together diverse groups of stakeholders to establish consensus on key efforts such as the establishment of research priorities or the design of performance standards.