How do advisory groups contribute to healthy public policy research?

This paper reflects on experiences of Australian public health researchers and members of research policy advisory groups (PAGs) in working with PAGs. It considers their benefits and challenges for building researcher and policy actor collaboration and ensuring policy relevance of research. Four research projects conducted between 2015 and 2020 were selected for analysis. 68 PAG members from Australian federal, state and local governments, NGOs and academics participated in providing feedback. Thematic analysis of participant feedback and researchers’ critical reflections on the effectiveness and capacity of PAGs to support research translation was undertaken. PAGs benefit the research process and can facilitate knowledge translation. PAG membership changes, differing researcher and policy actor agendas, and researchers’ need to balance policy relevance and research independence are challenges when working with PAGs. Strategies to improve the function of health policy research PAGs are identified. The paper suggests a broader adapted approach for gaining the benefits and addressing the challenges of working with PAGs. It opens theoretical and practical discussion of PAGs’ role and how they can increase research translation into policy.


Introduction
Research funding bodies increasingly require researchers to incorporate research translation strategies into grant applications. Research policy advisory groups (PAGs) are one approach frequently used in health policy research to improve research translation and build collaborative relationships between researchers and other policy actors. PAGs advise on how to engage with policy actors, how to ensure the policy relevance of research findings and increase the chances of research informing policy. Other strategies, such as developing policy briefs and holding policy-relevant research forums, are also used. Participation in PAGs by policy actors is voluntary, supporting member buy-in.
Despite a large literature about strategies to achieve research knowledge translation (Lavis et al. 2003;Lawrence 2006;Lomas 2000), there is limited discussion of the effectiveness of PAGs and the challenges in working with them.
We examine the PAGs of four Australian healthy public policy research projects in which the authors participated together as either researchers or policy actors.

Background
Two communities Caplan's (1979) 'two-communities theory' suggests that researchers and policymakers belong to two communities with different priorities and pressures. Researchers operate within long timeframes, motivated by the need to publish, maintain research rigour, demonstrate research translation, and win funding. In contrast, policymakers have short timeframes, changing priorities, and deadlines which sometimes take precedence over comprehensive assessment and utilisation of research evidence. Policymakers seek synthesised solutions to policy problems from which they can quickly draw high-level conclusions. They use multiple forms of evidence, including from the media, advocacy groups, and personal experience, and from research (Cairney and Oliver 2017;Choi et al. 2005). Public policy positions adopted by policymakers result from the interaction between their ideologies, interests and available information. This information may be practice-based evidence from policymakers' own values, interests and theories, and interpretations of research evidence, which is usually outweighed by ideology and interests (Weiss 1983).
The determination of policy solutions requires political judgement and is reactive to competing stakeholder demands. Research evidence is diluted by the significant influence of these competing interests (Bacchi 2008). For these reasons, Caplan (1979) suggested that the operation of researchers and policymakers in two communities makes it difficult to transfer research findings into policy.
Progress has been made to modify and mitigate the twocommunities divide. For example, Lomas (2007) proposes a 'linkage and exchange model', a focus on 'knowledge translation' and a 'knowledge broker' role. The knowledge broker role is intended to bridge the gap between researchers and policymakers and act as capacity builder for the exchange of ideas and understanding of the other's goals and professional culture. Knowledge brokers synthesise research evidence by translating policymakers' demands for evidence for researchers, and research evidence from researchers for policymakers (Lomas 2007).
Current research has continued to reconceptualise the initial binary view of the two-communities theory, arguing that there are multiple policy communities (Bowen et al. 2017;Wehrens 2014). Departmental policy officers, government ministers, and external policy advocacy groups have different agendas and priorities which can mean that gaps in understanding and imperatives between different policy communities can be as great as those between policymakers and researchers. These different policy communities can have conflicting interests, and experience different pressures (Cairney and Oliver 2017).
Research co-production Mackenzie and Bacchi (2010) found that authorised partnerships between researchers and policymakers where project decisions are made, such as PAGs, allow participants to 'mine the collaborative space' to develop understandings and debate ideas and policy practices, supporting the coproduction of knowledge. They suggest that 'mining the space' for collaboration through interaction within a PAG results in significant learning and may lead to policy change.
Research co-design engages policy actors in the research process to facilitate research use (Oliver et al. 2019). One approach to achieving research co-design occurs when researchers engage with policy actors as co-investigators in a research project from inception. This approach supports research co-design where power is shared between researchers and policy actors. Another approach involves researchers engaging with PAGs during the research process. Here researchers control research co-production, and determine PAG membership and when policy actors contribute within the research process (Oliver et al. 2019).

Facilitators and barriers for research translation
A systematic review of interview studies of facilitators and barriers to policymakers using research evidence found that personal contact between researchers and policymakers was a facilitator of research evidence uptake (Innvaer et al. 2002). Other factors included research timeliness and relevance, whether the research confirmed current policy, and community pressure for the research. Long-term relationships and sustained dialogues between researchers and policymakers were found to be important in helping researchers understand policymakers' priorities, and influence policy. However, maintaining long-term relationships was often difficult given high policymaker turnover (Innvaer et al. 2002).

Methods
Our policy research focuses on the social determinants of health and health equity (Baum and Friel 2017;Baum et al. 2014Baum et al. , 2018. It traverses multiple sectors (such as housing, transport, environment and health), systems and institutions, described as 'Healthy Public Policy' (Milio 1987). As a result, the intersectoral policy environment and translation of our research findings can be complex.
We selected the PAGs of four research projects for analysis of the experiences of researchers and PAG members (see Table 1). These PAGs were selected because they supported a national, state or local research project. National project PAG members were less likely to have prior contact than state or local PAGs, and national PAG meetings were more often held by teleconference. State and local project PAG members were more likely to be part of pre-existing networks. The state and local projects also more directly contributed to achieving members' organisational goals, whereas national projects were less directly applicable to members' work.
Sixty-eight participants from federal, state and local governments, NGOs and academics external to the research team provided feedback on the effectiveness of the PAGs. This feedback was sought at the final meetings of each PAG in a general discussion, usually led by the chair. When one PAG seemed not to be functioning well, based on meeting attendance and feedback, PAG members were also contacted individually to discuss how to improve its functioning and capacity to support research translation.
Feedback from PAG members and researchers was documented in minutes of PAG and research team meetings, and notes were made of individual discussions with members. Thematic analysis of member feedback and researchers' critical reflection on working with the PAGs was undertaken by the authors and treated as data.
While there was a risk that PAG members could feel they should provide positive feedback to the research team, we found that they provided useful critical comment. Table 1 provides an overview of the research and PAGs.

Description of the research projects and PAGs
Nationally, the Australian National Health and Medical Research Council (NHMRC) Centre for Research Excellence (CRE) research program involved a large body of research incorporating four work packages including multiple case studies. The research sought to elucidate different components of the policy process and understand how government policy can more effectively address the social determinants of health and reduce health inequities.
The Australian Research Council (ARC) project sought to understand how the policies of the Australian national and state/territory governments in sectors other than health could contribute to action on the social determinants of health and health equity. This project focused on policy in the justice, urban planning, natural environment and energy sectors and included policymakers as co-investigators. The CRE and ARC PAGs included practitioners and policymakers from different sectors, levels of government, and states/territories, as well as academics from outside the research team. These PAGs also included policy advocates and, in the final 2 years of the CRE research, a journalist who advised on social media use to support research dissemination.
The state NHMRC research project was an evaluation of the implementation of Health in All Policies (HiAP) in South Australia. The HiAP research project included South Australian policymakers as co-investigators. The PAG included senior state policymakers, and a policy advocate from a peak state non-government organisation (NGO). (A peak NGO has a membership of NGOs with aligned interests. It provides advice to government and advocates for its sector's interests.) The local research project, Healthy South, was a Medical Research Future Fund rapid translation research project which considered how to build local capacity for health promotion. It focused on undertaking the research and supporting research transfer within 12 months in partnership with local stakeholders. The research team included a government policymaker and health service manager as coinvestigators. The PAG consisted of stakeholders from state government and the local health service, urban planners and community development officers from local government, the state-wide Local Government Association, and relevant policy advocacy NGOs active in the region.
These four PAGs comprised policy experts selected by the research teams for their different perspectives, networks and expertise. The PAGs were intended to provide a link to the policy process, including advice on the policy relevance of the research and the changing context. They provided links to policy networks to which the researchers may not otherwise have had access, and advice on research questions, policy relevance and dissemination of research findings.
What have we learnt?-Analysis of the role of PAGs in research Haynes et al. (2018) differentiate between research ownership and buy-in. Ownership involves collaborating in the development of ideas, decision making and action. Buy-in  is endorsing someone else's proposal. Involving policy actors as co-investigators on research teams, as in the ARC, HiAP and Healthy South projects, can create increased opportunity for research co-design from the outset, and a greater sense of project ownership by policy actors (Greenhalgh et al. 2016). In the HiAP and ARC projects, the policymaker co-investigators helped develop the grant applications, including specifying the PAGs' roles. A policymaker chaired each PAG.

Policy expertise in a PAG
It is useful to reflect on what constitutes policy expertise in a PAG and the source of its legitimacy. In our four PAGs, policy expertise was mainly determined by the research teams, based on policy actors' depth of experience within particular policy areas. The research teams considered deep knowledge of policy issues to be important. The policy actors' connections with policy networks within and outside government were also important to extend the research teams' interaction with a broader field of policy actors. The researchers mainly determined policy expertise based on prior contacts and existing relationships with policy actors, and on advice from other policy actors. Given its regional and rapid translational nature, organisational representation was as important as individual policy expertise in the Healthy South project. Where the researchers did not have pre-existing relationships within an organisation, the organisation nominated a representative.
The process for inclusion in a PAG by researcher invitation raises questions of selection bias. However, an open call for membership would be unlikely to attract busy policymakers. There is also tension between an open call and researchers needing to develop long-term trust relationships with policy actors to facilitate research translation.
Advice was provided by policy experts during regular PAG meetings, on a one-to-one basis with researchers, and at public research translation events where they presented their perspectives and responded to research findings.
Our experience confirms that PAGs can provide a venue for building relationships between policy actors and researchers, sustaining dialogue and helping ensure research relevance and quality.

Benefits and challenges of PAGs
The research teams valued the PAGs' role in commenting on the research and providing input into translation strategies. The development of trust between researchers and PAG members supported sharing confidential contextual information which assisted the researchers to understand organisational, policy and system changes (Delany-Crowe et al. 2019). Table 2 summarises the benefits and challenges of PAGs from researchers' and PAG members' perspectives.

PAG membership changes
From our experience, PAG attendance, particularly for longer research programs, can decline over time because of shifting priorities in members' policy environments. Some PAG members moved to new positions, and continued their membership, extending the research impact into new organisations. However, more often members left the group if their role changed. Even in the 12-month Healthy South project, there were changes in membership with proxies attending for members.
PAG membership turnover can be challenging for researchers seeking to build long-term relationships with policy actors and maintain their awareness of the changing policy context (Innvaer et al. 2002). In our research projects, contextual change occurred continuously. This had implications for the HiAP research because the HiAP initiative was located within a department undergoing significant restructuring. Researchers had to understand internal bureaucratic changes. Policymakers who were co-investigators and PAG members provided helpful information.
Frequent changes in PAG membership could be disruptive, resulting in a need to induct new members into the group's culture. However, these changes also created opportunities to build understanding, opening new areas for discussion, and networks for researchers and PAG members.

Managing the different agendas of researchers and policy actors
When first establishing the PAGs, meetings focused on presenting research progress and findings. This allowed little time for members' input. Some of the PAG members said they felt under-utilised and overloaded by the volume of information provided. In the CRE program, we modified meeting agendas to reduce the proportion of time for reporting on research and focused each meeting on one or two research areas, allowing more discussion time. CRE PAG members subsequently said that this improved the meetings.
Most of the interaction in the CRE PAG was with the CRE co-directors who attended all meetings. Other chief investigators had little interaction with the PAG, and research team members generally only attended meetings when their research was discussed. In the ARC and Healthy South projects, all chief investigators and researchers interacted with the PAGs. This greater engagement meant PAG discussion often directly informed research team meetings, with a positive impact on the entire project.
The CRE research program differed from the other projects in not having a dedicated project manager. It was apparent to the CRE co-directors and PAG membership that this PAG was not functioning as effectively as it might. Members were less satisfied with their involvement than the other PAGs. A PAG coordinator was appointed for the last 2 years of the CRE to work with the PAG, liaise with the co-directors and PAG chair, and revise the agenda to make meetings more engaging. This improved meeting processes and the interaction within the PAG and received positive member feedback.
In the other projects, dedicated project managers liaised with the PAGs and their chairs, managed agendas and followed up issues arising at meetings. This helped build relationships between researchers and PAGs and ensured meeting processes were adapted in response to feedback.
We found that it is important to acknowledge the tensions between researchers' and policymakers' different priorities. For example, the priority that policy is pragmatic and fit-for-purpose is very different to the priority that research is evidence-based and independent. Recognising these different and sometimes irreconcilable priorities can help clarify the role of policy advice, the scope of discussions that could be productive, and the parameters within which it is reasonable for researchers to not accept policy advice.

PAG size and focus
PAGs need to be fit-for-purpose. Where research programs are large and intersectoral (the CRE and ARC projects), PAGs can be large and members may come from diverse policy areas with sector-specific knowledge. Consequently, members may not be interested in all the discussion at meetings or may feel they lack relevant expertise to participate. Some members indicated this, while others found these broader discussions helpful in identifying commonalities across policy sectors. They commented on the benefits of participating in the debates, and in staying abreast of cutting-edge policy research. They also indicated Policymakers sometimes struggled to find time to fully engage with research project given pressures of work and changing contexts and priorities Some felt overwhelmed and overloaded by large volume of information presented by researchers in agenda papers and during meetings Some not interested or felt had no expertise in all policy topics discussed at a meeting and so felt less able to participate How do advisory groups contribute to healthy public policy research? 1587 that they valued the opportunity to engage with policy actors from other sectors. For example, a member said: It's refreshing to have discussions across sectors to broaden perspectives. These kinds of opportunities are not common in the siloed government system.
We found that some policymakers do not work from the short-term environment suggested by the two-communities theory but are interested in quality, academic research and its policy implications. Many policy actors may share these interests, but are caught in a struggle to mediate against short-termism (e.g. of political imperatives and media cycles) while seeking opportunities for intervention from detailed research.
PAG membership composition, meeting structure and focus need to be appropriate to the research. For example, the Healthy South PAG was time-limited. Members were advised at its commencement that there would be three meetings over 12 months. The research was directly relevant to members' work and to the roles of their organisations. The focus on a local issue provided an incentive for participation.
In PAGs where members were nominees of their organisations, we found there was less engagement than where members were individually invited because of their known interest in the policy issues.
The CRE and ARC PAG meetings were mainly conducted by teleconference to enable participation of geographically dispersed members. For the CRE, one face-toface meeting and two teleconferences were held per year. The ARC PAG members could either teleconference into meetings or attend in person. While teleconferences enabled participation of members unable to personally attend, they also created challenges. It was more difficult to ensure all members had an opportunity to participate, particularly for the larger groups. A member explained: Face-to-face meetings are preferable. Teleconferences are difficult as it's harder to speak up in them. However, I recognise that they are practical and necessary sometimes.
The researchers and PAG membership considered faceto-face meetings desirable whenever possible.

Balancing policy relevance and research independence
Researchers who seek to influence policy often express concern about achieving balance between positive relationships with policy actors and ensuring research independence (Frenk 1992). Policymakers may feel defensive if research findings challenge policy, and researchers may be caught between wanting to maintain these relationships and ensuring research integrity.
We found that clear guidelines negotiated in advance help navigate this tension. These guidelines should maintain research integrity while protecting policy actors by making their PAG role clear and enabling them to be distanced from research findings where these cause them difficulties. For example, in the HiAP research, policymakers were collaborators in the research development and investigators. They were PAG members and co-authored publications. When research findings challenged a government policy position, there was negotiation to limit potential negative consequences for policymakers while not changing or undermining research findings (e.g. policymakers were not co-authors on papers that were critical of policy, and were advised prior to publication so they could brief their agencies) (Baum et al. 2014).
The institutional drivers from academia and funding bodies for researchers to demonstrate research impact and translation are an increasing driver for policy researchers to consider the relevance and application of their research. We noted that policy actors are not necessarily as motivated to generate policy-relevant research as policy researchers may be to ensure their research is policyrelevant.
Policy development is subject to multiple pressures from different stakeholders that compete with research evidence to influence policy. The length of the research process and the policy environment's short timeframes mean that policymakers may not find research findings useful when they are produced, even if they would have been useful when the research commenced. Policymakers sometimes only want research when they need to solve a complex policy problem, or to support an existing or proposed policy position (Innvaer et al. 2002). This means research will often not be timely for policy and its relevance not appreciated by policymakers when it is completed. Policy advocates outside the public sector who advocate for policy reform are more likely to value research that does not support government policy.
The PAGs were all considered successful by members, and the research teams also identified benefits from them. Members predominantly indicated they found meetings interesting and engaging. They appreciated the opportunity to debate emerging policy-relevant findings and valued the networks they developed. However, their feedback on what they found challenging, and the research teams' reflections on the way PAGs operate and their usefulness in fulfilling their role suggested to us that there is value in considering other approaches to gaining policy advice.

Discussion
How will we do it differently?

Maximising value from a research policy advisory panel
Based on our experience with PAGs, in Table 3 we propose a research policy advisory panel (PAP) as an alternative approach to gain policy advice for research through an ongoing advisory structure. In contrast to the PAG model, a PAP would meet at the beginning of the research for background and to give initial advice, and would then probably only meet annually for progress reports. It would participate in formal research translation activities. Oneon-one and targeted meetings with selected PAP members would be held throughout the research for targeted input, rather than the whole PAP attending scheduled regular meetings, as occurs for a PAG. If required, intermittent videoconference meetings could be organised.
Establishing a PAP requires researchers to identify policy actors with an interest in their research. To enable this, researchers need to establish long-term relationships with policy actors and learn about their needs and priorities. Other strategies for identifying PAP members include following up policy actors and civil society advocacy group members who attend policy engagement events; and formally inviting relevant government departments and civil society groups to attend research translation events, and join the PAP. For university academics, former students in policy positions can also be part of their policy network.
The PAP approach could be used to support co-designed research provided that policy actor engagement is part of a longer-term relationship that builds shared interest in research and evidence-informed policy. It is intended to enable policy input into research to support translation to policy and maximise policy relevance. It should increase the usefulness of policy advice to the research, and the value of involvement for members.
A PAP could create a longer-term multi-purpose collaborative space where researchers and policy actors can share and test their understandings of policy issues, making the boundaries between policy and research more flexible, where this space can be 'mined' in the midst of these exchanges to produce greater participant understanding and potentially lead to policy change (Mackenzie and Bacchi 2010). A single PAP may have advantages over serial PAGs by stimulating more systematic and long-term ways to break down Caplan's (1979) two communities of policymakers and researchers. By a deeper mining of spaces, the two communities may develop more permeable boundaries and a third space of trust and debate where there can be resolutions to their differential pressures.
Longer-term relationships may also better deal with Weiss' (1983) observation that social science research evidence is usually outweighed by ideology and interests. While a more permanent model cannot instantly resolve ideological differences, over time it should create Limiting the number of full research policy advisory panel face-to-face meetings and using electronic communication for comment reduces demand on members' time, with smaller meetings of expert sub-groups of the panel called as required for advice on specific topics. Determination of membership of sub-groups should be negotiated between researchers and panel members Some members may have broader interest in discussions that evolve beyond sectoral/discipline boundaries and may have expertise in working within systems and complexity or in getting ideas on the policy agenda or into public discourse. Separate meetings could be held to allow for broader discussions. Policy actors could be invited to join both the sectorspecific panel sub-group meetings and the broader policy system discussions opportunities to reframe or park evidence until policy actors detect a window of policy opportunity. In this sense, mining the third space becomes a variant of Lomas' (2007) knowledge broker role. A PAP would also more effectively deal with Bowen et al's (2017) proposal that there are multiple policy communities. By simultaneously involving more policy communities than serial PAGs, more varied opportunities for knowledge transfer may arise. A more institutionalised model may also provide greater opportunities for research co-production because the partnership does not stop with a single research project.

Conclusion
Our suggestion for a broader and more enduring research PAP is designed to respond to researchers' and policy actors' different drivers. PAGs have created spaces where researchers and policy actors can reflect together on research findings and their policy implications, and how best to support research translation. Based on our experiences, more can be done to gain the benefits and address the challenges of involvement with PAGs. In response to the identified challenges and consistent with the literature, we suggest an adapted approach involving research PAPs to enable improved incorporation of policy advice into research while engaging policy actors more effectively. This paper opens the theoretical and practical discussion of their role and how they may increase research translation into policy.
Funding This paper analyses the experience of research project advisory groups from 4 research projects that received funding from: Australian National Health and Medical Research Council (GNT 1078046)-Centre for Research Excellence in the Social Determinants of Health Equity. Australian Research Council Discovery Grant (DP160100244)-Understanding how the policies of Australian governments can promote health through action on the social determinants of health and health equity. Australian National Health and Medical Research Council (GNT1027561)-Does a Health in All Policies approach improve health, well-being and equity? Rapid Applied Research Translation for Health Impact Grant Scheme through funding allocation from the Australian Medical Research Future Fund-Healthy South: Testing the feasibility of the rapid translation of Health in All Policies ideas to create healthy urban environments, create health promoting health services and stem the non-communicable disease epidemic in the southern area of Adelaide.

Compliance with ethical standards
Conflict of interest The authors declare that they have no conflict of interest.
Ethical approval All the research projects on which this paper was based received ethical approval. All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and national research committees (see details below) and with the 1964 Helsinki Declaration and its later