RESEARCH FOR POLICY EVALUATION

In policy evaluation processes, key elements include beliefs, information, interests (stakeholders) and institutions which need to be examined in order to understand how policy is changing. This involves identifying how they interact with each other as they change under different perception. People play an important role in changing things, whether politicians, CEOs of particular organisations, influential lobbyists, MPs, trade unionists, newspaper editors, special policy advisors in the Home Office, etc. at the end of the day policy making appear to be more about the chance and timing rather than logical and systematic response. It should be recognized the importance of opportunities for policy change, when people are vigilant and looking for new ideas and argument. the paradigms, policies and practices relevant to research concerns need to be explored with enthusiasm along the path including above key elements. Other people’s support and advocacy on behalf of researcher need to be developed for credibility. Reliable research needs a good contextual analysis and integration that illustrate a broader picture.

‘Policy’ is not only public policy of governments, but also the policy of NGOs and local statutory bodies. The research process should be regarded as a change activity in itself and the process in itself can be made more relevant to policy, by enhancing participation in the research by the population concerned.

What is policy? Policy as a commodity is quite a diffuse thing, and the ways that research affects policy are equally complex. There is the issues of ‘where did our research go?’ – often it’s hard to track whose policy was informed by the research. The connection often gets lost to the actors involved, although a connection existed. For example, people talk about ‘knowledge-creep’, e.g. it took 20 years of good quality research on poverty reduction and empowerment before any serious changes were made to policies in the developing world.


How can you make research (usually a highly-focused exercise) relevant to policy and practice (usually broad agendas):

multiple written outputs,
knowledge management systems
opportunities to present and discuss the research
sharing research with ‘go-betweens’
working with ‘research translators’
user involvements
identify stakeholders

Critical views:

Policy makers are seen as:
􀂃Narrow-minded and ideological
􀂃Ridiculously impatient
􀂃Making decisions irrationally
􀂃Using research to rationalise what they already decided to do
􀂃Refusing to relinquish control
􀂃Excessively rule-oriented
􀂃Short-sighted
􀂃Knowing what they do not like, but cannot say what they could use

Researchers are seen as:
􀂃Narrow-minded and ideological
􀂃Taking too much time to do anything
􀂃not having all the answers
􀂃Making authoritative claims in areas outside the range of their expertise
􀂃Thinking they are always right and that there is no need to present the basis for their opinions
􀂃Doing what they want to do, not what they should do
􀂃Always wanting more money
􀂃Lacking any common sense

The emphasis is on the importance of keeping up with the policy world, including learning the language, knowing the people you are dealing with, talking to representatives at conferences and understanding the other forces that shape policy, and the other knowledge that is involved in policy-making.

Questions to be addressed:
‘Why should policy be like that?’, ‘When should policy be changed or commenced?’, ‘Where should policy take effect?’, ‘How can policy be executed?’ and ‘Who should carry out the policy or be the target of it?’, are more likely to produce practically relevant research that is taken on board by policy makers. There is a distinction between research literacy and research competence. Your own personal strengths and weaknesses lend you to different techniques.

Match your method (structured, unstructured and semi-structured interviews) to the nature of the research (inductive/deductive; critical social science or interpretive social science) and anticipated sensitivities regarding the topic and the participants. For example, unstructured interviews can be good for subject areas that deal with intense sensitivities as the participant is then entitled to determine the form, space and content of the interview. Interviewers can see themselves as non-participant and impartial, as co-participant and as co-agent of change. However, remember that not all participants want to participate, are interested, or want to have to ‘give out’ – participatory research is more demanding than other forms of research. You demand that the participant is confident to narrate and analyse their experience, and trusts you not to plunder their brains and run!
Regarding the politics of interviewing, researchers should anticipate the relational shape of the interview relationship. There is always an implicit hierarchy, an interview should be allocated just to chat.

To consider:
• Who is in the interview? Who are you? Your age, interviewee’s age, gender, cultural background etc. all have a bearing.
• What are you talking about? What are the politics of that topic? Where is the interview?

Evidence-Based Policy - the Centre for Advanced Study in the Social Sciences, Oxford University, May 2005