Contents
I Introduction
II When to use a priority-setting group process
III Generic structure of a good priority setting process
IV Roles and authority
V Dotmocracy
VI Paired comparisons
VII Quadrant analysis / decision box
VIII Grid analysis
IX Conclusion
X References
XI Resources
- submitted by Nancy van Boxmeer & Jodi Thesenvitz, Consultants, The Health Communication Unit (THCU)
I Introduction
Have you ever been in a meeting and found yourself surrounded by flipcharts with boundless ideas, and not known how to narrow them down to something credible, meaningful and actionable? Generally, it’s easy to generate lists of potential actions, programs, interventions, policy options, etc. The challenge, however, is how to decide which ideas should become priorities for action.
II When to Use a Priority-Setting Group Process
Prioritization is needed at many stages of the planning process. For example, priorities must be chosen about:
vision
mission
goals
audiences
activities
evaluation indicators.
In many cases, these decisions must be made through consultation with numerous people who represent your organization, your community, your funders, and others.
When you prioritize day-to-day activities you may not need tools to help; your priority setting process may be so well honed that you are not even conscious of your process. But when it comes to health promotion projects, we need to make our priority setting more transparent and we typically must engage groups of people in the process.
III Generic Structure of a Good Priority Setting Process
Whatever the decision required, or the types of people involved, the priority setting process starts when you already have a list of options/ideas. These options often arise during a situational assessment after consultation with community members, professionals, literature and other sources.
Once you have a list of options, a good priority setting process must have clearly defined:
criteria on which to compare options
processes to vote/score/rank
roles and processes to make the final choice/s.
The following sections describe four different group priority-setting processes. They vary in time required, level of rigour, and appropriateness for different kinds and sizes of groups.
IV Roles and Authority
Regardless of your process, there may be situations in which judgment calls are required. Thus it is critical to outline, before beginning a prioritizing process, the roles and expectations of all involved. For example, it is important to be clear with all participants whether their ‘votes’ are binding, or simply ‘recommendations’. It is also important to be clear about whether certain individuals have the power to make judgment calls (for example in ‘tie-breaker’ situations) and/or veto certain group decisions. Failing to clearly outline roles and expectations before beginning a priority setting process can result in conflict and damaged relationships.
In any priority-setting situation, the role of process facilitator should be carefully considered. It is important to have a neutral facilitator who is very familiar with the process, and who is skilled in navigating differences of opinions, drawing out those whose voice might at times be lost, and managing any conflict or issues that might arise during discussion. Being able to manage both the process and the people will help ensure that the priorities set are the right priorities and ones that will be endorsed by those involved.
V Dotmocracy
Dotmocracy is a technique that most people are familiar with. In its simplest form, you provide participants with one to three dots (usually stickers) and invite them to place a dot beside their top one to three options. It is a voting technique.
When to use it:
Dotmocracy works well with large groups (e.g., 20 – 30 participants), in situations when a quick ‘read’ of the group feelings are required and when participants are not very interested or able to engage in very rigorous, analytical ranking processes.
Limitations:
Dotmocracy is one of the more subjective priority setting processes. It is only useful in situations when the personal or professional judgment opinions of participants are an acceptable decision-making standard.
How it Works:
As with all prioritization methods, you first need a list of options. You then post the options on the wall and have people place their dots (or, checks or stars, etc.) beside their choice. Before people start placing their votes however, clarify criteria, scales and the process / rules.
- Establish voting criteria.
Give stakeholders a common direction on which to base their opinions, such as urgency, importance, reach, impact, etc. For example, if urgency is a criterion, ask participants to place a dot beside the issue most urgently in need of addressing.
- Establish the voting scale
Give stakeholders an explanation about what the dot(s) mean. For example, if you have more than one dot colour, you might tell participants to put red dots on the items they don’t support, green dots on the items they strongly support and no dot on the items they are neutral about.
- Establish the voting process/rules
Give stakeholders direction about how many dots they will get and what they are allowed, or not allowed, to do with them. This includes whether or not they can put more than one dot on an item, if there is a time limit, or if they have to use all their dots. For example, you might tell participants they can only use two green dots (for those items they strongly support) but as many red dots (for items they don’t support) as they want.
Clarify how the decision will be made, after the dots are posted. For example, does the option with the most votes win? The top three votes? Do ‘not support’ votes disqualify ‘strongly support votes’? Is the vote binding or is the process meant to serve as ‘advice’ for another group that will make decisions?
VI Paired Comparisons
When to use:
Paired comparisons work well when there is one criterion (for example, overall ‘importance’) and participants are not very interested or able to engage in very rigorous, analytical ranking processes. It is most appropriate for a small or mid-sized group (generally up to 10 – 15 people.) It is a great tool to use after a dotmocracy activity has narrowed options down to three or four choices that need to be further narrowed.
Limitations:
Compared to dotmocracy, paired comparisons has a higher degree of rigour because it forces people to quantify comparisons and get past their initial ‘gut reaction.’ It is still however, a fairly subjective voting process.
How it Works:
Paired comparisons work similar to a round-robin sports tournament. In a round robin tournament, each team plays the other teams in their pool and ends with an assigned score for each ‘head to head’ competition. In paired comparisons you do the same thing.
- List the options. Assign a letter to each. For example:
A – Subsidies to offset facility program fees
B – Buses with bike racks attached
C – Workplaces with flex hours to allow for workouts during the day
D – Mandatory training for teachers in daily physical activity (DPA) activities
- Mark the options as row and column headings on the table. Block out the cells where you will compare an option with itself and where you will duplicate a comparison.
For example:
Table 1
- Establish criteria and process rules
As with dotmocracy, decide on the critical factor/s you will use to compare options (for example, importance, urgency, reach, impact), and agree on how the final decision will be made.
Also decide how different scores among group members will be combined to reach a conclusion. After each person does his or her own scoring, the results can be discussed cell by cell. The final totals can be established either by taking an average of all individual scores, or by agreeing as a group (hopefully by consensus), on what the group ‘score’ should be.
- Compare the option in each row to the option in each column.
Based upon the criteria you chose:
Identify which of the two options ‘wins’ (for example, which is more important, more urgent, will result in greater reach.) In the appropriate cell, write the letter of the winner.
Then score the difference between the two options between 0-3. A score of zero means no difference and three is a major difference. Beside the letter of the winner write down the score. For example:
Table 2
- Add up the totals for each letter.
A = 2
B = 0
C = 3
D =7
In this case ‘D’ is the favoured choice.
VII Quadrant Analysis / Decision Box
When to use:
Quadrant analysis is useful if you have two clear criteria upon which to make a decision (for example, effort and impact), and those two criteria can be qualified in a dichotomous way (for example, high versus low.) The use of specific criteria means it is a slightly more rigorous and time-consuming method than the two previously described methods.
Limitations:
Quadrant analysis only allows for two criteria. Also, the results classify the options into broad categories, which may need to be further prioritized in resource limited situations.
How it Works:
- Choose and explain your two criteria.
Examples of criteria pairs include:
supply / demand
cost / benefit
effort / impact
internal environment / external environment
mandate / community need
- Identify your response categories.
Examples of response categories include:
yes / no
good / bad
high / low
- Name and provide a suggested action for each quadrant. For example:
Table 3
- Assign each option to a quadrant.
- Apply more rigorous prioritization process to remaining choices (for example, grid analysis, below).
VIII Grid Analysis
When to use:
Grid analysis is useful when you must or might have to defend your program decisions with ample evidence. Also known as a decision matrix analysis, pugh matrix analysis and MAUT (Multi-attribute utility theory), it is a great process for when you have many different criteria.
Limitations:
Compared to the other techniques listed above, grid analysis takes more time and requires a more sophisticated and engaged audience with adequate time available to complete a detailed, thoughtful ranking process. For example, a team of paid staff members within one organization, with a mandate to do evidence-informed planning for a particular topic would find this process useful.
How it Works:
- Identify your criteria.
There are a wide range of criteria are possible, for example, cost, effort, reach.
At The Health Communication Unit (THCU) we often talk about three criteria, each in a circle of its own: appropriateness, impact and capacity. Where all three circles overlap – it is the ‘right fit’ (For example, see the diagram at http://www.thcu.ca/ohpp/learning_centre.cfm?view=tmm_how_to_do)
THCU’s three criteria can be broken down further. For example, appropriateness may include: fit with mandate, fit with desires of funder, and fit with desires of stakeholders. Impact might include number reached and expected degree of change. Under capacity you could include both skills available and financial cost.
The choices are limitless, but keep to a reasonable number, ideally choose less than 10 criteria.
- Identify your scale.
Identify how you will rate your options against the criteria. You might choose a 1 – 3 scale of low, medium or high. You might choose a 1 – 4 scale of excellent, very good, good or poor.
- Optional – Choose Weights
Weighting is an optional step that involves determining whether some criteria are more important than others. A criterion that is very low in importance gets a weight of 1. Something very important gets a 5.
- Assign each option a letter.
For example:
A – Subsidies to offset facility program fees
B – Buses with bike racks attached
C – Workplaces with flex hours to allow for workouts during the day
D – Mandatory training for teachers in daily physical activity (DPA) activities
- Set up your grid.
Place your options on the rows, criteria in the columns and weights in the first row (if you choose to use weighting). Write your scale down across the top of the grid. For example:
Table 4
- Proceed with the scoring process.
First, assign ratings for each option against each criterion. Place the rating in the far left corner of each cell. For example:
Table 5
Second, proceed to finish the grid by multiplying the ratings by the weighting factor and totalling up the scores.
Table 6
In this scenario, without the weighting option D would have come out on top. With the weighting you can see that the priority shifted to option A.
IX Conclusion
These four techniques, with differing levels of complexity and rigour can help you be more strategic in your decision-making. Used in the right situations, they can help make decisions faster, justify your choices to your funder, and avoid conflict.