Chapter 4: Measuring Progress
An indicator is something that helps you understand where you are, which way you are going, and how far you are from where you want to be. An effective indicator alerts you to a problem before it gets too bad and helps you recognize what needs to be done to fix the problem. Indicators of sustainable forestry point to areas where the links between the forest resource, economy, and society are poorly understood. They allow you to identify issue areas and help show the way to improve those areas.
Indicators of sustainable forestry are different from traditional indicators of forest management. Traditional indicators change one part of the system if it were entirely independent of the other parts. For example, traditional forestry indicators simply measure the amount of timber harvested without linking it to the rate of growth or replenishment of this resource. Thus unsustainable practices such as overharvesting lead to decline in the forest resources (both forest cover and forest health). This in turn reduces the economic health and social benefits for the community, such as water quality and availability, recreation and enjoyment of forest beauty.
Regardless of where you are in the plan-act-evaluate process of developing sustainable forestry for your community, as depicted in Figure 4-1, developing indicators to measure progress is a useful and necessary step:
- During the planning phase, indicators can help to define the vision and goals for the community.
- During the acting phase, the indicators identify the data that should be gathered.
- During the evaluating phase, indicators are used to assess progress and determine what new programs or projects to undertake to ensure a sustainable future for the community's forest resources.
Figure 4-1. Developing Indicators as Part of Plan-Act-Evaluate Process
In some cases, a community may use indicators from the list of Montréal Process indicators provided in Appendix E. In others, the community may choose to develop their own indicators. However, to be effective, indicators need to be relevant, useful and useable for the community for which they are developed. While there is no 'one-size-fits-all' set of indicators, most projects chose some indicators used in the Montreal Process. In addition, the process of having diverse members of the community reach agreement on a common set of indicators for making decisions and measuring progress itself is a critical and necessary step in understanding how different aspects of the community are interrelated and how they are all needed to develop a viable long-term plan for community sustainability.
This chapter provides some general guidelines on how to develop sustainable forestry indicators. Figure 4.2 presents the main steps in this process.
Figure 4-2. Process of Developing Sustainability Indicators
Ideally, these steps should be done in conjunction with the Plan-Act-Evaluate stages where Steps A-C overlap the Plan stage, Steps C-E overlap the Act stage and Steps F-B overlap the Evaluate stage.
Step A: Agree on indicators' purpose and focus (issue area)
Regardless of whether you have used indicators previously and have collected large amounts of data, it is always important before initiating a sustainable forestry program to:
- understand what indicators are and how they help achieve your goals
- agree what is the purpose of the indicators you are developing (e.g., raise awareness, help decision-making, measure progress)
- agree on the focus of the indicators - will the indicator set be used just for forestry related issues or will it look at other aspects of the community such as education or public health?
Indicators can be developed for different purposes. For example, as described in the case studies in Appendix D, Gogebic County, Michigan, wanted to have sustainable forestry indicators to measure progress toward their vision and promote better natural resource management. On the other hand, Wallowa County, Oregon, was particularly interested in measuring the baseline conditions and using indicators to participate in local, regional and national forest policy decision-making. Understanding the purpose of the indicators will help narrow down the number of possible indicators, resulting in a more manageable final set.
Exercises and activities that could be useful in this step include:
1. Name that Indicator Exercise - Appendix C, Section C-1. This exercise is a useful starting place for those who are unfamiliar with indicators. It helps understand the difference between indicators and other items such as goals, issues and targets.
2. Trouble in River City Exercise - Appendix C, Section C-4. This exercise is a skill building exercise that helps community members gain experience using indicators for different purposes - to assess system conditions, make decisions and evaluate progress. It also demonstrates that different groups and organizations within a community will have different priorities and goals, highlighting the need to involve all stakeholders and respect others viewpoints.
3. Case Studies - Appendix D. Review the case studies in Appendix D to see how those communities defined the purpose of their indicators and compare to your community's needs.
Step B: Identify potential indicators and ways to organize them
Once the purpose of the indicators has been decided, the next step is to begin to identify potential indicators. Since there is, literally, an unlimited number of indicators from which to choose, it helps to have a structure or framework for organizing the indicators. The framework helps to ensure that important issue areas are not left out inadvertently and that the resulting set of indicators reflects a balanced view of the critical issue areas.
There are a number of frameworks that can be used to develop and organize indicators of sustainable forestry. This ToolKit focuses primarily on the Montréal Process but in some cases other frameworks may be more useful for a particular community. For more information on other frameworks refer to Appendix B.
Montréal Process is an example of goal-based approach for developing indicators. Its seven criteria are the basis for organizing indicators of forest sustainability. They ensure that a community looks at its forest resources in many different ways - as a source of timber and non-timber products, carbon sink, water, soil and biodiversity protection, as well as a source of social and economic benefits. The indicators developed also need to consider the institutional structure needed to deal with a community's forest resources.
The Montréal Process Criteria are as follows:
- Conservation of biological diversity
- Maintenance of productive capacity of forest ecosystem
- Maintenance of forest ecosystem health
- Conservation and maintenance of soil and water resources
- Maintenance of forest contribution to global carbon cycle.
- Maintenance and enhancement of long-term multiple socio-economic benefits to meet the needs of society.
- Legal, institutional and economic framework for forest conservation and sustainable management
Criteria 1 through 5 relate to the biological functions of the forest. Criteria 6 covers the numerous socio-economic functions that forests provide, such as jobs, recreation, aesthetics and better quality of life. Criteria 7 deals with how regulations and institutions affect forest sustainability - whether they actually promote more sustainable resource use or act as impediments to sustainability practices.
For each criterion the Montréal Process has several indicators that help measure that particular aspect and function of the forest. Some of the suggested indicators are more difficult to use at community level due to the lack of data but most of the indicators can be applied at different levels - local, regional and national. Which indicators are chosen will completely depend on the community's priorities, vision and goals. A complete list of Montréal Process Criteria and Indicators is included in Appendix E.
Activities that can be useful in this step include:
1. Frameworks for Organizing Issues and Indicators, Appendix C, Section C-2. The worksheets in this section can be useful for brainstorming potential indicators.
2. Indicator Frameworks, Appendix B and Appendix E. These appendices have several tables of sample indicators organized within different frameworks. Table B-3 shows indicators organized using the Pressure-State-Response framework. Table B-4 shows indicators and goals within an Input-Output-Outcome framework. Appendix E shows indicators organized in the Montréal Process Criteria Framework.
Step C: Select indicators to monitor
In many cases the initial brainstorming of indicators will lead to a huge list of possible measures.
Collecting data and measuring all 67 indicators listed under the Montréal Process C&I can be a Herculean task. For communities with limited resources it is particularly important to be able to prioritize and select a smaller number of effective indicators to measure sustainable forestry issues.
Once you complete the selection and/or brainstorming of indicators you can evaluate each indicator using some criteria. The Guide to Sustainable Community Indicators (Hart 1999) suggests some criteria such as:
- Relevant - an indicator must fit the purpose you have it for - help measure progress toward a goal, raise awareness about a critical issue, or help local decision-making regarding natural resource use, etc.
- Understandable - an indicator must be simple and easy for everyone to understand.
- Reliable - people must trust the information that an indicator provides.
- Provides timely information - an indicator must give information while there is time to act or correct the problem.
- Looks at the entire system rather than at isolated part of it - indicator should try to highlight the links among ecological, economic and social aspects of sustainable forestry.
- Clear and easy to measure - having available data is very important for an indicator to be used. However, do not fall into the trap of measuring only the things you have data for. In many cases you will need to do your own research and/or survey to collect the data. Although this can be time-consuming, it is possible to get assistance from local colleges or universities that have students who can work on projects. For example, Gogebic County was assisted by the Michigan State University Extension Program in conducting a survey on the social/cultural values that helped collect qualitative data for the indicator program.
Activities that can be useful in this step include:
1. Worksheets for Selecting Indicators, Appendix C, Section 6a, b, and c.
These worksheets can be used to select a small number of indicators from a long list. In small groups, have participants evaluate all the indicators that have been suggested. Worksheet C-6a uses standard indicator evaluation criteria. Worksheet C-6b uses sustainability indicator evaluation criteria. The two worksheets may be combined but will take additional time to complete since the participants will have to discuss each indicator and each evaluation criteria. (A list of 10 indicators could take an hour to evaluate using either worksheet or 1 ½ hours with the worksheets combined.) Worksheet C-6c is used to summarize the results of all the small groups. It works best when converted to a wall chart or overhead for tallying. Once a group has finished selecting a small number of indicators, they mark the selected indicators in the column for their group number. Once all the small groups have made their selections, the results are reviewed and discussed in the large group.
2. Evaluating Indicators in a Framework, Appendix C, Section 7.
A worksheet like this can be useful for determining how evenly a set of indicators covers the key issues. The example given uses the Montréal Process framework although other frameworks can be substituted.
Step D: Use Indicators to Set Targets for the Community
A target is a desirable value that you want an indicator to reach within a particular period of time. For example, if you are measuring the percent of forest-related jobs in your community and you want to increase these over time, you may set up a target of 10% increase over the next 10 years.
Santa Monica Sustainable City Program is an excellent example of indicator initiative which successfully developed indicators and set targets to evaluate progress toward their goals and vision over time. For each indicator a 1990 baseline was developed and a target for the year 2000 was set. In some cases indicator targets were chosen to reflect existing adopted or mandated goals, such as the target for landfilled solid waste which is mandated by state law. In other cases targets reflected established or informal City department goals. And others were chosen by the Program Task Force as aggressive and yet, realistic and achievable. For example, having measured 14.3 million gallons per day water usage in 1990, the Task Force came up with a 2000 target of 11.2 million gallons/day. Each of the indicators and targets was re-examined in year 2000 to find out if the target was met or not. In cases when the target was met, a new target was established for the period 2000-2010. In cases where the target was not met the reasons for this were investigated and corrective action was planned. In the cases where the indicators did not seem particularly useful anymore, they were replaced with new indicators and targets.
Setting targets using indicators is a very useful step because it allows tracking progress toward long-term goals and taking corrective action in timely manner and holding people/organizations accountable. However, not every community will be ready to set targets. Communities which have just begun to develop indicators will not have baseline data for the indicators in order to set specific targets. In other cases, reaching consensus for the targets might be difficult. If you have already collected some baseline data on the indicators, aim for setting targets as the next key step.
Step E: Collect data for the indicators
Collecting data to measure the baseline conditions (the first time when you measure an indicator) and trends is a critical step in using any indicator. In some cases the data will be readily available (e.g., Census data on population, employment, income); in others - you will have to put a lot of time and effort into this. Finding data for a particular indicator can sometimes be a serious obstacle and yet, it is always worthwhile to make the extra effort and collect data for the indicator you need instead of falling into the trap of measuring only the indicators you have data for.
In Appendix E, the last column of the table listing the Montréal Process indicators includes suggestions for data sources for some of the indicators.
Appendix F (Other Resources) lists some possible data sources for sustainable forestry indicators. The Guide to Sustainable Community Indicators (Hart 1999) includes suggestions for federal, state and local data sources as well as how to use Internet to find some data. The Sustainable Forest Management Handbook (McDonough et. al. 2002) provides a few general thoughts on gathering information and data. The Northeastern Area sustainability website www.na.fs.fed.us/sustainability/database.htm also includes information on data sources.
Other sources of data include:
- U.S. Census (www.census.gov) - provides good data at county/municipality level on population, employment, housing, etc. It is really easy to use but the data are available only once every ten years (e.g., 1970, 1980, 1990, and 2000).
- Forest Inventory and Analysis (FIA) Data Base Retrieval System. This is an excellent source of forest data available at state, county or geographical area. The system does not cover the entire U.S. yet but work is underway to include the Pacific Northwest FIA unit (www.srsfia.usfs.msstate.edu/scripts/ew.htm).
- OIK/OS (www.eco2eco.net) is an excellent online, map-based tool for getting economic trends information. It includes data on employment and income for every county in the Eastern U.S.
One way to make data searching easier is to make sure that the indicator project includes a diverse cross-section of the community. The more people involved, the more data sources will be known to the project.
Step F: Evaluate indicator usefulness
Once you have selected your indicators and collected data, you are ready to present the information in the form of a graph that should help you evaluate trends and find out whether you are moving toward achieving your targets, goals and vision. At this point it is important to evaluate the usefulness of selected indicators and drop or revise them accordingly, so you don't waste time collecting unnecessary data and tracking the wrong indicators.
In order to evaluate indicator usefulness you may ask the following questions:
- Is this indicator helping us see trends and evaluate progress toward our goals?
- Does is need to be measured with the current frequency? For example, most indicators are measured on annual basis but in some cases, changes are so small that it makes sense to track the indicator every 5 or even 10 years (e.g., forest cover, rate of fragmentation, number of rare and threatened species).
- Does the indicator provide timely information for initiating an action?
Once a community has developed and implemented a set of sustainable forestry indicators to measure its progress toward commonly agreed vision and goals, it needs to establish a mechanism to go back in order to review and revise its goals, targets and indicators if necessary. This last step in the process of using any sustainability indicators is in a way "aligning and readjusting its compass" to make sure that it points in the right direction.
In many cases an indicator's true usefulness becomes clear only after it has been implemented. This step allows revising the initial set of goals, targets and indicators. In many ways this looks like a spring cleaning at home - you have to go through all your stuff and get rid of the items that you don't need any more, otherwise you risk spending a lot of time looking for something. Similarly, keeping only a small number of useful indicators allows you to save time and resources for doing the actual work.
Here are some of the questions that you may want to ask when evaluating and revising the indicators, targets and goals:
- Does this indicator help evaluate progress toward a specific goal or target?
- Have we achieved the target? If the answer is "yes" you may either set up a higher target or focus on another issue of concern in the local community and therefore, select new indicators and targets to address this issue. However, if this is an important target then you may want to continue measuring it and not let the issue slip out of sight.
- Have there been some major changes in the community that led to new emerging priorities, goals and issues to include?
- Did the indicators help you uncover some unexpected problems and issues that you need to address?
Sustainable Forest Management Community Handbook for the Great Lakes Region, by Maureen McDonough, Leigh Ann Spence, and Wendy Hinrichs Sanders, May 2002. An excellent guide to developing a community-based, sustainable forestry initiative. In addition to the section who to engage in the process, the guide includes a number of case studies of communities in the Great Lakes Region that have developed sustainable forestry initiatives and has detailed information about sources of data for indicators.
Available at www.lsfa.org/pub_SFM_handbook.html.