Efficient use of resources -- Available resources should be used to maximize student performance. The costs and benefits of various approaches to education should be analyzed, and resources should be invested in things that make a difference in children's learning outcomes.

Performance incentives -- Systems should reward results. Teachers and other school personnel have a great ability to affect school performance, but nothing in existing systems encourages their creativity. Once goals and measurement systems are in place, those who take action to reach goals should benefit. Such a system focuses efforts on improving education by establishing incentives for educators rather than on processing paperwork or moving children through the grades.

Continuous learning and adaptation -- Because today's education system typically involves top-down, rigid structures, there is little opportunity for educators to take what they have learned and alter programs to improve them. The education system needs to have the flexibility to allow innovation, assessment of results and then feedback-prompted changes. This cycle of continuous improvement is well-recognized in the business world as a key to long-term success.

Another book echoes the business-world theme, focusing on educators as "front-line workers" who should have a clear understanding of goals, jobs that provide them with incentives and opportunities to contribute to solutions, and training needed to pursue solutions effectively. Progress can then be measured on a regular basis and lessons learned from experience can be incorporated in future approaches. The authors argue that when systems focus on these elements, the people within the system can use expertise and resources to move toward goals. When systems are focused elsewhere, such as on inputs, any new resources go toward simply providing more of the same services, whether those services are effective or not.⁽⁹⁷⁾

Other researchers place a strong emphasis on the linkage between standards, financing structures and accountability systems. In an article that examines how more resources could be best used in schools, the authors write that policy makers should focus on:⁽⁹⁸⁾

Specifying clear goals and high standards.

Supporting and facilitating local educational decision making designed to meet the individual needs of a school's specific student population.

Developing a funding system that ensures that all schools have the resources to provide their students with the educational opportunities necessary for success.

Creating an accountability system built around ongoing evaluation and improvement.

Two other researchers argue that because there have been no clear goals, schools have failed to focus on results. This has caused educators to concentrate more on providing services than on reaching educational goals. They write:

This orientation produces a focus on finding new areas where money must be spent -- such as student nutrition, parent involvement, professional development or child care -- rather than on direct education services. These areas may be very worthy, but they ignore the key question of how to allocate scarce current and new resources to boost student achievement.⁽⁹⁹⁾

These researchers conclude that the financing strategies of the past did little to meet the demands of equity and are even less structured to meet the future demands for results-oriented outcomes. Finance systems now focused on inputs need to be reconstructed to reinforce education agendas that center on outcomes at the school site. They advocate setting statewide fiscal equity; earmarking funding for development of teacher and organizational capacity to teach a high-standards curriculum and for student assessment; and directing most funds to the school level for maximum flexibility to meet clear goals.⁽¹⁰⁰⁾

Some states have already embarked on the types of changes described above, refocusing accountability through incentives and outcome measurements. For most, the results have been mixed and the process of refinement is still going on.

Other States

When education experts talk about outcome-based systems, Kentucky takes center stage. When the Kentucky Supreme Court threw out the entire education system as unconstitutionally inadequate (as described in the Background section of this report), the Kentucky Legislature made the most of the opportunity to start from scratch.

Recognized as one of the most sweeping reforms of education, the system is built around an assessment tool called the Kentucky Instructional Results Information System (KIRIS). Rand Corporation, which is providing an ongoing evaluation of the reform, describes the system:

KIRIS exemplifies several key themes of current assessment-based reform. It relies largely on "performance assessment" -- that is, assessment formats other than multiple choice. It measures student achievement against standards for expected performance, and those standards are intentionally set high relative to the current distribution of performance. It is a "high-stakes" assessment, although the direct consequences are for educators and schools rather than for students: financial rewards for schools whose KIRIS scores improve sufficiently, and (in the near future) sanctions for schools that fail to improve.⁽¹⁰¹⁾

The Kentucky system tests learning in reading, writing, math, science, social studies, arts and humanities, practical living and vocational studies, as well as assessing attendance, retention at grade level, dropout rates and post-secondary transitions to work or higher education. A high proficiency standard was set for all students and each school has been given twenty years to progress to proficiency, at a rate of at least one-tenth toward the goal each two years. Schools that improve at a faster rate receive financial rewards. Schools progressing below standard must develop improvement plans, and schools at the lowest level are judged in crisis and receive state leadership and improvement funds.⁽¹⁰²⁾

The implementation of the new Kentucky system, which includes not only multiple-choice testing but also essay questions, group performance and portfolios, is not problem-free. Rand's early report on attitudes of teachers and principals indicates that some parts of the assessment -- the group activities -- appear to have limited value. Others -- the portfolios -- are subject to variations in assessment.

In addition, Rand reports that nearly half the teachers they interviewed believe the curriculum frameworks are not specific enough to guide instructional activities in a way that is aligned with assessment. And many believe that schools are finding ways to inflate test score improvements without actually improving education.⁽¹⁰³⁾

Mississippi chose to focus its accountability system on districts rather than schools. Each district is given a ranking of one to five each year, with levels one and two considered inadequate. The rankings are based on test scores and process requirements. For a district to reach an acceptable ranking of three, students must have 70 percent correct answers on open-ended tests and must not fall below the thirty-second percentile for standardized, norm-referenced tests. In addition, districts must comply 100 percent with process regulations. Districts falling below the criteria receive intensive assistance from the state; districts at levels four and five win freedom from various regulations.

At least one powerful incentive for improvement in Mississippi was that the statewide association of athletics would not allow schools to compete if they lost accreditation. But critics of the state's system say that level three, where most of the districts sit each year, is a minimal standard that does little to raise academic achievement.⁽¹⁰⁴⁾

Like Kentucky, South Carolina is trying financial incentives. The state's School Incentive Award Program provides about $4 million to about 250 schools that make the largest achievement gains compared to similar schools. The program has been so successful in motivating improved performance that legislators there are considering releasing multiple-year winners from state regulatory requirements as a further reward.⁽¹⁰⁵⁾ Indiana has a similar program -- but there the incentive affect may be fairly minimal since almost all schools meet high enough standards to achieve awards. The state distributed $3.2 million in 1993-94 to 1,032 of the state's 1,077 schools.⁽¹⁰⁶⁾

South Carolina couples its incentive system with a sanction for low-performing districts. Those with poor test results, low attendance and/or high drop-out rates can be declared "impaired districts" that must follow state recommendations for improvement. Few districts fall into this category.⁽¹⁰⁷⁾

Programs like South Carolina that involve forced state-level intervention for poor academic results are referred to as "academic bankruptcy" provisions. The Education Commission of the States lists 20 states that have academic bankruptcy laws, with varying degrees of sanctions for failing districts. Most states provide for multiple warnings and increased financial aid for districts before any state takeover occurs. Many of the states that have academic bankruptcy provisions for low performance also have financial rewards for high performance.⁽¹⁰⁸⁾

Illinois places poor-performing schools on an Academic Watch List. After four years without improvement, schools may lose funding or the state may appoint an independent authority to operate the district. The state may redirect the assignment of students to other schools -- or it may "non-recognize" a school or district. A non-recognized district automatically dissolves and is realigned into another district. Absorbing a failing district into a more successful one is also a provision of the Arkansas, Texas, Oklahoma and Iowa academic bankruptcy programs.

The academic bankruptcy concept has been joined by what is referred to as reconstitution -- the replacement of superintendents, principals and teachers. In Missouri certificated staff can be placed under probationary contracts and in New Jersey both school board members and top administrators can be replaced when these states' academic bankruptcy laws are applied.⁽¹⁰⁹⁾ Under court desegregation agreements, both San Francisco and Cleveland have adopted policies of reconstituting the staff of schools that remain poorly performing after three years of oversight, and Texas has used the method in at least one school to turn results around. Reconstitution is a controversial sanction, however. Critics say that rarely are teachers or principals the whole cause of a school's problems. But many others believe the fresh start gives schools a chance to rise above whatever their problems are.⁽¹¹⁰⁾

More than 80 percent of the states say they are developing, piloting or implementing new approaches to accountability, according to researchers. But few states have actually moved ahead with wholesale reforms -- and those that have implemented change are still in a shake-down, revision period that keeps them from being a definitive role model for other states. Nonetheless, there are identifiable steps and considerations that can be taken into account when planning reform.

Path to Reform

While many experts are clear on the need to change accountability measures, advice on how to do so is usually general rather than specific. But to shift accountability successfully appears to involve two steps: redefining what schools should be held accountable for and then providing incentives and rewards for schools that perform well.

The first step, redefining accountability measures, requires clear goals and assessments to measure those goals. This is not an easy process. Assessments that measure items not connected with desirable outcomes can skew the decisions that educators make. This conclusion was supported by a Rand study based on interviews with school principals participating in a reform effort known as New American Schools at 140 schools across the nation. The study examined how school accountability systems aid or impede innovative practices. In general, the principals reported that standardized multiple-choice tests caused teachers to focus on test-taking skills and test drills rather than the knowledge, skills and thinking behaviors that education reform efforts are promoting. On the other hand, performance-based tests appear to help faculty focus on performance standards and student outcomes.⁽¹¹¹⁾

The principals also said that many existing school accountability systems require reporting on things like student attendance, teacher-student contact time, credit hours in each subject and teacher attendance rates. The desire to show "good" statistics in these kinds of categories may get in the way of some types of reforms, they said. For instance, teacher attendance at school may become a goal that impedes teacher training. Credit hours per subject is not reflective of multi-disciplinary teaching methods. And targets for the number of hours teachers and students spend together may discourage teacher preparation and coordination time. The principals urged that ill-fitting accountability requirements be replaced with indicators more in line with reform efforts.⁽¹¹²⁾

The second step, designing incentives, can be tricky. Incentives have to be specifically linked to the desired outcome or the results may go awry. One book provides the example of a Soviet nail factory that wanted to receive its reward for reaching its production quota of 10 metric tons of nails without having to work hard. It did so, producing 10 nails, each weighing one metric ton.⁽¹¹³⁾

Incentives can come at many different levels. Personal incentives include salary adjustments or bonuses. School-site incentives may include one-time awards that may be spent without restriction, increased autonomy from oversight and broad flexibility in designing programs. Districts may be given incentives similar to those for schools. Incentives may also be crafted to respond to different types of performances. Some may be straight awards for reaching a set goal; others may be granted for achieving some level of improvement over prior performance.

Moving forward with the two-step process described above requires political consensus that is often difficult to achieve. Experts on education accountability explain:

Decisions about accountability are intensely political. They carry within them policy makers' understandings of what state responsibility for education means: what schools must deliver and what the state must guarantee they deliver. Decisions about those indicators, and the consequences related to their achievement, are profoundly important to local educators, to key stakeholders and their associations, and to policy makers. One would expect the existing array of political interests to have difficulty adjusting to a new distribution of expectations, power and authority.⁽¹¹⁴⁾

These experts identify five main challenges for states in revamping their accountability systems:

Making systems understandable -- Teachers, students and others must understand what they have to do to meet standards. The system has to be direct enough for people to comply and straightforward enough to allow revision, when necessary. In addition, poorly understood systems are unlikely to generate sustained support, from either educators, politicians or the public.

Resolving issues of fairness -- Should schools be penalized or rewarded for performance when they are not in control of all factors that contribute to student success? If socio-economic backgrounds are taken into account, does that represent an unacceptable lowering of standards and expectations that is discriminatory? States need to answer these kinds of questions within the context of whether their accountability systems are designed to inspire better performance or to force the allocation of more resources to poor performing schools -- or both.

Focusing incentives for improvement -- Creating the proper incentives is a complex task. They can have unintended consequences. They lose their power if they are either too difficult to achieve or too easy to win. States must decide whether incentives will be aimed only at poor performers who improve or also at high achievers who sustain their success.

Developing state capacity -- There is a substantial investment required in creating and maintaining assessment systems so they are accurate and meaningful. Goals and capacities must be aligned for accountability systems to work.

Creating a stable political environment for reform -- Accountability that is results-based is a long-term project. Success depends on sustained political support for reforms, as they take hold, are measured and then revised.

Despite the difficulties highlighted, however, researchers found positive effects in the two states they studied, Kentucky and Mississippi. They wrote:

In both states, the attention to accountability has resulted in desperately needed additional state support for education. The attention to accountability has resulted in a new public dialogue around schools and student performances, in part because of close scrutiny by the press in its new role of messenger of results. Public attention on high-performing and low-performing schools and districts is highlighting model practices and is ensuring that poor practices begin to change.⁽¹¹⁵⁾

In short, while not perfect, the move to new accountability systems appears to be bringing positive results.

Summary

Education accountability is focused on inputs and processes not connected to the desired outcome -- graduates with solid skills and knowledge. Redirecting accountability can help schools and educators focus their attention on the end result. Such a shift cannot occur as long as the State has a system that generously rewards process compliance with dollars and pays scant attention financially to either academic success or failure.

But changing accountability systems is not easy. Goals need to be clearly described, assessments need to accurately measure results, and incentives and sanctions need to be linked in a meaningful fashion to actions that increase student learning.

Accountability that depends on inputs, like teacher-student ratios, and easily measured outputs, like attendance, is much easier to track and enforce -- and therefore is difficult for policy makers to give up. But these mechanisms divert resources and attention away from educational tasks. The challenge for the State is to shift from one system to another in a way that maximizes both citizen confidence in school performance and flexibility at the local level to meet educational needs.

Recommendations

Recommendation 5: Once academic performance standards and assessment systems are in place, the Governor and the Legislature should ensure that the State's education accountability system shifts to outcomes.

Educators should not have to struggle to meet the demands of two accountability systems: the existing one that focuses on processes that are largely unrelated to academic achievement and the new one that will surely be the natural consequence of implementing statewide standards and tests. Instead, the State should take steps to make sure that fiscal accountability is focused on meaningful activities. These steps could include creating rewards -- such as incentive bonuses -- and sanctions -- including an academic bankruptcy process -- to encourage better focus on academic performance.