Creating Accountability

  Performance data reported by the counties is glaringly defective. The bad data and the State's process-based performance review prevent the Department of Social Services from knowing whether local child support offices are serving families adequately.

The State uses these flawed evaluations to reward counties with incentive money. It does not hold counties to minimum standards or sanction those that perform poorly.

The Department of Social Services has not used its resources effectively to help counties improve programs, nor has it held county child support programs up to the light of public scrutiny.







Creating Accountability

Finding 2: The State does not hold county child support programs accountable for meeting minimum performance standards and depends on unreliable data to reward counties for undocumented successes.

The State holds a powerful tool for ensuring that district attorneys aggressively enforce child support: It controls the flow of federal money to the counties. But the State fails to use that tool, or any other tool, to effectively supervise county performance.

The problem begins with bad data. The counties keep track of their own performance numbers, at times defining statistics in ways that suit their needs or make them look good. As a result the State cannot even reliably say how many children are being served or not served by the program, let alone diagnose where the process is failing and needs to be improved.

These reams of unreliable data also lead to an annual internecine battle between program officials and their critics over just how bad or good California's efforts to enforce child support really are.

The problem does not end there. The State also uses this unreliable data -- along with performance reviews that favor procedures over results -- to award counties millions of dollars in incentive money. While it is difficult to fail this test, some counties do. The consequence for failure? They receive a slightly smaller fiscal reward than those counties that either are performing admirably or have figured out how to satisfy the state review.

Bad Data Begets Bad Management

Effective management begins with good data. Information is essential for managers to diagnose problems and routinize successes. Performance data bring accountability to process.

While some aspects of child support enforcement are similar to typical law enforcement activities, most of the functions are more akin to data processing. The better performing family support divisions are those that have found ways to efficiently process the most routine cases and standardize their approaches for solving the harder cases.(70) In doing so, they constantly monitor the performance of individual units or teams of employees -- encouraging innovation among the creative and holding under-performers accountable.

Similarly, federal and state regulations require that data be uniformly gathered so that cases can be tracked and performance measured. California aggregates this information in the Child Support Management Information System (CSMIS) annual report. For the most part, the State relies on the counties to submit the information: total caseloads, the number of welfare and non-welfare cases, the number of cases in which absent parents were located and paternities and orders were established, how much money was collected in support, and more.

No statistics, by the way, tell policy makers or program managers the bottom line: Of all of the families who have been referred by welfare officials or have asked for help from the child support enforcement program, how many are regularly receiving child support payments.

Instead, the statistics track cases in ways some family support directors said make little sense to anyone, can be deceiving to policy makers or the general public and are largely unreliable. The physical accounting can be grossly deficient. Counties that lack computerized systems count by hand. Those with automated systems use various methods depending on their software. Even many of the automated systems rely on caseworkers remembering to keep a hand-tally of procedural steps completed. The State does not audit data collection methods, but rather conducts a "desk check" to see if the mathematics are correct. In addition to the inconsistency engendered by this system, the numbers can be -- and are -- easily manipulated by counties to improve their performance record, at times to the detriment of the families who are entitled to help.

In some cases, the statistics take on an Alice in Wonderland surrealism. For example, the counties report when they "locate" missing parents or their assets. Finding parents and assets is an essential step toward establishing an order or enforcing it. But counties score a locate every time a computer finds a bank account or an address. As a result, the county may report several locates for each case, while none of them may have actually allowed the case to proceed to the next step. The address, for instance, may not be good. The parent may still not be served with legal notices. And an order may not have been established. The case, in child support parlance, may still be stuck in "locate." In the most recent CSMIS report, DSS reported that "locations" statewide had increased 36 percent between fiscal year 1994/95 and 1995/96. Unfortunately, that does not mean the State found one-third more missing parents than the year before.

In other cases, counties have adapted statistical definitions to suit themselves. While DSS has tried to make the counties keep uniform data, some counties have ignored the State. For example, according to the State, paternity does not need to be established in cases where the parents were married at the time of conception. In Los Angeles County, however, every case is assumed at the beginning to require paternity establishment. While that might save case workers the time required to decide whether a case needs paternity work, it might also make for more work down the line. Furthermore, if all cases are assumed to need paternity, then the county gets credit for having accomplished that task in those cases where paternity was never an issue and no effort was exerted to accomplish that purpose. Other counties count a paternity establishment once when they receive a voluntary declaration of paternity and again when they finalize that paternity in court.(71) In both instances the county's statistics look better than they would if paternities were tracked according to the State's definitions and counties could receive higher incentive payments as a result.

Of Cases and Children

An even more important discrepancy involves the number of cases in the system, which for starters does not represent the number of children in the system or the number of families in the system. In fact, the statistic does not even represent the number of cases in the system. The problem begins with bad definitions, is compounded by procedural deficiencies and finally is influenced by a desire to make the numbers look good.

Under federal rules, when a family is no longer receiving welfare benefits, and the government is still trying to collect past support from when the family was receiving benefits, the family is counted twice -- once as a welfare case and again as a non-welfare case. If the children in a family have different fathers or if both parents are absent and the children are in foster care, the family may be counted several times.

Cases can be double-counted again when families move from one county to another county. A case opened in Butte County, for instance, may remain open after the family has moved and opens a new case in Yuba County -- because Butte caseworkers do not know the family moved or do not have time to close cases. Among the five large Southern California counties it can take as long as a year for a case to be transferred to a new county even when caseworkers know of the move -- and by that time the family may have moved again. Whenever a family has a case open in more than one county it gets counted more than once.

Caseload Growth

Inaccurate or incomplete data prevents managers from understanding what is happening in the program and responding effectively to changes in clientele. Ideally, child support enforcement officials would have detailed information that allowed them to manage the caseload. But too often the data is incomplete or inaccurate.

The greatest challenge that child support enforcement officials have faced in recent years has been a dramatic rise in caseload. Between 1990 and 1995 the number of child support cases being worked by the counties doubled to nearly 2.4 million. The growth was blamed on the economic recession increasing welfare rolls and a rising tide of irresponsibility among parents. But those were only suspicions -- because the State did not have the data or the resources to definitively characterize why the caseload doubled, let alone assess how to respond to the caseload changes or project whether it would go up or down.

The unanticipated caseload growth created substantial management challenges. Among other things, it increased the costs of the Statewide Automated Child Support System by $21 million.(72)

By early 1996, however, child support officials were beginning to question their assumption that economic and social trends were responsible for the entire increase.(73) For starters, the welfare-related child support caseload had increased substantially faster than the welfare rolls. Many family support directors now believe that a significant portion of the case load growth was not an upsurge of new cases, but the inability to purge old cases from their files because of a change in the federal rules governing when cases can be closed.

Counties cannot control the number of cases that reach them -- the district attorneys are required to open cases when a single-parent applies for welfare or when a single-parent petitions the DA for help. But it can control how many cases it has "open" by how many cases it closes. In other crimes, law enforcement authorities close a case when it has been solved. But in child support, authorities close a case when it has been solved, or when they give up -- and historically authorities have given up on thousands of cases. Some counties aggressively close unproductive cases while others, constrained by resources or hoping for eventual results, keep those cases open.

In 1992, the federal government, concerned that local authorities were giving up on cases too quickly, issued new rules that required cases to be worked at least three years before local officials gave up and closed them. As a result, county officials said they were closing far fewer cases, contributing to the rise in the number of open cases.

Unfortunately, the statistics kept by the State do not allow for the kind of analysis that could definitively sort out the issue. What numbers are available show that the number of "new" cases to the system have not risen nearly as fast as the total number 0of cases. To the extent that the caseload did increase because fewer cases were closed, the higher caseload number appears to more accurately reflect the actual demand for family support services.

Some child support enforcement officials said that in the days before automation, they gave up on tough cases so they could spend available resources on promising ones. In some of these cases, the consequences to the child may have been muted because government was providing welfare. With automation and welfare reform, however, this dynamic changes. In a computerized system, unproductive cases can be left open for little expense and periodically matched against computer data bases with the hope of finding a missing parent or assets. And in an era of limited welfare benefits, child support may be the only financial help many families receive.

In any event, closing cases always improves a county's statistical performance record, because the open caseload is the denominator against which all successful efforts are compared. As one county family support director explained:

While this county official believes a better denominator would be county population, the reality is that absolute numbers must be compared to some base so comparisons can be made. Statistically, the problem is not that some counties close cases quicker than others, the problem is that counties have different standards for when to close cases -- and some counties close cases to make the statistics look good.

Some county family support directors concede that the statistics were making them look bad, and so they have started to close more cases -- shrinking the denominator and improving their success rate. Those county officials acknowledge that this may mean giving up on cases in which eventually the missing parent may be found or get a job, and as a result could be required to pay support. But they blame the critics for forcing their hand.

While public advocates, such as Children Now and the National Center for Youth Law, use total cases as the basis to gauge performance, so does the federal government.

For instance, the most recent federal assessment of state performance shows that in California the percentage of cases with orders is falling -- that California is losing ground in the effort to secure child support.(75) The trend is determined as much by how many orders are established as by how many orders California needs to establish. That is an important measure of success that also can be a reliable one.

Furthermore, California does not have a monopoly on this problem. Federal officials have struggled with incomparable and unreliable data reported by the states. The U.S. General Accounting Office lists the data inaccuracies as one of the primary challenges that federal child support enforcement officials face in developing strategies for improving the nation's performance.(76) Data that is uniformly and reliably collected, the GAO concluded, is especially important as management moves from focusing on procedures to focusing on results.

Public Accountability

In most years, the greatest public discussion about child support has resulted from the assessments issued by advocacy groups using state and federal data to report how individual counties have performed in the previous year. The nonprofit groups unleash their criticism and the counties discount the data. And in most counties this is the end of the public discussion that -- without judging the accuracy of the criticism -- is essential to making public agencies publicly accountable for their performance.

The Legislature recognized the importance of comparing the performance of counties in 1993, when it passed SB 606. The law requires DSS to produce specific statistics and distribute them to county officials.

While DSS appears to satisfy the letter of the law, it sidesteps the opportunity to tell community leaders and the public at large which counties are performing admirably and which are not. The county statistics are not reported in the program's annual report and are not reported on the program's Internet home page.

As a result, the State leaves under wraps potentially the greatest incentive locally elected officials would have to make improvements in their program -- avoiding an unfavorable public review of their performance.

Performance Reviews

Statistics on the number of orders a county establishes or dollars it collects is one way for policy makers and program managers to know how well a county is doing. The second instrument is the annual performance review.

The Legislature in 1983, concerned that there was no "consistency to the functions performed or the level of performance of the counties" directed the department to develop a method for gauging the performance of the counties.(77)

In 1990, after California failed an audit by the federal Office of Child Support Enforcement, the Legislature created a specific performance review process that was intended to make sure the State passed the next audit. The largest counties review themselves and report the results to the DSS, while the State visits the smaller counties and conducts the reviews. A review consists of pulling a sample of a county's cases and examining whether the cases are being processed in compliance with federal and state regulations.

Seven different procedural steps are examined. A passing grade is 75 percent. That is, in each category the county must have processed 75 percent of the applicable cases correctly in order to be found in compliance.

In fiscal year 1990-91, the first year of the performance review, only four of the 58 counties were found in compliance with program requirements. In 1994-95, the most recent review period completed, DSS reported that 32 counties had moved into "marginal or full compliance;" 18 of the remaining 26 counties were in compliance because they had corrective action plans and seven were in "hold harmless" status because staff resources have been diverted to implement the Statewide Automated Child Support System.(78) According to DSS, only one county remains out of compliance and state officials believe that is evidence that the performance reviews have accomplished their intended goal: preparing California for the next federal audit. How much of the improved compliance rate can be attributed to serving families better and how much to counties becoming more sophisticated at passing the review is the subject of contentious debate.

In any event, some of the performance review's deficiencies should erode the confidence of State officials that the program will pass the next federal audit easily. The performance review process also falls short of being the management tool that it could be. The reviews are plagued by three fundamental problems: They focus on process rather than performance and counties get credit for effort rather than results. So few cases are reviewed, that few solid conclusions can be reached. And deficient counties can be found repeatedly in compliance by preparing "corrective action plans" for categories in which they do not satisfy regulations.

Process Over Performance

The department fashioned the review to encourage counties to take procedural steps required by the federal government. While it is important to satisfy federal rules, satisfying procedures should not be confused with performance. Focusing on process rather than results in the performance review also fails to reveal much about how effective a county's program is at collecting child support for families. To pass a performance category, for example, counties need not achieve success, but rather need only try. As The National Center for Youth Law noted:

Small Sample

The department draws a sample that is large enough to be a reliable statistical reflection of a county's entire caseload. The problem is that it does not provide a large enough sample of cases at each stage of the process. DSS officials maintain they do not have the resources to take a larger sample or to stratify the sample to ensure there are enough cases in each category. As a result, reviewers often have too few cases to make a determination. But under department rules, if the sample is too small to be valid, counties are found in compliance anyway. For example, in the most recent review of Contra Costa County's nearly 70,000 child support cases, only three cases were reviewed to determine if the county was modifying orders correctly.(80) The three cases were in compliance, but clearly did not represent a statistically reliable sample. Nevertheless, the county was found to be in compliance.

A 1997 study by the Legislative Analyst of the performance review process concluded that because of the small sample sizes, the resultswere invalid far more frequently than the department conceded. While DSS maintains a sample is too small if it has fewer than 11 cases, the LAO concluded that the threshold should change from county to county.

Repeated Non-Compliance

If a county is found to be out of compliance in a category, it can be declared to be in compliance by preparing a corrective action plan. If the next year the county is still out of compliance in that category, it can prepare another corrective action plan and be found in compliance.

In 1994-95, for instance, more than half of the counties that were found to be in compliance relied on an least one corrective action plan to satisfy the minimum requirements. By one analysis, 14 counties have relied every year of the performance review program on at least one corrective action plan in order to be found in compliance and qualify for additional incentive money.(82)

Los Angeles

The 1995-96 self-review conducted by Los Angeles County demonstrates the shortcomings in the State's process. Los Angeles County has approximately 600,000 child support cases -- more than one-third of the State's entire caseload. The sample size for the review conducted in September 1996 was 288 cases. Of the seven procedural categories reviewed, three categories had too few cases to be evaluated -- and as a result the county was found to be in compliance. In three of the categories, the county was found to be out of compliance, but was declared in compliance because the county had instituted corrective action plans. In short, there was affirmative evidence that the county was complying with state and federal procedures in one of seven categories. Nevertheless, the county passed the performance review.(83)

Los Angeles County's Self Report Card

Category Evaluation Result
Order establishment 202 cases were reviewed
96 cases were in compliance
Success rate = 47.52 percent
Out of compliance, but a corrective action plan is in place		 In compliance
Modifying orders 2 cases were reviewed
0 cases were in compliance
Success rate = 0 percent
Too few cases to judge compliance		 In compliance
Enforcement 68 cases were reviewed
29 cases were in compliance
Success rate = 42.65 percent
Out of compliance, but corrective action plan		 In compliance
Collections and distribution 25 cases were reviewed
23 cases were in compliance
Success rate = 92 percent
In compliance		 In compliance
Interstate cases 2 cases were reviewed
0 cases were in compliance
Success rate = 0 percent
Too few cases to judge compliance		 In compliance
Obtaining medical support 12 cases were reviewed
2 cases were in compliance
Success rate = 16.67 percent
Out of compliance, but corrective action plan		 In compliance
Closing cases 0 cases were reviewed
Success rate = 0 percent
Too few cases to judge compliance		 In compliance


Paying the Incentives

The performance reviews are only the first half of the strategy intended to ensure that counties first meet minimum standards and then continually improve their performance. Based on the results of the performance review -- and the reported case statistics -- the department distributes millions of dollars in incentive payments. In 1994-95, DSS gave the counties $90 million in incentive money.

Under the program, defined in both statute and regulations, the penalties for poor performance are mild. The incentive structure is fashioned into two tiers. All counties, no matter how poorly they score in the annual performance reviews, receive a Tier I "base rate" incentive equal to 6 percent of the child support they collected the previous year. Counties that according to the performance review are found to comply with state and federal rules, can earn an additional Tier I "compliance rate" incentive equal to 5 percent of collections.

Counties that pass that hurdle are eligible to earn Tier II "performance standard" incentives equal to an additional 1 percent to 3 percent of their collections. The Tier II incentives are based on a point system determined by measuring the percent the county improved over the previous year in two performance areas: paternity establishment and support order establishment.

The incentive payments can make the difference between a county child support program covering all of its costs, or having to rely on tight county general funds to make up a portion of their operating budget.

In 1994-95, for example, Alameda County spent $13.5 million and the federal government reimbursed nearly $9 million of that. The county earned another $6.5 million in federal and state incentives, giving it a nearly $2 million "profit" that it could use to make further improvements in the program. Los Angeles County, which failed its performance review that year, had the opposite experience. It spent $82.4 million, was reimbursed nearly $60 million, and earned $11.5 million in incentives -- for a net loss of $11 million.(84)

Incentives Drive Programs

At best the State's system of awarding incentive payments deflects program goals away from collecting money for children toward doing whatever is necessary to pass the review and receive the incentive money. At worst, it invites manipulation of the numbers.

The Tier II incentives are calculated by comparing a county's progress in establishing paternities and support orders. The first step in the calculation is to determine how many paternities a county has established compared to how many paternities need to be established. A similar calculation is made in the area of order establishment. Counties long ago learned that having "dead wood" cases languishing in the files hurts success percentages -- encouraging them to close hard cases and concentrate on those most likely to yield results with the leasteffort expended. The incentive system therefore has the upside-down effect of punishing counties that do the right thing by not giving up on hard cases and rewarding those that do the wrong thing by jettisoning the hard cases in favor of easy collections. The hard cases, no less than the easy cases, represent children needing help in getting support. The Ventura County Family Support Director testified:

Counties also have found that the malleability of the rules allows them to move numbers around within caseloads -- shrinking denominators to simulate achievement where in fact none may have occurred.

Beyond any inducement to manipulate numbers, the Legislative Analyst questions the validity of the two variables -- paternity and order establishment -- as indicators of success. By the LAO's analysis there is not a statistically significant relationship between higher collections and more paternities or support orders. It suggests the state develop incentives built around those variables that gauge efficiency in the programs -- such as the cost-to-collections ratio.(86)

And finally, the performance reviews do little to reduce the energy-draining disputes over the State's performance.

The Chief of the State Office of Child Support points to the most recent performance review results as evidence that "counties have improved their productivity especially in establishing paternity and support orders and collecting child support. Once we began paying counties for performance productivity skyrocketed."(87)

But advocates for custodial parents, and the parents themselves, tell a different story. The directing attorney of the National Center for Youth Law testified:

Creating Outcome-Based Accountability

The State's emphasis on process in reviewing county child support programs has its roots in the way the federal government has evaluated state programs in the past. But the feds are changing.

The change began with recommendations from the U.S. General Accounting Office (GAO) and reforms initiated by the Office of Child Support Enforcement (OCSE) under the Government Performance and Results Act of 1993. Both efforts attempt to focus management on accountability and outcomes.

The Government Performance and Results Act requires federal agencies to develop annual performance goals, report on whether the goals have been met and develop incentives that create accountability for results. The GAO evaluations of the child support program have criticized the current incentive structure, which bases the rewards on child support collections relative to administrative costs rather than on program goals. The GAO said the reward plan allows all states to receive incentive payments regardless of how well they perform and does little to encourage improvements or sanction under-performance.

The GAO recommended in 1993 that the OCSE focus its management of state programs on long-term outcomes and that it redesign incentives to encourage improved performance. The OCSE accordingly has now set five-year national goals for increasing the number of paternities and support orders established. Building on those changes, the federal welfare reform act now requires a new incentive funding system for state child support enforcement programs based on performance.

As is the case nationally, the first step in California toward accountability is reliable and comparable information. The debate must be moved from the validity of statistics to the validity of strategies. The second step is for counties to be held accountable for minimum performance standards -- to be sanctioned when they do not reach minimum standards and rewarded when they exceed them.

The department already has been encouraged by the Legislature to develop a fair and comparable way of gauging the county performance so that program directors can be held publicly accountable for their successes and their failings. It already has the authority under law to sanction counties that do not meet minimum standards. The department has ample evidence that the performance review process needs to be revamped -- to become an accurate gauge of outcomes rather than an inaccurate measure of procedural effort. If not for the sake of California's children, this change will be needed to keep pace with federal rules.

As described in Finding 3, one option is for the State to take over those functions that it can clearly do better than some of the underachieving counties. But short of that, county officials have identified at least two ways that technical assistance could be better linked with program evaluations to give family support divisions more traction on the learning curve:

The State does have another tool that can be used to bring accountability to a system that has sidestepped the opportunity to make itself accountable: allowing parents to bring legal action to enforce existing law. In 1997, the U.S. Supreme Court, in the case of Blessing v. Freestone considered whether citizens have a right to sue in federal court to enforce federal provisions that require states to enforce child support. While the court found that citizens were not entitled to a satisfactory performance by government child support enforcement efforts, it let stand the ability of citizens to seek judicial redress if public agencies fail to meet clearly established regulatory obligations. The case was remanded back to the U.S. District Court and other legal challenges on the part of parent and children advocates are expected to force lower courts to make the distinctions drawn by the high court.

Regardless of the outcome of the federal cases, California lawmakers could provide parents the right to bring citizen suits against public agencies that are not following clearly stated state laws or standards.

Summary

Without better data -- without a process for translating case numbers into families and children, and without knowing what needs are not being met and why -- managers will not be able to improve child support enforcement programs. Without meaningful program evaluations, they have little basis for allocating fiscal and political resources to construct effective solutions. And without the will and the commitment on the part of managers and policy makers to act upon the knowledge gained, the child support program cannot meet the needs of the children it is intended to serve.

Recommendation 2: To develop an effective child support program, the State should collect reliable data from the counties, conduct sound evaluations and enforce minimum performance standards.

The county district attorneys want -- and should have -- the liberty to make all of the day-to-day decisions about how to administer local aspects of the child support enforcement program. In exchange for that freedom, however, counties should be required to report reliable data on program performance so that the public and state officials can hold locally elected officials accountable for that performance. The incentive system should be revamped to reward results and not excuses. Measures the State should take include the following:







Next Section   Previous Section   Table of Contents