Dear Governor and Members of the Legislature:

Each year, California spends about $4 billion on goods, services and construction projects, making purchases through a procurement process that is highly regimented, hemmed in by paperwork and costly for everyone involved. While the State's procurement laws pay lip service to the concept of cost-effectiveness, the actual process -- as it is designed and carried out -- hampers purchasing decisions that would result in the State receiving the best value for the money it spends.

For the past eight months, the Little Hoover Commission has examined the State's procurement process, paying particular attention to electronic data processing equipment purchases, the program that promotes state purchasing from disadvantaged businesses, and the Prison Industry Authority, which sells other parts of state government $150 million a year in products. Since the Commission's study focused on the policies and procedures used by the State in making decisions rather than on individual purchases, the Commission neither sought nor found the kind of waste exemplified by the $2,000 coffee pots or $500 hammers from federal purchasing scandals. Nonetheless, the evidence of something awry with the procurement process -- and a resulting higher cost to the State --abounds throughout the Commission's report:

• After spending years on a multiple-submission, much-evaluated and thoroughly discussed procurement process for a new internal telephone network, the State rejected two out of three bidders as non-responsive. The third bidder, granted the contract, is running almost two years late in implementing the system.

• The State's top high-technology expert has identified four information technology systems where hundreds of millions of dollars were spent on equipment and software that was not capable of meeting the State's needs.

• One state licensing board has purchased a computerized telecommunications system in bits and pieces over several years, with the system never becoming fully operational because of mismatched and/or missing equipment.

• A state university has been forced to pay higher-than-market prices to the Prison Industry Authority for dormitory mattresses that have worn out in just a few years.

The Commission's report contains seven findings and 26 recommendations, each of which is highlighted in the Executive Summary. The overall thrust of the report can be summarized in three areas:

1. Best Value. The State must switch its focus, energy and efforts from the procurement process itself to the end product that the buying department receives. Managers must be given the responsibility and authority to make best-judgment decisions that allow the weighing of many factors besides costs. Accountability will be assured through an open, documented process that involves a panel of people evaluating bids.

2. Streamlined Process. The high costs now incurred by both the State and the vendors should be slashed by streamlining the existing, multiple-submission procurement process. When the State's policy makers add complexity to the procurement process with programs designed around social goals, they should place particular emphasis on tightly administering and monitoring such programs to ensure goals are reached without wasting resources.

3. Program Accountability. The Prison Industry Authority should be held strictly accountable for meeting the goals outlined for the program in statute. These include saving the State money and training prisoners. Currently, the program shifts costs to other state departments that must purchase its goods and services, and even then only survives with the help of hidden subsidies. There is no documentation or tracking of the program's effectiveness in training prisoners in a useful manner for the outside world.

Sincerely,

Nathan Shapell, Chairman

Executive Summary

Introduction

Background

Findings and Recommendations

High Tech and Best Value
The Protest Process
Cultural Diversity
Prison Industry Authority

Conclusion

Appendices

State Procurement Subcommittee and Advisory Committee
State Procurement Public Hearing Witnesses
California Public Contract Project Summary of Recommendations
Excerpts from Sacramento Disparity Study Recommendations
Examples of Good-Faith Effort Evaluation Standards

Endnotes

Executive Summary

To buy $4 billion a year worth of goods, services and construction activities, the State has set up a procurement process that emphasizes fairness, low cost and achieving a set of social goals. None of these necessarily means that the State gets the best product to meet its needs or maximizes the use of its limited resources. Each of them adds cost (directly or indirectly) and complexity to the procurement process, resulting in delays and inefficiency.

In this report, the Little Hoover Commission acknowledges the need for the State to be fair in its dealings with suppliers as it spends public money; to be a comparative shopper in order to stretch dollars as far as possible; and to influence private actions through public policies encouraging small businesses, recycling and cultural diversity. The first priority, however, must be obtaining the best value: selecting the product that provides the most benefits for the lowest life-cycle cost. The procurement system should be designed to encourage officials to make best value choices rather than forcing them to focus on the paperwork-intensive process itself.

From this perspective, the Commission examined four areas of procurement: major computer and telecommunications equipment purchases; the protest process; the program designed to encourage minority, women and disabled veterans participation in state business; and the Prison Industry Authority. As a result of its investigations, the Commission has made seven findings and 26 recommendations.

High Tech and Best Value

Finding 1: The present state procurement system focuses on low cost rather than on best value for the State.

Procurement experts and academics believe that, to make the best information technology purchases, governments need procurement systems that rely on knowledgeable, well-trained decision-makers who have been given the leeway to determine which bids offer the best value and are trusted to use good judgment (as opposed to hemming them in with rules and processes designed to protect against bias and influence).

While the State has acknowledged this theory by creating a separate set of laws to address the purchase of electronic data processing equipment, in practice the State's procedures for buying complex computer systems continues to rely heavily on low-cost evaluations rather than best-value judgments. A major reason for this is the State's emphasis on accountability: Decisions made on the basis of objective data -- such as pricing -- are much easier to document and defend then are decisions made on subjective assessments of who might perform best or how much better one piece of equipment rather than another will meet the State's needs.

The result can be wasted expenditures for inappropriate information technology systems or the failure to maximize the use of emerging technology because of lack of government expertise.

Recommendations:

1. The Governor and the Legislature should enact legislation that declares that the primary goal in conducting state procurements is to obtain the best-suited product at the best price.

2. The State Administrative Manual should be changed so that state agencies may use a non-commodity, best-value evaluation procedure at their own discretion.

The State has a multi-step process for obtaining bids in response to Requests for Proposals for electronic data processing and telecommunications systems. The system was designed to ensure that the State could modify its concept of what was needed as new information emerged and that all suppliers would have an equally clear understanding of the RFP. Instead, the process has resulted in paper-intensive, costly, multiple submissions by suppliers and lengthy, costly, multiple evaluations by state personnel, with little evidence that the best interests of either the State or the suppliers are served.

Recommendations:

3. The Governor and the Legislature should direct the Department of General Services to streamline the procurement process to avoid multiple submissions.

4. The Governor and the Legislature should enact legislation that directs contract language negotiations to take place only after bids have been awarded.

The requirements laid out in a Request for Proposals control the end product that the State will eventually receive. Besides directly detailing what is being sought, the requirements -- known as specifications -- have many indirect affects: They define how large the pool of bidders will be, set parameters on how much creativity and technological know-how will be used in responses, and limit the criteria by which proposals will be evaluated. Badly formed specifications, therefore, have an overwhelming impact on a procurement. Yet the State makes little effort to ensure that specifications are written to reflect accurately the State's needs and to give the State flexibility to entertain the widest range of creative proposals. The result often is overly restrictive specifications that may be perceived as biased toward one supplier's products.

Recommendations:

5. The Governor and the Legislature should enact a resolution that would proclaim the State's intent to use functional specifications rather than detailed technical specifications in procurements.

6. The Governor and the Legislature should direct the Office of Information Technology to ensure that Requests for Proposals match the scope and intent of the Feasibility Study Reports.

7. The Governor and the Legislature should direct the Department of General Services to make increased resources available to those who write specifications for procurements.

While the broad philosophical underpinnings of the State's procurement system identified in the first three findings have a dramatic, long-term effect on what the State purchases, other less wide-ranging policies and laws have a day-to-day impact on the value the State receives for its expenditures. These include rigid standards imposed on departments buying equipment, disincentives for the purchase of reconditioned equipment, limited accessibility to the rules governing procurements, and the lack of standardized requirements for interactions between vendors and state departments. The result of policies and laws in these areas is to constrain officials from making the best purchasing decisions and to discourage wider vendor participation in state procurements.

Recommendations:

8. The Governor and the Legislature should direct the Department of General Services to maintain equipment standards matrices only as an advisory guideline for departments.

9. The Governor and the Legislature should direct a modification of procurement procedures that would allow departments to purchase reconditioned equipment at their own discretion.

10. The Governor and the Legislature should enact comprehensive legislation to reorganize, simplify and streamline statutes relating to procurement.

11. In consultation with vendors, state departments and other procurement interests, the Department of General Services should review contracting and invoicing procedures and create standardized formats to be used by all departments.

California's procurement protest process is spread among a variety of bodies. Where a bidder goes to complain about a procurement process or decision is dependent on the type of contract involved and the stage of the process being protested. The protest process in general has few of the procedural guidelines and structured policies that usually are essential for a system to have predictability and credibility. The "final" decisions of the protest system often involve no resolution of the problem and are tainted by an appearance of conflict of interest -- all of which result in a perception that the State's protest mechanism is unfair and/or ineffective.

Recommendations:

12. The Governor and the Legislature should enact legislation to centralize and provide uniformity to the bid protest process, as well as to upgrade the technical expertise of those involved in the process.

13. The Governor and the Legislature should direct the State Board of Control to institute formalized hearing procedures, record precedent-setting decisions, order remedies for bid award errors when appropriate and in other ways standardize the operation of the bid protest process.

14. The Governor and the Legislature should enact legislation to create an independent, binding arbitration process for those protestors who are willing to pay the costs of an alternative process.

15. The Governor and the Legislature should enact legislation that would require the release of all relevant records to bid participants within a time frame sufficient to allow the filing of a detailed protest.

16. The Governor and the Legislature should enact legislation to require documentation of reasons when all bids are rejected and a project is put out for rebid.

As the MBE/WBE/DVBE program enters its fifth year, almost all state departments are failing to reach the 15-5-3 percent goals for contracts. The program's administration is fragmented and its provisions are applied unevenly; in some cases, the law has simply been ignored while in others advantage has been taken of loopholes.

The program's good-faith effort and certification components and the lack of enforcement mechanisms all impose undue burdens on state departments, vendors and MBE/WBE/DVBEs, adding to state and private sector costs without producing the desired results. Although recent revisions promise some performance improvement, other sorely needed reforms pose a dilemma by threatening the program's viability.

Recommendations:

17. The Governor and the Legislature should pursue one of three options:

    Option A: Enact legislation to contract for a disparity study and a recommended proportionate remedy as a prelude to adopting an aggressive, anti-discrimination procurement program.

    Option B: Enact legislation that will recast the present MBE/WBE/DVBE program so that it operates similarly to the Small Business preference program.

    Option C: Enact legislation that centralizes the authority and accountability for the MBE/WBE/DVBE program and provides adequate resources for outreach and enforcement efforts.

18. The Governor and the Legislature should eliminate the good-faith effort component of the MBE/WBE/DVBE program or reform the process so it achieves its intended purpose.

19. The Governor and the Legislature should enact legislation to abolish self-certification and set up a single-point full certification process.

20. The Governor and the Legislature should direct the Department of General Services to embark on an aggressive enforcement program.

21. The Governor and the Legislature should enact legislation that allows firms to file "global" plans with the Department of General Services as an optional way of complying with MBE/WBE/DVBE requirements.

22. The Governor and the Legislature should enact Legislation to protect past and current vendors in the event the State's MBE/WBE/DVBE program is found unconstitutional.

The Prison Industry Authority has a captive customer base in other state departments, which are forced to buy its goods and services. These customers, who have no leverage over PIA's performance, contend the products are overpriced, deliveries are often delayed and that quality is sometimes poor. The PIA defends its record, claiming that prices are actually low for the quality of goods sold and that its activities save the State almost $48 million a year. But the PIA is unable to show success in preparing prison inmates for the outside world, and its claims of providing cost savings evaporate quickly under scrutiny.

Recommendations:

23. The Governor and the Legislature should enact legislation that allows state departments to purchase goods from the Prison Industry Authority on the basis of best value for the department.

24. The Prison Industry Authority should require its annualaudits to recognize and document the subsidies it receives.

25. The Governor and the Legislature should enact legislation to give the Prison Industry Authority the responsibility of creating a hiring process that reflects real-world conditions.

26. The Governor and the Legislature should enact legislation requiring the Prison Industry Authority to report on program outcome statistics.

Introduction

Where a person does the weekly grocery shopping is determined by a variety of factors: habit, proximity, prices, available selection of goods. Often without conscious consideration of each of these factors, a decision is made to go to one store rather than another.

Similarly, when a private business turns to its suppliers for goods or services, many factors beyond price may be considered: past performance, timeliness, quality, potential for doing business together in the future. The bottom line for any decision is a judgment of what works best for the company.

The State, when it acts as a buyer, faces the same considerations as an individual or a private business considering a purchase. What is the cheapest price it can pay? Will the product do the job it is being purchased to accomplish? Is the quality such that it will last for an appropriate number of years without requiring frequent repairs? Will it be delivered on time or be operational on schedule? In other words, what will work best for the State?

State's purchasing
decisions not based
solely on low cost
or best value

But unlike an individual or a private company, the State cannot simply gather comparative data and then make a decision to buy based on impulse, best guess or even best judgment. Instead, the State must also take into consideration other doctrines that have been laid out in statute or directed by policy makers. These include:

• Fairness to all potential suppliers. Many involved with procurement have expressed the belief that since the State is spending public funds, it has an obligation to see that no favoritism enters into the selection of who will receive the funds. This belief most often is expressed as the need to provide "a level playing field" so that all businesses are playing by the same ground rules and have an equal opportunity to become state suppliers.

• Competitive bidding. Although the State spends millions of dollars each year on "sole-source" contracts, for the most part who gets the State's business is decided in a competitive bidding process. Whether informally by telephone or more formally through voluminous documentation, the State seeks bids from suppliers that it can compare and use to make buying decisions.

• Achieving certain social goals, such as reflecting cultural diversity in suppliers, giving an edge to small businesses and encouraging the use of recycled products. In each instance, the State's policy makers have decided that the value of achieving the social goals outweighs whatever costs may be added to the procurement process. This value may be simply in terms of fairness or may be a more complex judgment that long-range costs are driven down by extra efforts to expand the pool of suppliers and increase competition over time.

Purchasing policies
are defensible despite added costs
and complexities

In the abstract, each of these elements is easy to defend. No one argues that it would be better for the State to be unfair in choosing suppliers. But the focus on fairness down to the last, myopic detail of each procurement process contributes to the growth of acrimonious protests and the multiplication of paperwork dedicated solely to proving that a just decision was made. A fitting analogy can be found in the medical world, where doctors argue they must order multiple, expensive tests and practice defensive medicine because of the heavy threat of malpractice. Similarly, a state procurement official may be caught up in the process of dotting the i's and crossing the t's to enable the defense of a purchasing decision rather than focusing on what would best suit the State's needs.

Competitive bidding also appears to be an inarguable "good." Why should the State choose any method other than making suppliers give it a price that can be compared to other suppliers' prices? But a system that works well when buying pencils, where it is easy to compare like products, falls somewhat short when it is used to buy complex, computerized systems that could work in any number of ways to fill a need. To be able to compare bids on equal products, the State must rigorously mandate detailed specifications. But such detailed specifications may preclude options -- including some of great potential value to the State -- that suppliers could offer if they were not hemmed in by rigid requirements.

Finally, the social goals represent public policy that has been set by the Legislature to redress past wrongs or shape the future. The goals, however, often are adopted without information available on added costs and other ripple affects. For instance, if businesses in targeted, economically disadvantaged areas of the State are given an edge in bidding, how much is the added cost to the State? And how many jobs do those added dollars yield -- or what other measure can be used to demonstrate the added value the State receives in return for the added expenditure?

The State, then, is not a simple consumer who will make purchases based on lowest cost or best value. Instead, the State has added other functions to its role as a purchaser. Each of these other functions adds cost, complexity, delay, a need for thorough documentation and an adversarial tone to the State's procurement process, none of which exist in the private sector.

Despite massive
purchasing power,
the State does not
always get best deal

The practical effect of these layers of considerations is that the State, despite its massive purchasing power, often does not get the best price, the best product or the best service. As the head of the State's procurement office has pointed out, a person can go to a discount retail store and buy a gross of pencils more cheaply than the supply officer of a state department is able to buy them for use by state workers.

The Little Hoover Commission believes that at a time when perpetual budget shortfalls are crippling state programs, it is critical that the State minimize wasted resources and maximize the economies of scale afforded by the State's bulk purchasing power. In mid-1992, the Commission therefore initiated a study of the State's procurement practices. The Commission convened an advisory body of state officials, private suppliers, lawyers, procurement experts and other interested parties (please see Appendix A for list of those who participated).

Commission's
scope and
methodology
for study

Based on information provided by the advisory committee and its preliminary investigation, the Commission chose to focus its study on four areas:

• Major procurements. The State spends about $300 million on telecommunications and electronic data processing equipment each year. Typically a purchase that involves cutting-edge technology, intense competition and high-dollar costs, major electronic equipment procurements during the past decade have proven contentious, sometimes ending in litigation, critical audit reports and unsatisfactory systems that cannot achieve the desired goals.

• The protest process. If a supplier does not win a contract from a private company, he is left with few options except to try harder the next time. The same contractor has more options when it comes to a lost state contract: He can use the State's protest mechanism or file a lawsuit. The current protest process is fragmented and is viewed with skepticism or outright scorn by many of those who make use of it.

• The minority, women and disabled veterans program. The State has set goals of 15 percent for minority owned businesses, 5 percent for woman-owned businesses and 3 percent for disabled veteran owned businesses as suppliers for the State. The program, which is only four years old, has yet to mature into a smoothly working mechanism that broadens the pool of suppliers.

• The Prison Industry Authority. Selling about $150 million in goods and services to state and local government entities, the Prison Industry Authority has often been characterized as holding state departments hostage to high prices and delayed deliveries. Current state law forces departments, with few exceptions, to buy products and services from the Prison Industry Authority regardless of price.

The study has resulted in this report, which begins with a transmittal letter, Executive Summary and this introduction. The following sections include a background, four chapters of findings and recommendations, and a conclusion. The report ends with appendices and endnotes.

Background

The State spends about $4 billion each year on goods, services and construction contracts. Several programs affect the selection of a winning bidder, including preferences for small businesses, disadvantaged areas and recycling. A pattern of troubled procurements and less-than-satisfactory purchases in recent years indicates that policies are not working well. The State's top priority appears to be conducting a "perfect" procurement process rather than getting the most cost-effective products and timely results.

The State spends about $4 billion to procure goods and services each year through 80,000 separate transactions. 1 These purchases take place in a variety of ways, with oversight mechanisms differing depending on the amount of money involved, the size of the department making the purchase and the type of purchase. Approximately one-quarter -- or $1 billion -- of the purchases are handled by the State Office of Procurement, a unit within the Department of General Services. Other procurements are handled independently by large state departments and the State's two university systems.

Among the mechanisms in place that allow departments to procure goods and services are:

• The State Stores, an operation that purchases, warehouses, resells and delivers to state departments supplies, such as paper, pens, printer ribbons and paper clips. More than 1,300 different items are stocked, including 750 general office and janitorial supplies, 500 state forms and 60 canned food items. Between $9 million and $10 million in stock is kept on hand at any one time. 2

• Master contracts and master service agreements, which establish set prices for repair services and items such as personal computers, copiers and other equipment. Departments order directly from the suppliers who have won the master contract rather than going through an individual procurement process.
• Purchase Orders, which allow departments to spend up to $100 with little or no paperwork after soliciting bids by telephone.

• Invitation for Bids (IFBs), which are used for commodities that can be easily compared to each other.

• Request for Proposals (RFPs), which in theory are supposed to detail a department's "business problem" and invite suppliers to submit creative solutions, including hardware, software, management, etc. In practice, RFPs often are heavily weighed down with precise specifications that may pre-ordain the eventual product that is purchased.

State has several
programs that affect
the selection of
contract winner

In addition to having different mechanisms for procurements and varying lines of authority for purchases, the State also has a variety of programs that affect which bidder will be designated as the winner of a contract. For instance, the Small Business preference gives California-based firms that have been certified as small businesses a 5 percent preference on bids, as long as the cost differential does not exceed $50,000 per contract. This means that if a large business bids $1 million and a certified small business bids $1,040,000, the small business wins the contract at an extra cost to the State of $40,000.

The Department of General Services tracks the effect of the Small Business preference and has concluded that on an annual basis this preference shifts who the contract is awarded to on about $30 million worth of contracts at an added cost of about $500,000. 4 In 1991-92, $367.7 million in contracts were awarded to certified small businesses, or 9.3 percent of the almost $4 billion in procurement contracts.5

A similar preference program is aimed at businesses in economically disadvantaged areas that have been designated by the Governor's Office of Planning and Research based on census data. Called the Target Area Contract Preference Act (TACPA), this preference gives a bidder a 5 percent edge on price. In addition, the bidder may receive an extra 1 to 4 percent preference for hiring people with a high risk of unemployment, provided that at least 50 percent of the labor required to perform the contract will be performed in or within commuting distance of a distressed area. (The actual scale is an additional 1 percent preference if the high-risk unemployed make up 5 to 9 percent of the work force, 2 percent if 10 to 14 percent, 3 percent if 15 to 19 percent and 4 percent if 20 percent or more.)

The total additional cost of TACPA to the State may not exceed $50,000 and the preference may not be used if its use would deny a small business bidder the contract.6 While not having statistics that include all state departments, the Office of Small and Minority Businesses indicates that this preference is used infrequently. In 1991-92, seven contracts worth $7.59 million were awarded based on TACPA preferences.7

On contracts for paper and paper products, a bidder may receive a 5 percent preference for using recycled paper at a maximum cost to the State of $100,000 on each contract. However, if granting a recycled-paper preference exceeding $50,000 would preclude a small business from receiving the award, then the recycled paper preference is limited to $50,000. (As an ecological side note, the State also prohibits agencies from contracting with businesses that have violated water pollution laws.)8

While not a preference program, the other major statutory requirement affecting contracts is the Minority Business Enterprise/Women Business Enterprise/Disabled Veterans Business Enterprise (MBE/WBE/DVBE) program. This program, which sets goals for participation in state contracts, is discussed at length later in this report.

Preferences may be combined, as long as the total added cost to the State does not exceed $100,000 for any one bid.

Limited resources
affect how widely
preference
programs are used

While the Commission did not choose to focus on the various preference programs, it did note that the extent to which they are used is affected greatly by resources. For instance, out of an estimated 700,000 small businesses in California, only about 12,000 are on the active certified list. The Office of Small and Minority Business identified the lack of funding to perform educational outreach as the reason for low participation in this and the TACPA preference program. In addition, strained resources have affected the office's ability to process small business applications, creating a four-month backlog of 4,000 small businesses waiting for certification or recertification. 9

Since the preferences were created to carry out state policies and reach certain -- although undefined -- goals, it appears counterproductive to underfund the efforts that could lead to a greater use of the preferences. If state decision-makers believe small business participation in state contracts is healthy for the State's economy, that state dollars flowing into disadvantaged areas creates jobs where they are most needed, and that the environment is assisted when state spending encourages recycling, then it seems logical that the State should invest the resources that would maximize the use of the preference programs. Instead, it appears that programs are created with the best intentions but are neither adequately carried out to assure that they have the desired affect nor monitored to assess their degree of success in fulfilling their intent. The result is to add to the complexity of state procurement processes without necessarily producing the desired benefits.

Procurement process
has been shaped
in response to past
practices and problems

The complex web of statutes, regulations and administrative procedures that govern procurement did not develop overnight. Those who have been familiar with state procurement practices over several decades have likened the present system to the far swing of a pendulum toward intensive accountability and well-documented regimentation as opposed to a less-regulated system in the late 60s and early 70s that was embroiled in controversies and scandals. Favoritism in procurement then was both the perception and the reality, according to those who track procurement practices.

In 1977, the Department of General Services created a task force of state officials, a study panel of outside procurement experts and a committee of American Bar Association contracting experts to examine and recommend ways to overhaul the State's procurement practices. Known as the California Public Contract Project, this extensive effort resulted in 31 recommendations about procurement practices in general and an additional seven recommendations focusing on electronic data processing purchases. Some of the recommendations included (please see Appendix C for the report's executive summary of recommendations):10

• Reorganizing all laws pertaining to purchasing into a single state act.

• Placing uniform contracting procedures into regulations, which require public input, rather than in internal policy manuals.

• Providing standardized model contract clauses.

• Setting up an independent commission to systematically review the procurement process on a periodic basis.

• Creating a scheme of selection criteria for personal services contracting that would include merit as well as pricing.

• Performing an economic and operational study of socio-economic programs (such as preferences for small businesses and recycled paper) to determine effectiveness and costs.

• Adopting uniform protest procedures for all types of contracts.

• Enabling the courts to direct an appropriate award of a contested contract.

• Creating a program for on-going research and training in government procurement law and procedures.

• Enacting a comprehensive, separate procurement act for electronic data processing.

• Developing sample contract provisions to permit flexibility to accommodate various sized procurements and degrees of complexity.

• Centralizing electronic data processing procurement functions.

With some exceptions, few of the recommendations have been implemented despite the high profile given to the study and its participants. Even the exceptions have not resulted completely in the changes sought by the study's authors. For instance, the first and most major recommendation was to recodify and reorganize statutes into a single act. This was done; however, many legal experts feel the recommended consistency, clarity and simplification that was supposed to accompany this overhaul of state law was never achieved. Similarly, separate statutes were created to address the unique aspects of purchasing electronic data processing systems. However, the underlying intent of having decision-makers use flexible mechanisms for making complex purchases on the basis of factors other than low cost has not been fulfilled.

The Commission draws attention to this previous, landmark study because the Commission has found, based on its own independent investigations, that many of the goals sought in the report remain valid today, 16 years later, although the recommendations themselves in many cases may need to be updated. These goals include creating a procurement system that is:

• Easily understood and managed.

• Fair to participants.

• Flexible enough to allow the State to make the best decisions as a well-informed consumer.

Pattern of troubled
procurements points
to problems with
existing process

That the State's system of procurement falls short of these goals is evidenced by a pattern of problem-plagued procurements over the past decade. The following highlights only a few of the controversial procurements:

California State University Computers: In the mid-1980s, the California State University system sought to revamp its computer systems with a procurement known as Educational Administrative Systems Environment (EASE). Control Data Corporation, one of two bidders on the $24 million system, accused the State of having a procurement process heavily biased toward IBM, the other bidder, through the use of detailed specifications that favored IBM equipment. Eventually, Control Data Corporation withdrew its bid after unsuccessful protests and requests for changes in specifications. IBM, which already had supplied other major systems to the university system, won the contract.

CALNET: In 1989, a long study and procurement process ended with the award of a contract to GTEL for approximately $109 million over 10 years to provide a new telephone system known as CALNET for state operations. This procurement, handled by the Department of General Services, prompted a full range of protests, audits, hearings and court filings. The State's Request for Proposals (RFP), issued in September 1987 and amended 26 times over the course of the procurement process, resulted in 17 months of meetings, submittals, evaluations and resubmittals. Each of the three eventual bidders submitted no fewer than five separate proposals during the process. When the State finished its evaluation, it threw out both the high and low bidders as non-responsive and announced its intent to award the contract to GTEL, the middle bidder.11

Protests were made using the State's bid protest process and the Superior Court system. In addition, hearings were held by the Legislature and the Little Hoover Commission during the procurement process and, after the bid had been awarded, an audit was conducted by the State Auditor General. Issues raised included:

• The State's decision to only solicit for systems it could purchase rather than also entertaining proposals for leased systems. While the courts upheld the right of the State to limit the bids to purchased systems, critics -- including the Auditor General 12 -- contended the decision was reached without adequate study and documentation.

• The decision that two out of the three bidders were non-responsive, despite the months of meetings and pre-evaluations that should have eliminated problems from the final bids. The unsuccessful bidders maintained that not only were they responsive and that deviations were not material, but that GTEL's successful bid suffered from the same kinds and degree of defects.

• The lack of a Feasibility Study Report, a document required by the Office of Information Technology (within the Department of Finance) that examines the needs of an agency and explores the long-term effect of different options to meet those needs. While the project did not legally require such a study because telecommunications does not fall under the jurisdiction of the Office of Information Technology, the lack of such thorough planning was criticized in the Auditor General's report, which concluded that the State may not have obtained the most cost-effective solution to meet its needs.13

None of the protests were successful in altering the outcome and on November 28, 1989, GTEL was awarded the contract, with an initial cost of $66,987,694 to cover the first three years. Since then, the Department of General Services reports that there have been 10 amendments to the contract with an additional cost of $325,906. Implementation dates, which originally called for three phases to be completed in January 1991, October 1992 and May 1992, have been revised many times. The first phase was completed on July 1, 1992 (18 months late) and the last two phases have been combined and are now expected to be complete in June 1994 (more than two years late). 14

Department of Motor Vehicles: The Department of Motor Vehicles has been the focal point for several controversial procurements. During the early 1980s, both a computerized database system and a reflectorized license plate contract were criticized for alleged favoritism. More recently, the department conducted a procurement for "credit card" style driver's licenses that could store data about the driver on a magnetic strip. Like the CALNET procurement, the driver's license procurement involved multiple submissions of detailed technical proposals, draft proposals and final proposals. Seventeen final proposals from six different suppliers were submitted in June 1989. After an evaluation and benchmarking process, National Business Systems Inc. (NBS) was announced as the intended awardee.

Four other bidders filed protests with the State, which were rejected, and one bidder -- National Information Systems Inc. (NIS) -- has continued to pursue the case in court. In addition, the Auditor General issued a report in February 1990. 15 Legislators also slated hearings and in other ways sought to intervene in the process. Issues raised included:

• Whether there were actually two or more bidders as required by law since all bidders but the successful one had deviations from requirements, although the Department of Motor Vehicles did not determine whether the deviations were material or immaterial.

• The fact that the successful bid was based on a single density magnetic stripe with abbreviated data rather than on a higher density magnetic stripe that could hold a minimum of 268 characters of data as specified in the RFP and as bid by the other suppliers. The other bidders were not alerted that the State would allow abbreviated information and a capacity for fewer than 268 characters.

• The quality of the benchmarking demonstrations. Of 77 objectives that the bidder was supposed to prove ability to meet, only 64 were demonstrated; verbal assurances on the other objectives were accepted by the State. In addition, tests that were conducted and showed deficiencies were not deemed important enough by the State to deny the contract to the bidder.

The complexity of this procurement and the difficulty of resolving the issues raised is demonstrated by the different conclusions reached by the various entities that reviewed the procurement thoroughly and rendered judgments:

• The Auditor General concluded that the Department of Motor Vehicles and the Department of General Services "did not always follow state procurement procedures and, as a result, may have to rebid the contract to ensure that the procurement is legal and in the best interests of the State...the State may not have received the best product at the lowest cost."16

• The administrative law judge who conducted 39 days of hearings with 31 witnesses and 244 items of evidence recommended to the State Board of Control that all protests be denied. He wrote:

  The procurement process was not perfect and all participants made mistakes, particularly the State. ...Despite the mistakes and imperfections, there was no proof that the procurement was not conducted in an entirely even-handed manner, with all bidders having the same rules applied to them and a level competitive field. ...It was not established that there was any favoritism, inequality, unfairness or prejudice to one bidder over another in this procurement. ...Although the procurement was not mistake free, no bidder received an advantage or suffered a disadvantage relative to the other bidders as a result of State action or failure to act. The State did not accept a final bid containing a material deviation and the protestants failed to prove such a contention. The State did not conduct the demonstration improperly, and although substitutions and nonmaterial deviations were revealed at the demonstration, NBS successfully demonstrated the capability to perform the State's work in substantial accord with the starred mandatory requirements of the RFP.17

• Sacramento County Superior Court Judge Richard K. Park ordered the State Board of Control to set aside its decision rejecting the protests and to reconsider the case, citing only a few instances in the long and complicated decision where he believed the administrative law judge had used the wrong standard or reached the wrong decision.18

• The State appealed the Superior Court decision to the State Court of Appeal, Third Appellate District. That court reversed the Superior Court judge, concluding, "NIS fails to show abuse of discretion in the Board's decision in any regard. It was not put at a competitive disadvantage by the actions of the State. Therefore, the superior court erred by issuing the peremptory writ."19

California Highway Patrol: The procurement of a statewide Computer Aided Dispatch (CAD) system for the California Highway Patrol included three separate RFP processes, a state finding that the contractor finally selected was in default, and the award of the contract to the "losing" bidder from the third procurement. A timeline is provided in the table on the following page:

SUMMARY OF EVENTS IN CHP COMPUTER PROCUREMENT

DATE	STATUS
August 1988	First Request for Proposals released.
March 1989	Three final proposals received with bids ranging from $7.9 million to $8.6 million.
May 1989	All three rejected for material deviations. The procurement is cancelled. A second RFP is issued.
June 1989	Three final proposals received with bids ranging from $7.7 million to $9.1 million.
July 1989	Only one bid determined to be responsive; intent to award issued.
August 1989	Protest filed by one of the losing bidders.
September 1989	After review of the protest issues, the State cancels the procurement.
October 1989	A third RFP is issued.
May 1990	Three final proposals are received with bids ranging from $8.7 million to $10.5 million.
July 1990	High bidder is found non-responsive. Intent to award issued for middle bidder, who scored more technical points in the evaluation process than the low bidder. Protest filed by low bidder.
August 1990	State issues response to protest.
September 1990	Protest withdrawn.
September 1990- March 1991	Alterations are made to the contract. The winning bidder contends that three system capabilities required by CHP are outside the scope of the contract. The State disagrees verbally, in written memos and finally in a formal ruling. The delivery date of an operational system falls into dispute. Finally, the State declares the supplier in default. The supplier fails to convince a court to issue a temporary restraining order against the State.
May 1991	The contract is issued to the low and only remaining responsive bidder from the third RFP.
July 1991	The original supplier files suit, seeking more than $100 million in damages.
September 1992	Trial begins in San Diego court.
December 1992	A jury concludes that the firm breached its contract and owes the State more than $1 million, twice the amount given to the firm in partial payments during the time the contract was in effect.

Source: Department of General Services

As the table shows, it took almost three years from the time the first RFP was issued until a bidder who could meet the State's needs was selected.

This list of troubled procurements could continue. A $100 million system for child abuse case management for the Department of Social Services was the focus of protests, all of which were denied. A system to track child support payments, also for the Department of Social Services, has met with protests. A third system for the department, this one to automate the welfare application process, has been contentious. And newspapers recently highlighted the procurement process for the Department of Justice's Statewide Integrated Narcotics System when potential bidders claimed specifications were tailored after the products of one company. Eventually all bidders dropped out except that company, and Digital Equipment Corporation ended up with the $50 million contract.20

Commission focusing on policy issues
rather than on faults
with specific purchases

The Little Hoover Commission did not examine each of these procurements to determine whether errors were made or bad decisions were reached, leaving those functions to the entities involved in the protest process, the Auditor General and the court system. The Commission instead focused its attention on the policy perspective of procurement, noting that the examples above raise serious questions about whether the State's procurement process meets the reasonable goals of being easily understood and managed, fair to participants, and flexible enough to allow the State to make the best decisions as a well-informed consumer.

The State itself has shown no smugness about the contentious and adversarial tone that has been cast over procurements. While insisting that procurements are conducted without bias, state procurement experts see the system as burdened with requirements and shaped in response to their belief that the Legislature places a higher priority on process than on results. The head of the Department of Motor Vehicles summed up the situation in a letter to the Commission:

Everyone involved in the State's procurement process, including the vendor community as well as State agencies, agrees that the current time-consuming and costly procedure is in dire need of an extensive overhaul. The basic problems cannot be resolved by making minor procedural changes. Basic change is required at a policy level. A decision must be made regarding priorities: Is it more important to conform to current procedures with the inherent delays associated with accommodating a wide range of vendors and subjecting them to unnecessary expense? Or is it more important to obtain cost-effective and timely results? In today's procurement environment, these are competing themes.21

In the course of its investigations, the Commission addressed these policy issues, coming to the conclusions described in the following four chapters of findings and recommendations.

High Tech And Best Value

Information technology purchases are difficult to assess based on price alone, yet the State's procurement system is slanted heavily toward selecting low-cost rather than high-value bids. The procurement system is cumbersome, slow and costly for the State and the vendors. Recommendations: Declare that the top priority in procurement practices is obtaining the best value. Empower managers to make selections on factors other than price. Streamline the process to avoid multiple submissions. Involve the Office of Information Technology in the creation of well-drafted, functional specifications.

Among the largest individual procurements made by the State each year are those that involve electronic data processing and telecommunications equipment. The State spends about $300 million annually on computer and telecommunications technology. Between 900 and 1,000 transactions are for equipment costing less than $100,000, while approximately 60 procurements annually involve more than $100,000 -- usually millions of dollars.22

These highly technical, complex procurements are among the most contentious that the State deals with, largely because apple-to-apple comparisons are difficult and cost/benefits are not simply a matter of looking at how much an item costs and how long it will last. Those simple comparisons can be made easily and with little controversy when off-the-shelf commodities are involved. When the State buys a commodity, such as pencils or even a personal computer, the specifications can be rather narrow and definitive. Bids that meet the specifications can be weighed, one against another, and the contract awarded to the responsive supplier who provides the State with the lowest price.

When it comes to
high tech, price
is not the most important factor

In contrast, electronic data processing systems often involve much more than hardware and software. Management of data, the development of the system itself and other non-tangible services enter the picture, as does the question of ability to perform as promised. Price, the most easily measured component of a bid, may be one of the least important factors in comparing bids. As one person sketched it out for the Commission:

Consider two cases involving the purchase of a small piece of wood. In the first case, the piece of wood is manufactured into a pencil, a number two lead pencil. One pencil is virtually indistinguishable from another and the best buy on a dozen pencils is the box obtainable at the lowest cost. It makes little difference whether a box of pencils is manufactured by Acme or Allied.

In the second case, the piece of wood is made into a musical instrument, a violin or a classical guitar. The quality of sound produced by a musical instrument is greatly dependent upon the experience and skill of the master craftsman who selects the wood and hand makes the sounding board. It makes a great deal of difference if a classical guitar is made by Vicente Arias -- and the value of an Antonio Stradivarius violin is self-evident.23

Because high technology can be enhanced by the choices made in system design or by the way it is installed, managed and integrated, who provides the equipment and services to the State can sometimes be as important as what is provided. In addition, alternative solutions can be provided that may meet the State's needs equally but in very different ways. In both cases, a low-cost bid process does not allow the State the latitude it needs to pick and choose among bids.

The Commission has determined that while state law acknowledges the differences between buying simple products and complex technology, state practice does not always reflect that acknowledgement. This chapter details the Commission's four findings and 11 recommendations pertaining to procurement processes.

Finding 1:

The present state procurement system focuses on low cost rather than on best value for the State.

Procurement experts and academics believe that, to make the best information technology purchases, governments need procurement systems that rely on knowledgeable, well-trained decision-makers who have been given the leeway to determine which bids offer the best value and are trusted to use good judgment (as opposed to hemming them in with rules and processes designed to protect against bias and influence). While the State has acknowledged this theory by creating a separate set of laws to address the purchase of electronic data processing equipment, in practice the State's procedures for buying complex computer systems continues to rely heavily on low-cost evaluations rather than best-value judgments. A major reason for this is the State's emphasis on accountability: Decisions made on the basis of objective data -- such as pricing -- are much easier to document and defend then are decisions made on subjective assessments of who might perform best or how much better one piece of equipment rather than another will meet the State's needs. The result can be wasted expenditures for inappropriate information technology systems or the failure to maximize the use of emerging technology because of lack of government expertise.

Academics focus on
results over process, long-term relationships
and best-value buying

Academic studies tend to focus on federal procurement practices but in many ways are relevant to California's policies and problems. A review of literature on the subject highlights three areas:

• Most experts agree that when it comes to purchasing information technology -- computerized systems and telecommunications equipment -- governments place too much emphasis on the actual procurement process and too little on the desired results.

• Some experts believe that governments overlook the advantages of turning suppliers into "partners" and of having long-term relationships with vendors.

• Experts also believe governments fail to understand the benefits inherent in the "best value" approach to purchasing.

The problem with the current system is that public officials cannot use common sense and good judgment in ways that would promote better vendor performance....I am suggesting that too much weight is currently placed on equity and integrity at the expense of other values, such as the substantive quality of procurement performance.24

The present system
hinders managers
who want to make
rational choices

Kelman argues that there are other ways to halt fraud, corruption and kickbacks than the multiple layers of regulations that tie the hands of bureaucrats who are trying to make purchasing decisions. The present system, he says, does not allow bureaucrats to make the rational decisions they would make in their own life as private consumers, such as returning to a store where the service has been good or buying a product that has worn well over time.

The fear of discretion makes it more difficult to select the right vendor because public officials cannot use important information that could help predict vendor performance if that use requires judgment that the system forbids the officials to exercise. The most dramatic example of the information that may not be used is information regarding the past performance of vendors on earlier contracts with the organization. Such information is at the heart of countless decisions in everyday life about the future performance of others. Yet it is ruled out because biased or corrupt decision makers might pick and choose from a vendor's past actions as an excuse for an unfair contract award.

Moreover, unwillingness to allow officials the discretion to depart from "free and open competition" can prevent the government from obtaining more value from vendors. Officials cannot offer as an incentive for good performance a promise to award future contracts. For the same reason, vendors are less likely to invest in developing creative ideas that the customer had never conceived, as opposed to ideas for effecting what the customer has already developed. Indeed, the greatest costs of the current system may well be those least recognized by participants -- the creative ideas and suggestions that are never made, that suggest that participants may not know what they have missed. 25

One remedy: Replace
rules with written
justifications and
multi-member panels

Kelman advocates eliminating most procurement rules in favor of two broad procedural requirements: written justification for each procurement decision and multiple-member evaluation panels to reach decisions. The first should supply enough information to ensure that fraud has not played a role, and the second should make bribery or corruption more difficult since there is no single player.

 A special study and workshop on strategies for purchasing information technology by the John F. Kennedy School of Government, Harvard University, echoed many of the same themes:

...procurements have become too "rule-bound" -- that is, excessively mechanistic and controlled. They give too much weight to contracting values and not enough to program values. They handcuff all participants in a misguided attempt to insure that all potential vendors are given scrupulously equal treatment.

Rigid rules have not only failed to control favoritism, they have hindered the exercise of front-line judgment essential for success with complex, innovations-oriented procurements. 26

The study found that overemphasis on prior specification and documentation has led to procurements that are too slow, incapable of taking into account a vendor's past performance and unable to incorporate information learned during the evaluation process. As one of the Harvard workshop participants put it:

It's hard to say when it happens, but you can put it down as an immutable rule. Somewhere in the course of complex procurements, while trying to determine how best to spend a substantial amount of the taxpayers' money, government ceases to be regarded as anything remotely resembling a customer and becomes instead the referee in a competition among prospective vendors: " These specs are proprietary. " "These specs aren't clear." "We don't have enough time."

So it goes. What was it we were trying to buy again?

The present system prescribes one of two approaches. Either you ask in advance for what you want with sufficient specificity that a choice can be based solely on respondents' prices, or you ask for what you want in more general terms, in which case the choice is based upon broader, but still very objective weighted criteria, which are likewise specified in advance. Either process can take weeks or months. In the meanwhile, if you learn anything new which is important to your decision, either forget it or start over. Your job is to impartially apply the rules that you have previously laid down.

Having created this situation, it is little wonder we are frequently stuck with what we asked for instead of what we want and need. 27

The Harvard study advocated greater flexibility and discretion for purchasing agents, but it also recommended relying more on relationships and partnerships with private industry to meet its information technology needs. For example, a competitive selection process could be used to allow governments to select a vendor to meet a well-defined need. The actual method for meeting that need could then be developed cooperatively. Or a joint partnership could be formed where the government entity and a private firm develop a program to meet some specific need and then share the risks, costs and/or profits. This would be distinctly different from today's procurements, which hold vendors at arms' length from the government/customer. But not everyone is comfortable with the concept or can envision how it would be structured. One of the Harvard workshop participants called it "an exceedingly important topic whose resolution is fraught with peril" and another called it "antithetical to both the project basis of procurement and the concept of competition."28

"Best value" systems
allow a trade-off
between price and
desirable features

The third area highlighted in academic circles is "best-value" procurements compared to "full and open competition." Procurement authorities Ralph C. Nash and John Cibinic, professors at George Washington University, write that best value refers to a system where a contract award decision is based on a trade-off between the price offered and other features of the proposal, such as quality, technical aspects, management potential and timing.

Such a trade-off is sometimes referred to as a "cost/benefit" analysis. It requires that judgment be made as to whether the price differences between competing proposals are justified by their other aspects....

The only alternative to best value source selection is the "lowest cost, technically acceptable" method. Under that technique, the proposals are separated into two categories: those that are technically acceptable and those that are not. Award is then made to the responsible offeror with the lowest cost, technically acceptable proposal. The source selection official is not required, or permitted, to make cost/quality or technical tradeoffs....While this method assures that the Government will not pay a higher price for slight technical advantages between proposals that are technically acceptable, it also requires the Government to accept significantly inferior items at small monetary savings. What is worse, it only has the appearance of objectivity. The determination of technical acceptability is highly discretionary and a technically unacceptable proposal can be excluded from the competitive range without consideration of cost.29

Out of three million contracts since 1991, the Defense Logistics Agency has awarded almost 120,000 contracts to quality vendors. Only 594 involved the payment of a price preference (for a total of $240,000 extra). Writing in a handbook explaining the program, the federal agency said:

Our statistics show that quality vendors usually offer the lowest price as well. While the number of times that awards have been made at a price preference seems small, our willingness to pay the preference has indicated our commitment to quality and has had an impact far beyond those few times we actually awarded to other than the low price....The benefit of best value programs to the Department of Defense is reflected in the reduced cost of contract administration, fewer late deliveries, fewer quality defects, reduced stock levels and costs as a result of more reliable delivery, and perhaps most importantly, greater confidence in the quality and reliability of the items acquired. 30

Best-value systems
increase competition
by allowing a wider
range of proposals

Some critics have worried that best-value purchases limit competition. But rather than being anti-competitive, the two George Washington University professors argue that best-value procurements actually increase competition. Because the decision maker is free to pick and choose among many options rather than tied to a single, overly defined set of specifications, a wider range of proposals will be submitted from a larger number of suppliers. Nash and Cibinic add that decision makers should be required to explain their selection regardless of whether a higher- or lower-priced offer is selected.

Because some have the misguided belief that the lower-priced offer is always better, much of the focus has been on justifying the advantages received by award at a higher price. We think that if a lower-priced offer is selected, the source selection official should be required to justify why the dollar savings are sufficient to make up for the inferior performance the Government will receive. The only award that does not need explanation is when the best proposal is also the lowest priced -- the source selection official's dream. In real life, the selection decision is more likely to resemble a nightmare. 31

In California, many people who participate in the procurement process and who spoke to the Commission feel that nightmare is not too strong an adjective for the State's process. But California at least has taken steps toward implementing current theories in the three areas -- best-value procurements, partnerships, and process- rather than result-orientation -- although in practice the outcome may fall short.

State law specifically addresses the standard by which bids will be awarded. Section 10300 of the Public Contract Code says in part that it is the intent of the Legislature that there will be "full compliance with competitive bidding statutes as a means of protecting the public from misuse of state funds;" that all qualified bidders will have "a fair opportunity to enter the bidding process, thereby stimulating competition in a manner conducive to sound state fiscal practices;" and that "favoritism, fraud and corruption" will be eliminated. To achieve this intent, contracts shall be made or entered into with the lowest responsible bidder meeting specifications (emphasis added)."32

State law sets
a cost-effective
standard for
high-tech purchases

But new statutes were enacted in 1982 when the State recognized the need to handle computer purchases differently. Beginning with Section 12100 of the Public Contract Code, the law says:

The Legislature finds that the unique aspects of electronic data processing systems and telecommunications systems and the importance of such systems to state programs warrant a separate acquisition authority for electronic data processing and telecommunications goods and services. The Legislature further finds that such separate authority should enable the timely acquisition of goods and services in order to meet the state's needs in the most cost-effective manner (emphasis added).33

The law also lays out the standard for selecting a winning bid for high-technology equipment, saying the award should go to:

The proposal which provides the most cost-effective solution to the state's requirements, as determined by the evaluation criteria contained in the solicitation document. These evaluation procedures may provide for the selection of a vendor on an objective basis other than cost alone (emphasis added). 34

While the law comes close to the "best-value" concept, allowing for a selection on criteria other than price, it still continues to demand objective decision-making. This means that elements such as management expertise and past track records must be quantified and compared -- an exercise that creates the pretense that subjective decisions will be made objectively.

Effect of law
is undermined
by bureaucracy
and mishandling

The Commission found that the original intent of the law -- to provide flexibility and better decision making -- is undermined in two ways. First, the use of the mechanism provided by the law is discouraged by provisions of the State Administrative Manual and by procurement officials at the Department of General Services; and second, when the mechanism is used, it is usually handled in such a way that there is little difference between it and a standard, low-price procurement.

Despite the "cost-effective" enabling language, most Request for Proposals stick to low-price selection. One procurement official for a state licensing agency who was handling the procurement of a computerized telecommunications system wanted to develop a point-based system that would allow the agency to determine which bid gave the most functional system for the least amount of money. He envisioned setting up a panel from the Office of Information Technology, the Office of Procurement, the Division of Telecommunications and his own agency to evaluate the bids. The Office of Procurement turned him down, he said, based on advice from the office's legal experts that a point system would be too vague and arbitrary.35

Another procurement expert from the California State University system said that despite state statutes, most procurements focus on what is least expensive rather than what is the most functional for the least expense. The least expensive alternative usually is also the least functional in terms of meeting the State's needs, he said. He advocates setting up a model point scoring method that could be frequently reviewed for amendment, well documented and accessible to everyone who participates in procurements.36

One or two people's experience, of course, is not definitive. However, the State Administrative Manual, which lays out the accepted procedures for complying with state procurement laws, clearly comes down in favor of low-price evaluations:

For purposes of awarding a contract pursuant to a Request for Proposals, there are two methods for evaluating proposals and awarding contracts. The preferred and primary method (emphasis added) requires that the contract be awarded to the lowest responsible bidder meeting the standards. The second method requires evaluation of proposals by a committee with the award being made to the bidder with the highest score. 37

It is not easy for departments to win the right to use a weighted evaluation mechanism. Writes an official for the Employment Development Department:

Unfortunately, the Department of General Services requires departments to justify why they need to weight cost along with quality factors even though no such requirement is apparent in statute. Thus, DGS becomes the arbiter as to when quality considerations exceed price alone. Certainly bid specifications need to be written clearly and fairly. But the State's business needs should not be sacrificed in order to ensure that low bids -- which may, in fact, result in greater total cost -- are always used.38

Cost continues to
dominate the process
even when other factors
are allowed

Those Requests for Proposals that do incorporate a multi-factor scoring method are almost always still heavily weighted in favor of low cost. For instance, an RFP may state that 100 points will be given during the evaluation for technical competency, past experience and innovation, while 200 points will be given for low price. By practice, the Office of Procurement says it allows at most a 50-50 scoring split between points for technical competence and points for low cost, but prefers a 40-60 or lower split so that price plays a bigger role.39 This may mean that a company that could offer an innovative, higher-initial-cost concept that might pay off for the State in the long run will not do so, instead sticking with a more traditional, competitively priced option.

Another criticism of the seldom-used multi-factor scoring method is that the points scored for non-cost factors are handed out almost equally to all bidders, in essence converting the evaluation to a low-bid process. For instance, a recent procurement for fiscal intermediary services for Medi-Cal gave 200 points for technical merit and 300 points for cost. Of the two bidders, one had a commendable record of service for the past five years as the current contract holder and the other -- who had previously held the contract -- had been the focus of much criticism for backlogged payments and poor service. The technical points given to each proposal were only five points apart, according to the successful bidder, who added, "The track records alone should have made the proposals more than five points apart."40 The result was that the contract went to the low-cost bidder (who, fortunately for the State, did have the better track record).

Why is the State loathe to rely on points for technical merit -- or when it does, eager to pass them out equally? Bidders speculate that state procurement officials are uncomfortable with subjective criteria that are difficult to "prove" and leave them open to protests and potential legislative criticism. No one can argue when one price is determined to be higher than another, but when one bidder gets more points than another for something that requires a value judgment or opinion -- such as whether they have a good record of past reliability -- then complaints can and do arise. The head of the Office of Procurement confirmed this viewpoint for the Commission. "Our concern is meeting all of the statutory requirements for true competitive bidding and also we want to be able to defend our decisions against protests," he said. 41

The partnership
concept has not
been popular
with the State

His comments also explain why the State has not rushed to embrace partnerships and long-term relationships with single suppliers, although the concept is not a new one in California and some teamwork has been tried for exploring the uses of advanced technology. Computer and telecommunications technology firms have long advocated such partnerships, including in testimony to the Little Hoover Commission.

For instance, one company believes that the Employment Development Department could save $4 million to $6 million annually if consolidated call centers were installed with a centralized database and computerized screening processes so that people who need to apply for unemployment benefits could be handled by telephone. The company would like to install a small system at its own expense for a field trial to prove the value to the State but could not make the investment knowing that it might be the loser in any eventual RFP process.42 Not only would the company lose money, but it would also receive no reward for the creativity and innovation it brought to the State.

But when allowing such partnerships was recommended by a council of state departments, the Office of Procurement was blunt:

We oppose this concept because we are unable to conceive of a fair, ethical manner by which to select such a partner. It appears to encourage vendors to buy into a procurement in order to benefit from future sole sources and deprive other vendors of an opportunity to bid on significant business. 43

The benefits of
partnerships:
expertise, creativity
and good service

While the Office of Procurement has defined well the pitfalls associated with relationships/ partnerships in procurement, the benefits could be substantial. First, it would provide governments with the in-depth, cutting edge expertise that they now lack when they try to make informed information technology decisions. Second, knowing in advance that creativity and innovation would be rewarded, companies would be more willing to propose solutions to problems in ways government has not been able to conceive. Today, such creativity results only in the government amending a Request for Proposals so all supliers can equally bid on whatever the innovation is. Third, companies would have an incentive to provide good service and treat government as a valued customer since future choices of vendors would be influenced by what kind of "partner" the company is.

What emerges from an examination of the State's attitudes toward best-value procurements and the potential of partnerships is a clear answer to the question of where the State focuses: on the process or on the results of procurements. The answer is that the process through which procurements are handled is of paramount concern. The deputy director in charge of the Office of Procurement testified to the Commission: "The Legislature has told us they want a fair and level playing field above all else. Until they give us some other direction, that's what we are focusing on. "44

As shown in the cases described in the background section of this report, the State's attitude toward procurement -- and the great emphasis on getting the process "right" -- often is an impediment to maximizing the State's resources, obtaining systems without delay and ensuring that the right technology choices are made.

Perhaps even more telling evidence that there is too great an emphasis on process and not enough on results is recent testimony given by the head of the Office of Information Technology. He named four procurements where the State tried to take advantage of new and emerging technologies, only to end up with expensive systems that are not capable of meeting the State's needs. These included a relational database for the Department of Motor Vehicles, a system intended to be a statewide repository of health records, a system to deal with corporate records, and the Department of Insurance's computerized consumer protection system. In each case, the State's procurement process failed to result in a system that fulfilled the expectations of the purchasing departments.45

In addition, the Commission was told by two independent sources about the purchase of a computerized telecommunications system in a state agency that has never worked properly because it was bought piecemeal in separate procurements that resulted in components that could not interact.

Neither design nor
implementation of
procurement process
results in best value

While the Commission has neither the expertise nor the resources to examine fully individual procurements, it is difficult to look at the pattern of troubled procurements and unusable systems without concluding that the State's procurement process is not designed to achieve the best value for the State. And best value is an important goal. As an industry leader described the effect of best value:

It doesn't matter if we're buying for the government, business or our personal life -- there is nothing more expensive than buying a product that doesn't work, regardless of the price we pay. Buying from the lowest bidder is tempting and easy. One can always rationalize that the difference in price between two levels of quality can be used to fix the lower quality if required. If not required, then we're ahead. Unfortunately, it's never that simple. In addition to repairs or reworks, there are costs associated with the impact of the poor quality. Furthermore, a consistent practice of buying on price regardless of performance tends to encourage low quality and poor performance. It takes time and effort to do it right the first time. Pure price competition discourages the extra effort, especially if the supplier knows that poor quality can be rewarded by additional funds to fix the problem.

Upgrading the State's procurement practice to include the best-value concept will not only improve the quality of the purchased goods and services, it will also upgrade the quality of the "core" supplier base. This base will be better equipped to compete for commercial orders both within and outside of California.46

Recommendation 1:

The Governor and the Legislature should enact legislation that declares that the primary goal in conducting state procurements is to obtain the best-suited product at the best price.

This would change the existing emphasis on "cost-effectiveness as determined by objective criteria" to a best-value system. The new law should allow state departments flexibility in determining whether what they are purchasing is a well-defined commodity that should be selected on the basis of cost alone or a non-commodity that should be judged on factors besides price. When dealing with a non-commodity, the purchasing entity should create a multi-disciplinary panel from several areas of state expertise to evaluate the costs and benefits of each bid. Once a selection is made, the purchasing entity should provide a written justification for the choice, including such factors as incremental costs versus added value. The law should explicitly provide for the use of subjective judgment by the panel in the pursuit of best-value purchasing decisions.

Hand-in-hand with the declaration of legislative support for best-value procurement and subjective evaluations would have to be an implicit commitment on the part of legislators to refrain from ordering hearings and/or audits at the first complaint from losing vendors. State procurement officials have made it clear that much of the defensive posture embedded in the current procurement system stems from their experiences with legislative reaction to charges of bias and unfairness. Instead, legislators should rely on the protest process and the court system to guard against corruption and malfeasance.

Recommendation 2:

The State Administrative Manual should be changed so that state agencies may use a non-commodity, best-value evaluation procedure at their own discretion.

The present system ties the hands of state department decision makers. Instead, the system should allow decision makers the discretion to make the best choices. While the new system would lack the rigid internal controls now exercised, several factors will continue to guard against waste and the unwise use of state funds:

• The written justification required in a best-value procurement will allow for public, media and competitor scrutiny of all decisions.

• The multi-disciplinary nature of the selection panel will ensure checks and balances on any predisposition toward a specific vendor.

• The added complexity of evaluating bids and justifying decisions in a best-value procurement will be a disincentive for department officials to use the system except when it would truly be beneficial.

• Managers will face the same budgetary constraints they always have: If more dollars are spent on one piece of equipment, then fewer dollars are available for something else. Therefore, managers will only spend added dollars on enhancements that add value to what the department receives.

Finding 2:

The procurement process, particularly when it pertains to electronic data processing and telecommunications systems, is needlessly complex, time-consuming and costly for the State and the suppliers.

The State has a multi-step process for obtaining bids in response to Requests for Proposals for electronic data processing and telecommunications systems. The system was designed to ensure that the State could modify its concept of what was needed as new information emerged and that all suppliers would have an equally clear understanding of the RFP. Instead, the process has resulted in paper-intensive, costly, multiple submissions by suppliers and lengthy, costly, multiple evaluations by state personnel, with little evidence that the best interests of either the State or the suppliers are served.

State's process
may require up to
six different
bid submittals

In any electronic data processing procurement, there are potentially two phases that involve up to six different steps. There is always a Final Phase, which includes two steps: a Draft Bid and a Final Bid. There also may be a Compliance Phase prior to the Final Phase. The Compliance Phase may have a Conceptual Proposal, a Detailed Technical Proposal and revisions of either or both.

The Compliance Phase allows the State, working together confidentially with each bidder, to assess and discuss the bidders' proposed methods of meeting the State's needs. "It is a radical departure from the rigid `either accept or reject' philosophy of traditional competitive bidding, yet it is highly competitive in nature," according to the State's outline of the procedures. "It provides the flexibility needed for the bidder to test his solution prior to formal submittal of his final bid, and it facilitates the correction of defects before they become fatal to the bid. " 47

In a Conceptual Proposal, a bidder may be as innovative as the RFP leaves room for, knowing that if he is on a track that is totally unacceptable to the State, he will be told well before the Final Bid is due. After the proposals are submitted, the State evaluates each, identifying non-responsive elements or areas where additional clarifications are needed. In a confidential meeting, these are discussed with the vendor, along with proposed support plans, implementation plans, validation plans, demonstration plans and proposed contracts.

The State's procedures manual explains:

The State will not make counter proposals to a bidder's proposed solution to the requirements. The State will only identify its concerns, ask for clarification, and express its reservations if a particular requirement is not, in the opinion of the State, appropriately satisfied. The primary purpose of this discussion is to ensure that the bidder's final bid will be responsive. 48

The outcome of the meeting is a Discussion Memorandum that notes all problem areas and agreed-upon plans for resolution or details of further steps to be taken.

The State at this point may notify the bidder that the proposal has been rejected if the State believes that there are defects that could not be remedied ever or in time for the final submission deadline. If the State believes more clarity is needed following the meeting and the issuance of a Discussion Memorandum, then it may ask for a revised Conceptual Proposal.

As a next step, the State may require a Detailed Technical Proposal, allowing each bidder to provide a detailed technical description of his or her plans. If there has been a Conceptual Proposal phase, then the Detailed Technical Proposal must match whatever was proposed in the first step. The submission is followed by the same procedures as the Conceptual Proposal: a state evaluation, a confidential meeting and a Discussion Memorandum, as well as a revised Detailed Technical Proposal if the State feels it is needed. Once again, the bidder may be told his proposal has been rejected if the State believes it is completely off track.

The Final Phase begins with the Draft Bid, an "almost final" bid that will be reviewed for faulty administrative aspects that might cause the Final Bid to be rejected if not corrected. The State's evaluation of the Draft Bid does not include a review of technical responsiveness, although the bidder is notified about any technical defects that are noted. Passing the Draft Bid stage does not guarantee that there are no material deviations or other defects in the bid, and the State reserves the right to reject the Final Bid as non-responsive. The Draft Bid must conform to the Conceptual Proposal and Detailed Technical Proposal (if those steps were part of the procurement).

The Final Bid is the only document that includes cost information. It must conform to the prior submissions in the process and it must have all the elements required in the RFP.

Multi-step process is
designed to assist
flow of information
but critics find flaws

In theory, then, the multi-step process is an opportunity to have an exchange of information flow between the State and potential bidders. Critics complain, however, that in practice the process:

• Is an inefficient method of designing information technology systems. Other states put out an RFP to design a system and then issue a second RFP for implementation.49 California gets its design work "free" through constantly tinkering with the RFP during the Compliance Phase -- but the "free" design is actually costly in terms of time and state staff efforts, as well as having the potential of being a conglomeration of compromises rather than someone's best effort at maximizing the opportunities of high technology.

• Allows bidders to pressure the State to make changes in specifications that may be beneficial to them or detrimental to other bidders. One vendor told the Commission his company spent thousands of dollars preparing preliminary submissions only to have a last-minute RFP amendment make it impossible for the company to submit a Final Bid.

• Requires bidders to jump through the same hoops repeatedly, but with no assurance that their bid will be deemed responsive in the end.

• Is getting longer and longer, and costlier and costlier. One witness told the Commission it is not unusual for a company to spend more than $1 million to put together the many responses required for a multi-million dollar project. The RFP itself may fill one or two 3-inch binders, while the responses may take up seven or eight of the binders -- for each separate step of the two phases.50 The timeline from the issuance of an RFP to the date for a Final Bid can run more than a year.

SUMMARY OF EVENTS IN CALNET PROCUREMENT

DATE	STATUS
June 1987	State issues Request for Information, an option that allows the State to seek supplier input on possible solutions to the State's defined problem.
September 1987	An RFP -- later amended 26 times -- is issued.
November 1987	Conceptual Proposals are submitted by five vendors.
April 1988	Two suppliers drop out and only three Detailed Technical Proposals are submitted.
September 1988	Suppliers submit revised Detailed Technical Proposals.
November 1988	State provides each bidder with a detailed review and evaluation. Draft bids are submitted.
February 1989	Final bids are submitted after each draft bid has been reviewed for compliance with specifications and discussions have been held with bidders.

Source: Department of General Services

Despite the multiple submissions, many evaluations and frequent meetings, the end result of this 17-month process was that two out of the three vendors were found to be non-responsive. The third (neither the high nor the low bidder) was given the contract.

In addition to the multi-step process, the State also requires all bidders to negotiate contract language before the contract is awarded. This means that if there are five bids submitted by five separate vendors, then state lawyers negotiate with five firms to arrive at final contract language even though only one contract will ever be issued.

Contract negotiations
with each vendor
before bid selection
adds to costs

Critics complain that this method, not used in other states, is a complete waste of money and staff time, both for the unsuccessful companies and the State. State officials, however, maintain that negotiations must be carried on with everyone before a bid award so that there is no question of advantage being gained by a firm after a bid is awarded through the acceptance of contract language different from what other bidders thought they could get.

The State's system of handling electronic data processing procurements is a well-intentioned effort to save costs, give the State flexibility and ensure vendor responsiveness. But in practice, the system adds costs, delays and complexity without providing the State an easily managed, easily understood procurement system.

Recommendation 3:

The Governor and the Legislature should direct the Department of General Services to streamline the procurement process to avoid multiple submissions.

The new process could include the following, simplified steps:

1. A draft RFP.

2. Vendor input on modifications for the RFP.

3. Issuance of a final RFP.

4. A single submittal of a technical proposal without cost data.

5. A formal clarification process (if needed). 6. Evaluation of the proposals and rejection of non-compliant bids.

6. Notification of compliant vendors to submit cost proposals.

7. Publication of the ranking of technical proposals (which may include the State's evaluation of how much added cost it is willing to bear in return for different levels of added value).

8. Opening of cost proposals.

9. Evaluation of cost proposals in relation to technical ranking.

10. Selection of vendor.

This system avoids multiple submissions, yet still allows for clarification of errors or misunderstandings. It removes cost considerations from the early evaluation of technical details, but still provides best-value flexibility by allowing the State to weigh technical factors against added costs before reaching a decision.

Recommendation 3:

The Governor and the Legislature should enact legislation that directs contract language negotiations to take place only after bids have been awarded.

The law would provide the State with the option of going to the next highest best-value bidder if it wishes to reject the original successful bidder's contract language proposals. Vendors have indicated that such a system allows them to avoid the cost of contract negotiations on bids they will never win, while giving the State an added "club" to win arguments over contact language. Vendors are more likely to be flexible if they fear the loss of a multi-million-dollar contract that they know they have been selected to receive.

Finding 3:

Specifications in state Requests for Proposals are sometimes poorly drafted, too restrictive and not conducive to the State receiving the best product to meet its needs.

The requirements laid out in a Request for Proposals control the end product that the State will eventually receive. Besides directly detailing what is being sought, the requirements -- known as specifications -- have many indirect affects: They define how large the pool of bidders will be, set parameters on how much creativity and technological know-how will be used in responses, and limit the criteria by which proposals will be evaluated. Badly formed specifications, therefore, have an overwhelming impact on a procurement. Yet the State makes little effort to ensure that specifications are written to reflect accurately the State's needs and to give the State flexibility to entertain the widest range of creative proposals. The result often is overly restrictive specifications that may be perceived as biased toward one supplier's products.

A good definition
of what the State
needs is the key to
a good procurement

The most important step in any procurement is defining what the purchasing agency is looking for. Academics write about this, suppliers intuitively know it and state officials recognize it -- and to some limited extent, even state policies and procedures reflect it.

Writing about the problems of low-cost purchasing and the focus on procurement processes rather than outcomes, procurement expert Steven Kelman says:

The current system encourages the government to try to describe in advance too many of the features and applications of the system that is to be developed, rather than realizing that it is foolish to believe one can understand all the potentials and pitfalls of a brand new system in advance of its implementation ...Grand designs are at best a recipe for not getting all one can from a new computerization project and at worst an invitation to costly disaster.51

Kelman adds that the driver behind overly detailed specifications is the focus on a level playing field for all bidders, which requires purchasing agencies to as much as possible reduce evaluations of bids to objective, easily compared facts, such as price, rather than allowing evaluators to make subjective comparisons.

The philosophy of reining in discretion establishes an elaborate process of relating specifications and evaluation criteria to government requirements and encourages a structured effort to set down requirements in advance, often with a view that more detail is better. 52

Kelman points out that a side-effect of overly detailed specifications is that the process of putting together RFPs is so burdensome that many agencies try to wrap together all of their information technology needs into one gigantic procurement. This also works against getting the best end product, as noted by the John F. Kennedy School of Government at Harvard University in their study:

Instead of the long-term informal relationships favored by corporations, governments tend toward long-term formal contracts, often called "mega-contracts" or "grand designs." Specifications and contracts sometimes run forward for 10 or more years.

Almost always this is wrong. It results in Rube Goldberg approaches of bewildering complexity and botched performance. For most major systems, "Do it, try it, fix it!" is a better approach.

The grand designs have not risen by accident, however. They are partly a response to burdens imposed by the contracting process. There is strong temptation to spread the high fixed costs of procurement justification and audit over as large a project as possible.53

Specifications need
to focus on State's
problem rather than
defining a solution

Academics favor two solutions to the problems caused by detailed specifications. The first is to write functional specifications rather than detailed technical specifications. This means that specifications should describe what needs to be accomplished in general rather than how it will be accomplished specifically. The second -- which is actually an echo of the first -- is to ensure that RFP itself focuses on the problem that the purchasing agency is trying to address rather than on some favored solution.

What does it mean to focus on a problem rather than specifying a solution? One example might be a licensing agency that wants to be able to handle more public inquiries about the status of license holders. The agency might presume that what it needs is a telephone system that will allow callers to be sorted according to the questions they have and then directed to an operator that can handle that question. If it constructs an RFP around that premise, it may never learn about a more sophisticated telephone system that allows callers to tie directly into a centralized databank and access the information needed without the use of more operators.

Vendors say
specifications tie
their hands and
are drafted poorly

The real-world affect of detailed specifications and "grand designs "was brought up repeatedly by vendors who communicated with the Commission. Their comments included:

• The current process places the burden of determining the best solutions on the State -- yet the State does not have the expertise to know all the options or to determine the best combinations of technology.

• Detailed specifications encourage protests of bid awards because the State must use subjective judgment to decide whether deviations are material, and therefore a bid will be thrown out, or non-material, and therefore a bid is acceptable.

• Vendors are discouraged from being creative. Companies that believe they have a better approach to solving the State's problems than that called for in the RFP may find that their solution does not fall within the parameters of the specifications. They therefore do not even get a foot in the door to try to sell their approach.

• Specifications may subtlely favor one firm or exclude others by requiring equipment or performance standards that can only be met by some companies.

• State workers rely on advice and input from vendors because they lack the technical expertise to write detailed specifications. This allows vendors to try to influence specifications so that they will favor their firms or exclude competitors.

• The evaluation process is not always conducted with adequate expertise. Vendors hire the best technical people possible to put together their proposals, but the proposals usually are evaluated by people with little in-depth technical knowledge. The subtleties of different bids and nuances of how systems will perform are often evident to competitors but are missed by those who are supposed to be weighing the relative value of different proposals.

Detailed specifications
block State from
pursuing other options
as new ideas emerge

The practical result of a system that tries to equalize all factors in bids before they are received is that the State is not allowed to have second thoughts about what it wants. Nor is it allowed to change direction if it receives new information after the RFP has been finalized. Nor can it suddenly decide to add a gadget it overlooked if it finds out that the added cost is slight but the added value is great. All of these options are open to individual consumers and to private businesses.

For instance, if a person is shopping for a videotape player, he or she may begin with a list of functions that are desirable. This list may lengthen or shorten as salesmen are consulted about price differentials, added options and expected quality. Then a purchasing decision is made based on a balance between what is needed, what is desired and what can be afforded. A rigid procurement system, instead, requires the State to figure out in advance exactly what it thinks will meet its needs, to estimate about how much it will cost so it can determine what features it can afford to seek, and then to write specifications that will force bidders to give the State what it has determined that it wants.

Although the State has conducted dozens of procurements with highly technical specifications that could be used as examples, perhaps the one that most easily shows how micro-management-minded procurements can become is the State's RFP for the State Computer Store. This procurement was for a master contract that allows departments to purchase personal computers and other equipment from a single supplier without going through separate bid processes. The RFP, which filled a 3-inch binder, contained one section of 75 separate requirements that bidders must commit to meet, including:

• Maintaining a showroom space of 1,500 square feet.

• Scheduling at least three sales representatives to work in the store each work day.

• Providing at least one hour free parking for State employees using the store.

• Training the telephone receptionist to answer product pricing questions.

This detailed RFP with its solution-oriented approach guaranteed that the State would get precisely what it asked for, including free parking for shoppers. But a problem-oriented RFP -- one that explained the State's goals of providing a convenient place for state departments to get top-notch service, technical advice and reliable products -- would have allowed bidders to take their best shot at convincing the State that their plan, with their own creatively designed details, would provide the best value in terms of costs and services. The State might have ended up with options that had never occurred to it, such as "house calls" for outlying state departments or free debugging programs.

Despite the evidence of overly detailed procurements, the State is not unaware of the academic theories, the vendor concerns and the real-world results discussed above. In response, the State has -- to some extent -- set up a system to focus on meeting the State's needs rather than on solutions.

Feasibility Study
Report forces
departments to
focus on needs

The RFP and its specifications are not the first steps in the chain of events that allows a state department to purchase new information technology. When a department decides that a new computerized system will streamline its workload and increase efficiency, the first external hurdle it must pass is the Feasibility Study Report (FSR). This document, completed with oversight by the Office of Information Technology, is designed to take a business-like approach to governmental functions. The FSR, when properly done, lays out the department's "business problem" and explores alternative solutions. It addresses issues such as: What are the mission and goals of the department? How does the need it is seeking to meet inter-relate with other functions? What are the short-term and long-term implications of the proposed solution?

By law, the Office of Information Technology, a unit of the Department of Finance, is in charge of policy and coordination for information technology throughout state government. (In contrast, the Office of Procurement, a unit in the Department of General Services is in charge of overseeing and/or carrying out information technology procurements. A separate General Services' Unit, the Division of Telecommunications, is in charge of policy and procurement for telecommunications systems.) The Office of Information Technology reviews Feasibility Study Reports with the perspective of ensuring that needs are well-defined and options are reasonable and thoroughly assessed. The Feasibility Study Report is not intended as a system design document but, since it is used in determining budget appropriations for new information technology projects, it does propose an intended solution.

Once the Feasibility Study Report has been approved and funding budgeted, the department can move ahead with the procurement process. The RFP and the specifications are developed by the department that is making the purchase, rather than by the Office of Procurement in its role of procurement oversight or by the Office of Information Technology in its role of information technology coordination. The supposed advantage of this system is that the department "knows what it wants" and so is in the best position to formulate the RFP and specifications. The disadvantages include:

• Since most departments make large information technology purchases only rarely, few departments have the in-house technical expertise necessary to draft an RFP and specifications that will result in the best possible procurement process. Even for small computer acquisitions, technical assistance is largely unavailable for those who must put together the RFPs. Small departments that call either the Office of Information Technology or the Office of Procurement are told that they do not provide assistance on drafting specifications.

• While the RFP that is developed is supposed to be based on the Feasibility Study Report, no such linkage is ever assured. The State's top technical experts sign off on the Feasibility Study Reports, but none of them are involved in fashioning specifications or double-checking