JENNIFER S. SHANE, S.ASCE; DOUGLAS D. GRANSBERG, M.ASCE; KEITH R. MOLENAAR, A.M.ASCE;

AND

JOSEPH R. GLADKE, P.E. ABSTRACT: The U.S. highway industry has witnessed increased use of design/build project de-

livery and best-value procurement in the last decade. The industry is experiencing the growing pains of venturing into territory that was previously left to other sectors of design and construction. Inexperience with the system and the processes behind these systems by all parties, state highway agencies, designers, and contractors has occasionally led to protest filed with the court system. The precedents set by the decisions of the court will have a lasting effect on the industry as the exploration of delivery and procurement systems continue. One state highway agency that utilized a two-step best-value procurement process received a protest from one of the firms that was not successful in passing the qualifications phase of the procurement. The court found evidence to uphold the decision of the state highway agency through the transparent and documented evaluation process established by the state highway agency. Continued exploration of new and different technologies and ideas for procurement and delivery of highway projects will result in legal challenges to the system as all parties involved work to gain an understanding of the processes. This paper is an examination of one state highway agency’s experience with an alternative to the highway industry’s typical low-bid procurement system.

D

esign/bid/build is the traditional method of project delivery for highway projects in the United States. This system is composed of three distinct phases with the project plans being 100 percent designed prior to advertising the project for bids. The low-bid method of procurement has become the norm for design/bid/build projects. The highway method of procurement has become the norm JANUARY 2006

..........................................................

Downloaded from ascelibrary.org by 37.44.207.175 on 01/18/17. Copyright ASCE. For personal use only; all rights reserved.

Legal Challenge to a Best-Value Procurement System

for design/bid/build projects. The highway industry is now exploring alternative project delivery methodologies including design/build. The design/build delivery method allows the owner to award the project with little design complete to an entity that is responsible for completing the design and carrying out construction of the project. Design/build projects are often procured either through low bid or through a bestvaluestructure.Best-valueprocurementconsidersnotonlythe cost proposal but qualifications and technical criteria as well.

20 䊏

Leadership and Management in Engineering

Leadership Manage. Eng., 2006, 6(1): 20-25

Downloaded from ascelibrary.org by 37.44.207.175 on 01/18/17. Copyright ASCE. For personal use only; all rights reserved.

Evaluation criteria Legalandfinancialqualifications Organizationandexperience Keypersonnel Projectunderstanding Projectapproach Projectmanagement Total

Possible points Pass/fail 15 30 10 25 20 100

The type of qualifications and technical information requested is dependent on the owner’s needs and how these attributes will be included in the selection process 共Beard et al. 2001; Gransberg and Senadheera 1999; Molenaar and Gransberg 2001;Molenaaretal.1999;Tenah2001兲. The Minnesota Department of Transportation 共Mn/DOT兲 chose to use the design/build project delivery system in conjunction with a two-step best-value procurement method on a project that required the addition of a lane in each direction along eight miles of a well-traveled arterial highway. Mn/DOT completed 30 percent of the design and required the wining proposer to complete the remainder of the design and construct the project. Mn/DOT estimated the project to cost$110million.Thefirststepofthebest-valueprocurement method, established by Mn/DOT, required Mn/DOT to release a request for qualifications 共RFQ兲. Interested parties were required to submit a statement of qualifications 共SOQ兲. The submittals were evaluated and based on this evaluation all oralimitednumber共threetofive兲oftherespondentsweresent a request for proposals 共RFP兲 in the second step. In response to the RFP the proposers were then required to submit technical and price proposals for evaluation. The evaluations from the firststepwerenottobecarriedovertotheevaluationprocessin thesecondstep. TheRFQincludedinformationregarding: 1. Minimum qualifications and experience of design/ builders; 2. Scope of work and schedule; 3. Documents defining the project requirements; 4. Form of contract; 5. Weighted selection criteria for the SOQ; 6. Description of the RFP; 7. Maximum time allowed for design and construction; and 8. Cost estimate. The evaluation criteria for the SOQ totaled to one hundred possible points, as detailed in Table I. Leadership and Management in Engineering

....................................................................................................................................................................................................................................................................................................................................

Table 1. Evaluation Criteria Established for Mn/DOT SOQ

The RFQ stated: “The selection team shall evaluate the design/build qualifications of responding firms and shall compileashortlistofnomorethanfivemosthighlyqualifiedfirms in accordance with qualifications criteria described in the request for qualifications 共RFQ兲.” Each evaluation criteria included the main points of assessment. Under the organization and experience category listed in Table I, consideration pointsincluded: • Effective project management authority and structure • Design and construction management structure • Effective utilization of personnel • Owner/client references • Experience on projects of similar scope and complexity • Experience with timely completion of comparable projects • Experience with on-budget completion of comparable projects • Experience with integrating design and construction activities • Experience of design/build team members working together This section specifically directed the respondents to “describe at least one but a maximum of four design/build projects that the Submitter and each Major participant has managed, designed and/or constructed.” The RFQ also stated that respondents should “Highlight experience relevant to the Project that the Submitter/Major Participants have gained in the last ten years. Cite projects with levels of scope comparable to that anticipated for the Project.” Under the “key personnel” category the RFQ requested the SOQ include the name and resume of eighteen required positions. The evaluation criteria for this section included team members with experience and qualifications that cover project scope and key management/staff experience, capabilities, and functions on similar projects. After release of the RFP, Mn/DOT released an addendum that replaced the description statement presented earlier with the following: “Describe at least one but a maximum of four design/build or major highway projects that the Submitter and each Major Participant has managed, designed, and/or constructed.” This second statement request thus eliminated theneedforanydesign/buildexperience. Five SOQs were received by Mn/DOT. Upon receipt Mn/DOT convened the evaluation committee, composed of six members. Each member was trained on how to score the SOQs.Anevaluationplanhadpreviouslybeencomposedwith the stated purpose of providing the methodology and procedures for the evaluation of the SOQs. The plan called for each SOQ to be evaluated by each committee member. Each committee member evaluated each section of the SOQ based on the degree of satisfaction of the requirements set forth in the RFQ. Each committee member was also asked to identify potentialstrengthsandweaknessoftheSOQ.Eachcommittee member was asked to assign an adjectival rating of excellent,

21 䊏

JANUARY 2006

Leadership Manage. Eng., 2006, 6(1): 20-25

Submitter

Downloaded from ascelibrary.org by 37.44.207.175 on 01/18/17. Copyright ASCE. For personal use only; all rights reserved.

A B C D E

Average score 85.7 83.0 82.7 71.9 69.4

very good, good, fair, or poor to each category of each SOQ. The evaluation plan included how each member was to assign the adjectival rating based on the number and level of strengths and weakness of the submittal. Based on the adjectival rating a percent score was determined; this percent score was then multiplied by the maximum score for that category to determine the final score for that category. Each category was then combined for the committee members’ total score for each SOQ. The averages of the combined scores for each SOQ werethendetermined,asshowninTableII. The evaluation team determined that the top three SOQ respondents would receive an RFP due to the large break between the scores 共over ten points兲 between the top three and the bottom two submitters. Submitter D, with the forthhighest score, submitted a protest to the short list outcome. Theprotestwasfiledforthefollowingstatedreasons: 1. Mn/DOT violated the state design/build statue by requiring that the evaluation would rely on and emphasize previous design/build experience, which would restrict competition. 2. Mn/DOT engaged in unpublicized rule making. 3. Judgment regarding who was short-listed was arbitrary and capricious fact-finding, and the conclusions were not substantiated by the evidence. The courts found that the plaintiff, Submitter D, did not prove that Mn/DOT created a restricted competition situation by requiring design/build experience. The RFQ did state that design/build experience would be considered, but not required. Mn/DOT also never directly or indirectly instructed the evaluation committee to weight design/build experience differently than similar project experience. The court did note that individual evaluators might have come to this conclusion basedontheirownjudgment. Given the limited experience with design/build by Mn/DOT and the unique features of the project the court found that establishing rules for evaluation on a case-by-case basisisreasonable.Also,therulesestablishedonthisprojectdo not formally establish policies for any future Mn/DOT design/ buildprojects. Finally, the court system found that Mn/DOT did not act arbitrarily and upheld the Mn/DOT short list decision. The JANUARY 2006

....................................................................................................................................................................................................................................................................................................................................

Table 2. Results of SOQ Evaluation

evaluation process was documented with expressed standards and while the process does not include the safeguards of lowbidcompetitivebidding,thecourtcouldnotconcludethatthe process did not include reasoned decision making. The court could also find no reason to challenge the members of the evaluation committee as not being qualified and experienced enoughtomaketheshortlistdecision. The court concluded that no one submitter was favored over another, and while the evaluation technique may not be perfect it is reasonable. The court ruled in favor of Mn/DOT and left the short list decision intact with no restitution for the protestingsubmitter. COMMENTARY I Douglas D. Gransberg, M.ASCE This case sets a very important precedent for not only design/ build projects but also all projects procured on a best-value basis. No matter how careful an owner is in articulating the details of their best-value award evaluation system, there will always be the possibility that one or more of the potential proposers will be dissatisfied with the outcome and file a protest. The courts in this case upheld the right of the owner to determine what makes one offer a better value than another offer. I believe that this case was won for three major reasons:

1. The SOQ evaluation criteria were transparent to all offerors. 2. The owner followed its evaluation plan as published. 3. The owner could defend its decision logically. Transparency in evaluation plans is a key element of bestvalue success. Parvin, a lawyer who defends construction contractors, wrote an insightful article regarding design/build evaluation planning for highway construction projects 共Parvin 2000兲. The article discussed the legal view of the need for fair and open evaluation processes in the transportation industry. In it, he highlighted the following two points that are of interest in this case: 1. “Clearly state the evaluation criteria and weight given for each item and ensure that the evaluation team uses them.” 2. “Leave no doubt about the honesty and integrity of the public agency’s evaluation team.” He argued that without a transparent evaluation plan, the owner would find itself constantly defending award protests. In this case, Mn/DOT clearly articulated the definition of success. It helped its case by stating that it would narrow down the field to a short list of the “most highly qualified firms in accordance with qualifications criteria described in the request for qualifications 共RFQ兲.” The requirements for making the short list were clear, and each potential competi-

22 䊏

Leadership and Management in Engineering

Leadership Manage. Eng., 2006, 6(1): 20-25

Leadership and Management in Engineering

Downloaded from ascelibrary.org by 37.44.207.175 on 01/18/17. Copyright ASCE. For personal use only; all rights reserved.

....................................................................................................................................................................................................................................................................................................................................

tor could compare itself to the competition and make an informed decision as to its ultimate competitiveness in the known field of players. This transparency serves to reduce the element of subjectivity that is inherent to best-value award evaluation systems by spelling it out rather than hiding it. Secondly, as the requirements for making the short-list were clear, it was easy for the courts to find that Mn/DOT had followed its own evaluation plan. This speaks to the second Parvin quote cited above. Once an owner publishes its evaluation plan, it loses all flexibility in applying it to the proposals that are received. If it wants to be able to defend against a protest, it must follow its own rules. If it does, the second Parvin quote regarding leaving no doubt about the honesty and integrity of the evaluation team is satisfied. Finally, the use of the terms “most highly qualified” gave Mn/DOT great latitude about determining the final size of the short list. The resultant scores showed that three competitors fell within a range of three points of each other, while Submitter D was ten points below the third-highest score. This created a solid argument that the three competitors on the short list were indeed the most highly qualified. In fact, Mn/DOT was unintentionally doing Submitter D a favor by not being conservative in forming the short list with four firms. The cost of preparing a technical and price proposal for the $110 million project would probably be in the range of $300,000 to $500,000. If Mn/DOT had arbitrarily set a minimum point score of seventy to be considered “qualified” 共i.e., minimally qualified兲, then three instead of two firms would have had to invest that significant amount of money in a losing effort. Thus, the logic of short-listing the only “most highly qualified” instead of all the minimally qualified is compelling in an economic sense and in the long run is much more fair to the design/build industry. Thus, several lessons can be learned from this case. First, the owner must publish transparent evaluation criteria and its best-value award algorithm. This puts all the competitors on an even footing and makes the defense against a possible protest stronger. Secondly, once published, the owner must follow its evaluation plan to the letter, collecting documentation along the way to prove that the decisions made for the project flow directly from the published evaluation plan and its attendant criteria. Finally, the plan must be logical and the decisions that flow out of it must also be based on defensible logic. In application, the use of the term “most highly qualified” probably is advisable in any RFQ evaluation plan as it gives the owner the greatest amount of latitude to develop a short list while being fair to those firms that could not possibly win the project on a basis of qualifications alone.

COMMENTARY II Keith R. Molenaar, A.M.ASCE

This case illustrates why two-phase best-value procurement is necessary for design/build delivery and also illustrates the challenges that are inherent in the system. Since design/build highway procurements often require substantial design effort, the proposals are costly to prepare. Public sector agencies must keep industry proposal costs to a minimum if they wish to make design/build a sustainable alternative delivery method in the transportation sector. Two-phase procurement provides a simple and low-cost method to minimize industry proposal costs by creating a shortlist of only those proposers who have the highest potential for success. In issuing an RFQ and short listing to only three proposers, Mn/DOT minimized proposal costs to the industry. Two-phase design/ build procurement has been proven successful by many other government agencies and is endorsed by the Federal Acquisitions Regulations 共FAR 2005; Molenaar et al. 1999兲. A primary challenge in using two-phase best-value procurement is that the use of subjective evaluation criteria is unavoidable. The use of subjective evaluation criteria is commonplace in consultant selection, but is foreign to most highway contractors. The subjective criteria that Mn/DOT chose to use 共i.e., design and construction management structure, effective utilization of personnel, experience on projects of similar size and complexity, etc.兲 are very similar to the criteria that engineers encounter when preparing project proposals. Since design/build projects contain substantial design work, these criteria must also be applied to design/builder selection in addition to the actual cost of the project. The judge commented on the subjective nature of the management evaluation criteria in his ruling on the arbitrary and capricious allegation. He stated that the members of the technical review committee 共TRC兲 were qualified and any errors in scoring “would not have changed the outcome.” The judge found that “it was ultimately the responsibility of the submitters to provide the necessary information and to convince the committee that it was the most qualified Submitter for the 关design/build兴 Project,” and while the TRC “drew conclusions that, while not perfect, were reasonable, … their decision does not reflect an error of law, arbitrary and capricious fact-finding, or conclusions unsubstantiated by the evidence, … 关and they did兴 not exceed 关their兴 statutory authority or abuse 关their兴 discretion.” However, the judge did state that the evaluation process could be improved. “It would be impossible to use people in the process and filter out subjective evaluations … the fact that the process could be improved does not make the process used in this case arbitrary or capricious.”

23 䊏

JANUARY 2006

Leadership Manage. Eng., 2006, 6(1): 20-25

Downloaded from ascelibrary.org by 37.44.207.175 on 01/18/17. Copyright ASCE. For personal use only; all rights reserved.

COMMENTARY III Joseph R. Gladke, P.E. This case demonstrates the importance that owners must place on having a procurement process that is well documented, in compliance with state statutes, and contains clearly stated evaluation criteria. Design/build projects are not typical construction projects. There are specific reasons why projects are chosen as design/build projects, and specific goals established for each project. These project goals, along with weighted evaluation criteria, are stated in the RFQ to help potential design/build teams understand what the department is trying to achieve with the project. Through this understanding, contractors and designers form teams comprised of individuals with the experience to meet the goals of the project. The teams submit SOQs detailing their qualifications, organization, and project approach. Following the evaluation criteria, the department must determine which teams, typically three to five, are most likely to meet the project goals and deliver a successful project. It is in the best interest of the public to have a well-qualified design/build team to help insure the project goals are met. Through the short-listing process the most highly qualified teams are selected to prepare project price and technical proposals. Using a traditional design/bid/build process the experience and past performance of the contractor is not considered. The only requirement that must be met to bid on the project is havJANUARY 2006

....................................................................................................................................................................................................................................................................................................................................

If highway agencies wish to use design/build project delivery, there will need to be some level of subjectivity in the procurement process. Design/build contracts are contracts for both services and real assets. The assessment of services to be performed is by nature a subjective evaluation. It is up to the highway agency to be sure that the evaluation is reasonable and directly relates to the published evaluation criteria. Owners must use experienced design and construction professionals to evaluate proposals and then provide a documented evaluation plan for them to follow. The judgment of these professionals is an acceptable method in which to choose a design/build team given that they follow the evaluation plan and document the reasoning behind their judgment. Finally, if the owner is going to evaluate a specific factor, such as past design/build experience, this should be stated explicitly in the published evaluation criteria. Although the judge did find that “experience on projects of similar scope and complexity” did equate to design/build experience, the owner and the industry would have been better served if the RFP directly stated design/build experience in the description of the evaluation criteria. Owners are best served by making evaluation criteria as transparent as possible at the time of evaluation and not deviating from these criteria during evaluation.

ing the necessary bonding capacity, and thus the likelihood of the contractor delivering a successful project that meets the goals is unknown. One of the claims in the lawsuit was that Mn/DOT violated the state design/build statue by requiring that the evaluation would rely on and emphasize previous design/ build experience, which would restrict competition. Mn/DOT did not violate the design/build statute. Mn/DOT followed the statute that outlines the requirements of the RFQ, including “requirements for construction experience, design experience, financial, personnel, and equipment resources available from potential design-builders for the project and experience in other design-build transportation projects or similar projects, provided that these requirements may not unduly restrict competition” 共Minnesota Revisor of Statutes 2001,2004兲. Short-listing three to five teams does not unduly restrict competition, and it increases the likelihood of having a design/build team that can deliver a successful project that meets the project goals. Mn/DOT has no interest in restricting competition. Mn/DOT has an interest in having more highly qualified teams. In an effort to obtain more highly qualified teams, Mn/DOT offers debriefing meetings with unsuccessful teams to help them understand what improvements they could have made in their SOQs or technical proposals to obtain a higher score. Mn/DOT also makes SOQs, technical proposals, and price proposals from past projects available in its library for review. Finally, design/build procurement is not fully understood or accepted by some contractors. Some of them have been very successful in obtaining projects under the design/bid/ build procurement approach, yet they may need to modify their business processes, procedures, personnel, and bidding approach if they are to be successful on design/build projects. Owners are well served by helping new design/build teams understand improvements that they can make on past SOQs and technical proposals in an effort to have more teams that better understand the design/build procurement process and can improve on their scores for future projects. Design/build is not about limiting competition; rather, it is about having well-qualified teams that can meet project goals and ensure successful projects for a reasonable price 共not necessarily the lowest initial bid兲. REFERENCES Beard, J. L., Loulakis, M. C., and Wundram, E. C. 共2001兲. Design build: Planning through development, McGraw-Hill, New York. Federal Acquisitions Regulations 共FAR兲. 共2005兲. “Subpart 36.3—Two-phase design-build selection procedures.” Federal Acquisitions Regulations, 具http:// www.acqnet.gov/far/典 共July 5, 2005兲.

24 䊏

Leadership and Management in Engineering

Leadership Manage. Eng., 2006, 6(1): 20-25

Leadership and Management in Engineering

Downloaded from ascelibrary.org by 37.44.207.175 on 01/18/17. Copyright ASCE. For personal use only; all rights reserved.

............................................................................................................

Gransberg, D. D., and Senadheera, S. P. 共1999兲. “Designbuild contract award methods for transportation projects.” J. Transp. Eng., 125共6兲, 565–567. Minnesota Revisor of Statutes 共MRS兲. 共2001兲. Minnesota session laws 2001, 1st special session, chapter 8, article 3, section 5, MRS, Saint Paul, Minn. Minnesota Revisor of Statutes 共MRS兲. 共2004兲. Minnesota Statutes 2004, section 161.3420, subdivision 3, MRS, Saint Paul, Minn. Molenaar, K. R., and Gransberg, D. D. 共2001兲. “Designbuilder selection for small highway projects.” Eng. Manage. J., 17共4兲, 214–223. Molenaar, K. R., Songer, A. D., and Barash, M. 共1999兲. “Public sector design/build evaluation and performance.” Eng. Manage. J., 15共2兲, 54–62. Parvin, C. 共2000兲. “Design-build: Evaluation and award,” Roads and Bridges, 38共12兲, 12. Tenah, K. A. 共2001兲. “Project delivery systems for construc-

tion: An overview.” Cost Engineering, AACE International, 43共1兲, 30–36. Jennifer Shane is a PhD candidate in the Department of Civil, Environmental, and Architectural Engineering at the University of Colorado, Boulder, and can be reached via e-mail at [email protected]. Douglas D. Gransberg is associate professor in the Construction Science Division at the University of Oklahoma. Keith R. Molenaar is assistant professor in the Department of Civil, Environmental, and Architectural Engineering at the University of Colorado, Boulder, and can be reached via e-mail at [email protected]. Joseph R. Gladke is design/build program director at the Minnesota Department of Transportation, and can be reached via LME e-mail at [email protected].

25 䊏

JANUARY 2006

Leadership Manage. Eng., 2006, 6(1): 20-25