Tender evaluation through efficiency analysis for public construction contracts

Abstract

Given the European Public Procurement Directive 2014/24/EU, policymakers have ordered the inclusion of various criteria, such as the price, life-cycle costs, environmental, and social aspects, in the evaluation of tenders for public construction projects. Consequently, the relevance of non-monetary award criteria has gained significant value. However, the established evaluation formulas, which are used to obtain the best value for money procurement, have resulted in legal disputes. The existing evaluation formulas exhibit mathematical weaknesses, wherein scoring indices do not express economic efficiency adequately. Thus, a conflict is observed between the political requirement of non-monetary award criteria and their evaluation by contracting authorities. To overcome such dilemma, an extensive literature review is conducted. Specifically, this study explores the essential problems of existing evaluation formulas and develops a more reliable method. The technique from the field of efficiency analysis, i.e., Data Envelopment Analysis (DEA), is adopted in this study. For contract awarding, the DEA is extended by introducing a decision theoretical framework. For public procurement, the proposed method combines two advantages. First, the proposed method ensures the derivation of a robust tender ranking given that with respect to clients’ preferences, irrelevant and insufficiently tailored tenders do not influence the scoring. Second, the proposed method supports the intention of policy makers to promote public goals, such as sustainable aspects. By disclosing the strengths and weaknesses of bidders with respect to their competitors, all bidders can obtain a precise overview of their performance regarding the award criteria. In sum, the proposed method allows a targeted improvement of certain criteria values in future tenders and consequently leads to an enhancement of public goals.

Graphical abstract

Keywords

bid evaluation / public procurement / best value for money / data envelopment analysis / contractor selection

1 Introduction

Public construction projects put forward by the members of the European Union (EU) that exceed an investment of more than €5.350 million (e.g., traffic, water, and wastewater infrastructures, governmental offices, educational facilities, or hospitals) are procured at a supranational level. This process enables the contracting authorities (i.e., client) to evaluate all bidders regarding the project’s qualitative, environmental, social, and innovative aspects. These issues are summed up as “sustainable development goals” (SDGs) in the World Economic Forum (2017), which has occurred on 17 January 2017 in Davos (Switzerland). The potential effects of these aspects, e.g., the energy consumption of construction machinery or the volume of waste due to work processes, are given wider focus in public procurement. Considering the relevance of these projects, the client asks for non-monetary award criteria in addition to the price. Therefore, the European Procurement Council Directive 2014/24/EU strengthens the existing criteria and the life-cycle costs (LCC). For contract awarding, multidimensional criteria have to be aggregated to a scalar scoring index. This index is intended to express the best value for money to identify the most appropriate contractor for the construction project. In current public procurement procedures, evaluation formulas are used. However, the existing formulas have mathematical weaknesses, which can result in legal complaints by unsuccessful bidders. Failing to start a project because of procurement-related deficiencies is an obstacle hindering the request of multidimensional award criteria. Thus, a conflict is observed between legally correct tender evaluation on behalf of the client and the consideration of SDGs on behalf of policymakers. To avoid this conflict, contracts for public construction projects are still awarded in favor of the tender with the lowest price (Ballesteros-Pérez et al., 2016; Tran et al., 2016; Eke et al., 2017). Thus, the chance of promoting SDGs is often overlooked (European Commission, 2012; Bratt et al., 2013).

This study develops a new tender evaluation method following a four-stage research method: Introduction → Methodology → Results → Discussion. After providing a brief overview on the status quo of tender evaluation, the boundary conditions for the proposed method will be formulated. Subsequently, the method development will be presented, starting with the theoretical principles up to the extension of the Data Envelopment Analysis (DEA). Prior to drawing conclusions, a scenario analysis will be conducted to test the reliability and validity of the results.

2 Tender evaluation procedure for public construction contracts

2.1 Contract award criteria and its weighting

An increasing number of European member states are obligated to request SDG criteria in addition to the price or to the LCC. For example, the introduction of the Council Directive 2014/24/EU (European Parliament and Council of the EU, 2014) into Austrian law requires at least one supplementary criterion in addition to the price. The UK government also requests LCC and quality criteria. The policy paper (UK Cabinet Office, 2012) describes this awarding mechanism as “whole life value for money”. However, in SDG award criteria, practical recommendations emphasizing comprehensibility and controllability in the construction stage are rare. Therefore, policymakers from different countries, such as Austria (Faire Vergaben, 2016) and Sweden (National Agency for Public Procurement, 2019), have published catalogs with various criteria for clients. For example, execution time, which is assessable as a competitive factor, can be used as a major criterion for construction work on main roads. For urban construction projects, noise emission can also be used as an evaluation criterion. For specific structures, such as traffic tunnels, particular collections have been published (International Tunneling Association, 2014). All these documents mainly contain ordinal criteria, which are scored by an evaluation committee of the client. In addition, the client is granted a juridical scope in which the comprehensibility of the evaluation ultimately guarantees legal certainty, as emphasized by the European Court of Justice (2013). Cardinally determinable criteria, such as LCC, overcome this fact. Life-cycle costs can be used to quantify criteria such as “quality” with follow-up costs and “sustainability” with external costs. According to Vogt and Thewes (2012) and Vogt (2013), the calculation of follow-up costs is associated with technical and financial uncertainties. Thus, the client has to provide information on the reliability of the technical components, the anticipated level of safety, the interest rates, and other related factors.

After establishing the award criteria, a percentual weighting expressing the preferences of the client has to be specified (European Parliament and Council of the EU, 2014). However, Alhola (2012) has noted that non-monetary award criteria rarely influence the award decision. Generally, the price receives a significant weighting that marginalizes the importance of other factors. Moreover, the weighting factors are commonly determined directly, i.e., percentual values are assigned intuitively by the client (Mateus et al., 2010). From the perspective of decision theory, the direct determination of weights is justifiable if the number of criteria is small and the client has already estimated the effects on criteria and its interactions. A methodological support can be useful when complex issues, such as environmental influences and user impairment, are part of the decision and valuable for counteracting the criticism of arbitrariness (Shyur and Shih, 2006; Lorentziadis, 2010; Liu et al., 2017).

2.2 Established evaluation formulas

If SDG criteria are requested in addition to the price, an evaluation method must be applied. In European countries, such as UK, Germany, or Austria, formulas that sum up the weighted price term (p) with the weighted SDG term (g), such as Eq. (1), are used. Consequently, each tender receives an aggregated scoring index (s) that indicates the best value for money (Dini et al., 2006).

(1)s=p\pm g.

Cardinal criteria, such as the price or the construction time, have to be converted into evaluation points to calculate the scoring index. Linear interpolation is widely used in such calculation. Exemplarily, the bidder with the cheapest price receives 10 points, whereas a fictitious offer that is twice as expensive receives 0 point. All intermediate offers are interpolated linearly. Table 1 shows an effect of the given example, which may occur in the application of this formula. The client queries the price and an SDG criterion, which for example assesses the quality of the environmental concept for site processes. For the SDG criterion, the client describes which services have to be offered to receive a certain number of evaluation points. If a bidder submits the minimum standard, which is defined normatively, 0 point is provided. Up to 10 additional points can be received, for example, if the quality of noise protection, waste disposal, or the use of low-energy construction machinery have been dealt with advantageously. By multiplying the points with the weighting factors, the results of weighted price points and the weighted SDG points are obtained, as shown in Table 1. In Case (a), two bidders apply for the construction work by submitting the tenders T1 and T2, respectively, but only T1 has been awarded a contract. In Case (b), a third bidder submits the tender T3. If the submission of variants is allowed, T3 can also be considered an alternative tender besides the main tender T2. However, in Case (b), T2 is supported by T3. Thus, the price points and the ranking of all tenders are based on the cheapest bid, which may not qualify for the award.

With linear interpolation, the relevance of the price can be reduced artificially by introducing an additional tender. Evidently, this effect leads to legal disputes, which have been discussed in various jurisdictions by German courts (Higher Regional Court of Düsseldorf, 2010; Federal Court of Germany, 2017; Higher Regional Court of Munich, 2017). These court decisions have provided different views with regard to the conversion of the price into evaluation points. Some courts even emphasize that no correct and undisputed formula exists. Consequently, the scoring index can hardly be used for measuring the best value for money. Such argument is in line with the opinion of the Dutch policymakers. In the Netherlands, they use an approach where the price is not converted into evaluation points, but the SDGs are monetized. Good and bad performance in SDGs reduces and increases the bid price, respectively. Thus, the client has to use pricing factors, e.g., carbon prices. Such approach is called “price correction mechanism” (Dreschler, 2008). However, combining various multidimensional award criteria is a transnational problem. Such legal uncertainty has been recently tackled by the European Court of Justice (2016). In its jurisdiction, the Supreme Court stipulated that the method might not change the values of the award criteria or their weighting.

To overcome the weaknesses of scoring formulas with linear interpolation, various scientifically proven approaches (i.e., advanced methods) have been developed.

2.3 Advanced methods for tender evaluation

Different decision making approaches have been adopted to deal with the specific requirements of public procurement tender evaluation. Methods from the field of multiattribute decision making are suitable for application, given that a fixed number of objectives (i.e., the award criteria) can be ranked by a discrete number of alternatives (i.e., the tenders) (Thewes and Kamarianakis, 2012). Considerable researches have adopted the Analytical Hierarchy Process (AHP) for tender evaluation (Cheung et al., 2010; Polat, 2016; Darko et al., 2019). Quantitative and fuzzy criteria are introduced in the award decision through the AHP. Hasnain et al. (2018) have introduced a decision support system based on the Analytic Network Process (ANP) for best value contractor selection in road construction projects. The ANP is based on the AHP and includes interdependent relationships among decision alternatives. Other approaches are based on multivariate decision making. These methods comprise decisions with several predefined objectives to be achieved in compliance with restrictions. Consequently, these tasks are often solved with the help of linear programming. A comprehensive summary of decision making methods used for contractor selection is introduced by Erdogan et al. (2019).

In addition to decision making approaches, efficiency analyses have been developed in production economics. Although efficiency analysis compares various alternatives with one another, its focus lies on benchmarking and subsequently the improvement of alternatives. Compared with other industries, efficiency analyses are of little significance for construction contractor selection. McCabe et al. (2005) and Yang et al. (2016) have proposed an approach for the prequalification of contractors. Costantino et al. (2011), Falagario et al. (2012), Cheaitou et al. (2019), and Niewerth (2019) have applied efficiency analysis in their respective studies on public tender evaluation. However, an unambiguous decision can only be made by further methodological development. For example, Falagario et al. (2012) have proposed a method to compute the cross efficiency as a scalar scoring index. However, their proposed approach does not consider the client’s weighting, which is required by European law (European Parliament and Council of the EU, 2014).

In summary, the above methods provide several advantages compared with the established formulas. Multiattribute and multivariate decision making allows the derivation of robust rankings. Efficiency analysis enables the identification of the strengths and weaknesses of alternatives through benchmarking. Thus, a method that combines decision making and efficiency analysis with valid procurement law has been developed.

3 Requirements for the proposed evaluation method

The European Supreme Court also demands for the identification of the benefits of a tender. According to the European Court of Justice (2018), the client has to inform all bidders, whose tenders have been rejected. Upon request, the bidder has to be informed about the reasons for the loss of the award and its disadvantages with respect to the commissioned tender. Thus, a tender evaluation method needs to indicate whether a tender with better SDG values can be successful, even though this tender results in additional costs for the client. Eder (2012) has recommended that non-monetary criteria must be capable of overcoming a price difference in the range of 10%–20%. The award process must not be influenced negatively by a third participant or a tactical variant, as shown in Table 1. Thus, the proposed method needs to guarantee a transparent and comprehensible award decision.

To meet the above requirements, an evaluation method that aggregates multidimensional award criteria is presented in this study. Based on the Nerlovian Efficiency proposed by Chambers et al. (1998), the proposed method ensures the derivation of a robust ranking and compares the quantitative performance of all bidders. In this way, the proposed method identifies value for money and reflects SDG criteria values, enabling a bidder to identify the potential areas that can be addressed and further improved. This mechanism is equal to a pricing process. To achieve a competitive price for a project, sufficient knowledge of the market is necessary. If the price is not the only award criterion, a bidder must also have experience with rival SDG performance. If bidders are aware of their own weaknesses, they will be able to improve the construction process to work out a more competitive tender.

4 Tender evaluation using the DEA

4.1 Theoretical principles

Efficiency analysis is introduced by production economists. Efficient analysis has been used to evaluate and compare production processes on the basis of the transformation of input into outputs. Farrell (1957) has refined efficiency as a relative indicator to quantify the performance of Decision Making Units (DMUs), e.g., production processes, firms, divisions, and activities. A DMU is considered to be technically efficient if it possesses the best possible ratio between inputs and outputs, considering a preset type of return to scale. Developed by Charnes et al. (1978) and Banker et al. (1984), the DEA is the most widely accepted technique used to calculate the Technical Efficiency (TE) value of a DMU. In the DEA, a number of j (= 1, 2, …, n) DMU_j will be benchmarked by considering a number of i (= 1, 2, …, m) inputs x_i and r (= 1, 2, …, s) outputs y_r. For the quantification, Banker et al. (1984) have defined TE by measuring the distance between the DMU under investigation (DMU₀) and a radial input-oriented reference on a convex frontier function in the hyperspace of dimension ℝ^m+s, as shown in Fig. 1. Given the distance measurement, the efficiency value possesses the dimension ℝ¹. To achieve an aggregated efficiency value for inputs and outputs with different units, an endogenous weighting of the input and output values is conducted using linear optimization.

In the DEA, the frontier function is approximated by the sectional linear linking of all technical efficient DMUs. These units possess no distance to the frontier function. Consequently, the efficiency value of technically efficient DMUs is TE = 0 (Chambers et al., 1998). By contrast, the efficiency value of technically inefficient DMUs lies within the interval (0, ∞). In the evaluation method, only technically efficient tenders will be considered for the award of the contract.

Given the small amount of subjective information on how to compute TE, the DEA is suitable for public procurement tender evaluation (Falagario et al., 2012; Yang et al., 2016). Accordingly, each tender is considered to be a DMU. An award criterion with a minimization task (e.g., price, LCC, execution time, reduction of transport distances, or the amount of waste during construction) will be treated as an input. By contrast, an award criterion with a maximization task (e.g., qualification of the workers, noise protection, or enhanced occupational safety measures) will be treated as an output.

4.2 Decision theoretical framework for the DEA

Given that all technically efficient DMUs possess the same efficiency value (TE = 0), a decision theoretical extension of the DEA becomes necessary in deriving a ranking. The concept of profit maximization is used in this study. Initially, Farrell (1957) has reported that the maximization of the achievable profit of every DMU_j can be derived by multiplying the input and the output values by the market prices (i.e., w_xi for input prices and w_yr for output prices). This process leads to a more specific differentiation of the technically efficient DMUs. Based on this approach, the linear profit function p(w_yr, w_xi) in Eq. (2) is introduced into the DEA, as shown in Fig. 2. For profit maximization, the function defines the supreme bound of the unit set T.

(2)p(w_{yr},\mathrm{}{w}_{xi})=\sup \{w_{yr}{y}_{j,r}-w_{xi}{x}_{j,i}:(x_{j,i},\mathrm{}{y}_{j,r})\in T\},\mathrm{\hspace{0.17em}}\mathrm{\hspace{0.17em}}\mathrm{\hspace{0.17em}}(x_{j,i},\mathrm{}{y}_{j,r})\ge \mathrm{0.}

The point of tangency between the profit function and the envelopment yields the DMU that pursues the maximum profit, as shown in Fig. 2. This micro-economically based approach is used to identify the best value for money.

In procurement law, the client is required to indicate a percentual weighting with respect to the award criteria. By using values, a profit function can be derived. However, the weighting does not represent absolute quantities, such as market prices; instead, they are relative values. After introducing this percentage data into the DEA, a specific DEA model has been developed. Award criteria do not have the same restrictive characteristics of production goods. Thus, a physical transformation of criteria with a minimization task (i.e., inputs) into criteria with a maximization task (i.e., outputs) is unnecessary. Given that fewer restrictions apply for award criteria than for production goods, the DEA model can be structured using various possibilities.

In this tender evaluation method, the structural properties of the envelopment are used to convert the percentage weighting into absolute values. Therefore, the envelopment has to represent the clients’ preferences, i.e., the weighting. If the preference of the client reflects a high relevance for SDG and consequently a lower weighting of the price (a_p<45%), tenders with weak values of SDG criteria must not be considered for the contract (Case I in Fig. 3). The case distinction leads to non-increasing returns to scale. By contrast, if the client prefers a high weighting of the price (a_p>75%), expensive tenders must not be considered for the award (Case III in Fig. 3). This situation leads to a frontier function with the property of non-decreasing returns to scale. A moderate weighting of the price leads to a balanced relevance of the price and the SDG criteria (Case II in Fig. 3). The function retains the property of variable returns to scale. The percentual limits for the weighting have considered the German court decisions regarding the lowest and highest admitted thresholds for the weighting of the price.

For the implementation of the percentual weighting, the minimum [min(x_{j,i}^{\mathrm{TE}}); min(y_{j,r}^{\mathrm{TE}})] and maximum [max(x_{j,i}^{\mathrm{TE}}); max(y_{j,r}^{\mathrm{TE}})] values of the technically efficient DMUs are used, as shown in Fig. 3.

Accordingly, the percentual weights a_xi specified by the client are transformed into absolute weights w_xi for the award criteria with a minimization task (Eq. (3)), whereas the weights a_yr are transformed into w_yr for the award criteria with a maximization task (Eq. (4)). Finally, the sum of a_xi and a_yr is equal to 100%.

(3)w_{xi}={\displaystyle \frac{a_{xi}}{\max (x_{j,i}^{\mathrm{TE}})-\min (x_{j,i}^{\mathrm{TE}})}},

(4)w_{yr}={\displaystyle \frac{a_{yr}}{\max (y_{j,r}^{\mathrm{TE}})-\min (y_{j,r}^{\mathrm{TE}})}}.

4.3 Setup of the DEA model

In addition to the structural property of the frontier function, a distance function has to be defined. The calculation of efficiency according to Banker et al. (1984) is entirely oriented in the input direction, as shown in Fig. 1. Given that all award criteria have weighting factors, a multitask improvement of inputs and outputs must be feasible. Therefore, a directional distance function according to Chambers et al. (1998) has been used. This individual distance function reflects the shortest radial distance between the DMU₀ and the profit function, as shown in Fig. 4. Consequently, the direction vectors d_yr and d_xi of the distance function represent a normal vector with regard to the profit function p(w_yr, w_xi).

Thus, the direction vector for minimization (d_xi) and maximization (d_yr) criteria can be derived from Eqs. (5) and (6).

(5)d_{xi}={\displaystyle \frac{\mathrm{1}}{w_{xi}}},

(6)d_{yr}={\displaystyle \frac{\mathrm{1}}{w_{yr}}}.

The implementation of the case distinction for the structural properties and the directional distance function based on the clients’ preferences has finally led to a specific DEA model. These properties can be specified by a linear programming task, as indicated in Eq. (7). A number of j (= 1,…, n) DMU_j, which is composed of input x_j,i and output y_j,r values, can be evaluated. The restriction states that the TE value of a DMU under investigation (DMU₀) is increased until the distance between the DMU₀ and the envelopment is as short as possible. Therefore, TE expresses the distance between the actual position of the DMU₀ and the envelopment, which is measured along the defined directional distance function. DMU₀ is located on the envelopment, only if TE reaches a maximum amount of 0.

\max \mathrm{\hspace{0.17em}}TE

subject to

{\displaystyle {\sum }_{j=\mathrm{1}}^n{\lambda }_j\cdot y_{j,r}}\ge y_{\mathrm{0},r}+TE\cdot d_{yr},

{\displaystyle {\sum }_{j=\mathrm{1}}^n{\lambda }_j\cdot x_{j,i}}\le x_{\mathrm{0},i}+TE\cdot d_{xi},

{\lambda }_j\ge \mathrm{0},\mathrm{}\forall j=\mathrm{1},\dots ,\mathrm{}n,

Case discrimination:

\mathrm{Case\; II}:\mathrm{\hspace{0.17em}}\mathrm{\hspace{0.17em}}\mathrm{\hspace{0.17em}}{\displaystyle {\sum }_{j=\mathrm{1}}^n{\lambda }_j=\mathrm{1}},\mathrm{}\forall j=\mathrm{1},\dots ,\mathrm{}n\mathrm{\hspace{0.17em}}\mathrm{\hspace{0.17em}}\mathrm{\hspace{0.17em}}(4\mathrm{5}\%\mathrm{}\le a_{\mathrm{p}}\le 75\%),

(7)\mathrm{Case\; III}:\mathrm{\hspace{0.17em}}\mathrm{\hspace{0.17em}}\mathrm{\hspace{0.17em}}{\displaystyle {\sum }_{j=\mathrm{1}}^n{\lambda }_j\ge \mathrm{1}},\mathrm{}\forall j=\mathrm{1},\dots ,\mathrm{}n\mathrm{\hspace{0.17em}}\mathrm{\hspace{0.17em}}\mathrm{\hspace{0.17em}}(a_{\mathrm{p}}>\mathrm{75}\%).

By using the DEA model (Eq. (7)) and the profit function (Eq. (2)), the Nerlovian Efficiency is determined.

4.4 Nerlovian Efficiency as an indicator of “value for money”

TE does not qualify for the derivation of a ranking, because all technically efficient DMUs receive a value of TE = 0. Therefore, the profit function and the specific concept of Nerlovian Efficiency are applied for the tender evaluation method. Chambers et al. (1998) have defined the Nerlovian Efficiency (NE) value as the distance between the DMU under investigation (DMU₀) and the profit function. With regard to TE, NE = 0 expresses the “maximum maximorum of profit”. In the context of tender evaluation, this value represents the best offered value for money. All inefficient DMUs lie within the interval (0, ∞). The measurement of the distance to the profit function finally yields the ranking. Due to the perpendicularity between the profit and the distance function, the equation of Chambers et al. (1998) can be transferred to Eq. (8). The ranking results are obtained from the ascending order of NE of all tenders.

(8)NE={\displaystyle \frac{\mathrm{1}}{m+s}}[\sup (w_{yr}{y}_{j,r}-w_{xi}{x}_{j,i})-(w_{yr}{y}_{\mathrm{0},r}-w_{xi}{x}_{\mathrm{0},i})].

In Eq. (8), the difference between the supremum of profit from Eq. (3) and the actual profit of DMU₀ is normalized by the total number of award criteria (m + s). This process is necessary to overcome the size differences of the multidimensional criteria and find the reference on the profit function, as shown in Fig. 4. As an achievement, the difference between NE and TE in Eq. (9) indicates how well a bidder has adapted its tender to the preferences given by the client. This third type of efficiency is referred to as the Allocative Efficiency (AE).

(9)AE=NE-TE.

In summary, the structure of this evaluation method is shown in Fig. 5.

5 Scenario analysis

To verify the reliability and validity of the proposed method, different construction scenarios have been evaluated. In this scenario, the client demands the following award criteria: 1) x₁: LCC (given in €), 2) x₂: construction time (given in weeks), and 3) y₁: environmental concept (given in evaluation points, ranging from 0 to 10). As shown in Table 2, seven tenders have been turned in. The LCC and the construction time describe a minimization task, whereas the environmental concept represents a maximization task.

Niewerth et al. (2017) have used the AHP to ensure a transparent weighting. The LCC, construction time, and environmental concept have been weighted as a_x1 = 72%, a_x2 = 19%, and a_y1 = 9%, respectively. Thus, the structural properties have been specified to a variable return to scale, as shown in Case II in Fig. 3. Through linear programming of Eq. (7), Case II must be considered a constraint. First, the technically efficient tenders are identified through the linear optimization of the DEA. Thus, tenders T2, T4, and T5 have an efficiency value of TE = 0. These tenders have been identified according to objective, mathematical information and therefore apply for the award of the contract. The minimum and maximum values of these DMUs are used to implement the percentual weighting. The absolute values of the weighting (i.e., w_xi and w_yr) are derived using Eqs. (3) and (4). Simultaneously, the direction vectors d_xi and d_yr are calculated using Eqs. (5) and (6). The results are shown in Table 3.

Table 4 shows the results according to Eqs. (8) and (9) and the DEA model in Eq. (7). The efficiency values disclose whether the deficits have resulted from the specific weighting of the client or from the poorly offered award criteria values. A high TE value is obtained from comparatively tenuous criteria values. A high AE value indicates a long distance to the profit function, which includes the weighting of the client.

From the efficiency values listed in Table 4, a bidder can derive the potential of criteria improvement, i.e., the shortfalls to the references of the functions. If the criteria values of a bidder are reduced or increased by the values specified in Table 5, the offer will become a part of the profit function and consequently the bidder will receive the contract. Accordingly, an envelopment extrapolation can be calculated.

A graphical presentation of the results can be helpful to illustrate the concept of efficiency. The envelopment of the unit set and the profit function of the weightings of the clients is shown in Fig. 6. Among the seven tenders, three are located on the envelopment. These tenders are T2, T4, and T5, which have implemented the award criteria in the best possible way with minor subjective influences. Therefore, these tenders basically qualify for the award. By adding a project-specific profit function, the tender that satisfies the preferences of the client in the best combination receives the contract. As shown in Table 5, T2 exhibits Nerlovian-efficient criteria values. The DMU itself (T2), the reference of the profit function (T2'), and the reference on the envelopment (T2") exhibit the same values. By contrast, the references of T6 are not situated in the same point as the DMU itself (T6 ≠ T6' ≠ T6").

In addition to the ranking, a quantitative report of the award decision may be handed out to the bidders. Exemplarily, the shortfalls of the inefficient tender T6 in Table 5 are analyzed to identify the potential areas for improvement. The LCC of T6 must have been 1.31% lower (i.e., |-472743.06 €/35980150 €|), the construction time must have been 4.11% shorter, and the environmental concept must be improved by 34.71% (or 2.43 points). If the bidder identifies a regular repetition of these deficits in several procurement procedures, a revision of work or management processes must be aspired. In case of minor deficits for one certain criterion, a moderate adjustment may be enough, e.g., a slight revision of the environmental concept or a moderate change in site logistics. Larger deficits may require more extensive customizations, e.g., buying low-energy construction machinery or applying of lean construction techniques to reduce the execution time. In this case, the intention of policymakers to cause an improvement of SDGs will be achievable.

In the described scenario, the evaluation method is applied to an open procurement procedure. However, the proposed method can also be used for other public procurement procedures, such as the negotiation procedure or the competitive dialog, as described in Directive 2014/24/EU (European Parliament and Council of the EU, 2014). In the competitive dialog, contractors must be involved in the early stages of planning. All tenders have to be adapted to the latest stage of planning continuously. Contractors with the weakest tenders at the end of the planning stage —the technically inefficient tenderers (i.e., TE≠ 0)— will be rejected from the project. After completing the planning stage, the award decision can be made with the help of the predefined clients weighting.

According to Hanák and Serrat (2018), the use of electronic auctions can also support the client in achieving the best value for money. The combination of electronic auction with the proposed evaluation method allows bidders to optimize all award criteria simultaneously. Consequently, the client can transparently seek out the best value for money.

6 Conclusions

Multidimensional award criteria can be aggregated and compared through the proposed tender evaluation method. When evaluating supranational tenders for construction projects, complying with the legal scope and ensuring a high degree of objectivity are of utmost importance. These conditions are guaranteed through the DEA. The developed method uses the strength of the DEA, which is the value-free identification of technically efficient DMUs. Different from conventional evaluation formulas, only the tenders that possess realistic opportunities are being considered. Therefore, the proposed method is more robust against artificial manipulation of the scoring than the established formulas. In sum, Nerlovian Efficiency is a suitable key figure to identify best value for money. Moreover, the use of the method leads to greater legal certainty. These aspects are decisive advantages of the developed evaluation method.

In addition to the tender ranking, the proposed method can be used to support SDGs in the long-term. By emphasizing bidders’ deficits, a comparison of their performance is guaranteed. In this way, a bidder can evaluate whether deficits in award criteria are repeated regularly and therefore can be used as performance indicators.

However, the calculation of NE and the determination of shortfalls are obtained from a complex mathematical approach compared with established evaluation formulas. To comprehend the calculation method, the client needs essential knowledge of efficiency analysis. Basically, the user needs to know how the envelopment is established. Knowledge about the definition of the efficiency value as a synonym for the distance between a DMU and the envelopment is required. The use of proven calculation sheet and comprehensible text modules for procurement documents is necessary to introduce the method in construction practice. For example, Niewerth (2019) has prepared a computational application for the calculation and instructions for use. Consequently, a valid and fast tender evaluation process is guaranteed.

Further research must be conducted to examine various aspects that have emerged during method development. First, the integration of the client’s weighting, which is a legal requirement in Europe, needs to be explored. Given the intuitive determination of percentage values, a certain amount of subjectivity is retained. Different approaches must be explored to objectify this process. Further research must also investigate which award criteria are best suited for performance assessment in the long-term and across different projects.

References

[1] Alhola K (2012). Environmental criteria in public procurement: Focus on tender documents. Monographs of the Boreal Environmental Research, 40. Aalto: Department of Real Estate, Planning and Geoinformatics, Aalto University

[2] Ballesteros-Pérez P, Skitmore M, Pellicer E, Zhang X (2016). Scoring rules and competitive behaviour in best-value construction auctions. Journal of Construction Engineering and Management, 142(9): 04016035

[3] Banker R D, Charnes A, Cooper W W (1984). Some models for estimating technical and scale inefficiencies in Data Envelopment Analysis. Management Science, 30(9): 1078–1092

[4] Bratt C, Hallstedt S, Robert K H, Broman G, Oldmark J (2013). Assessment of criteria development for public procurement from a strategic sustainability perspective. Journal of Cleaner Production, 52: 309–316

[5] Chambers R G, Chung Y, Färe R (1998). Profit, directional distance functions, and nerlovian efficiency. Journal of Optimization Theory and Applications, 98(2): 351–364

[6] Charnes A, Cooper W W, Rhodes E (1978). Measuring the efficiency of decision making units. European Journal of Operational Research, 2(6): 429–444

[7] Cheaitou A, Larbi R, Al Housani B (2019). Decision making framework for tender evaluation and contractor selection in public organizations with risk considerations. Socio-Economic Planning Sciences, 68: 100620

[8] Cheung S O, Lam T I, Leung M Y, Wan W W (2010). An analytical hierarchy process based procurement selection method. Construction Management and Economics, 19(4): 427–437

[9] Costantino N, Dotoli M, Falagario M, Fanti M P, Mangini A M, Sciancalepore F (2011). Supplier selection in the public procurement sector via a data envelopment analysis approach. In: Proceedings of the 19th Mediterranean Conference on Control and Automation (MED). Corfu: IEEE, 236–241

[10] Darko A, Chan A P C, Ameyaw E E, Owusu E K, Pärn E, Edwards D J (2019). Review of application of analytic hierarchy process (AHP) in construction. International Journal of Construction Management, 19(5): 436–452

[11] Dini F, Pacini R, Valletti T (2006). Scoring rules. In: Dimitri N, Piga G, Spagnolo G, eds. Handbook of Procurement. Cambridge: Cambridge University Press, 293–321

[12] Dreschler M (2008). Analysis of price correction award mechanisms applied in the Dutch construction industry. In: Proceedings of the 3rd International Public Procurement Conference. Amsterdam, 423–440

[13] Eder M (2012). VIP—Tender model for infrastructure projects. Geomechanics and Tunnelling, 5(6): 708–717

[14] Eke G, Wedawatta G, Elgy J (2017). A quantifiable method of assessing the risk of selecting the lowest bidder in construction projects: A literature review. In: Proceedings of the 13th International Postgraduate Research Conference. Salford, 637–646

[15] Erdogan S A, Saparauskas J, Turskis Z (2019). A multi-criteria decision-making model to choose the best option for sustainable construction management. Sustainability, 11(8): 2239

[16] European Commission (2012). Guide to Research and Innovation Strategies for Smart Specialisations (RIS3). Luxembourg: Publications Office of the European Union

[17] European Court of Justice (2013). Judgment of the Court (Second Chamber) from 2013–01–29 in Case T-532/10. ECLI:EU:T:2013:38

[18] European Court of Justice (2016). Judgment of the Court (Fourth Chamber) from 2016–07–14 in Case C-6/15. ECLI:EU:C:2016:555

[19] European Court of Justice (2018). Judgment of the Court (Fourth Chamber) from 2018–05–03 in Case C-376/16. ECLI:EU:C:2018:299

[20] European Parliament, Council of the EU (2014). Directive 2014/24/EU of the European Parliament and of the Council of 26 February 2014 on public procurement and repealing Directive 2004/18/EC. Official Journal of the European Union, 94(57): 65–242

[21] Faire Vergaben (2016). Criteria for Value for Money Procurement. Vienna: GBH-Presse (in German)

[22] Falagario M, Sciancalepore F, Costantino N, Pietroforte R (2012). Using a DEA-cross efficiency approach in public procurement tenders. European Journal of Operational Research, 218(2): 523–529

[23] Farrell M J (1957). The measurement of productive efficiency. Journal of the Royal Statistical Society: Series A (General), 120(3): 253–281

[24] Federal Court of Germany (2017). Resolution of 2017–04–04. Reference X ZB 3/17

[25] Hanák T, Serrat C (2018). Analysis of construction auctions data in Slovak public procurement. Advances in Civil Engineering, 3036340

[26] Hasnain M, Thaheem M J, Ullah F (2018). Best value contractor selection in road construction projects: ANP-based decision support system. International Journal of Civil Engineering, 16(6): 695–714

[27] Higher Regional Court of Düsseldorf (2010). Resolution of 2010–03–03. Reference VII–Verg 48/09

[28] Higher Regional Court of Munich (2017). Resolution of 2017–03–30. Reference Verg 10/16

[29] International Tunneling Association (2014). Recommendation of a Tender Model for Infrastructure Projects. Vienna: Austrian National Committee of ITA

[30] Liu B, Huo T, Liao P C, Yuan J, Sun J, Hu X (2017). A special Partial Least Squares (PLS) path decision modelling for bid evaluation of large construction projects. KSCE Journal of Civil Engineering, 21(3): 579–592

[31] Lorentziadis P L (2010). Post-objective determination of weights of the evaluation factors in public procurement tenders. European Journal of Operational Research, 200(1): 261–267

[32] Mateus R, Ferreira J A, Carreira J (2010). Full disclosure of tender evaluation models: Background and application in Portuguese public procurement. Journal of Purchasing and Supply Management, 16(3): 206–215

[33] McCabe B, Tran V, Ramani J (2005). Construction prequalification using data envelopment analysis. Canadian Journal of Civil Engineering, 32(1): 183–193

[34] National Agency for Public Procurement (2019). Sustainability criteria. Available at:

[35] Niewerth S (2019). A Decision-Theoretic Extension of the Data Envelopment Analysis for Tender Evaluation in Public Construction Contracts. Dissertation for the Doctoral Degree. Bochum: Ruhr-University (in German)

[36] Niewerth S, Vogt P, Thewes M (2017). Evaluation of non-monetary award criteria and life-cycle cost for procurement of construction contracts. Bauingenieur, 92(9): 398–405 (in German)

[37] Polat G (2016). Subcontractor selection using the integration of the AHP and PROMETHEE methods. Journal of Civil Engineering and Management, 22(8): 1042–1054

[38] Shyur H J, Shih H S (2006). A hybrid MCDM model for strategic vendor selection. Mathematical and Computer Modelling, 44(7–8): 749–761

[39] Thewes M, Kamarianakis S (2012). Multi-criteria decision making of construction methods using the analytical hierarchy process based on fuzzy scales. In: Proceedings of the 13th World Conference of the Associated Research Centers for the Urban Underground Space (ACUUS)—Advances in Underground Space Development. Singapore

[40] Tran D, Molenaar K R, Gransberg D D (2016). Implementing best-value procurement for design–bid–build highway projects. Transportation Research Record: Journal of the Transportation Research Board, 2573(1): 26–33

[41] UK Cabinet Office (2012). Government Construction—Common Minimum Standards for procurement of the built environments in the public sector. London

[42] Vogt P (2013). Model for Life Cycle Cost Analysis of Road Tunnels Considering Technical and Financial Uncertainties. Dissertation for the Doctoral Degree. Bochum: Ruhr-University (in German)

[43] Vogt P, Thewes M (2012). The calculation of life-cycle costs for road tunnels under the influence of uncertainties. In: Life-Cycle and Sustainability of Civil Infrastructure Systems: Proceedings of the 3rd International Symposium on Life-Cycle Civil Engineering. Vienna: CRC Press, 1036–1043

[44] World Economic Forum (2017). Sustainable Development Impact Summit 2017 Report – Global Agenda. New York

[45] Yang J B, Wang H H, Wang W C, Ma S M (2016). Using data envelopment analysis to support best-value contractor selection. Journal of Civil Engineering and Management, 22(2): 199–209

RIGHTS & PERMISSIONS

The Author(s) 2020. This article is published with open access at link.springer.com and journal.hep.com.cn