Decision support based on performance data using the analytic hierarchy process without expert judgement

Goal: This article proposes a decision model based on the Analytic Hierarchy Process that allows carrying out the evaluation of alternatives in a multicriteria problem, without expert judgement.
Design / Methodology / Approach: The algorithm is based on AHP. The novelty is the transformation of a performance data matrix into pairwise evaluation matrices, instead of using experts’ judgement.
Results: The algorithm was applied in a defense procurement problem for the choice of a light 4x4 vehicle for amphibious operations. The results allowed ranking the 17 models based on catalog data.
Limitations of the investigation: the algorithm depends on the availability of catalog data, not always available in open sources in the defense industry.
Practical implications: Decision support involves several activities in Operations Management and AHP has been frequently applied to solve problems in this sector. The proposed algorithm allows performing deterministic or probabilistic evaluations, depending on the degree of uncertainty and precision involving the systems’ performance data. These assessments are composed of scenarios to facilitate decision making.
Originality / Value: AHP typically uses experts for pairwise judgments. However, human judgment is subject to outcomes that involve bias and cognitive distortions. Few studies have modeled the AHP without experts, even so they used human judgment in some part of the process. The approach proposed here does not require human judgment and returns two different results, based on the database precision. This new approach gives decision makers a different perspective and can alter the final choice.


INTRODUCTION
Human decisions are subject to bias, inconsistencies, systemic errors and cognitive distortions (Abatecola et al., 2018;Hilbert, 2012;Lovallo and Sibony, 2006).If the decision is restricted to a private problem, an error in this process remains within the scope of decision maker's environment.However, if the decision is related to the public sector, the burden of error can bring undesirable consequences to society or third parties.
Operations management (OM) focuses on the organizational activities involved in the production of goods and/or provision of services required by its customers (Radnor and Barnes, 2007).Several technical processes for choosing services, equipment or even for improving Among the various decision support techniques suitable for OM processes, the Analytic Hierarchy Process (AHP) is very popular and has been applied in a wide variety of OM topics including planning, selection of best alternatives, allocation of resources and conflicts resolution (Chawla et al., 2021;Ho and Ma, 2018;Subramanian and Ramanathan, 2012).The scope of this research deals with decision support with an AHP variant to be applied in the public sector and uses the defense industry as an example.
In the defense industry errors in choosing a system or product can compromise the security of the nation and aggravate budget issues for a significant period of time due to the long development and construction periods implied (Clowney et al., 2016;Franck and Udis, 2017).In general, three actors have a direct influence on the defense procurement process: the manufacturers that develop the systems, the armed forces that employ them, and the government, which supports the defense sector (Gholz and Sapolsky, 2021).The interests of these parties in the acquisition process can bias choices (Maser and Thompson, 2011).
In this sector, is usual to seek for the support of experts to assess the performance of defense systems (Kadish et al., 2006).However, it would be interesting for the defense ministry or department to carry out an evaluation of defense products in a process without human judgment.
The essentially technical result of this evaluation, based exclusively on the "factory" performance of the systems under analysis, can bring greater transparency, coherence, and convergence to the final decision.This does not prevent other judgments of a political-strategic or economic nature, among others, often involved in a decision process of this magnitude.However, knowing and comparing the technical performance of the systems can help improve the final decision by providing a counterweight to the mentioned interferences.This article proposes a decision support model without the participation of experts, to identify defense systems or products based on their technical performance.The nominal (factory) characteristics of these items are usually presented at defense industry events or through direct inquiries by countries interested in their acquisition.These data can also be found in catalogs, used as technical references by the defense industry.The tabulation of the systems of interest and the performances in their respective characteristics composes the database for the application of a multicriteria decision support model (Almeida, 2013;Gavião et al., 2017).
The choice for the AHP as the basis of the proposed model is due to the wide spectrum of applications for managerial decision-making, in the most diverse application areas, which include defense procurement processes (Abastante et al., 2019;Agápito et al., 2015;Balusa and Gorai, 2019).In addition to the mathematical rigor of the AHP, based on the principles of linear algebra, the method brings its own indicator for validating the process, something rare in multicriteria decision methods .Assis et al. (2022), Camilo et al. (2020) and Silva et al. (2010) applied the AHP to prioritize aerospace projects of the Brazilian Air Force (FAB).Salgado (2021) evaluated ships for polar research, in support of the Brazilian Antarctic Program.Hamurcu and Eren (2020) evaluated unmanned aircraft (UAV) models for defense acquisition.AHP was employed by the Portuguese Navy to prioritize projects (Simplício et al., 2017).Regarding the Pakistan Air Force, Ali et al. (2017) applied the AHP for the choice of attack aircraft.Kim and Lee (2019) for South Korea; Wei and Shi (2022) for China; and Bimo et al. (2022) for Indonesia, applied AHP in the defense context.In all these works the support of specialists was used for the evaluations.
Very few studies using AHP have addressed models without the participation of specialists.We performed searches in the ISI Web of Science and SCOPUS databases using combinations of terms "Analytic Hierarchy Process" or "AHP" and "without expert".The search was updated on September 5, 2023, searching these expressions in the abstracts, as shown in Table I.
Table 1 -literature search After a brief presentation of these five papers, the calculation algorithm, called "Performance-Based AHP (PB-AHP)", was detailed.Finally, the PB-AHP was applied to a problem of choosing a light tactical 4x4 vehicle, among 17 vehicles available in military catalogs.

3/15
AHP, created by Saaty (1980), is a multicriteria decision support method that depends on pairwise comparisons between criteria, subcriteria and alternatives that make up the structure of the problem.The method uses a nine-point scale for these assessments, based on psychometric perceptions for comparison.In most applications, AHP receives support from experts to perform pairwise comparisons and form the initial database for algorithm modeling.A variant of AHP with fuzzy logic, generating models called "Fuzzy AHP", seeks to deal with the uncertainty of the decision and the bias resulting from human judgment, but it still requires the opinion of specialists (Chang, 1996).
After filtering the ISI Web of Science and SCOPUS databases, we turned to Google Scholar in the search for other studies of interest.Finally, five publications, that employed AHP with an effective procedure without specialists, remained for analysis: Bulut et al. (2012), Vats and Vaish (2014), Szűts and Krómer (2015), Dos Santos et al. (2021) and Kozłowska et al. (2023).However, a detailed reading revealed that the absence of specialists occurred only in part of the decisionmaking process.This corroborates the relevance of this research since the model proposed here does not use experts at all.In addition, PB-AHP is innovative as literature is scarce on AHP applications under these conditions.Bulut et al. (2012) proposed the generic fuzzy AHP (GF-AHP), without initial consultation with specialists, although they still resorted to them during the process.In addition to pairwise judgments, experts indicated their career time and professional experience, aspects that were considered to prioritize their opinions by a lambda parameter.The authors also argued that there are preference problems with variables capable of assigning the relative importance between the alternatives, without the need for pairwise comparison.For example, the result of the financial analysis allows evaluating an investment, since the results embed the perception of cost or benefit.In this context, the higher rate of return on investment directly indicates the superiority of one strategy over another.In summary, the authors still had to rely on experts for their model.Vats and Vaish (2014) associated AHP and VIKOR to select the ideal temperature for the ceramic sintering process, based on piezoelectric parameters.The authors applied the AHP without specialists, justifying that the performance parameters can be measured exclusively based on their physical properties, as evidenced in the scientific literature.Nine physical properties were associated with criteria and their values for the analysis of four alternatives, related to processing temperatures (1060 o C, 1080 o C, 1100 o C and 1120 o C).The approach to transform this decision matrix (alternatives, criteria and performance values) into AHP pairwise evaluation matrices was simple and logical.However, Vats and Vaish (2014) still resorted to specialists to weight the criteria using the MDL technique, developed by Dehghan-Manshadi et al. (2007).Szűts and Krómer (2015) that proposed a fuzzy AHP approach without experts, called Hybrid Fuzzy AHP, justified the proposal based on the high number of criteria and subcriteria necessary to solve a problem in the construction industry, which would require 90 judgments per evaluator.Thus, this effort was transferred to fuzzy inference systems in the MatLab software.However, the participation of a group of specialists was still necessary to carry out the pairwise evaluations at the criterion level, to define their weights.
Recently, Dos Santos et al. (2021) developed the AHP-Gaussian.This model has received applications in different areas, including the defense area itself (Dos Santos et al., 2021;Soares et al., 2022), sensor evaluation (Pereira et al., 2023) and other applications (Barroso, 2022).In common with the PB-AHP, three aspects stand out: (1) the hierarchical structuring of the problem; (2) the use of an objective technique for assigning weights to criteria, based exclusively on the decision matrix database; PB-AHP uses entropy, as described in Pomerol and Barba-Romero (2012), and the AHP -Gaussian uses a "Gaussian factor", which is in fact is called Coefficient of Variation (CV) proposed by Karl Pearson (Kvålseth, 2017), which measures the ratio between the standard deviation and the mean of the data; (3) both methods do not require Saaty's nine-point scale to generate their results.However, the PB-AHP maintains fidelity to some pillars of the AHP theory, which include the use of pairwise comparison between the alternatives of the problem (according to Equation (6) of Section 2.1) exploring the approach of Vats and Vaish (2014), the calculation of the final weights of the alternatives by eigenvectors (according to Equation (7) of Section 2.1), and the indication of the consistency ratio of the process without experts (according to Equation (11) of Section 2.1).Furthermore, the PB-AHP allows the generation of deterministic or probabilistic results, depending on the accuracy and reliability of the catalog, report, or market data of the evaluated systems.Kozłowska et al. (2023) applied the AHP in the energy sector, without resorting to human judgement.The Authors created a score reference, called "importance level (IL)", to replace experts' opinions in the AHP pairwise comparison.The algorithm that generates the IL deviates from the AHP evaluation procedure, so we disregard this reference for analysis.Alrasheedi et al. (2023) was also discarded, as the procedure "without experts" did not directly involve AHP, but remote image analysis.

MATERIAL AND METHODS
The PB-AHP methodology is described by three steps.The 1 st Step consists of collecting data, which are organized in a decision matrix (M).The rows and columns of this matrix are composed, respectively, of the alternatives of the problem and the evaluation criteria, according to Equation (1).The internal values of the matrix correspond to the performance measures of each alternative in each criterion, obtained from catalogs, other technical documents or even in the scientific literature, without considering specialist judgments about the performance values.The 2 nd Step applies the algorithm for ranking the alternatives.The "Performance-based AHP (version 3.0)" library is available in open access at "Zenodo.org" (Gavião, 2023).To generate the results, there are two functions in R code, depending on the degree of uncertainty or inaccuracy of the data.For a lower degree of uncertainty, one should use the "AHP.Perf.Exac" function, which considers performance data as exact values.For a greater degree of uncertainty, the "AHP.Perf.Prob" function should be used, which considers the data collected as representing the most probable values (modes) of triangular distributions.Table II shows the pseudocode used to calculate the PB-AHP, in the R software.In the 3 rd Step the results are analyzed.In the case study, we used statistical correlation to compare results of the deterministic and probabilistic functions.In case of significant differences between them, three scenarios (greater uncertainty, neutrality, and greater precision) were suggested to aggregate the results, facilitating the final analysis for the decision-maker if there are difficulties in matching his/her problem to the most suitable function.
In the 1 st Step, the weights (w) of the decision matrix criteria are obtained through the concept of entropy (E), proposed by Zeleny (1982), based on the information theory of Shannon (1949).Zeleny (1982) used the data without expert judgments to objectively determine the weights.He resorted to the principle of entropy, associating the highest weight to the j th criterion that presents the greatest dispersion in the evaluations of the alternatives.Thus, the most important criteria are those that have the greatest discriminating power among the alternatives.The sequence of calculations in this step involves normalizing the decision matrix (Equation 2), calculating the entropy (Equation 3), calculating the dispersion measure (D) (Equation 4) and defining the weights (p) with the normalization of the dispersions (Equation 5).These equations were presented by Pomerol and Barba-Romero (2012).The entropy weight method is often used in multicriteria decision problems with AHP (Shen et al., 2022;Shi et al., 2022;Wu et al., 2022;Yue-ming et al., 2020;Yuna and Lei, 2021).
In the 2 nd Step, the data of each criterion of the decision matrix (M) are transformed into a pairwise matrix (M'), by dividing each row element by the corresponding column element, according to the procedure described by Vats and Vaish (2014).Equation ( 6) illustrates this procedure for the j th criterion.In the 3 rd step, the AHP is calculated for each matrix M' generated in the previous step, based on the eigenvectors and eigenvalues technique used in Liu and Lin (2016).In this step, the weights (w) of the alternatives in each criterion and the AHP consistency ratios (CR) are generated, for the purpose of validating the results.Equations ( 7) to (11) describe these calculation steps.
The mathematical notations in these equations indicate: In the 4th Step, the matrix multiplication of the results of the 1 st Step (criteria weights -p) and the 3 rd Step (AHP weights by criterion -w) generates the final weights (r) of the alternatives, according to Equation (12).

Function "AHP.Perf.Prob"
This function applies the same steps described in Section 3.1, with the addition of a new procedure between the 1 st and 2 nd Steps.Before generating the matrix M', described in Equation ( 6), the evaluations of the initial decision matrix (M) are converted into probabilities, based on the Composition of Probabilistic Preferences (CPP) (Sant'Anna, 2015).This multicriteria decision support method has been widely explored in operations research for the solution of problems involving uncertainty in relation to the database, either due to the imprecision of the sources or the collection process, the variance of the performance of the alternatives in the face of different scenarios, among other aspects that allow assuming that the evaluations of the alternatives are not exact and constant values (Gavião et al., 2020(Gavião et al., , 2021)).
In this intermediate procedure, the two initial steps of the CPP are performed.First, the values of the decision matrix (M) are assumed to be the modes of probability distributions, representing the most frequent but variable values.The triangular distribution is usual in CPP and requires only two additional parameters to define it, in this case the minimum and maximum value of a data sample.Fig. 1 illustrates the transformation of performance data into the corresponding triangular distribution, which is called randomization.The minimum and maximum parameters of the triangular distributions are extracted from the performance data sample of the alternatives in each criterion, here illustrated by the values "0" and "10", respectively.
Then, the probabilities of each alternative being higher in relation to the others are calculated for each criterion (PMax).Equation ( 13) defines the value of these probabilities, where the notation F indicates the cumulative distribution function (CDF) and f the probability density function (PDF).The index (i) represents the alternative that one chooses to calculate the PMax and the index (-i) the other alternatives in the same criterion j.The cdf and pdf functions of a triangular distribution can be imported from the R-package "triangle" (Carnell, 2022) or "EnvStats" (Millard, 2013).The request for this library is already in the PB-AHP code.

APPLICATION AND RESULTS
PB-AHP functions were applied to rank preferences of a sample of 17 light tactical 4x4 vehicles, for employment in amphibious operations.The Jane's Land Warfare Platform catalog lists technical features of different manufacturers, as depicted in Table IV.The vehicle designations are numerical, to maintain the confidentiality of the manufacturers and their models.Although the consulted catalog presents a greater number of vehicles, only the 17 vehicles listed presented complete data in the indicated criteria and subcriteria.
The three subcriteria related to ship boarding and unloading have a negative impact, that is, the lower the absolute value of the vehicle in the subcriterion, the better for amphibious operations, as they improve boarding and unloading conditions.Thus, when modeling in the R software, the three initial columns of data in Table V need to be inverted, so that the smallest numerical values are the most representative to the algorithm.The use of negative values (-x), instead of their inversion (1/x), cannot be applied in the AHP.Perf.Exac function, because the algorithm performs the division of these values during the calculation process, which nullifies the effect of the negative values.The AHP.Perf.Prob function is indifferent to the two scale adjustment procedures, both by inversion and by negation.Sources: Brazil (2009Brazil ( , 2015)).Tables VI and VII show the deterministic and probabilistic results, respectively, referring to the application of the AHP.Perf.Exac and AHP.Perf.Prob functions.The values of the vehicles in each subcriterion are not weighted by entropy and correspond to the results of Equation ( 8).It is also possible to verify that the consistency ratios (Equation 11) of each pairwise matrix generated in each subcriterion was less than 10%, validating the calculations based on the logical consistency of AHP.
Decision support based on performance data using the analytic hierarchy process without expert judgement   The matrix multiplication of the weights by the results of each vehicle produced the results and the rankings of the models, as shown in Table VIII.The results of the rankings were submitted to Kendall's correlation.For a 95% confidence level, the tau index = 0.4559 (p-value = 0.01034) indicates that the preferences of the deterministic and probabilistic models are significantly different.This motivated the creation of scenarios to support decision making, depending on the degree of uncertainty surrounding the collected performance data.Three scenarios were then established to weight the results: greater uncertainty, neutrality, and greater precision.The scenario with greater uncertainty weights the probabilistic result with 75% of relevance and the deterministic with 25%.The neutral scenario weights both with 50% importance in the results.The most accurate scenario weights the probabilistic result with 25% relevance and the deterministic one with 75%.The scenario results are shown in Table IX.Vehicles 16, 13, 17 and 5 stood out from the others, as indicated by the results and ranks in the three scenarios.Their performances remained consistent in the best positions, in which vehicle 16 achieved the best performance.This result was obtained without any interference of judgment in the evaluations, even with the weighting of the scenarios.
Regardless of the interference of political-strategic and economic factors, among others, this technical reference can guide decision-making to maintain consistency with the best performance in the final choice for the acquisition, and seek convergence in the application of government resources.If the final result represents the choice of one among these four vehicles with the best global performance, it is possible to consider that the cost-benefit ratio was satisfactory.

CONCLUSION
This paper proposes a model, based on the AHP that allows an essentially technical evaluation of alternatives to be carried out, without the intervention of specialists.The PB-AHP algorithm performs deterministic or probabilistic evaluations, depending on the degree of uncertainty and precision involving the systems' performance data.The model was applied in a defense procurement problem for the choice of a light tactical 4x4 vehicle for amphibious operations.The results allowed ranking the 17 models, based on catalog data.
We believe this proposal is unprecedented in the literature on AHP and relevant for decision support.Human opinion or judgment is subject to outcomes that involve bias, inconsistencies, systemic errors, and cognitive distortions.These aspects can lead to substandard decisions, which become more serious if the problem involves public administration, bringing undesirable consequences to society and possible damage to third parties.Political-strategic and economic aspects, among others, are already taken into account in problems related to the country's security and defense, so the result of an essentially technical process, based exclusively on the performance of the defense systems considered, may be relevant to support the final decision.
The R-codes are open and available for consultation in the supplementary material.
International Journal of the Analytic Hierarchy Process, Vol. 9, No.

Figure
Figure 1 -Randomization procedure square meters (m2), occupied by the vehicle in the boarding area on ships or aircraft, corresponding to the product of the length by the width.Here the Stowage Breakage Factor is not considered, only the area effectively occupied by the vehicle.The smaller this measure, the better for boarding and loading.Unladen weight Weight, in kilograms (kg), of the unladen vehicle.The smaller this measure, the better for loading and unloading of ships or aircraft.Turning radius Measure, in meters (m), of the smallest space needed for the vehicle to make a change of direction (180 degrees).The smaller this measure, the better for loading and unloading of ships or aircraft.by the vehicle, measured in kilograms (kg).The greater this measure, the better mobility and tactical support in military operations.Maximum speed Maximum speed reached by the vehicle, measured in km/h.The greater this measure, the better mobility and tactical support in military operations.Fuel capacity Fuel tank capacity, in liters (l).The greater this measure, the better mobility and tactical support in military operations.Capacity to override obstacles Fording Measured in meters (m), representing the vehicle's transposition limit of a watercourse.The greater this measure, the better the overcoming of obstacles in that indicates the limit of the angle of attack of a vehicle on a ramp.The greater this measure, the better the overcoming of obstacles in military operations.Angle of departure Measure in degrees that indicates the limit of the departure angle of a vehicle on a ramp.The greater this measure, the better the overcoming of obstacles in military operations.Side slope Measure in percentage (%) that indicates the limit of lateral inclination of a ramp for safe arrangement or parking of the vehicle.The greater this measure, the better the overcoming of obstacles in military operations.Gradient Measure in percentage (%) that indicates the limit of frontal inclination of a ramp for safe displacement or parking of the vehicle.The greater this measure, the better the overcoming of obstacles in military operations.

Table 8 -
results and rankings