Technological Institute of Aeronautics – ITA, São José dos Campos, São Paulo, Brazil.
Technological Institute of Aeronautics – ITA, São José dos Campos, São Paulo, Brazil.
Goal: The purpose of this paper is to build the structure of a multicriteria decision model that supports definition and prioritization of strategic initiatives by an institution performing in the prevention of natural disasters.
Design/Methodology/Approach: The Multicriteria Decision Analysis (MCDA) methodological process is adopted to build a multicriteria evaluation model. A two-phase process is followed, employing a top-down approach, based on Value-Focused Thinking (VFT), in combination with the Multi Attribute Value Theory (MAVT) method. The participants of the MCDA process were experts of the control room in the case study institution.
Results: The proposed methodology not only helped decision makers to enumerate a number of strategic initiatives to accomplish the organizational objective, but also helped them to establish a structured procedure to prioritize these initiatives.
Limitations: Absence of a criterion related to the organization budget in the model; limited scope of participation in the process; absence of quantitative criteria.
Practical implications: The major practical contributions are as follows: an structured model that supports the strategic planning process; a better allocation of resources (human, financial, and materials) in projects that are truly aligned with the strategic objective of the organization; the organizational learning coming from the exercise of reflection on values, objectives and preferences; and the legitimacy of decisions as a result of the participative character of the construction process of the model.
Originality/Value: In this study, a multicriteria evaluation model is structured and applied as support for strategic decision making in the context of a natural disaster early warning system. The model has a significant application potential, since it encourages the adoption of structured decision support methods rather than traditional empirical decision making. Thus, the value of the study lies in the contribution that the proposed model can offer to more effective disaster prevention.
Keywords: Disaster prevention; Early warning system; Value-Focused Thinking; Multicriteria decision analysis.
In the last decade, the world witnessed a series of natural disasters which had devastating consequences for the population and led to economic losses for the affected countries. Among them, it can be cited the earthquake in Haiti in 2010; the tsunami in northeastern Japan in 2011, which also reached the Fukushima nuclear power plant; the Sandy storm on the east coast of the USA in 2012; and the typhoon in the Philippines in 2013.
In Brazil, most of the natural hazards with risk of disaster are related to climatic factors. Generally, in rainy periods, there are floods and landslides. On the other hand, risk of droughts and forest fires increase during dry period. In the last decade, the country was hit by more than 40 natural disasters of hydro meteorological origin, in a total accumulated of approximately 1,700 dead and 36 million people affected, besides economic damages in the order of 10 billion dollars (EM-DAT, 2018).
The worst natural disaster ever recorded in Brazil was in the mountainous region of Rio de Janeiro in 2011, when a series of landslides and floods killed about 900 people. Since then, it has become indispensable for Brazil to have a national strategy for managing and reducing the risk of natural disasters. Among the initiatives coming from this strategy is the implementation of a national system for monitoring and issuing warnings of natural disasters in 2011.
Thus, the Brazilian Center for Monitoring and Early Warning of Natural Disasters (Centro Nacional de Monitoramento e Alertas de Desastres Naturais – Cemaden) was created in December 2011. It is a federal organization whose mission is to conduct research and technological innovations that contribute to the improvement of its early warning system and monitor natural threats in Brazilian municipalities with risk areas susceptible to the occurrence of natural disasters.
The Federal Government Program 2040 - Management of Risk and Disasters, present in the 2016-2019 Pluriannual Plan (Plano Plurianual - PPA), has as one of its objectives to increase the effectiveness of natural disaster warnings whose responsibility is attributed to Cemaden. Therefore, it is important for the Center to have means to assist it by defining and prioritizing the actions to be taken in order to achieve its long-term strategic objective.
Thus, the following research question arises: How to decide which initiatives are strategic to achieve the organizational objective and how to establish means to prioritize them?
The literature presents several studies about the application of multicriteria methods for prioritization of alternatives in several decision contexts. Miyamoto et al. (2014) use the AHP (Analytic Hierarchy Process) multicriteria method with the SWOT (Strength, Weakness, Opportunity, and Threat) matrix to prioritize interventions that experts believe are necessary to implement a flood warning system in Bangladesh. Ferretti and Comino (2015) use the multicriteria MAVT (Multi-Attribute Value Theory) method to aid in the planning and definition of sustainable solutions for public heritage management. Van den Honert (2016) uses a multi-attribute value model to develop a decision-making framework for allocation and selection of Australian government funds to mitigate natural disasters and reduce risks. Vieira et al. (2017) use the AHP method to evaluate prospective scenarios in the context of corporate social responsibility. Santis et al. (2017) use a decision model based on the AHP Fuzzy method for maintenance supplier selection in a large Brazilian railway operator.
Thus, this work proposes to structure a multicriteria decision model that supports definition and prioritization of strategic initiatives that can contribute to the accomplishment of the Cemaden’s organizational objective. The present study is an extended version of the paper entitled “Evaluation of Strategic Initiatives with MCDA for Issuing Natural Disaster Alerts” (Silva and Belderrain, 2019), which was presented at the Production and Operations Management Society (POMS) Conference, in 2018.
The research process is divided into two phases: in the first phase (structuring the problem), a top-down approach, based on VFT (Value-Focused Thinking), is used to support decision makers in identifying relevant attributes and alternatives by considering the values and objectives of the institution; in the second phase (structuring the decision model), an MCDA (Multicriteria Decision Analysis) methodological process, based on MAVT, is adopted to build a multicriteria evaluation model. MAVT is used in this phase to support decision makers in determining preferences and evaluating decision alternatives.
The major practical contributions of this paper are stated as follows: the construction of a structured model that supports the strategic planning process; a better allocation of resources (human, financial, and materials) in projects that are truly aligned with the strategic objective of the organization; the organizational learning coming from the exercise of reflection on values, objectives, and preferences; and the legitimacy of decisions as a result of the participative character in the process of the model construction. Moreover, the model has a significant application potential, since it encourages the adoption of structured decision support methods rather than traditional empirical decision making. Thus, the value of the study lies in the contribution that the proposed model can offer to more effective disaster prevention.
Finally, this paper is organized in seven sections. Following this Introduction, in Section 2, it is provided a literature review about Early Warning Systems, VFT method, MCDA and MAVT. In Section 3, the methodology used in the research and the procedures for data collection are presented. Next, in Section 4, it is described the context (institution description) and the method application, which is divided into two phases: structuring the problem and structuring the decision model. In Sections 5 and 6 the results and a brief discussion about them are presented respectively. Finally, Section 7 concludes this paper with some recommendations for future improvements.
The following subsections present the theoretical framework on which the development of this work is based.
There is no consensus on common terminology for disaster. Therefore, this work adopts the United Nations Office for Disaster Risk Reduction (UNISDR) definition: disaster is a "serious disruption of the functioning of a community or a society at any scale due to hazardous events interacting with conditions of exposure, vulnerability and capacity, leading to one or more of the following: human, material, economic, and environmental losses and impacts" (UNISDR 2017, online). A disaster may occur at a small or large scale and can be classified as slow-onset (hazardous event emerges gradually over time) or sudden-onset (hazardous event emerges quickly or unexpectedly).
An important part of a holistic approach to natural disaster risk management is the implementation of Early Warning Systems (EWS) (Alfieri et al., 2012). According to an internationally accepted terminology, EWSs are integrated disaster risk monitoring, forecasting and assessment systems, which also cover activities and processes to communicate and prevent adverse events, enabling those exposed to them to take timely and early action to reduce disaster risks (UNISDR, 2017). Thus, the successful functioning of an EWS depends on the effective alignment of the four elements composing it: (1) risk knowledge, based on systematic data collection and risk assessment; (2) detection, monitoring, analysis, and prediction of hazards and possible consequences; (3) dissemination and communication by official sources of authorized, timely, accurate and actionable alerts; and (4) preparation at all levels to respond to received alerts (UNISDR, 2006).
The relevance of EWS is emphasized in guidelines and priorities of international agreements among countries in a joint effort to reduce the risk of disasters in the world. Among these agreements, the best-known ones are the Hyogo Framework for Action 2005-2015 for disaster risk reduction (UNISDR, 2005) and the Sendai Framework for Disaster Risk Reduction 2015-2030 (UNISDR, 2015). The former has as one of its priorities for disaster risk reduction the identification, assessment, and monitoring of risks and the improvement of EWS. The latter highlights the importance of investing, developing, maintaining, and strengthening people-centered multi-hazard EWS (UNISDR, 2015), in order to enhance disaster preparedness for an effective response.
Thus, EWS comprise an integrated and coordinated action of multiple mechanisms, capabilities and stakeholders in diverse levels. Alfieri et al. (2012) advocate that an appropriate integration of EWS in national and local levels requires not only that the information be accessible and understandable for all stakeholders. It is necessary that resources for risk reduction actions be available and that there are management plans for crisis situations. In addition, major efforts need to be devoted to train all stakeholders (including population) on emergency procedures and on security measures.
From 1989 (year of publication of the first edition of Jonathan Rosenhead's book: Rational Analysis for a Problematic World) to today, the Problem Structuring Methods (PSM) are becoming increasingly popular. The best known of them are SODA (Strategic Options Development and Analysis) (Eden, 1988), SSM (Soft Systems Methodology) (Checkland and Tsouvalis, 1997), and SCA (Strategic Choice Approach) (Friend and Hickling, 1987).
PSM is a fundamentally analytical approach to Soft Operational Research, where the participatory process is its key component (Rosenhead, 2001; Rosenhead, 2006). Its purpose is to support the understanding and structuring of complex and poorly structured decision problems (Rosenhead, 1996), using alternative methods to conventional rational analysis (Rosenhead, 2001).
Simpson and Hancock (2009) argue that PSMs are intuitively adequate to capture insights into the highly dynamic and wicked nature of problems arising from an emergency response context. Amideo et al. (2019), when discussing optimization models for shelter location in disasters situations, argue that identification and involvement of stakeholders can be achieved through PSMs. Thus, such methods can be used to solve route evacuation planning problems. Kovacs and Moshtari (2019), when discussing methodological aspects of studies on humanitarian operations, point out the importance of including the structuring of problems in the conduction of research in this area, since the humanitarian context contains complex problems, whose formulations and/or resolutions are difficult, in addition to a large number of humanitarian organizations with different and often conflicting objectives.
Giordano et al. (2013) employ a methodology based in SODA to collect, structure, and analyze the stakeholder’s perceptions on drought impacts, and, thereby, build a set of risk indicators based on local knowledge and physical indicators. The ultimate goal is to support the design of a people-centered drought monitoring system. SSM is employed by Walker et al. (2014) as a collaborative situational analysis tool for disaster recovery projects, and by Wienke and Mustafee (2015) for the development of conceptual models in the context of disaster operations management (DOM) and emergency operations research (EOR). Cerqua and Rapicetta (2014) discuss the importance of involving the community and other stakeholders in the phases of mitigation and recovery of the disaster management cycle. To achieve this, they propose using SCA as a participatory approach. The primary goal is to analyze the problems arising from improper urban planning in risk-prone areas and suggest improvements in the disaster management cycle.
Keeney (1992) indicates the Value-Focused Thinking (VFT) as a potential PSM, as can be demonstrated in the studies of Morais et al. (2013), Keisler et al. (2014), and Bernardo et al. (2018). With a top-down and value-focused approach, VFT is a method that seeks to identify the values that a decision maker should use as a basis for decision-making. Thus, VFT can be seen as a path for identifying decision opportunities, and for guiding strategic thinking and collection of information.
Values are what really matters. They are the guiding force in the decision-making process in the VFT approach. In essence, values mean “purposes, desires, ends, “what is important”, “what is of concern”, “what satisfies”- in short, and what the person wants to achieve through the decision” (Leon, 1999, p. 214). According to Keeney (1992), the general principle of thinking about values is to discover the reason for each objective and how it relates to other objectives.
VFT basically consists of two activities: deciding what one wants and finding out how to reach it. Therefore, the work is guided towards the determination of values involved in the decision-making process and the generation of alternatives, respectively (Hammond and Keeney, 1999).
The objectives represent a statement of what is needed to be achieved. They are also characterized by a decision-making context, an object and a direction of preference (Almeida, 2013). The VFT distinguishes “fundamental objectives” from “means objectives”. Fundamental objectives in a decision-making context are the ends that the decision makers wish for. Means objectives represent the way to achieve these ends. For strategic decisions, fundamental objectives are defined as strategic objectives. Strategic objectives are those fundamental ones that guide organizational decisions and form the basis for more detailed fundamental objectives appropriate for specific decisions (Keeney 1996).
Nevertheless, “the challenge is to distinguish between objectives that are means to an end (having leather seats in your new car) and those that are ends in themselves (having a comfortable and attractive interior)” (Hammond et al., 1999). One possible way is to use the WITI test ("Why Is That Important?"). If the answer characterizes a reason for interest in the decision-making, it is a fundamental objective. If the answer is important as a way to reach another objective, it is a mean objective. Thus, we move from the means objectives to fundamental ones. Another way is to go through the reverse path, that is, move from the fundamental objectives to means ones. In this case, the following question must be asked: "How can this objective be achieved?" (Keeney, 1992).
Identifying objectives requires creativity and hard thinking (Keeney, 1992) in a decision-making situation. In order to guide this process, Keeney (1992) suggests some techniques that can stimulate the identification of possible objectives: prepare a wish list, consider problems and weaknesses, predict consequences, identify goals, constraints, and directions, and consider different perspectives, etc. Finally, it has recommended constructing a means-ends objectives network.
The process of structuring objectives results in a more precise and in-depth understanding of what is important in the decision-making context. If someone continues to structure a means-ends objectives network to lower levels, eventually they will get alternatives (the raw material of decision making (Hammond et al., 1999)). Thus, with a means-ends objectives network, the natural breakpoint of the specification is alternatives or classes of alternatives. They represent the range of potential choices the decision-makers will have for pursuing their objectives (Leon, 1999; Hammond et al., 1999).
Keeney (2012) demonstrates an application of the VFT, combined with traditional brainstorming (a hybrid version called Value-Focused Brainstorming), to the definition of public policies, aiming to improve the emergency evacuation of large buildings, based on the World Trade Center disaster. Other VFT applications can be found in Morais et al. (2013) (water management, information system/information technology (IS/IT) in the support of business strategies, and disposal of plaster waste), in Nachtmann and Pohl (2013) (transportation readiness in emergency operations planning), in Lienert et al. (2015) (sustainable water infrastructure planning), and in Paiva and Daher (2016) (improving productivity and sustainability in industries). Ultimately, a comprehensive review of VFT applications can be found in Parnell et al. (2013).
Multicriteria Decision Analysis (MCDA) is a methodology used to support decision making when multiple, and often conflicting, objectives are present. This approach should not be viewed as a tool for solving problems, but rather as a process (Belton and Stewart, 2010). According to Gomes et al. (2004), the multicriteria methods have a scientific character and they are able to aggregate multiple quantitative and qualitative characteristics in the same model. In this way, MCDA methods make it possible to systematize the process referring to decision-making problems.
In the classification of the MCDA methodology, two groups of discrete methods can be highlighted: the aggregation methods using additive models and the outranking methods.
The aggregation methods using additive models are based on the hypothesis in which the decision maker is able to identify several discrete alternatives to be compared, and to structure hierarchically the criteria by which the alternatives will be evaluated (Gomes et al., 2004). Examples of methods that make up this group are: MAVT (Multi-Attribute Value Theory), MAUT (Multi-attribute Utility Theory), Macbeth (Measuring Attractiveness by a Categorical Based Evaluation Technique) and AHP (Analytic Hierarchy Process).
The outranking methods accept a more flexible model of the problem, since they do not suppose the comparison between alternatives and do not require that the criteria be structured in a hierarchical manner (Gomes et al., 2004). From this group, the most used methods are those of the Prométhée family (Preference Ranking Organization Method for Enrichment Evaluations) (Mareschal et al., 1984) and those of the Electre family (Elimination et Choix Traduisant la Réalité) (Roy, 1968).
The MAVT (method used in this work) uses value functions to help decision makers to numerically express their preferences in relation to each sub-objective. To aggregate the partial value functions and calculate the overall performance of the alternatives, the MAVT includes different aggregation models. The additive model, presented at Equation 1, is the simplest and the most used (Belton and Stewart, 2002).
In which,
V(a): Overall performance value of alternative a;
wi: Weight of the i-th attribute, also called substitution rate;
vi (a): Partial value function representing the performance of alternative a in the attribute i;
n: Number of attributes of the model.
According to Equation 1, the MAVT associates an overall performance value V(a) with each alternative a. V(a) is the result of the sum of the local performance of A in each attribute. The local performance, in turn, is the result of the product of the partial value function vi(a), representing the performance of alternative a in the attribute i, by the weight of the attribute i.
The main requirement for employing the additive value function is the preferential independence between the attributes. Two attributes i and j are preference-independent if the substitution rates between i and j are independent of all other attributes (Ferretti, 2016).
Ferretti (2016) lists a number of advantages of the MAVT in relation to other multicriteria methods: it presents justifiable results, since they are based on fundamental axioms of rational analysis; it is able to handle a large number of alternatives without an increase in the elicitation effort when compared to a smaller number; and it allows both quantitative and qualitative parameters to be assessed, thus playing a crucial role in environmental decision-making and in policy-making where many aspects are often intangible.
Regardless of the method employed, the MCDA process comprises a sequence of phases, as shown in Figure 1.
Figure 1. The MCDA process
Source: Adapted from Belton and Stewart (2002)
In order to solve a multicriteria decision problem, in general, one must: (i) detect the problem to be solved; (ii) define the values, objectives and uncertainties through the process of structuring the problem; (iii) specify the alternatives and define the criteria together with the decision makers; (iv) apply an analytical model to evaluate and select alternatives; (v) and finally develop an action plan. As the process is cyclical, at each step it is possible to go back to the previous phase to make changes in the model, if necessary.
There is extensive literature on the use of MCDA, and more specifically MAVT, in various disciplines. The ensemble of applications ranges from decision processes in public policy making (Ferretti, 2016) to health technology assessment (Angelis and Kanavos, 2016). Discussions involving MCDA in disaster management applications can be found in Peng (2015), as it integrates the results of different multicriteria decision support methods to provide a regional earthquake vulnerability assessment, in Chitsaz and Banihabibi (2015), which employ MCDA for dealing with complex issues of prioritization within the context of flood management, and in Han and Deng (2018), who use MCDA to identify critical success factors in high-risk emergency systems. Marttunen et al. (2017) present a literature review of the main studies involving the combination of problem structuring methods with decision aiding methods. These authors conclude that the PSM-MCDA combination produces a richer view of the decision situation and allows a more effective support for the different phases of the decision-making process.
The approach adopted is a qualitative research, because it was considered the values and preferences that influence decision-making and that cannot be numerically quantified. From the technical procedure point of view, this work is a case study, of empirical character, with the goal of investigating a current phenomenon in a specific context in real life (Miguel et al., 2010). The study took place at Brazilian Center for Monitoring and Early Warning of Natural Disasters (Centro Nacional de Monitoramento e Alertas de Desastres Naturais – Cemaden), an organization of the Brazilian Federal Government responsible for the prevention of natural disasters, located in São José dos Campos, São Paulo, Brazil. Data collection happened through the study of documents, interviews and consultation with experts of the Cemaden’s control room.
The proposed method for solving the problem of prioritization strategic initiatives to meet the strategic objective of Cemaden is composed of two phases:
Phase 1 - Structuring the problem: a top-down approach, based on VFT, was used in this phase to support decision makers in identifying relevant attributes and alternatives by considering the values and the objectives of the institution. In this phase, data collection happened throughout the month of May 2018, mainly through the study of documents, and was supplemented with expert consultations. VFT was chosen due to its philosophy of support decision making, where the focus is on thinking first about values and objectives, not alternatives, at the risk of causing conflicts of decision implementation, by not include some of the key concerns of stakeholders. Several studies have revealed the significant contribution of VFT in the field of decision analysis, as pointed out in the study by Parnell et al. (2013). These authors argue that the VFT-MAVT combinations are quite common in the literature, since VFT provides the chance to use values not only to create better decision alternatives, but also to evaluate them. Thus, in the present work, the VFT is applied with the MAVT to elucidate the objectives of delivering more effective natural disasters warnings (with regard to prevention actions), to create alternative solutions and to evaluate them by means of a multi-attribute value function built by stakeholders.
Phase 2 - Structuring the decision model: an MCDA methodological process, based on MAVT, was adopted for building a multi-criteria evaluation model. MAVT was used in this phase to support decision makers in determining preferences and evaluating decision alternatives. In this phase, data collection and analysis happened through interviews with experts in the first two weeks of July 2018, and the decision support software Visual Interactive Sensitivity Analysis (V.I.S.A.) was used. MAVT was chosen because it is a conceptually simple approach, quite understandable even for laymen, theoretically well-grounded and well validated in a large number of real cases. This provides a good understanding of how to conduct a decision-making process using this method (Marttunen et al., 2015). Moreover, since MAVT is based on fundamental axioms of rational analysis, its results are justifiable, which is vital for strategic and/or political decisions (Ferretti and Comino, 2015). The option for the V.I.S.A. software was mainly due to its nature: a multi-criteria decision support system, based on a multi-attribute value function. Another reason is "its extensive facility for visual interactive sensitivity analysis (hence the name), which enables decision makers to explore the implications of changing or differing priorities and values", as pointed out by Belton et al. (1997, p. 118).
The following subsections present the intervention context (institution description), as well as a detailed step-by-step application of the proposed method.
Cemaden, established by federal decree in July 2011, was created as part of the National Strategy for Natural Disaster Management in Brazil. It is a Science and Technology Institute and it is tied to the Brazilian Ministry of Science, Technology, Innovation, and Communications (Ministério da Ciência, Tecnologia, Inovações e Comunicações - MCTIC). Cemaden’s mission is to monitor natural threats in Brazilian municipalities with risk areas susceptible to the occurrence of natural disasters, and conduct research and technological innovations that contribute to the improvement of its early warning system.
Cemaden has an observational network composed of rainfall gauges, radars, and hydrological stations, among others, which constantly provide environmental data for its database. This database is supplemented by different types of data obtained from other government agencies and research institutes, with which the center maintains a partnership. The continuous monitoring of this database (24h a day) allows the identification of imminent risk situations that can trigger the issuance of a natural disaster warning. Currently, Cemaden issues warnings of hydrometeorological nature, including landslides, mudslides, floods and flash floods for 958 municipalities monitored.
The decision whether or not to issue a warning is taken by a multidisciplinary team of seven to eight members that include geologists, hydrologists, meteorologists, and natural disaster specialists. The first three are responsible for analyzing the technical aspects of the risk, while the natural disaster specialist is responsible for analyzing the environmental and socio-economic impact of the potential disaster.
Cemaden’s warnings are sent to the National Center for Risk and Disaster Management (Centro Nacional de Gerenciamento de Riscos e Desastres - CENAD), which is responsible for forwarding the warnings to the State and Municipal Civil Defenses. These agencies initiate preparatory actions for an imminent disaster, such as the mobilization of residents for evacuation, as well as activate response actions, such as victim’s relief operations.
Phase 1 – Structuring the problem
The process of applying the VFT for this work began by structuring the values and identifying the objectives of the organization. For this purpose, official documents of the Federal Government and internal documents of the organization were analyzed, which allowed identifying the values of the organization and converting them into objectives. Next, the objectives were classified and related to each other using the tests “Why is that important?” and “How can this objective be achieved?” Thus, the means-ends objectives network was constructed, as illustrated in Figure 2.
Figure 2. A Means-ends objectives network for Cemaden
Source: The authors themselves
The organization’s strategic objective ("Increasing the effectiveness of natural disaster warnings") is situated on the left of the network, while the fundamental objectives are located in the center (gray area). Table 1 presents the hierarchy among the fundamental objectives. These are the objectives that, if met, can lead to the accomplishment of the strategic objective.
Table 1. Hierarchy of fundamental objectives
Finally, on the right of the network one can see the means objectives, which are translated into ways of achieving the fundamental objectives. To measure the impact of adopting each decision alternative on the overall objective or value concern, performance descriptors, known as attributes, were defined and later validated by the decision makers, as shown in Table 2.
The participants selected to validate the attributes construct the decision model (considered decision makers) and evaluate the decision alternatives were the experts of Cemaden’s control room. The reason for this choice is that they represent the Players, i.e. the stakeholders with the highest power and the highest interest in decision-making, according to the Power and Interest Grid from Ackermann and Eden (2011). Four experts working in the control room were involved in the method application and the choice of these experts was random, according to their availability and interest.
Table 2. Description of attributes
Ferretti and Comino (2015) suggest the definition of a panel of experts to evaluate attributes of a multicriteria model because the use of this panel expands the knowledge bases and avoids possible biases. Since such stakeholders are the experts of each area involved in the decision-making process of the alert (hydrology, geology, meteorology, and natural disasters), the approach of Ferretti and Comino (2015) corroborates their choice.
The last step of the VFT approach was to stimulate the creation of decision alternatives, through open discussion and reflection based on the values of the organization. In the context of this intervention, the alternatives are translated in the form of initiatives that can contribute to the achievement of the fundamental objectives and, consequently, increase the effectiveness of natural disaster warnings. As many alternatives were generated (some of them obvious; others redundant), decision makers chose the most strategic alternatives, according to their judgment value. Thus, the decision alternatives agreed for phase 2 were:
A1: Promoting seminars and workshops with the scientific community and the Civil Defense;
A2: Implementing a Project Office dedicated to the management of research, development and innovation programs and projects;
A3: Developing flood forecasting tools using nowcasting;
A4: Implementing a mobile application with features of crowdsourcing in order to offer a channel of communication with society and operate as a data collector;
A5: Defining the performance indicators for alerts;
A6: Creating national networks for scientific and technological cooperation in disaster management;
A7: Operating the product of satellite precipitation estimates derived from NASA's Global Precipitation Measurement Mission.
Phase 2 – Structuring the decision model
The first step in constructing the decision model is to structure all the attributes in the form of an organized schematic representation known as value tree.
The hierarchy of fundamental objectives presented in Table 1 highlighted two main areas of interest: continuous improvement and technological innovation. Decision makers believe that investments in these two areas can increase the effectiveness of natural disaster warnings. In the value tree shown in Figure 3 these areas are presented as the criteria of the model. The attributes identified by the VFT are situated at the bottom of the value tree and they measure the fulfillment of the criteria. At the top of the tree is the organizational objective which is being pursued.
Figure 3. Value tree
Source: The authors themselves
The properties that the criteria and attributes of the value tree must satisfy to ensure a robust process were validated with the decision makers in order to maintain the rigor of the method. Thus, it is desirable that both criteria and attributes be unambiguous, comprehensive, direct, operational, understandable, and preference-independent (Keeney and Gregory, 2005; Franco and Montibeller, 2010).
As the attributes presented are all from a qualitative nature, the method known as Direct Rating was used to obtain the value functions. In this case, the decision makers define and characterize the levels of attributes directly. As suggested in Ensslin et al. (2001), the worst acceptable situation (N1 - the alternative makes a negligible contribution to the achievement of the objective) was associated with the value 0 of the scale, while the best feasible situation (N5 - the alternative makes a very high contribution to the achievement of the objective) was associated with the value 100. Both levels represent the lower and upper limits of the variation of each attribute and serve as anchor to evaluate the relative value of the other levels. Table 3 presents the description of the levels, which were used in a common way by all attributes of the model, and their respective scales converted into value functions.
Table 3. Description of attribute levels and value functions
Once the value function associated to a descriptor has been determined, an evaluation (sub) criterion was constructed, thus allowing the measurement of the performance of the actions according to a particular evaluation pillar (Ensslin et al., 2001). Therefore, Graph 1 presents a partial evaluation of the alternatives (A1, A2, A3, A4, etc.) to each attribute (from now on referred to as subcriteria). The values presented in Graph 1 are obtained as follows: decision makers select alternative A1 and, by consensus, evaluate their performance in relation to subcriteria (Performance management, Risk knowledge, Models and systems, Partnerships, and Applied research), based on the value functions presented in Table 3, that is, an evaluation between N1 and N5, on a discrete scale. This process is repeated for the other alternatives.
Graph 1. Partial evaluation of the alternatives to each subcriterion
Source: The authors themselves
After that, substitution rates of the multicriteria model were elicited, in order to allow a later aggregation of all the evaluation dimensions.
Also referred to as trade-offs or scale constants, substitution rates are measures that express the loss of performance that a potential action must take in one criterion to compensate for the performance gain in another (Ensslin et al., 2001). Popularly, substitution rates are known as weights.
The method used to determine the weights presented in the Chart 1 was the Swing Weights, based on the compensation concept. Following the step-by-step described in Ensslin et al. (2001), a fictitious action is created with neutral level in all (sub) criteria, and then the decision makers have the opportunity to improve the performance of this action, from neutral to good, in only one of the (sub) criteria. The (sub) criterion chosen receives a jump (swing) of 100. The same procedure is repeated for the other (sub) criteria, in the order of preference of the decision makers, with the value of the swing evaluated in relation to that (sub) criterion that received the value 100. The weights are obtained from the normalization of these values.
Taking the group of subcriteria Models and systems, Partnerships and Applied research as an example, the values presented in Chart 1 were obtained as follows. First of all, the main author of this study, called facilitator, asked the decision makers to think of a fictitious action that can affect organization strategic objective. After, the participants had to choose to improve the performance of this action, from neutral to good, in only one of the subcriteria. The chosen subcriterion was Models and systems, which received a swing of 100 points. The facilitator then asked the decision makers to choose the following subcriterion to receive a swing, from neutral to good. In addition, they had to evaluate the value of this swing in relation to the first one. Decision makers judged that the jump should occur in Applied Research subcriterion, and that this should be worth 80 points. Finally, participants evaluated the value of the jump for Partnerships subcriterion (also in relation to the first one), which received a swing of 40 points. The last step was to normalize these values, so that their sum was equal to 1. This is calculated by dividing the value of the swing by the sum of all points. Thus, the subcriteria weights were obtained as demonstrated below:
It is worth clarifying that the order of choice of subcriteria, as well as the swing values, were defined on a consensual basis by the group of decision makers.
Chart 1. Weights of the criteria and subcriteria
Source: The authors themselves
After a partial evaluation of the alternatives to each subcriterion and definition of weights of the criteria and subcriteria, the information was inserted in the V.I.S.A. software for mathematical modeling.
In Figures 4 and 5, it is possible to observe the partial performance of each alternative in the Continuous improvement and Technological innovation criteria, respectively.
In Figure 4, the alternative A4 (Implementing a mobile application with features of crowdsourcing in order to offer a channel of communication with society and operate as a data collector) was the one that obtained the highest score, which, in this case was 92 (the number was shown in the left of the alternative label). This score is calculated based on an additive model, represented by the following equation:
In which,
vi (A): Partial value function representing the performance of alternative A in the criterion i;
wj: Weight of the j-th subcriterion;
vj (A): Partial value function representing the performance of alternative A in the subcriterion j;
n: Number of subcriteria of the criterion i.
Thus, based on Equation 2, the score of the alternative A4 in relation to the Continuous improvement (CI) criterion was calculated as follows:
w1: Weight assigned to subcriterion Performance management (Chart 1);
w2: Weight assigned to subcriterion Risk knowledge (Chart 1);
v1 (A4): Value function of A4 in the subcriterion Performance management (Graph 1);
v2 (A4): Value function of A4 in the subcriterion Risk knowledge (Graph 1).
Therefore,
Figure 4. Partial performance of the alternatives in Continuous improvement
Source: The authors themselves
Figure 5 shows the partial performance of the alternatives in relation to the Technological innovation (IT) criterion. For this criterion, the alternative better evaluated by the decision makers was A2 (Implementing a Project Office dedicated to the management of research, development and innovation programs, and projects). The score of A2 was obtained as described below:
w1: Weight assigned to subcriterion Models and systems (Chart 1);
w2: Weight assigned to subcriterion Partnerships (Chart 1);
w3: Weight assigned to subcriterion Applied research (Chart 1);
v1 (A2): Value function of A2 in the subcriterion Models and systems (Graph 1);
v2 (A2): Value function of A2 in the subcriterion Partnerships (Graph 1);
v3 (A2): Value function of A2 in the subcriterion Applied research (Graph 1).
Therefore, based on the Equation 2:
Figure 5. Partial performance of the alternatives in Technological Innovation
Source: The authors themselves
Figure 6 illustrates the final performance of decision alternatives in relation to the main objective. From the aggregation of partial performances, the overall performance of the alternatives in the multicriteria model was obtained, resulting in the final ranking presented in Table 4. The final score of A4 was obtained from the aggregate additive model presented in Equation 1, and is detailed as follows:
w1: Weight assigned to criterion Continuous improvement (CI) (Chart 1);
w2: Weight assigned to criterion Technological innovation (TI) (Chart 1);
vCI (A4): Partial value function representing the performance of A4 in the criterion CI (Figure 4);
vTI (A4): Partial value function representing the performance of A4 in the criterion TI (Figure 5).
Thus,
Figure 6. Performance of the alternatives in relation to the main objective
Source: The authors themselves
Table 4. Final ranking of alternatives
A4 was the most preferable alternative to the others (Implementing a mobile application with features of crowdsourcing in order to offer a channel of communication with society and operate as a data collector). Thus, this is the initiative that most contributes to the achievement of the strategic objective of the organization, according to the value judgment of the decision makers.
On the other hand, A2 was the least preferable alternative (that is, that received the lowest score - Implementing a Project Office dedicated to the management of research, development and innovation programs and projects).
The sensitivity analysis was performed as the last step of the process for evaluating the alternatives. One of the uncertainties of the decision-making process is the trade-offs related to the fundamental objectives. Thus, the sensitivity analysis consisted of performing slight controlled variations in the weights of the criteria (X-axis) and observing possible changes in its final result (Y-axis).
Figure 7 illustrates the performance of the alternatives as a function of variations in the weight of the Continuous improvement criterion (the dotted line represents the weight originally defined by decision makers).
Figure 7. Sensitivity analysis for Continuous improvement
Source: The authors themselves
From the analysis of the graph presented in Figure 7, it can be seen that as the weight value of criterion increases, the overall evaluation of A1, A4 and A7 improves significantly (represented by the red, blue, and black lines, respectively), whereas the overall evaluation of A2 drops to zero when the weight value of criterion reaches 1 (represented by the green line). This means that if Continuous improvement is the only criterion considered in the model, A2 would receive value zero in the overall evaluation. Variations around 10% above and below the weight current value of the criterion do not change its overall value, as the order of preference of the alternatives remains the same. Thus, it can be concluded that the model is robust in relation to the Continuous improvement parameter.
The sensitivity analysis was also performed for the Technological innovation criterion, as a function of the variation of its weight (Figure 8).
Figure 8. Sensitivity analysis for Technological innovation
Source: The authors themselves
The analysis of the graph in Figure 8 shows that, unlike what occurred in the previous criterion, as the weight of the Technological innovation criterion increases, the overall performance of A2 improves substantially. A1, A4, and A7 also present an inverse behavior: higher weights for the Technological innovation criterion generate lower performances for these alternatives.
Thus, it is concluded that the model is robust in relation to this parameter, since variations around 10% above and below the weight current value of the criterion do not change the final result.
The scoring of alternatives, obtained by applying the multicriteria model, allows quantifying the degree of contribution of each initiative to the organization's strategic objective. Thus, this result may justify the definition of actions that will be a priority in the strategic planning of the institution.
Of the seven alternatives, four of them were considered the most important because of the high scores they obtained in the overall evaluation.
The best evaluated alternative, A4 (Mobile application with characteristics of crowdsourcing capable of offering a channel of communication with society, acting as a data collector), has its importance justified in three aspects: first, a mobile application with characteristics of crowdsourcing can provide real-time field information capable of supporting risk analysis; secondly, it can act as a mechanism for the community to participate in the disaster risk reduction efforts; and thirdly, it can be used for sharing local knowledge, and thus improve an eventual limited access to data.
The second alternative in the ranking, A3 (Development of flood forecasting tools using nowcasting), is essential insofar as forecasting tools using nowcasting can reduce the warning issuance time and, consequently, enable early action by civil defense agents. An effective and timely response action is able to substantially reduce the number of fatalities.
The third best-evaluated alternative, A1 (Promotion of seminars and workshops with the scientific community and civil defense), is important for the organization, since seminars and workshops are a space for discussions among practitioners and scientists about recommendations for the strengthening of early warning systems. These events also open space for reflections on how prediction of geo-hydrological risk can help actions for preparation and prevention of natural disasters.
Finally, the fourth alternative in the overall performance ranking, A6 (Creation of national networks for scientific and technological cooperation in disaster management), is essential, according to experts, as it collaborates to the development of interdisciplinary and interinstitutional research. It is an opportunity to deepen knowledge on risks, vulnerabilities, and impacts of natural disasters.
The aim of this paper is to structure a multicriteria decision model that supports definition and prioritization of strategic initiatives that can contribute to the accomplishment of the Cemaden’s organizational objective. Cemaden is a Science and Technology Institute whose main activities are to monitor natural threats and issue early warnings for Brazilian municipalities with risk areas susceptible to the occurrence of natural disasters.
The research process was divided into two phases and it had as a scope the participation of experts from the Cemaden’s control room, mentioned throughout this work as decision makers.
The first phase comprised the structuring of the problem with the VFT method. The means-ends objectives network was constructed from the study of documents and was supplemented with expert consultations. At the end of this phase, the decision makers were able to identify relevant attributes and alternatives by considering the values and the objectives of the institution.
The second phase comprised the structuring of the decision model and the application of the MAVT method. Data collection and analysis happened through interviews with decision makers and the V.I.S.A. software which was used for the mathematical modeling step. At the end of this phase, decision makers were able to determine preferences, evaluate decision alternatives and define priority initiatives for the organization strategic objective.
In the context of the problem addressed, the VFT was chosen in combination with the MAVT, since there are arguments that a multicriteria decision analysis should employ to a value-based decision making, where alternatives must be seen as mere devices for organizations to achieve their objectives (Franco and Montibeller, 2010).
The proposed methodology not only helped decision makers to enumerate a number of strategic initiatives to accomplish the organizational objective, but also helped them establish a structured procedure to prioritize these initiatives. Thus, it is considered that the research questions that guided the development of this work were answered in a satisfactory way.
In short, in this study a multicriteria evaluation model is structured and applied as support for strategic decision making in the context of a natural disaster early warning system. The main contribution of the work is that the structured model can serve as a managerial tool for a better allocation of resources (human, financial, and material) in projects that are truly aligned with the strategic objective of the organization. Appropriate allocation of financial resources, which will really bring results, is vital for any institution in a limited public budget scenario. Moreover, the model has a significant application potential, since it encourages the adoption of structured decision support methods rather than traditional empirical decision making. Thus, the value of the study lies in the contribution that the proposed model can offer to more effective disaster prevention.
Finally, it is recognized that the present study has some limitations. Thus, the following future improvements are pointed out:
Ackermann, F. and Eden, C. (2011), “Strategic management of stakeholders: Theory and practice”, Long Range Planning, Vol. 44, No. 3, pp. 179-196.
Alfieri, L. et al. (2012), “Operational Early Warning Systems for Water-Related Hazards in Europe”, Environmental Science & Policy, Vol. 21, pp. 35-49.
Almeida, A. T. (2013), Processo de Decisão nas Organizações: Construindo Modelos de Decisão Multicritério, Atlas, São Paulo.
Amideo, A. E. et al. (2019), “Optimising shelter location and evacuation routing operations: The critical issues”, European Journal of Operational Research, Vol. 279, No. 2, pp. 279-295.
Angelis, A. and Kanavos, P. (2016), “Value-based assessment of new medical technologies: towards a robust methodological framework for the application of multiple criteria decision analysis in the context of health technology assessment”, Pharmacoeconomics, Vol. 34, No. 5, pp. 435-446.
Belton, V. and Stewart, T. J. (2002), Multiple Criteria Decision Analysis: An Integrated Approach, Kluver Academic Publishers, Massachusetts.
Belton, V. and Stewart, T. J. (2010), “Problem Structuring and MCDA”, in Ehrgott, J.; Figueira R.; Greco S. (Eds.), Trends in Multiple Criteria Decision Analysis, Springer, New York, pp. 209-239.
Belton, V. et al. (1997), “Integrated support from problem structuring through to alternative evaluation using COPE and V·I·S·A”, Journal of Multi‐Criteria Decision Analysis”, Vol. 6, No. 3, pp. 115-130.
Bernardo, H. et al. (2018), “A Combined Value Focused Thinking-Soft Systems Methodology Approach to Structure Decision Support for Energy Performance Assessment of School Buildings”, Sustainability, Vol. 10, No. 7, p. 2295.
Cerqua, A. and Rapicetta, S. (2014), “A Proposal to Improve the Disaster Management Cycle Model: The Importance of Community Participation”, in Bostenaru Dan M.; Armas I.; Goretti A. (Eds.), Earthquake Hazard Impact and Urban Planning. Environmental Hazards. Springer, Dordrecht, pp. 249-257.
Checkland, P. and Tsouvalis, C. (1997), “Reflecting on SSM: The link between root definitions and conceptual models”, Systems Research and Behavioral Science, Vol. 14, No. 3, pp. 153-168.
Chitsaz, N. and Banihabib, M. E. (2015), “Comparison of different multi criteria decision-making models in prioritizing flood management alternatives”, Water Resources Management, Vol. 29, No. 8, pp. 2503-2525.
Eden C. (1988), “Cognitive mapping”, European Journal of Operational Research, Vol. 36, No. 1, pp. 1-13. EM-DAT (The OFDA/CRED International Disaster Database) (2018), available from: http://www.em-dat.net/ (access 30 Nov. 2018).
Ensslin, L. et al. (2001), Apoio à Decisão: Metodologias para Estruturação de Problemas e Avaliação Multicritério de Alternativas, Insular, Florianópolis.
Ferretti, V. (2016), “From stakeholders analysis to cognitive mapping and Multi-Attribute Value Theory: An integrated approach for policy support”, European Journal of Operational Research, Vol. 253, No. 2, pp. 524-541.
Ferretti, V. and Comino, E. (2015), “An integrated framework to assess complex cultural and natural heritage systems with Multi-Attribute Value Theory”, Journal of Cultural Heritage, Vol.16, No. 5, pp. 688-697.
Franco, L. A. and Montibeller, G. (2010), “Problem Structuring for Multicriteria Decision Analysis Interventions”, in Cochran, J. J. (Ed.), Wiley Encyclopedia of Operations Research and Management Science, John Wiley and Sons.
Friend, J.K. and Hickling, A. (1987), Planning Under Pressure: The Strategic Choice Process, Elsevier, New York.
Giordano, R. et al. (2013). “Integration of local and scientific knowledge to support drought impact monitoring: Some hints from an Italian case study”, Natural hazards, Vol. 69, No. 1, pp.23-544.
Gomes, L. F. A. G. et al. (2004), Tomada de decisões em cenários complexos: introdução aos métodos discretos do apoio multicritério à decisão, Cengage Learning, São Paulo.
Hammond, J. S. and Kenney, R. L. (1999), “Making smart choices in engineering”, Journal IEEE Spectrum, Vol. 36, No. 11, pp. 71-76.
Hammond, J. S. et al. (1999), Smart choices: A practical guide to making better decisions, Boston, Massachusetts, Harvard Business Review Press.
Han, Y. and Deng, Y. (2018), “A hybrid intelligent model for assessment of critical success factors in high-risk emergency system”, Journal of Ambient Intelligence and Humanized Computing, Vol. 9, No. 6, pp. 1933-1953.
Keeney, R. L. (1992), Value Focused Thinking: A Path to Creative Decision making, Harvard University Press.
Keeney, R. L. (1996), “Value-focused thinking: Identifying decision opportunities and creating alternatives”, European Journal of operational Research, Vol. 92, No. 3, pp. 537-549.
Keeney, R. L. (2012), “Value-focused brainstorming”, Decision Analysis, Vol. 9, No. 4, pp. 303-313.
Keeney, R. L. and Gregory, R. S. (2005), “Selecting attributes to measure the achievement of objectives”, Operations Research, Vol. 53, No.1, pp. 1-11.
Keisler, J. et al. (2014), “Value-focused thinking for community-based organizations: objectives and acceptance in local development”, EURO Journal on Decision Processes, Vol. 2, No 3-4, pp. 221-256.
Kovacs, G. and Moshtari, M. (2019), “A roadmap for higher research quality in humanitarian operations: A methodological perspective”, European Journal of Operational Research, Vol. 276, No. 2, pp. 395-408.
Leon, O. G. (1999), “Value-focused thinking versus alternative-focused thinking: Effects on generation of objectives”, Organizational Behavior and Human Decision Processes, Vol. 80, No. 3, pp. 213-227.
Lienert, J. et al. (2015), “Structured decision-making for sustainable water infrastructure planning and four future scenarios”, EURO Journal on Decision Processes, Vol. 3, No. 1-2, pp. 107-140.
Mareschal, B. et al. (1984), “Prométhée: A new family of outranking methods in multicriteria analysis”, in Brans, J. P. (Ed.), Operational Research’ 84, North-Holland, Amsterdam, pp. 477-490.
Marttunen, M. et al. (2015), “How to design and realize participation of stakeholders in MCDA processes? A framework for selecting an appropriate approach”, EURO Journal on Decision Processes, Vol. 3, No. 1-2, pp. 187-214.
Marttunen, M. et al. (2017), “Structuring problems for Multi-Criteria Decision Analysis in practice: A literature review of method combinations”, European Journal of Operational Research, Vol. 263, No. 1, pp. 1-17.
Miguel, P. A. C. et al. (2010), Metodologia de Pesquisa em Engenharia de Produção e Gestão de Operações, Elsevier, Rio de Janeiro.
Miyamoto M. et al. (2014), “Development of an Integrated Decision-Making Method for Effective Flood Early Warning System”, Journal of Disaster Research, Vol. 9, No. 1, pp. 55-68.
Morais, D. C. et al. (2013), “Using value-focused thinking in Brazil”, Pesquisa Operacional, Vol. 33, No. 1, pp. 73-88.
Nachtmann, H. and Pohl, E. A. (2013), “Transportation readiness assessment and valuation for emergency logistics”, International Journal of Emergency Management, Vol. 9, No. 1, pp. 18-36.
Paiva, M. L. D. U. G. and Daher, S. D. F. D. (2016), “Abordagem VFT para estruturação de problema para melhoria da produção mais limpa em empresa de confecções do Agreste Pernambucano”, paper presented in the XLVIII SBPO - Simpósio Brasileiro de Pesquisa Operacional, Vitória, ES, Sept 27-30, 2016.
Parnell, G. S. et al. (2013), “Invited Review—Survey of Value‐Focused Thinking: Applications, Research Developments and Areas for Future Research”, Journal of Multi‐Criteria Decision Analysis, Vol. 20, Nos. 1-2, pp. 49-60.
Peng, Y. (2015), “Regional earthquake vulnerability assessment using a combination of MCDM methods”, Annals of Operations Research, Vol. 234, No. 1, pp. 95-110.
Rosenhead, J. (1989), Rational Analysis for a Problematic World: Problem Structuring Methods for Complexity, Uncertainty and Conflict, John Wiley & Sons, Chichester.
Rosenhead, J. (1996), “What’s the Problem? An Introduction to Problem Structuring Methods”, Interfaces, Vol. 26, No. 6, pp. 117-131.
Rosenhead, J. (2001), “Introduction: old and new paradigms of analysis”, in Rosenhead, J. and Mingers, J. (Eds.), Rational analysis for a problematic world revisited: Problem structuring methods for complexity, uncertainty and conflict, 2nd ed., John Wiley & Sons, Chichester, pp. 1-20.
Rosenhead, J. (2006), “Past, present and future of problem structuring methods”, Journal of the Operational Research Society, Vol. 57, No. 7, pp. 759-765.
Roy, B. (1968), “Classement et choix en présence de points de vue multiples”, Revue française d'informatique et de recherche opérationnelle, Vol. 2, No. 8, pp. 57-75.
Santis, R. B. et al. (2017), “Multi-Criteria Supplier Selection Using Fuzzy Analytic Hierarchy Process: Case Study from a Brazilian Railway Operator”, Brazilian Journal of Operations & Production Management, Vol. 14, No. 3, pp. 428-437.
Silva, G. F. P. and Belderrain, M. C. N. (2019), “Evaluation of Strategic Initiatives with MCDA for Issuing Natural Disaster Alerts”, in Leiras, A. et al. (Eds.). Operations Management for Social Good. 2018 POMS Conference International in Rio.
Simpson, N. C. and Hancock, P. G. (2009), “Fifty years of operational research and emergency response”, Journal of the Operational Research Society, Vol. 60, No. 1, pp.126-139.
United Nations International Strategy for Disaster Reduction - UNISDR (2006), “Platform for the Promotion of Early Warning”, available from: http://www.unisdr.org/2006/ppew/whats-ew/basics-ew.htm/ (access 12 Feb. 2018).
United Nations International Strategy for Disaster Reduction - UNISDR (2015), “Sendai Framework for Disaster Risk Reduction 2015-2030”, available from: http://goo.gl/E6lM74 (access 12 Dec. 2017).
United Nations International Strategy for Disaster Reduction - UNISDR (2017), “Terminology on disaster risk reduction”, available from: http://www.preventionweb.net/english/professional/terminology/v.php?id=478 (access 12 Dec. 2017).
van den Honert, R. C. (2016), “Improving Decision Making about Natural Disaster Mitigation Funding in Australia-A Framework”, Resources, Vol. 5, No. 3, p. 28.
Vieira, J. A. M. ET AL. (2017), “Development of a Scenario Prospecting Model with the Use of Multicriteria Decision Aiding: Importance of Environmental Variables”, Brazilian Journal of Operations & Production Management, Vol. 14, No. 2, pp. 210-217.
Walker, D. et al. (2014), “Stakeholder voices through rich pictures”, International Journal of Managing Projects in Business, Vol. 7, No. 3. pp. 342-361.
Wienke, A. and Mustafee, N. (2015), “An investigation of'soft'operations research methods to inform hybrid simulation studies on environmental disasters”, paper presented in the 48th Annual Simulation Symposium, Alexandria, Virginia, April 12-15, 2015.
Received: 05 Mar 2019
Approved: 19 Jul 2019
DOI: 10.14488/BJOPM.2019.v16.n3.a10
How to cite: Silva, G. F. P. and Belderrain, M. C. N. (2019), “Prioritization of strategic initiatives in the context of natural disaster prevention”, Brazilian Journal of Operations & Production Management, Vol. 16, No. 3, pp. 473-489, available from: https://bjopm.emnuvens.com.br/bjopm/article/view/817 (access year month day).