A JOINT MARKOV DECISION PROCESS - TWO-STAGE STOCHASTIC OPTIMIZATION FRAMEWORK FOR CAPACITATED SUPPLY ACQUISITION AND DISTRIBUTION IN HUMANITARIAN OPERATIONS

ABSTRACT

Relief operations involve many actors, from government entities to humanitarian organizations. Each has its own objectives and procedures within the overarching goal of aiding as many victims as possible. However, as their objectives often overlap, organizations may compete for the same scarce resources and thus jeopardize the efficiency of the operation. This paper aims to develop an inventory management framework for collaborative acquisition and distribution of goods in a dual-sourcing system during a disaster relief operation. Our framework uses a model that combines a Markov Decision Process with Two-Stage Stochastic Programming to manage sequential decisions within an uncertain environment. It improves inventory visibility between the different levels of the relief chain, thereby promoting cooperation and coordination whilst minimising inventory and shortage costs. To illustrate the applicability of the framework, we conducted experiments to demonstrate how different demand scenarios and different available action sets can affect acquisition and distribution policies in humanitarian operations.

Keywords:
humanitarian logistics; markov decision processes; two-stage stochastic problem

1 INTRODUCTION

From 2018 to 2023, the international disasters database¹ registered 3.365 disasters, 568 in 2023 only. They disrupted the lives of over 714 million people and caused almost US$ 1.217 trillions worth of damages. Over the same period, however, only approximately 46% of the disaster relief funding appeals were met². This underscores the need for humanitarian organizations (HO) to improve cost efficiency and reduce operational costs (Besiou & van Wassenhove, 2020).

From 2018 to 2023, 72 disasters were reported in Brazil, including 50 floods and landslides. In addition, 24 flood and landslide events were recorded from January to August 2024, affecting the lives of over 120 million inhabitants³. This study is motivated by the disaster response operations for landslides and floods that affected the city of Petrópolis, Brazil, in February 2022. Located in the mountainous region of the state of Rio de Janeiro, the city encompasses hundreds of communities susceptible to landslides. Municipal and State authorities acted alongside dozens of organizations on the disaster response, with virtually no coordination. Furthermore, the response was heavily reliant on donations: an unreliable source of supplies.

Importantly, both the city and the state set up set an annual budget to respond to disasters that could strike several times within the planning horizon. Given the recurring characteristics of disasters, it is clear that a coordinated stochastic framework for disaster response could improve the efficiency of humanitarian operations. It is such a framework that is designed and proposed in this paper.

A humanitarian operation often comprises hundreds of supply chains that overlap, interact, and even compete for resources. They aspire to meet uncertain and highly dynamic demands with little to no information on what is needed, when or where it is needed (Holguín-Veras et al., 2014). Haiti’s 2010 earthquake response, with over 200 organizations involved in relief operations, highlights this complexity (Lewin et al., 2018). The need for coordination to improve efficiency and promote better utilisation of resources is well-known and widely acknowledged (Dowty & Wallace, 2010; Dubey & Altay, 2018; Li et al., 2019).

As disasters can damage local infrastructure, disaster relief supply chains must consider uncertain road network availability and possible disruptions of basic services such as water, power, communications, and fuel supplies. These may impair the relief effort, interfere with or disable machinery and vehicles used in relief operations (Kaatrud et al., 2003; Kovács & Spens, 2007). Disasters can also affect ports, airports, public buildings, schools and other facilities typically used as shelters or relief centres, hindering or preventing their use in the response (Rahmani et al., 2018). Finally, disasters may affect local suppliers and logistics operators’ supply chains, limiting their ability to contribute to the relief effort (Kovács & Spens, 2007; Murray, 2005).

Since purchasing from local suppliers supports the affected region’s long-term recovery, minimizing costs is not the sole objective. Instead, HOs can focus on providing equal opportunities for distinct suppliers in an effort to drive economic recovery (Balcik et al., 2016). This involves finding the right compromise between promoting local services and ensuring a reliable and sustainable supply chain. To that end, we propose a novel framework that includes dual source procurement under uncertainty whilst optimising the collaboration among HOs or demand points; it includes joint procurement from multiple suppliers and centralised warehousing and distribution of relief supplies. Recognised for their relatively low implementation costs and technological requirements, these mechanisms are conducive to the relief effort (Balcik et al., 2010).

Due to possible disruptions, each local supplier is a viable, albeit unreliable source of relief supplies. Hence, there is a probability that a local supplier will not deliver the procured items in a timely manner. A dual sourcing model ensures that another supplier outside the affected region can always provide the procured items at the required time, but at a higher price. To pursue a trade-off between costs and reliability, the model penalises unmet demands and missed deliveries, which cannot be timely refunded.

Our framework also supports inventory and distribution decisions from governmental agencies responding to disasters. They typically set aside a Distribution Center (DC) close to the shelters/demand points but still accessible from the outside world (in the case of a large-scale disaster), or strategically located (in the case of frequent small-to-medium disasters). The DC can be an existing permanent warehouse or a set of mobile storage units (when a permanent warehouse is not available), and it serves multiple shelters and/or demand points. The government itself can be treated as a reliable supplier that typically procures items for prepositioning/preparedness, and can quickly deploy supplies from other regions, whereas local suppliers can be collectively seen as unreliable suppliers as they may be affected by the disaster. The framework integrates a two-state stochastic programming (TSSP) into a Markov decision process (MDP), to create a Parameter Evaluation Algorithm. While the MDP prescribes an optimal purchasing policy (how much to buy?), the TSSP splits the optimal purchasing policy into individual orders for each supplier (where to buy?) and identifies an optimal distribution policy in accordance with the number of acquired goods to supply the operations (where to allocate or send?). The innovative idea is that we use TSSP to translate the long-term perspective of an MDP solution into several short-term solutions. By combining the two methodologies, the framework avoids the combinatorial explosion of scenarios imposed by multi-stage stochastic programming models for large time horizons and circumvents the limitations on the number of possible actions within an MDP that includes distribution and acquisition decisions, without forgoing optimality. Exploiting the strengths of both approaches, we derive an inventory management model for multi-sourcing and integrated distribution of relief supplies that is suited for three settings: (1) disaster response or recovery operation with finite random duration, (2) long-term disaster response or recovery operation, (3) optimising the average long-term cost for organizations that set periodic budgets (e.g. annual planning) for continuous disaster response operations.

Our main contributions are twofold. Methodologically, the combination of MDPs and TSSP for solving long-term stochastic planning problems is new and allows the decision maker to combine the MDPs adaptability to long-term decisions with he TSSP’s suitability for short-term analysis to simplify the search space and improve computational tractability. From an application standpoint, we propose a new integrated inventory management and distribution model for HL that considers unreliable suppliers and enables a centralised distribution centre to coordinate the relief effort and promote cooperation among HOs.

This paper is organised as follows. Section 2 presents a brief literature covering general aspects and collaborative efforts in humanitarian logistics, as well as operational research and management science applications. Section 3 details the framework and introduces the mathematical formulation and algorithms. Section 4 includes numerical experiments, using Petrópolis as background, to illustrate the model’s applicability and analyse the results and insights. Finally, Section 5 concludes the paper.

2 LITERATURE REVIEW

Comprising around 80% of disaster relief efforts, logistics is vital for humanitarian operations (Van Wassenhove, 2006; Campos et al., 2012; Çelik et al., 2014). Such a prominent role led HO organizations to adapt conventional logistics techniques to disaster mitigation, with limited success (Besiou & van Wassenhove, 2020). This is due to major differences between conventional logistics and the logistics of humanitarian operations - humanitarian logistics (HL), as the latter involves uncertain intermittent demands that vary with time, location and disaster magnitude. HL includes destroyed or compromised infrastructure around the disaster site and unconventional objectives, such as mitigating suffering. Such specificity renders humanitarian supply chains difficult to manage (Kovács & Spens, 2007; Habib et al., 2016).

Although under drastically different conditions, HL and commercial logistics involve similar activities. Therefore, their key success factors may overlap (Power et al., 2001; Lu et al., 2006; Pettit & Beresford, 2009). Abidi et al. (2013) maps 8 success factors for relief chains: strategic planning; inventory management; transportation and capacity planning; information management and technology utilisation; human resources management; collaboration; continuous improvement; and supply chain strategy.

By appropriately planning and responding rapidly to disasters, HL aims to promote effective recovery for the affected population and to reduce loss to human life and property (Mishra et al., 2020). It encompasses procurement, warehousing, inventory management, transportation, and distribution of products and/or resources during a disaster relief operation (Balcik et al., 2016). Despite the adverse conditions, analysts and decision makers must evaluate the needs, establish functional relationships with suppliers and consumers, and map the required infrastructure (Tabbara, 2008).

Humanitarian supply chains comprise two parts (Habib et al., 2016): relief supply chain and relief distribution chain. The former comprises upstream activities, such as acquiring and transporting products from suppliers to warehouses or distribution centres. The latter involves the transportation of products from distribution centres to HOs and affected areas. Figure 1 depicts a conceptual framework of the humanitarian supply chain.

Figure 1
Humanitarian Supply Chain framework.

Our study highlights the need for coordination and collaboration among multiple organizations to optimise the distribution of relief supplies, two aspects considered in only 13% of the humanitarian logistics and supply chain literature from 2011 to 2017 (Behl & Dutta, 2018). In contrast, coordination and collaboration are top practitioner’s priorities (Besiou & van Wassenhove, 2020). We address this mismatch by proposing a collaboration framework wherein multiple organizations share the same suppliers and transport network. The approach centralises purchasing, warehousing and distribution decisions within a distribution centre (DC). The DC acquires and stores goods from multiple suppliers to later distribute them to different organizations, thereby linking the relief supply chain with the relief distribution chain, see Figure 1. This reduces inventory and distribution costs and promotes a better usage of the available resources to improve the assistance to the victims. Further, the collaboration mechanism prevents the competition among organizations with similar objectives over scarce resources, which can lead to increased prices. This study focuses on the evaluation of the benefit of this approach using a mathematical model.

After COVID-19 highlighted the perils of a super lean supply chain, there is growing concern about the inventory of vital supplies in humanitarian responses, such as drugs and medical equipment (Mishra et al., 2020; Perdana et al., 2022). Our approach promotes a balanced operational policy that avoids overstocking and wasting resources whilst maintaining enough inventory to minimise shortages due to peak demand.

We include a local supplier to tackle another practitioners’ rising concern: the need to rehabilitate and develop the local economy after a disaster (Heaslip et al., 2018). To hedge against local supply chain disruptions, the model includes a reliable outside supplier to ensure that HOs can properly assist the disaster victims even when local supply is unavailable. Whilst the model includes a single reliable supplier matched by a single unreliable counterpart, it is easily adaptable to multiple suppliers (both reliable and unreliable).

To tackle supplier unreliability, Iakovou et al. (2014) propose a model that includes a second supplier contractually remunerated to keep a number of items in stock, in case the main supplier is unavailable. While they assume a two-tier network design and contractual obligations, our framework considers a three-tier supply chain network with centralised decisions at the DC. Furthermore, our framework does not imply any contractual obligation and allows the decision maker to freely choose among suppliers based on the conditions at each decision epoch. As the model is stochastic, it considers failed deliveries.

Other studies considering supply chain disruptions mostly focus on facility locations, often considering that disruptions may affect facilities such as distribution centres and shelters (Rahmani et al., 2018; Hamdan & Diabat, 2020; Mohammadi et al., 2020). These studies consider that the disrupted locations are completely inoperative due to damaged infrastructure or compromised accessibility. Our framework is more general and does not assume that disruptions necessarily render suppliers completely inoperative. Rather, they cause the supplier to miss specific delivery deadlines with a given probability. Moreover, whilst two-tier networks are relatively common in the literature, three-echelon networks, while existent, are hardly deployed (Balcik et al., 2016; Behl & Dutta, 2018). The present study shortens this gap by proposing a three-tier network that models the flow of supplies from their source (suppliers) to their final destination (humanitarian organizations/victims), including the distribution centre.

Markov decision processes provide a natural framework for modelling HL uncertainty. Ferreira et al. (2018) used MDPs to derive optimal inventory policies for perishable items in slow-onset disasters, whereas Meraklı & Küçükyavuz (2019) used a value at-risk MDP formulation to tackle a similar problem under parameter uncertainty. The models, however, do not include distribution decisions as these would enlarge the state and action spaces and render the approach intractable for moderately complex distribution networks (e.g., Powell, 2011). To include distribution decisions, researchers often utilise two-stage stochastic programming (TSSP) (Barbarosoğlu & Arda, 2004; Alem et al., 2016; Noyan et al., 2019) or robust programming (Rezaei-Malek et al., 2016; Rahmani et al., 2018; Hamdan & Diabat, 2020). These models, however, present a combinatorial growth with the time horizon, which renders them impractical for longer-term problems such as slow-onset disasters. Furthermore, as the time horizon grows, the scenarios become increasingly unlikely and hardly representative (e.g., Snyder, 2006).

Fuzzy sets can also be utilised to tackle uncertainty. Kabra et al. (2015) use a fuzzy analytic hierarchy process to prioritise barriers for coordination in humanitarian supply chains based on perceived severity. Representing the demand by a fuzzy parameter, Goli & Malmir (2019) introduce a Vehicle Routing Problem (VRP) covering tour approach to distribute essential goods.

With a similar goal, Mohammadi et al. (2020) combine fuzzy logic and robust optimisation to route vehicles and distribute products in a multi-echelon relief distribution network.

Uncertainty can be addressed via simulation to test a limited number of possible solutions. Beamon & Kotleba (2006) simulate complex emergencies in South Sudan and study the interrelationships among parameters in the relief chain design, such as reorder, lot size and distribution costs. Similarly, McCoy & Brandeau (2011) simulate interactions between a central warehouse and downstream operations, and test distinct distribution and budget allocation decisions. Krejci (2015) studies decision maker’s behaviours to find the influence of collaboration mechanisms in the operation. Using COVID-19 as a background, Malmir & Zobel (2021) propose a simulation-based approach to generate problems of different sizes with uncertain demand whilst considering deprivation and equity in humanitarian operations. Although simulation allows us to propagate uncertainties and compare a limited number of prescribed policies, it does not embed optimisation.

As MDPs are specially suited to long-term operations (Putterman, 1994) and TSSP can effectively prescribe shorter term distribution policies, this paper proposes a novel framework that combines these approaches to effectively solve problems that involve longer-term strategic/tactical and short-term operational decisions. For example, governmental disaster management agencies typically need to set an annual budget for their inventory planning and make operational decisions to distribute the inventory to demand points whenever disaster strikes. Additionally, when a large-scale disaster happens and the government agency knows that the response operation will take a long time, they will need to plan the inventory for the length of the operation and make short-term decisions to distribute relief items to victims on a regular basis (e.g. daily). To address the need for longer-term decision, we use an infinite horizon MDP formulation to derive the optimum purchasing strategy; short and medium-term operations are covered via the discounted cost MDP formulation that considers the random duration of the response. We then use a TSSP optimisation model to break down the MDP’s optimum purchasing strategy (how much to buy) into acquisition (where to buy) and distribution (where to send) policies. It is worth highlighting that, while purchasing quantities and distribution strategies are decoupled for computational ease, all possible acquisiton and distribution policies are considered and therefore the approach is guaranteed to converge to the optimal solution.

The approach ensures a simple and easy to solve MDP that prescribes the number of items to purchase given the current inventory level at the DC at the start of the planning horizon (e.g. the start of the budget year, the start of a response operation). Considering the initial inventory at the DC and the number of items to be acquired, the stochastic programming (TSSP) formulation refines the purchasing decisions at each epoch by assigning quantities to suppliers, establishing the distribution strategy under uncertainty. As it only encompasses the uncertainties of the current decision epoch, the TSSP is also simple and easy to solve. Therefore, the framework takes advantages of the strengths of the MDP and TSSP formulations to derive a fast and reliable algorithm for inventory management and distribution. Our approach addresses a real problem faced by city or state agencies which need to set a longer-term budgeting policy and short-term decisions to respond to disasters. To the best of our knowledge, this is the first approach that combines MDP and TSSP to tackle a combined long-term inventory management problem and short-term procurement/distribution problems with multiple suppliers and recipients within a three echelon supply network.

3 PROBLEM DESCRIPTION AND MATHEMATICAL MODELING

To support a disaster relief distribution effort, we study the inventory management within a distribution center (DC) that acquires items from suppliers and distributes them to distinct humanitarian organizations. Suppose a local supplier is a viable source of relief goods, but an external supplier is considered to ensure reliability (a local supplier here can also represent a group of local suppliers). The local supplier will henceforth be called unreliable supplier, as it may be affected by the disaster and fail to deliver the acquired items with a given probability. In contrast, the external supplier, henceforth called reliable supplier, always delivers in time.

The DC stores acquired products and distributes them to multiple HOs under its area of influence, incurring inventory and shortage costs. Shortage penalties may differ for different HOs. The centralised decision making in the DC also helps to coordinate the operations of multiple HOs in the relief effort.

Given the demand volatility following a disaster, our framework assumes that the demand for relief items at each HO is stochastic and follows a probability distribution. Under demand and supply uncertainty, the framework finds the optimal purchasing quantities in a dual sourcing system, as well as the optimal distribution policies to the multiple HOs. A policy comprises actions that must be taken at every decision epoch, given the system’s state (Putterman, 1994). Figure 2 synthesises the flow of goods in the described humanitarian supply chain.

Figure 2
The flow of supplies in the studied problem.

The proposed framework combines a two stage stochastic programming model (TSSP), incorporated in a Parameter Evaluation Algorithm, and a Markov Decision Process (MDP). The output of the MDP is an optimal purchasing policy that specifies the number of items to be acquired. In addition, the TSSP’s output describes the optimal distribution policy and specifies the exact amount to be bought from each individual supplier.

Figure 3 presents the conceptual flowchart for the proposed framework. λ _t is the system’s state (inventory at the DC) at period t and α the total number of products to be acquired (purchase policy) given that the state is λ _t . It is worth emphasising that the MDP’s state and action spaces are unidimensional, hence the formulation avoids by design the curse of dimensionality. To further specify the purchases, the TSSP finds an optimal policy u(λ), that prescribes the acquisition policy (where to buy) and the distribution policy (where to send), and establishes the transition probabilities $p_{{\lambda }_t{\lambda }_t\text{'}}^{\alpha }$ and average costs associated with the given policy C(λ _t , u(λ _t )) for all possible realisations of the uncertainties. These transition probabilities and average costs are used as parameters in the MDP formulation. Note that, since all possible distribution policies are considered for any given purchase quantity (TSSP), and since all purchasing quantities are compared within the MDP, the search space is preserved and the approach converges to an optimal solution.

Figure 3
Proposed framework conceptual flowchart.

3.1 Model characteristics and assumptions

We make the following assumptions.

• The demand of each organisation is stochastic and follows a known probability distribution.

• Product acquisitions happen at the beginning of each decision period. Distribution occurs after the realisation of the uncertain parameters (demand of organizations and reliability of suppliers).

• Lead times for both suppliers are sufficiently small not to encumber the system. Hence, when delivered, products acquired at a decision epoch will arrive in the same decision epoch. This is a common assumption in the literature (e.g., Goli & Malmir, 2019; Meraklı & Küçükyavuz, 2019; Malmir & Zobel, 2021).

• There are two suppliers for the distribution centre: one reliable, which always delivers the procured products, although at a higher cost, and one unreliable, which fails to deliver (or delivers only a portion of) the procured products, with a given probability.

• The distribution centre and the organizations have limited inventory capacity. Each supplier’s capacity is also limited.

• The demand lost in a period is not backlogged. That is valid for many products used in short-term disaster recovery, such as water bottles, since nutrition needs do not directly accumulate between decision epochs (day, week, etc.) when the item is not available. This assumption is regularly applied in the literature (e.g., Rezaei-Malek et al., 2016; Cook & Lodree, 2017)

These assumptions are not intended to restrict the problem. Instead, these are assumptions that are rather common in the specialised literature and reflect specific characteristics of humanitarian operations.

The parameter evaluation algorithm (TSSP) in Section 3.5 evaluates all possible acquisition and distribution strategies, given that a specific number of items must be acquired. The minimised cost of the TSSP purchase and distribution policy at each state is the one-step cost of the MDP for the respective state-action pair. Note also that the distribution policies for each scenario are used to calculate state transitions and probabilities, for each state-action pair, see Section 3.6.1.

The MDP’s solution prescribes the optimal overall purchase for each initial inventory level at the DC, while the TSSP determines the split between suppliers and the distribution policy. Each scenario used by the TSSP comprises of the joint realisation of the demands for all HOs and the ability of the unreliable supplier to deliver the procured goods.

Since our framework does not include routing decisions, only distribution of goods, the distances between suppliers, DC and organizations do not constrain the problem. They only affect supply costs (including transportation costs). But the TSSP is generic enough to enable the inclusion of distance constraints for suppliers, DC or organizations, if needed.

3.2 Decision epoch

Acquisition and distribution decisions take place on a regular basis (e.g. daily, weekly, etc.) at the beginning of each decision epoch, for the whole planning horizon.

3.3 State of the system

The system states represent the available inventory in units of product (blood packs, bottles of water, etc.). Therefore, the state space is discrete and denoted by Λ = {0, 1..., W _cd }, where W _cd is the DC’s maximum capacity.

3.4 Actions, cost function and long-term objective

The MDP aims to find an optimal stationary purchase policy π∗: Λ → A from the set of feasible purchasing policies Π. Any policy π ∈ Π specifies a purchase α ∈ A _λ for each inventory level λ ∈ Λ. A _λ denotes the set of available actions at state λ, and A = ∪_λ∈Λ A _λ is the set of available actions. For each inventory level λ ∈ Λ, and purchase amount a ∈ A _λ , a parameter evaluation algorithm distributes the purchase amount between the competing suppliers and specifies the optimal distribution policy given the state action pair (λ, α).

For each state-action pair (λ , α), the parameter evaluation algorithm solves a two-stage stochastic programming problem (TSSP). Considering the overall order α, the first stage finds the optimal orders from the reliable and unreliable suppliers, resp. a and b such that α = a + b. Given the initial inventory λ and individual orders to the suppliers (a and b), the second stage of the TSSP algorithm defines the amount of products to be distributed to each individual humanitarian organisation after the realisation of the uncertainties. The TSSP algorithms aims to to minimise the overall single-period purchase, inventory holding, distribution and shortage cost C(λ, α).

This overall cost is then fed to the MDP for multi-stage optimisation. Let t denote the time period and (λ _t , a _t ) ∈ Λ × A, t ≥ 0 be the MDP’s state-action pair at period t. The MDP’s transition probabilities - $p_{\lambda {\lambda }^{\text{'}}}^{\alpha }=P({\lambda }_{t+1}={\lambda }^{\text{'}}|{\lambda }_t=\lambda ,a_t=\alpha )$ - are determined by the TSSP algorithm, see Figure 3. The TSSP model used is detailed in Section 3.5. The evolution of this process is controlled by a Markov Chain {Z _t , t ≥ 0} under a control policy π ∈ Π, where Z _t = λ _t is the inventory level at time t. Let τ > 0 be a random planning horizon, to account for the random duration of a disaster response. The MDP’s objective is to find a policy π ∈ Π that minimises the overall cost of the disaster response, defined as:

$v_{\pi }\left(\lambda \right)=E_Z\left\{\sum _{t=0}^{\tau }C({\lambda }_t,\pi ({\lambda }_t\left)\right)|Z_0=\lambda \right\},\forall \lambda \in \Lambda$ (1)

where τ is assumed to be a geometrically distributed random variable with parameter 0 < p = 1 −γ < 1, which means that P(τ > t) = γ ^t , t ≥ 0. Hence:

$v_{\pi }\left(\lambda \right)=E_Z\left\{\sum _{t=0}^{\infty }{\gamma }^{t}C({\lambda }_t,\pi ({\lambda }_t\left)\right)|Z_0=\lambda \right\},\forall \lambda \in \Lambda .$ (2)

Eq. (2) shows how to design an equivalent infinite-horizon discounted cost MDP to optimise a finite-horizon response for a system with random response time. For finite-duration disaster responses, the aim to find a policy π* ∈ Π such that

$v^{*}\left(\lambda \right)=v_{{\pi }^{*}}\left(\lambda \right)\le v_{\pi }\left(\lambda \right),\forall \pi \in \Pi ,$ (3)

under mild conditions, the solution to (3) exists and is unique (Putterman, 1994).

Alternatively, for long-term responses or for systems in which an annual budget is prescribed to deal with disasters in a certain region, we can use an infinite-horizon MDP with average cost criterion. In that case, each policy π ∈ Π is associated to a long-term average cost given by

${\eta }_{\pi }=\underset{T\to \infty }{\lim }\frac{1}{T}E\left\{\sum _{t=0}^{T}C({\lambda }_t,\pi ({\lambda }_t\left)\right)\right\}.$ (4)

The objective is to find an optimal policy π∗ ∈ Π such that:

${\eta }_{{\pi }^{*}}\le {\eta }_{\pi },\forall \pi \in \Pi .$ (5)

3.5 Two-stage stochastic model

The goal of the TSSP model is to find an optimal purchase quantity a (resp. b) from the unreliable (resp. reliable) supplier, as well as the distribution policy that minimises the single-period cost C(λ, α) for an initial inventory λ and a total purchase α = a + b. At each time period, the unreliable supplier fails to deliver the requested products with a given probability, in which case the acquisition cost is lost. The probability that the request is not met is a model parameter, as is the demand distribution for each individual humanitarian organisation.

Tables 1 and 2 summarise the TSSP’s variables and parameters.

Table 1
Decision variables.

Variable	Description
at	amount purchased from supplier a (reliable) on period t
bt	amount purchased from supplier b (unreliable) on period t
xit	amount sent to organization i on period t
yit	amount of shortage allowed to the organization i on period t

Table 2
Parameters and sets.

Parameter	Description
I	set of humanitarian organizations
qa	purchase cost per unit from supplier a
qb	purchase cost per unit from supplier b
qh	inventory holding cost
ht	inventory level at the beginning or period t
α	total purchased amount
qei	delivery cost per unit to organization i
q fi	shortage cost per unit to organization i
Wi	storage capacity of organization i
Wa	capacity of the reliable supplier
Wb	capacity of the unreliable supplier
s	Scenario, comprised of the joint realization of demand and unreliability

Table 3
Uncertain Parameters.

Parameter	Description
R t	Realization of the delivered goods by the unreliable supplier
D t	set of the demands of all organizations on a given period t
d it	demand of organization i on a given period t
ξ t	random vector with finite realizations ${\xi }_t^1,...,{\xi }_t^s$ on period t, where ${\xi }_t^s=(D_t^s,R_t^s)$ is a random vector that represents the joint realization (s) of D t and R t for a given scenario s

The two-stage model described below decides the purchase quantities a ₁ and b ₁ at the first state and the distribution policy at the second stage:

$\text{Minimise}{q}_a{a}_1+q_b{b}_1+q_h{h}_1+E\left[Q_2^s\right(a_1,b_1,{\xi }_2^s\left)\right]$ (6)

$$\begin{gathered} \text{subject to:} \\ {a}_1+b_1=\alpha \end{gathered}$$ (7)

$a_1\le W_a$ (8)

$b_1\le W_b$ (9)

$a_1,b_1\in {\mathrm{\mathbb{Z}}}^{+},$ (10)

where:

$Q_2^s(a_1,b_1,{\xi }_2^s)=\min \sum _i\left(q_{e_i}{x}_{i2}\left({\xi }_2^s\right)+q_{f_i}{y}_{i2}\left({\xi }_2^s\right)\right)$ (11)

$$\begin{gathered} \text{subject to:} \\ {a}_1+b_{1}R\left({\xi }_2^s\right)+h_1-\sum _i{x}_{i2}\left({\xi }_2^s\right)\ge 0 \end{gathered}$$ (12)

$x_{i2}\left({\xi }_2^s\right)+y_{i2}\left({\xi }_2^s\right)=d_{i2}\left({\xi }_2^s\right),\forall i\in I$ (13)

$x_{i2}\left({\xi }_2^s\right)\le W_i\forall i\in I$ (14)

$x_{i2}\left({\xi }_2^s\right),y_{i2}\left({\xi }_2^s\right)\in {\mathrm{\mathbb{Z}}}^{+}$ (15)

The objective (Eq. (6)) is to minimise inventory holding and purchase costs. Equation (7) ensures that the number of items purchased matches the value obtained by the MDP algorithm. Equations (8)-(9) represent the supplier’s capacity constraints. Equation (10) establishes the variables domains.

At the second stage, Eq. (11) represents the total distribution and shortage costs. Equation (12) ensures that the amount of distributed goods cannot exceed the amount purchased plus the initial inventory. The balance between the amount sent to each organisation and the shortage allowed for each organisation is enforced by Eq. (13). Equation (14) ensures that the inventory capacity of each organisation is respected. Equation (15) establishes the variables domain. The cost C(λ, a) returned to the MDP is the solution of (6).

3.6 The parameter evaluation algorithm

As previously detailed, the parameter evaluation algorithm uses a TSSP to find the optimal acquisition and distribution policy, as well as the MDP parameters for each state-action pair. The detailed acquisition and distribution policies, as well as the one-period cost C(λ, α), are obtained from (6). Now let $p_{\lambda {\lambda }^{\text{'}}}^{\alpha }$ be the probability of transitioning from inventory level λ ∈ Λ to λ’ ∈ Λ in two subsequent periods, given that the total purchase is α ∈ A.

The reasoning behind the development of the algorithm is twofold. Firstly, it avoids the combinatorial explosion of the number of possible actions when dealing with acquisition from multiple suppliers and distribution for multiple organizations, which could render the MDP intractable. Secondly, it provides scalability to the proposed framework, supporting the addition of more suppliers and HOs with simple changes in the TSSP, without sacrificing the MDP’s performance.

3.6.1 Execution

Assume that TSSP ^λ,α is the two-stage stochastic problem described with parameters h _t = λ and the total acquisition of α items. Then C(λ, α) = Z ^λ,α , where the latter is the value of the objective function for the TSSP ^λ,α . Finally, let

$h_{t+1}\left({\xi }_t^s\right)=a_t+b_{t}R\left({\xi }_t^s\right)+h_t-\sum _i{x}_{it}\left({\xi }_t^s\right)$ (16)

be the inventory at period t + 1, given scenario ${\xi }_t^s$. Then

$p_{\lambda {\lambda }^{\text{'}}}^{\alpha }=\sum P\left({\xi }_t^s\right){𝟙}_{\left\{h_{t+1}\right({\xi }_t^s)={\lambda }^{\text{'}}|h_t=\lambda \}}$ (17)

where 𝟙_{·} is the indicator function and $P\left({\xi }_t^s\right)$ is the probability of such scenario. Thus, $p_{\lambda {\lambda }^{\text{'}}}^{\alpha }$ is the overall probability that the next state will be λ’ considering all possible scenarios. Each realisation ${\xi }_t^s$ comprises a demand realisation $D=D_t^s$ and a realisation of the delivered purchase $R=R_t^s$, and has probability $P\left({\xi }_t^s\right)$.

To solve the MDP and obtain the optimal purchasing total for each initial state λ ∈ Λ, we input the costs and transition probabilities for each state-action pair into the respective MDP formulation, i.e. (3) for finite-horizon planning and (5) for the long-term average cost formulation and solve the MDP via classical value or policy iteration (Putterman, 1994).

Finally, assuming that π∗(λ) = α is the optimal total purchase at state λ ∈ Λ, the detailed purchase plan a and b such that a + b = α and the detailed distribution plan for each realised scenario will be obtained from the corresponding TSSP ^λ,α formulation (6).

4 EXPERIMENTATION

The two-stage stochastic programming (TSSP) model was implemented in Xpress Workbench 3.3.2, running in Windows 10 (64 bits), with 16 GB RAM and an Intel Core i7-8650U CPU (8 CPUs) with approximately 2.11 GHz. The Value Iteration algorithm was programmed in C#, using the Xpress Workbench library to interface with the TSSP model. Xpress Workbench was available to the researchers via institutional license and C# was preferred due to the ease of integration with the Xpress workbench suite.

4.1 Experiment design

To demonstrate the model’s applicability, our experiments explore the distribution of water supplies from a distribution centre (DC) to landslide relief Support Points (SP) in Petrópolis (Brazil). SP are typically schools and public buildings that provide shelter to the local community during emergency situations. Petrópolis is located in the mountainous region of the state of Rio de Janeiro, approximately 70 km west of the state capital. Comprised of steep slopes subjected to tropical weather, with no drought season and a high volume of precipitation throughout the year, the region is highly susceptible to mass movements (landslides) that pose a serious threat to the population. The problem is aggravated by illegal hillside occupation and deforestation. Between 1991 and 2013, 18 landslides were registered in the city, with a death toll of 146 people, which corresponds to 27% of the total of deaths due to landslides in Brazil in that period (Ferreira et al., 2017). More recently, in February 2022, a precipitation volume of 258 ml in only 2 hours triggered landslides across different regions of the city, leading to 233 fatal victims and over 3.000 families displaced from their homes (CNNBrasil, 2022). Such a background highlights the importance of proper preparations and quick landslide response.

Landslide response operations are very challenging, as the surroundings of the affected areas become unstable and new landslides can follow even after the rainy period is over. This jeopardises rescue operations, hinders or even prevents the return of families to their homes. Moreover, even non affected locations that are considered ‘risk areas’ will require evacuation until weather conditions improve. Hence, the relief operation may last from a few days to weeks after the event, and analysis of the needs of the region needs to be performed constantly.

For our experiment, we assume that a distribution centre (DC) supplies different SPs in each district of the city, which in return will provide the required assistance to the affected communities. Since new landslides or evacuation orders can be triggered constantly, the decision on the number of items to be supplied at each period needs to be constantly revised, based on the probabilities of events in each region. This makes our framework a good match for the needs of the landslide relief operations.

The city is divided in 5 different districts (Centro, Cascatinha, Itaipava, Pedro do Rio and Posse). The city centre (Centro) is the most populous area, with higher number of areas prone to land-slides (Vulnerable Communities - VC). The Petrópolis Municipal Plan of Mass Movement Risk Reduction (Theopratique, 2017) identified 233 VCs in the city, distributed across the districts as detailed in Table 4.

Table 4
Number of vulnerable communities (VC) per district.

	Centro	Cascatinha	Itaipava	Pedro do Rio	Posse
# of VC	101	39	35	32	26

Each VC in the city can be divided into Risk Areas (RA) (Theopratique, 2017). Each RA assigned one of seven possible categories of risk (CR), classified according to the probability of a landslide event and the magnitude of an impact in the RA, based on the geological and physical characteristics of the site. Table 5 shows the probabilities of landslide and the respective impact for each category of risk.

Table 5
Categories of risk.

Category of risk	I	II	III	IV	V	VI	VII
Probability	0.083	0.13	0.11	0.25	0.19	0	0.22
Risk factor	0.3	0.5	0.4	0.9	0.7	0	0.8
New category of risk	VI	IV	V	I	III	VII	II

To simplify the interpretation of the experiment and the analysis of the results, we reorganise the CRs by Risk Factor, as shown in the Tables 5 and 6, where CR I has the highest Risk factor, and so on. We assume that each CR has as an impact scenario and a corresponding probability of evacuation (Table 5). The demand for aid is a function of the total population of the corresponding RAs.

Table 6
Number of houses per CR per district .

Category of risk	I	II	III	IV	V	VI
Centro	3748	1913	1495	2523	4691	747
Cascatinha	234	171	1133	501	2389	1334
Itaipava	23	258	880	84	1682	387
Pedro do Rio	287	110	404	29	601	292
Posse	362	44	323	60	476	393

Figures 4, 5 and 6 illustrate the risk area (RA) composition of Oswero Vilaça, one of the communities with highest risk of landslides in the city, as can be observed in the level curves in Figure 4. The coloured areas in Figure 4 represent the respective CR.

Figure 4
Risk Areas for Oswero Vilaça Community.

Figure 5
Satellite view of Oswero Vilaça Community.

Figure 6
Overview of Oswero Vilaça Community.

We assume that if an area with a given risk factor is impacted, all areas with a higher risk factor will also need evacuation. For instance, if the RAs with CR II are evacuated (risk factor = 0.8), the RAs with CR I (risk factor = 0.9) will also need evacuation. The rationale is simple: if the precipitation is powerful enough to trigger the evacuation of an area, it must be powerful enough to trigger the evacuation of any other area that is more susceptible to landslides. Finally, we assume an average number of 4 people per residence in each RA. Hence, the total demand for any given area is four times the number of houses in that area. Table 6 depicts the total number of houses per district per CR.

It is also important to note that CR VII has Risk factor 0, hence no evacuation or impact is expected, which is why this CR has been ommitted in Table 6.

To support the evacuees from risk areas during landslides, we assume that each district installs a Support Point (SP), to provide shelter, food and to support rescue operations. Each SP is supplied by the municipal DC, which centralises the distribution of supplies. The DC receives donations (unreliable supplier) or acquires the required goods from a commercial supplier (reliable supplier).

Our experiment addresses the distribution of one commodity: water bottles. It is believed that the human body needs around 2l of water daily to survive. Since 2l water bottles are common in Brazil, we assume a daily need of one water bottle per person per day (decision epoch). At each decision epoch, the DC will choose the amount of goods to acquire from the reliable supplier and the amount of goods it will request from the community as donations. The number of water bottles donated per period after the DC’s appeal varies according a binomial distribution X ∼ ℬ(3, 0.5), see Table 7.

Table 7
Probability of percentage of donations received.

% of requested donnations received	25	50	75	100
Probability	0.125	0.375	0.375	0.125
More reliable scenario	0.216	0.432	0.288	0.064
Less reliable scenario	0.064	0.288	0.432	0.216

To analyse how unreliability affects the optimal acquisition policies, we present the optimal acquisition policies for both suppliers under the same action set (A ₁), assuming a more reliable flow of donations, which follows a binomial distribution X ∼ ℬ(3, 0.6) and a less reliable flow of donations, which follows a binomial distribution X ∼ ℬ(3, 0.4), as presented in Table 7.

Therefore, for every CR (impact scenario), we have 4 different problem scenarios. Hence, our experiment has a total of 24 scenarios, which accounts for the different risk areas (6 CRs) and percentage of donations received. For each support point (SP) we assume an arbitrary storage capacity. Table 8 summarises stock capacity per SP in units of product.

Table 8
Storage capacity in units of product for each SP.

Storage capacity	Support Points (SP)
	Centro	Cascatinha	Itaipava	Pedro do Rio	Posse
	70,000	25,000	15,000	10,000	10,000

The DC has a storage capacity of 120,000 units of product. The supply capacity for the reliable supplier is 100,000. There is no limitation on the amount of goods the DC can request as donation, hence, we assume it up to the total DC capacity of 120,000 units. The logistical costs assumed in the experiment are presented in Table 9, where the values are arbitrarily chosen, with shortage costs higher than delivery costs, in order to penalise shortages.

Table 9
Experiment costs.

Cost set	Value
Purchase + transport + handling per unit from the reliable supplier (a)	$ 04.00
handling + sorting per unit from donations - unreliable supplier (b)	$ 01.60
Inventory holding cost per unit	$ 01.00
Shortage cost per unit to a SP	$ 40.00
Delivery cost per unit to a SP	$ 05.00

We analyse our experiment under 3 different action sets A, recalling that each element in α ∈ A represents the total number of items to be acquired at a given period. The first action set assumes 11 possible actions, ranging from acquiring 0 units to acquiring 100,000 units, with increments of 10,000; this yields

$A_1=\{0;10,000;20,000;30,000;40,000;50,000;60,000;70,000;80,000;90,000;100,000\}.$

The second set (A ₂) also ranges from 0 to 100,000 units, but with increments of 5,000 units, yielding 21 possible actions. Finally, the third set (A ₃) ranges from 0 to 100,000 units with increments of 2,500 units, leading to 41 possible actions. This not only acts as sensitivity analysis for the proposed model, but allows us to assess how the model behaves regarding the time and memory requirements for different problem sizes.

To determine the duration of response operations in the region, we gathered information from Non-Governmental Organizations (NGO) that acted in the relief operations from the last land-slide disaster in the city, February 2022. Governmental information was not yet available during the execution of this experiment.

We applied a curve fitting method to the data (executed in python) and found that it fits a geometric distribution with parameter p = 0.0185 and mean = 66 days. Figure 7 compares the empirical cumulative probability distribution with the fitted distribution. The discount factor is then set up as γ = 1 − p = 0.9815 in the objective function - Eq. (2).

Figure 7
Comparison of empirical and geometrical cumulative probability distribution

The Q-Q plot presented in Figure 8 compares the empirical and the fitted geometric distributions and evinces a good match overall, with a minor distortion after the 8-th decile, where empirical data seems to be a little higher than the exponential distribution. The trend line for the Q-Q dispersion also demonstrates that the empirical data does not stray too far from the x=y line and that the empirical data gathered is more dispersed than the geometrical distribution. However, it is noteworthy that not all organizations acting on the disaster provided sufficient data and official data is still not available, as more data points lead to a smoother fit.

Figure 8
Q-Q graph for comparison of deciles between empirical and geometrical distributions

4.2 Results

This section discusses and summarises the experimental results. Firstly, we present the information that the framework offers to the decision maker in order to optimise the relief distribution. Then we briefly analyse the efficiency of the proposed algorithms in solving relatively large-scale models, demonstrating that the model provides optimal results quickly enough to be used in real operations.

4.2.1 Decision making

Table 10 shows the optimal purchase policy for the DC, assuming a set of 11 available actions (A ₁). The table displays the optimal purchase amount for each range of inventory levels. For instance, with an inventory level ranging from 0 to 19.999, the decision maker should acquire 100.000 units of product.

Table 10
Optimal policy for A ₁.

Inventory level (units)	Optimal purchase amount (units)
0 - 19,999	100,000
20,000 - 29,999	90,000
30,000 - 39,999	80,000
40,000 - 49,999	70,000
50,000 - 59,999	60,000
60,000 - 69,999	50,000
70,000 - 79,999	40,000
80,000 - 89,999	30,000
90,000 - 99,999	20,000
100,000 - 120,000	10,000

Along with Table 10, Figure 9 shows how the purchase policy varies with the DC inventory, for 11 available actions. They provide guidance to the decision maker, suggesting the optimal amount to acquire for the current inventory level in the DC. Observe that, as expected, the optimal purchase quantities decrease as the inventory level grows.

Figure 9
Optimal policies 11 available actions.

Figures 10 and 11 show the optimal policies assuming 21 (A ₂) and 41 (A ₃) actions, respectively. They show the same overall behaviour, with the order sizes decreasing as the stock level grows. It is also important to note that the more actions the decision maker can take, the faster the number of purchased items diminishes. This can be easily explained by the fact that the decision maker can make more precise acquisitions accordingly to his/her needs, avoiding overstocking whilst also avoiding shortages for the Support Points.

Figure 10
Optimal policies for 21 available actions.

Figure 11
Optimal policies for 41 available actions.

Table 11 complements Figures 9, 10 and 11, and unfolds the expected average costs for each action set. It demonstrates that with more available actions one can reduce the long-term cost, as additional purchasing actions can lead to smaller purchase costs.

Table 11
Average costs ($) for action sets studied.

	11 actions	21 actions	41 actions
Average costs	507,426.61	506,818.76	505,884.84

Figure 12 summarises and compares the optimal policies for all studied action sets. It shows that adding granularity to the action space allows for more flexible policies which, as depicted in Table 11, yield better long-term costs.

Figure 12
Summary of optimal policies for all available action sets.

The optimal distribution policies and detailed acquisition policies for both suppliers, are obtained through the execution of the TSSP, with inventory level and total purchase action taken as input parameters. This execution takes place in the parameter evaluation algorithm, as detailed in Section 3.6. Figure 13 shows the variation on the acquisition amounts from both suppliers with the inventory level of the DC, for A ₁ action set.

Figure 13
Acquisition variation per supplier for A ₁ action set.

For the decision maker, this information allows the determination of the amounts to be acquired from each supplier. It complements the information presented in Figure 9, giving precision to the purchase action, for each inventory level.

It is noticeable that as the inventory level rises, the amount of purchased items from the reliable supplier diminishes and the amount acquired from the unreliable supplier increases. This can also be explained by the amount of items the decision maker has at hand, as items in stock can replace items acquired from the unreliable supplier, allowing the decision maker to choose to buy from the unreliable supplier at a smaller cost.

Figure 13 also allows us to observe a saw-tooth behaviour for the acquisition policies of both suppliers, with the unreliable supplier being more stable whilst the reliable supplier sees their requests decreased. The spikes in the acquisitions from the reliable supplier match the decrease of the total amount of products purchased. With a smaller quantity purchased, the decision maker has a smaller slack to deal with unreliability, which leads to small increases in the acquisitions from the reliable supplier when the total purchase amount is reduced.

Finally, it shows that from 80,000 items in stock onward, all the requests can be made from the unreliable supplier, since the DC already have enough stock at hand, and therefore can take higher risks with the acquisition policy without jeopardizing the operation. Here, it is important to note that in Figure 13, the “Total” line is superposed by the “Unreliable supplier” line, because all the purchases are made to the unreliable supplier.

Figures 14 and 15 represent the purchase policies for reliable and unreliable supplier (donations), for both the less and more reliable donations scenarios, respectively. Alongside Figure 13, they display how the acquisition policy for the reliable supplier diminishes faster with in a more reliable flow of donations. This is an expected behavior, since donations are much cheaper than acquisitions from a reliable supplier, leading to smaller overall costs, presented in Table 12, if the decision maker can rely on them to run the operation.

Figure 14
Acquisition variation per supplier for A ₁ action set with less reliable donations (ℬ(3, 0.4)).

Figure 15
Acquisition variation per supplier for A ₁ action set with more reliable donations (ℬ(3, 0.6)).

Table 12
Average costs ($) for different levels of reliability of donations, assuming A ₁ action set.

	Less reliable scenario	Original scenario	More reliable scenario
Average costs	514,641.47	507,426.61	488,141.65

Finally, an example of distribution policy is given in Figures 16, 17, 18 and 19 which convey an optimal distribution policy for A ₁, assuming an inventory level equal to 0, for which Table 10 presents an optimal purchase policy of 100,000 items

Figure 16
Units of product sent to each SP per scenario for 25% of requested donations received.

Figure 17
Units of product sent to each SP per scenario for 50% of requested donations received.

Figure 18
Units of product sent to each SP per scenario for 75% of requested donations received.

Figure 19
Units of product sent to each SP per scenario for 100% of requested donations received.

Figure 16 presents the amount of goods sent to each SP for scenarios 1 to 6 (CR 1 to CR 6). In this example we assume that only 25% of the requested donations were received, hence, the DC could not meet the demand of all SP. However, we can see that the received goods were distributed according to the SP demands and that the overall shortage of the system is low.

This information is of vital importance for the decision maker. It guides the decision maker on how much to send to each SP in order to supply the affected location, considering the current situation of the system.

Figures 17, 18 and 19 represent a percentage of delivery of donations of 50%, 75% and 100%, respectively. We can observe that the overall shortage of the system reduces with increased number of donation requests met, with the smaller SP receiving less products in case of shortages. Observe, however, that even with 100% of requests met, the system still can not fulfil the demand of all SPs when the DC inventory level is 0 and demands are at their peak (CR6). Although the peak demand probability is low, at 8.3%, the results indicate that the decision maker may need to include higher purchase options in the system.

4.2.2 Computational remarks

Table 13 below summarise the computational results for the experiments. It shows the execution times (in seconds) and amount of physical memory (RAM, in megabytes) required to execute both algorithms that comprise the proposed framework, for each action set studied.

Table 13
Computational results.

KPI	11 actions			21 actions			41 actions
	TSSP	MDP	Total	TSSP	MDP	Total	TSSP	MDP	Total
Time (s)	3,251.69	60.33	3,312.02	6,234.51	108.94	6,343.45	11,855.16	207,03	12,062.19
RAM (MB)	324	69	393	601	85	686	1,153	332	1,485

One can observe that both algorithms perform efficiently, using few resources and low execution times, even for relatively large problems such as the scenarios with 41 available actions. Both processing times and physical memory usage grow linearly with the number of actions considered. The TSSP which carries the heaviest load of the processing time, because it is responsible for the execution of all integer models and calculation of future states of the systems and the transition probability matrix, takes an average of 3 hours to run for 120.000 states and 41 available actions, and the MDP takes around 4 minutes to run.

Furthermore, the parameter evaluation algorithm takes more time for a high number of available actions, once the number of executions of the TSSP directly depends on the number of actions being considered. However, since the framework needs to be executed only once during the lifetime of the operations, the total execution time of the framework is not prohibitive, as can be observed in Table 13.

It is noticeable that the TSSP executions require more physical memory than the MDP, for every scenario studied, and the amount of physical memory required to run the framework is stacked up. However, since the MDP adds a very little load to the overall framework, it is not a limitation for current computers.

Furthermore, the proposed experiment uses the maximum granularity for inventory management, which leads us to 120,000 possible states of the system, which greatly increases the problem size. The decision maker can choose a smaller granularity for the system, and discretize the inventory levels in bigger intervals. Simply changing the inventory level interval to 2 units instead of one already reduces the problem size by half, which reduces the amount of resources used and allows the decision maker to increase the number of actions in the action set, if needed.

5 CONCLUDING REMARKS

In this study we developed an inventory management framework that aims to minimise the average operational costs for a DC that acquires and distributes supplies in a disaster relief operation. We assume a dual-sourcing system, with one reliable and one unreliable supplier, and multiple supply points with stochastic demands that receive products from a centralized distribution centre. To the best of our knowledge, this is the first attempt to assume unreliability of a supplier as a constraint in humanitarian operations.

The framework is comprised of a Markov Decision Process and a parameter evaluation algorithm with a two-stage stochastic model. The MDP allowed us to determine the optimal ordering policy based on the inventory level of the system. The parameter evaluation algorithm is responsible for identifying the amount of goods to be acquired from each supplier, based on the ordering policy identified, and the amount of goods to be distributed to each Supply Point. The proposed framework is easy to implement and the results can be readily analysed by practitioners.

A dual-sourcing system allows us to assume donations as a viable source of goods, while adding a reliable supplier to guarantee efficiency of operations. Donations can be easily replaced by a local supplier, in order to help reestablish the local economy, without compromising the operations in case the local (unreliable) supplier is unable to fulfil the orders. Furthermore, by assuming a centralised decision making structure, where the DC coordinates with suppliers and SP, we implement both horizontal and vertical coordination mechanisms. This allows organizations, acting on different levels of the supply chain, to improve information and resource sharing and to avoid overlap, thus leading to a more reliable decision making process.

The experiments suggest that when the flow of donations is steady, or the unreliable supplier has a bigger confidence margin, the DC can refer to a cheaper solution to replenish its stockpile, greatly reducing the overall costs of the operation. It also allows us to conclude that the reliability of suppliers tends to have a bigger impact in the overall costs of the operations than the number of available actions.

The experiment also explores the impact of different action sets available to the decision maker, and finds that with more available options the decision maker can take more precise actions, reducing the overall costs the operations and avoiding overstocking while keeping a safe inventory level to minimise shortage. However, the study suggests that the number of available actions has a smaller impact on the average expected costs than the reliability of donations or unreliable suppliers.

The presented framework has room for improvement. We assume that both reliable and unreliable supplier have no lead-times, and the difference between them is only in the supply costs. Hence, the model can be extended to incorporate lead-times from both suppliers as a constraint, especially assuming that both suppliers can be in very different locations, and could have different lead-times and that donations may arrive in very erratic patterns.

Furthermore, our model assumes an unreliable source of goods, which may not deliver all the expected items in a specific decision epoch. Hence, DC must also decide how much shortage is allowed for each organisation when the inventory is insufficient to supply all the demand, considering only the impact of shortage in the overall costs. However, we do not assume an equitable distribution of goods across the HOs. Therefore, the model can be improved by assuming equity constraints.

Finally, we assume arbitrary demand fulfilment probabilities for the unreliable source of items. We do not focus on identifying the actual reliability of suppliers. Therefore, our framework could be improved by identifying a more realistic measure of the reliability of individual suppliers to incorporate it to proposed solution.

Data Availability

All the data utilised in the experiments are provided in the article.

References

• ABIDI H, LEEUW SD & KLUMPP M. 2013. Measuring Success in Humanitarian Supply Chains. International Journal of Business and Management Invention, 2(8): 31-39.
• ALEM D, CLARK A & MORENO A. 2016. Stochastic network models for logistics planning in disaster relief. European Journal of Operational Research, 255(1): 187-206.
• BALCIK B, BEAMON BM, KREJCI CC, MURAMATSU KM & RAMIREZ M. 2010. Coordination in humanitarian relief chains: Practices, challenges and opportunities. International Journal of Production Economics, 126(1): 22-34.
• BALCIK B, BOZKIR CDC & KUNDAKCIOGLU OE. 2016. A literature review on inventory management in humanitarian supply chains. Surveys in Operations Research and Management Science, 21(2): 101-116.
• BARBAROSOĞLU G & ARDA Y. 2004. A two-stage stochastic programming framework for transportation planning in disaster response. Journal of the Operational Research Society, 55(1): 43-53.
• BEAMON BM & KOTLEBA SA. 2006. Inventory management support systems for emergency humanitarian relief operations in South Sudan. The International Journal of Logistics Management, 17(2): 187-212.
• BEHL A & DUTTA P. 2018. Humanitarian supply chain management: a thematic literature review and future directions of research. Annals of Operations Research, 283(1-2): 1001-1044.
• BESIOU M & VAN WASSENHOVE LN. 2020. Humanitarian operations: A world of opportunity for relevant and impactful research. Manufacturing and Service Operations Management, 22(1): 135-145.
• CAMPOS V, BANDEIRA R & BANDEIRA A. 2012. A Method for Evacuation Route Planning in Disaster Situations. Procedia - Social and Behavioral Sciences, 54: 503-512.
• CNNBRASIL. 2022. Tragédia em Petrópolis completa um mês e mais de 600 seguem desabrigados. Available at: https://www.cnnbrasil.com.br/nacional/petropolis-tragedia-completa-um-mes-e-mais-de-600-seguem-desabrigados/
  » https://www.cnnbrasil.com.br/nacional/petropolis-tragedia-completa-um-mes-e-mais-de-600-seguem-desabrigados/
• COOK RA & LODREE EJ. 2017. Dispatching policies for last-mile distribution with stochastic supply and demand. Transportation Research Part E: Logistics and Transportation Review, 106: 353-371.
• DOWTY RA & WALLACE WA. 2010. Implications of organizational culture for supply chain disruption and restoration. International Journal of Production Economics , 126(1): 57-65.
• DUBEY R & ALTAY N. 2018. Drivers of coordination in humanitarian relief supply chains. In: The Palgrave Handbook of Humanitarian Logistics and Supply Chain Management. chap. 10, pp. 297-325. London: Palgrave Macmillan.
• FERREIRA CO, DE QUEIROZ ED & RICHTER M. 2017. A realidade de risco de movimentos de massa em Petrópolis, RJ: uma contribuição da Educação Ambiental para a Proteção Civil. Revista Percursos, 18(36): 34-65.
• FERREIRA GO, ARRUDA EF & MARUJO LG. 2018. Inventory management of perishable items in long-term humanitarian operations using Markov Decision Processes. International Journal of Disaster Risk Reduction, 31(May): 460-469.
• GOLI A & MALMIR B. 2019. A Covering Tour Approach for Disaster Relief Locating and Routing with Fuzzy Demand. International Journal of Intelligent Transportation Systems Research, 18(1): 140-152.
• HABIB MS, LEE YH & MEMON MS. 2016. Mathematical Models in Humanitarian Supply Chain Management: A Systematic Literature Review. Mathematical Problems in Engineering, 2016: 1-20.
• HAMDAN B & DIABAT A. 2020. Robust design of blood supply chains under risk of disruptions using Lagrangian relaxation. Transportation Research Part E: Logistics and Transportation Review , 134(January 2018): 101764.
• HEASLIP G, KOVÁCS G & HAAVISTO I. 2018. Cash-based response in relief: the impact for humanitarian logistics. Journal of Humanitarian Logistics and Supply Chain Management, 8(1): 87-106.
• HOLGUÍN-VERAS J, JALLER M, VAN WASSENHOVE LN, PÉREZ N & WACHTENDORF T. 2014. Material Convergence: Important and Understudied Disaster Phenomenon. Natural Hazards Review, 15(1): 1-12.
• IAKOVOU E, VLACHOS D, KERAMYDAS C & PARTSCH D. 2014. Dual sourcing for mitigating humanitarian supply chain disruptions. Journal of Humanitarian Logistics and Supply Chain Management , 4(2): 245-264.
• KAATRUD DB, SAMII R & VAN WASSENHOVE LN. 2003. UN Joint Logistics Centre: A Coordinated Response to Common Humanitarian Logistics Concerns. Tech. rep.. UNJLC, Rome.
• KABRA G, RAMESH A & ARSHINDER K. 2015. Identification and prioritization of coordination barriers in humanitarian supply chain management. International Journal of Disaster Risk Reduction , 13: 128-138.
• KOVÁCS G & SPENS KM. 2007. Humanitarian logistics in disaster relief operations. International Journal of Physical Distribution & Logistics Management, 37(2): 99-114.
• KREJCI CC. 2015. Hybrid simulation modeling for humanitarian relief chain coordination. Journal of Humanitarian Logistics and Supply Chain Management , 5(3): 325-347.
• LEWIN R, BESIOU M, LAMARCHE JB, CAHILL S & GUERRERO-GARCIA S. 2018. Delivering in a moving world. . . looking to our supply chains to meet the increasing scale, cost and complexity of humanitarian needs. Journal of Humanitarian Logistics and Supply Chain Management , 8(4): 518-532.
• LI C, ZHANG F, CAO C, LIU Y & QU T. 2019. Organizational coordination in sustainable humanitarian supply chain: An evolutionary game approach. Journal of Cleaner Production, 219: 291-303.
• LU DK, PETTIT S & BERESFORD A. 2006. Critical success factors for emergency relief logistics. WHAMPOA-An Interdisciplinary Journal, 51: 177-184.
• MALMIR B & ZOBEL CW. 2021. An applied approach to multi-criteria humanitarian supply chain planning for pandemic response. Journal of Humanitarian Logistics and Supply Chain Management , 11(2): 320-346.
• MCCOY JH & BRANDEAU ML. 2011. Efficient stockpiling and shipping policies for humanitarian relief: UNHCR’s inventory challenge. OR Spectrum, 33(3): 673-698.
• MERAKLI M & KÜÇÜKYAVUZ S. 2019. Risk aversion to parameter uncertainty in Markov decision processes with an application to slow-onset disaster relief. IISE Transactions, 52(8): 811-831.
• MISHRA JL, CHIWENGA KD, MISHRA N & CHOUDHARY S. 2020. Understanding Humanitarian Supply Chain Through Causal Modelling. South Asian Journal of Business and Management Cases, 9(3): 317-329.
• MOHAMMADI S, AVAKH DARESTANI S, VAHDANI B & ALINEZHAD A. 2020. A robust neutrosophic fuzzy-based approach to integrate reliable facility location and routing decisions for disaster relief under fairness and aftershocks concerns. Computers and Industrial Engineering, 148(January): 106734.
• MURRAY S. 2005. How to deliver on the promises. Financial Times, .
• NOYAN N, MERAKLI M & KÜÇÜKYAVUZ S. 2019. Two-stage stochastic programming under multivariate risk constraints with an application to humanitarian relief network design. Mathematical Programming, .
• PERDANA T, ONGGO BS, SADELI AH, CHAERANI D, ACHMAD ALH, HERMIATIN FR & GONG Y. 2022. Food supply chain management in disaster events: A systematic literature review. International Journal of Disaster Risk Reduction , 79: 103183.
• PETTIT S & BERESFORD A. 2009. Critical success factors in the context of humanitarian aid supply chains. International Journal of Physical Distribution & Logistics Management , 39(6): 450-468.
• POWELL WB. 2011. Approximate Dynamic Programming: Solving the Curses of Dimensionality. 2nd ed.. Wiley Series in Probability and Statistics. Hoboken, NJ, USA: Wiley.
• POWER DJ, SOHAL AS & RAHMAN SU. 2001. Critical success factors in agile supply chain management - An empirical study. International Journal of Physical Distribution & Logistics Management , 31(4): 247-265.
• PUTTERMAN ML. 1994. Markov Decision Processes Discrete Stochastic Dynamic Programming. British Columbia: John Wiley-Intersciencie.
• RAHMANI D, ZANDI A, PEYGHALEH E & SIAMAKMANESH N. 2018. A robust model for a humanitarian relief network with backup covering under disruptions: A real world application. International Journal of Disaster Risk Reduction , 28: 56-68.
• REZAEI-MALEK M, TAVAKKOLI-MOGHADDAM R, ZAHIRI B & BOZORGI-AMIRI A. 2016. An interactive approach for designing a robust disaster relief logistics network with perishable commodities. Computers & Industrial Engineering, 94: 201-215.
• SNYDER LV. 2006. Facility location under uncertainty: a review. IIE Transactions, 38(7): 547-564.
• TABBARA L. 2008. Emergency Relief Logistics : Evaluation of Disaster Response Models. Ph.D. thesis. Oxford Brookes University.
• THEOPRATIQUE. 2017. Plano municipal de redução de risco de movimentos de massa. Petrópolis. 1-218 pp.
• VAN WASSENHOVE LN. 2006. Humanitarian aid logistics: supply chain management in high gear. Journal of the Operational Research Society , 57(5): 475-489.
• ÇELIK M, ERGUN O, JOHNSON B, KESKINOCAK P, LORCA A, PEKGÜN P & SWANN J. 2014. Humanitarian Logistics. chap. Chapter 2, pp. 18-49.

¹

EM-DAT, https://www.emdat.be

²

https://fts.unocha.org/appeals/overview/2023

³

EM-DAT, https://www.emdat.be

AI Declaration

No AI was used.

Funding

No funding was received.

*

Corresponding author

Editor responsible for the review

Editor-in-Chief: Antônio Augusto Chaves.

Publication Dates

• Publication in this collection
  28 Aug 2026
• Date of issue
  2026

History

• Received
  06 Feb 2026
• Accepted
  20 Apr 2026