An Agent-based Approach To Modeling The Beer Game

An agent-based model of the beer distribution game. Each of the players in the game is modeled as an agent.
Keywords

agent-based modeling, system dynamics,abm, bptk, bptk-py, python, business simulation

An Agent-based Approach To Modeling The Beer Game

An agent-based model of the beer distribution game. Each of the players in the game is modeled as an agent.

This page explores the game using deterministic agents; Training AI to Play the Beer Game lets the agents learn by themselves using a reinforcement learning approach.

Agent-based Modeling with BPTK-Py explains how agents are built in general; the rules, the costs and the dynamics of the game are on Understanding the Beer Game.

The Agents

Agents

Each player in the game is modeled by an agent, so there are five agents: the consumer and the agents that are part of the supply chain, i.e. the retailer, the wholesaler, the distributor and the brewery.

The supply chain agents are essentially all the same - they have an internal state that captures:

  • Inventory, i.e. how much beer the agent has in stock.
  • Outstanding Orders, i.e. how much beer the agent has ordered from its supplier but has not received yet.
  • Backorder, i.e. how much beer the agent’s customers have ordered but which the agent hasn’t delivered yet.

In each timestep, the agent has to react to the following events:

  • Delivery. The beer delivered from the agent’s supplier.
  • Order. The beer ordered by the agent’s customer.

At the end of each timestep, after the agent has received deliveries and orders, the agent has to make an order decision, which is then passed on to its supplier as an order.

Agents
from BPTK_Py.bptk import bptk 

bptk = bptk()

Steady State

bptk.plot_scenarios(
    scenario_managers=["smBeergame"],
    kind="area",
    scenarios=["steady"],
    agents=["brewery","distributor","wholesaler","retailer"],
    agent_states=["active"],
    agent_properties=["outstanding_orders"],
    agent_property_types=["total"],
    return_df=False, format="axes"
)

Typical Player Behavior

The policy is explained on Understanding the Beer Game.

Typical player behaviour (which leads to the “whiplash” effect).

bptk.plot_scenarios(
    scenario_managers=["smBeergame"],
    kind="area",
    scenarios=["typical"],
    agents=["brewery","distributor","wholesaler","retailer","consumer"],
    agent_states=["active"],
    agent_properties=["outgoing_order"],
    agent_property_types=["total"],
    return_df=False, format="axes"
)
bptk.plot_scenarios(
    scenario_managers=["smBeergame"],
    kind="area",
    scenarios=["typical"],
    agents=["brewery","distributor","wholesaler","retailer"],
    agent_states=["active"],

    agent_properties=["surplus"],
    agent_property_types=["total"],
    return_df=False, format="axes"
)
bptk.plot_scenarios(
    scenario_managers=["smBeergame"],
    kind="area",
    scenarios=["typical"],
    agents=["brewery","distributor","wholesaler","retailer"],
    agent_states=["active"],
    agent_properties=["inventory"],
    agent_property_types=["total"],
    return_df=False, format="axes"
)
bptk.plot_scenarios(
    scenario_managers=["smBeergame"],
    kind="area",
    scenarios=["typical"],
    agents=["brewery", "distributor","wholesaler","retailer"],
    agent_states=["active"],
    agent_properties=["cost"],
    agent_property_types=["total"], format="axes"
)
bptk.plot_scenarios(
    scenario_managers=["smBeergame"],
    kind="area",
    scenarios=["typical"],
    agents=["controlling"],
    agent_states=["active"],
    agent_properties=["supply_chain_cost","target_supply_chain_cost"],
    agent_property_types=["total"], format="axes"
)

Ignore Backorder

The policy is explained on Understanding the Beer Game.

bptk.plot_scenarios(
    scenario_managers=["smSmartBeergame"],
    kind="area",
    scenarios=["ignore_backorder"],
    agents=["retailer"],
    agent_states=["active"],
    agent_properties=["outgoing_order","inventory","incoming_order"],
    agent_property_types=["total"],
    return_df=False, format="axes"
)
bptk.plot_scenarios(
    scenario_managers=["smSmartBeergame"],
    kind="area",
    scenarios=["ignore_backorder"],
    agents=["retailer"],
    agent_states=["active"],
    agent_properties=["total_cost","target_cost"],
    agent_property_types=["total"],
    return_df=False, format="axes"
)

Include Supply Line

The policy is explained on Understanding the Beer Game.

bptk.plot_scenarios(
    scenario_managers=["smSmartBeergame"],
    kind="area",
    scenarios=["include_supply_line"],
    agents=["retailer"],
    agent_states=["active"],
    agent_properties=["outgoing_order","inventory","incoming_order"],
    agent_property_types=["total"],
    return_df=False, format="axes"
)
bptk.plot_scenarios(
    scenario_managers=["smSmartBeergame"],
    kind="area",
    scenarios=["include_supply_line"],
    agents=["retailer"],
    agent_states=["active"],
    agent_properties=["total_cost","target_cost"],
    agent_property_types=["total"],
    return_df=False, format="axes"
)
bptk.plot_scenarios(
    scenario_managers=["smSmartBeergame"],
    kind="area",
    scenarios=["include_supply_line"],
    agents=["retailer"],
    agent_states=["active"],
    agent_properties=["surplus"],
    agent_property_types=["total"],
    return_df=False, format="axes"
)
bptk.plot_scenarios(
    scenario_managers=["smSmartBeergame"],
    kind="area",
    scenarios=["include_supply_line"],
    agents=["brewery","distributor", "wholesaler", "retailer","consumer"],
    agent_states=["active"],
    agent_properties=["outgoing_order"],
    agent_property_types=["total"],
    return_df=False, format="axes"
)
bptk.plot_scenarios(
    scenario_managers=["smSmartBeergame"],
    kind="area",
    scenarios=["include_supply_line"],
    agents=["controlling"],
    agent_states=["active"],
    agent_properties=["supply_chain_cost","target_supply_chain_cost"],
    agent_property_types=["total"], format="axes"
)

Slow Inventory Adjustment

The policy is explained on Understanding the Beer Game.

bptk.plot_scenarios(
    scenario_managers=["smSmartBeergame"],
    kind="area",
    scenarios=["slow_inventory_adjustment"],
    agents=["brewery","distributor", "wholesaler", "retailer","consumer"],
    agent_states=["active"],
    agent_properties=["outgoing_order"],
    agent_property_types=["total"],
    return_df=False, format="axes"
)
bptk.plot_scenarios(
    scenario_managers=["smSmartBeergame"],
    kind="area",
    scenarios=["slow_inventory_adjustment"],
    agents=["controlling"],
    agent_states=["active"],
    agent_properties=["supply_chain_cost","target_supply_chain_cost"],
    agent_property_types=["total"],
    return_df=False, format="axes"
)
bptk.plot_scenarios(
    scenario_managers=["smSmartBeergame"],
    kind="area",
    scenarios=["slow_inventory_adjustment"],
    agents=["brewery","distributor","wholesaler","retailer"],
    agent_states=["active"],
    agent_properties=["total_cost"],
    agent_property_types=["total"],
    return_df=False, format="axes"
)
bptk.plot_scenarios(
    scenario_managers=["smSmartBeergame"],
    kind="area",
    scenarios=["slow_inventory_adjustment"],
    agents=["retailer"],
    agent_states=["active"],
    agent_properties=["total_cost","target_cost"],
    agent_property_types=["total"],
    return_df=False, format="axes"
)
bptk.plot_scenarios(
    scenario_managers=["smSmartBeergame"],
    kind="area",
    scenarios=["slow_inventory_adjustment"],
    agents=["brewery","distributor", "wholesaler", "retailer"],
    agent_states=["active"],
    agent_properties=["inventory"],
    agent_property_types=["total"],
    return_df=False, format="axes"
)
bptk.plot_scenarios(
    scenario_managers=["smSmartBeergame"],
    kind="area",
    scenarios=["slow_inventory_adjustment"],
    agents=["retailer"],
    agent_states=["active"],
    agent_properties=["surplus","target_surplus"],
    agent_property_types=["total"], format="axes"
)

Order Balance Strategy

One simple strategy is to manage the order balance, i.e. the difference between incoming orders (those coming from the customer) and outgoing orders (those going to the supplier). Put simply, we would expect the balance to be a constant equal to the desired inventory, so a sensible ordering strategy is:

Int(Max(Incoming Order + (2 * Incoming Order + Target Inventory - Order Balance) / Inventory Adjustment Time, 0))

This policy is nice and simple, and it is almost good enough to keep the supply chain costs below the target of $29,300.

bptk.plot_scenarios(
    scenario_managers=["smSmartBeergame"],
    kind="area",
    scenarios=["order_balance"],
    agents=["brewery","distributor", "wholesaler", "retailer","consumer"],
    agent_states=["active"],
    agent_properties=["outgoing_order"],
    agent_property_types=["total"],
    return_df=False, format="axes"
)
bptk.plot_scenarios(
    scenario_managers=["smSmartBeergame"],
    kind="area",
    scenarios=["order_balance"],
    agents=["brewery","distributor", "wholesaler", "retailer"],
    agent_states=["active"],
    agent_properties=["inventory"],
    agent_property_types=["total"],
    return_df=False, format="axes"
)
brewery_active_total_cost_total distributor_active_total_cost_total wholesaler_active_total_cost_total retailer_active_total_cost_total
t
1.0 200.0 200.0 200.0 200.0
2.0 400.0 400.0 400.0 400.0
3.0 600.0 600.0 600.0 800.0
4.0 800.0 868.0 1140.0 1125.0
5.0 1185.0 1580.0 1637.0 1725.0
6.0 2110.0 2316.0 2484.0 2225.0
7.0 2807.0 3459.0 3275.0 3025.0
8.0 3320.0 4264.0 4404.0 3725.0
9.0 3683.0 4792.0 5135.0 4725.0
10.0 3923.0 5092.0 5539.0 5294.0
11.0 4123.0 5292.0 5739.0 5507.0
12.0 4323.0 5492.0 5939.0 5707.0
13.0 4523.0 5692.0 6139.0 5907.0
14.0 4723.0 5892.0 6339.0 6107.0
15.0 4923.0 6092.0 6539.0 6307.0
16.0 5123.0 6292.0 6739.0 6507.0
17.0 5323.0 6492.0 6939.0 6707.0
18.0 5523.0 6692.0 7139.0 6907.0
19.0 5723.0 6892.0 7339.0 7107.0
20.0 5923.0 7092.0 7539.0 7307.0
21.0 6126.0 7292.0 7739.0 7507.0
22.0 6331.0 7492.0 7939.0 7707.0
23.0 6539.0 7692.0 8139.0 7907.0
24.0 6749.0 7892.0 8339.0 8107.0
as_table(bptk.plot_scenarios(
    scenario_managers=["smSmartBeergame"],
    kind="area",
    scenarios=["order_balance"],
    agents=["brewery","distributor","wholesaler","retailer"],
    agent_states=["active"],
    agent_properties=["total_cost"],
    agent_property_types=["total"],
    return_df=True
))
bptk.plot_scenarios(
    scenario_managers=["smSmartBeergame"],
    kind="area",
    scenarios=["order_balance"],
    agents=["controlling"],
    agent_states=["active"],
    agent_properties=["supply_chain_cost","target_supply_chain_cost"],
    agent_property_types=["total"],
    return_df=False, format="axes"
)
bptk.plot_scenarios(
    scenario_managers=["smSmartBeergame"],
    kind="area",
    scenarios=["order_balance"],
    agents=["retailer"],
    agent_states=["active"],
    agent_properties=["backorder"],
    agent_property_types=["total"],
    return_df=False, format="axes"
)
brewery_active_order_balance_total distributor_active_order_balance_total wholesaler_active_order_balance_total retailer_active_order_balance_total
t
1.0 600 600 600 600
2.0 600 600 600 675
3.0 600 600 693 740
4.0 600 717 772 797
5.0 746 813 839 847
6.0 862 892 896 891
7.0 955 957 945 929
8.0 1028 1010 986 962
9.0 1086 1053 1021 991
10.0 1130 1088 1050 1017
11.0 1164 1116 1075 1039
12.0 1190 1139 1096 1059
13.0 1209 1157 1114 1076
14.0 1223 1171 1129 1091
15.0 1233 1182 1141 1104
16.0 1239 1191 1151 1116
17.0 1243 1197 1160 1126
18.0 1244 1202 1167 1135
19.0 1245 1206 1173 1143
20.0 1244 1209 1178 1150
21.0 1243 1211 1182 1156
22.0 1241 1212 1185 1161
23.0 1238 1212 1188 1165
24.0 1235 1212 1190 1169
as_table(bptk.plot_scenarios(
    scenario_managers=["smSmartBeergame"],
    kind="area",
    scenarios=["order_balance"],
    agents=["brewery","distributor","wholesaler","retailer"],
    agent_states=["active"],
    agent_properties=["order_balance"],
    agent_property_types=["total"],
    return_df=True
))
bptk.plot_scenarios(
    scenario_managers=["smSmartBeergame"],
    kind="area",
    scenarios=["order_balance"],
    agents=["retailer"],
    agent_states=["active"],
    agent_properties=["surplus","target_surplus"],
    agent_property_types=["total"], format="axes"
)