Real-world decision making through MABs and multi-level reinforcement learning on agent-based models

Harshavardhan Kamarthi

Presented at Data Seminar (Oct 8)

State Level Epidemic Model

uses a variant of SIR model that accounts for vaccination

Transmission rate is location and time specific based on stringency level

Vaccination rate proportional to population

\beta_{i,t} = \beta_i^{\pi} \pi_{i,t-d} + \beta_0

\beta_{i,t} = \beta_i^{\pi} \pi_{i,t-d} + \beta_0

\Delta V_i = v_i n_i

\Delta V_i = v_i n_i

State Level Econ. Model

Unemployment proportional to stringency level

U_{i,t} = \text{softplus}\left( \sum_{k=1}^K w_{i,k} \tilde{U}_{i,t}^k \right) + U_i^0

U_{i,t} = \text{softplus}\left( \sum_{k=1}^K w_{i,k} \tilde{U}_{i,t}^k \right) + U_i^0

\tilde{U}_{i,t}^k = \sum_{t'=t-L}^t e^{(t'-t)/\lambda_k} \Delta\pi_{i,t'}

\tilde{U}_{i,t}^k = \sum_{t'=t-L}^t e^{(t'-t)/\lambda_k} \Delta\pi_{i,t'}

Economic output depends on available workers and federal subsidies

\omega_{i,t} = c(n_i - D_{i,t} - \eta I_{i,t}) - U_{i,t}\\ P_{i,t} = k\omega_{i,t} + T_{i,t}

\omega_{i,t} = c(n_i - D_{i,t} - \eta I_{i,t}) - U_{i,t}\\ P_{i,t} = k\omega_{i,t} + T_{i,t}

Rewards

Condiser health index and economic productivity

r_{i,t} = \alpha_i \Delta H_i + (1-\alpha_i) \Delta E_i\\ \Delta H_{i,t} = - \Delta D_{i,t}\\ \Delta E_{i,t} = crra\left( \frac{P_{i,t}}{P_i^0}\right)

r_{i,t} = \alpha_i \Delta H_i + (1-\alpha_i) \Delta E_i\\ \Delta H_{i,t} = - \Delta D_{i,t}\\ \Delta E_{i,t} = crra\left( \frac{P_{i,t}}{P_i^0}\right)

At federal level aggregate state-level productivity except penalize for subsidies

P_{f,t} = \sum_{i} P_{i,t} - c T_{i,t}

P_{f,t} = \sum_{i} P_{i,t} - c T_{i,t}

Real-world decision making through MABs and multi-level reinforcement learning on agent-based models Harshavardhan Kamarthi Presented at Data Seminar (Oct 8)

AI Economist

By Harshavardhan Kamarthi

AI Economist

4 years ago
135

Harshavardhan Kamarthi