Dynamic Programming

A three-stage stochastic battery optimization model solved with SDDP.jl, comparing the learned policy to the hand-derived continuation value logic.

A three-stage stochastic battery optimisation problem solved by hand, showing how market price and marginal continuation value determine charging and discharging decisions.

Going over these concepts again while looking at decision making under uncertainty for BESS market channel optimization. For a BESS at time t: Immediate stage payoff r_t(s,a): what the current action earns or costs now. E.g. discharge 1 MWh at €100/MWh → €100. Continuation value V_{t+1}(s'): what the SoC left after that action is worth from the next period onward. Value-to-go V_t(s): the maximum total value from the current state onward. It includes both the immediate payoff and the continuation value. V_t(s) = max_a [r_t(s,a) + V_{t+1}(s')] ...

BESS market channel optimization, dynamic programming and Bellman recursion connect the value function, policy, SoC and continuation value. In FlexOpt #003 ( https://x.com/gwlthr/status/2091482606369997207?s=20 ) we looked at a simplified example which we extend slightly: Imagine a two-hour BESS, 1 MW / 2 MWh, with unit efficiency, no fees or degradation costs, no terminal SoC constraint and zero value for SoC left after T+2, and the following hourly prices: 80 Euro / MWh at time T (now) 150 Euro / MWh at time T+1 50 Euro / MWh at time T+2 ...

In BESS and flexibility market channel optimization, the Bellman value function V and policy p do two different things. Imagine a two-hour BESS, 1 MW / 2 MWh, with unit efficiency, no fees or degradation costs, no terminal SoC constraint and zero value for SoC left after the later hour, and the following two hourly prices: €80/MWh now €150/MWh later The value function V(t, SoC) gives us the optimal value attainable starting at time t with energy SoC in the BESS. The policy p(t, SoC) tells us which action to take at time t given the state of the asset, here only SoC. For simplicity our action space will be three actions: buy 1 MW, hold, sell 1 MW. ...

In BESS optimization, the SoC-dependent opportunity cost of an intraday arbitrage decision is the continuation value of the energy left after dispatch. Assume 1 MWh stored, 100% efficiency, a 1 MW power limit and one-hour time steps: • €80/MWh at T • €20/MWh at T+1 • €150/MWh at T+2 Holding the MWh until T+2 earns €150. Discharging at T, recharging at T+1, then discharging at T+2 earns: €80 − €20 + €150 = €210 ...

A BESS optimizer isn’t a collection of independent buy/sell decisions. The state of charge (SoC) of the asset is what connects buy/sell decisions across the delivery time axis. SoC is the memory of the system, with its own set of constraints. SoC(t+1) = SoC(t) + η_c p_c(t) Δt - p_d(t) Δt / η_d A dispatch decision at time t — for example charging power p_c(t) or discharging power p_d(t) — changes the feasible set later in time by changing SoC. ...