Optimization Problems for Interacting Particle Systems and Corresponding Mean‐field Limits

We summarize the relations of optimality systems for an interacting particle dynamic in the microscopic and in the kinetic description. In particular, we answer the question if the passing to the mean‐field limit and deriving the first order optimality system can be interchanged without affecting the results. The answer is affirmative, if one derives the optimality system on the kinetic level in the metric space (𝒫2, 𝒲2). Moreover, we discuss the relation of to the adjoint PDE derived in the L2‐sense. Here, the gradient can be derived as expected from the calculus in Wasserstein space.


Introduction
Interacting particle systems and their kinetic descriptions were extensively studied in the last decades. The applications range from formation of swarms of birds or schools of fish to dogs herding sheep, consensus formation and follower-leader dynamics [1][2][3][4][5][6]. Recently, even various particle swarm schemes for global optimization were proposed [7,8]. Followerleader dynamics or dogs herding a large amout of sheep naturally lead to a kinetic description involving a probability measure representing the positions and velocities of the crowd [9][10][11]. Naturally, there arises the question of optimal evacuation, leading to a constrained optimal control problem. This problem can be formalized as optimal control problem in the space of probability measures with finite second moment P 2 supplemented with the Wasserstein-2-metric W 2 which is a non-standard setting for optimal control, since (P 2 , W 2 ) is not a Hilbert space. The challenge is therefore to derive optimality conditions in this new framework. This article gives a glimpse on the recent advances in this field of study.

Optimal Control Problem and Relations of the Optimality Systems
We begin with a general problem and restrict the considerations to first order systems. In fact, we consider a fixed time interval t ∈ [0, T ] and N interacting particles collected in the vector t → x(t) = (x i (t)) i=1,...,N , x i (t) ∈ R dN that encounter some control represented by the variable t → u(t) ∈ R dm . This leads to the general system Passing to the mean-field limit with the empirical measure µ N (t, Additionally, we introduce the cost functional which completes the optimization problem The cost functional may for example involve the center of mass or the variance w.r.t µ. For further details we refer to [10,12,13]. The interesting question is now Can we interchange the limit N → ∞ and the optimization procedure without affecting the results?
The answer is affirmative and the main findings are summarized in the flow chart in Fig. 1. Indeed, starting from the state ODE, one can compute the corresponding adjoint ODE for the adjoint state p using the standard L 2 -calculus. Considering now the coupled system of state and adjoint ODEs with its empirical density 1 ) and passing to the mean-field limit following the steps in [14][15][16][17] leads to coupled PDE in R 2d and containing both, state and adjoint information, at once. The high-dimensional PDE is supplemented with both, initial and terminal data. In fact, the initial data is posed in the states and the terminal data for the adjoints. The PDE for the coupled system was found independently in [18,19] as result of the Pontriagyn Principle and the Hamiltonian system corresponding to (Opt).

of 2
Section 19: Optimization of differential equations Fig. 1: Flow chart summarizing the relations of the optimization of the particle and the mean-field level.
Passing to the limit N → ∞ on the state ODE yields the corresponding state PDE. Further, an adjoint calculus in (P 2 , W 2 ) introduced in [13] leads to a corresponding mean-field adjoint which is an evolution equation for a vector-valued measure. The computations are based on theory discussed in [20][21][22][23]. The mean-field adjoint can also be derived from the evolution equation for ν by taking the first moment w.r.t. the adjoint variable.
On the other hand, one can assume that µ has an L 2 -density and apply standard L 2 -calculus to the state PDE. Then, one obtains a scalar adjoint equation which does not preserve mass. This is expected since the mass-conservation of the transport equation is not included in this metric. Nevertheless, taking the gradient of this adjoint equation and evaluating it along the characteristics of the state ODE leads to the corresponding adjoint particle system. In our opinion, this relation is a justification for using the L 2 -adjoint, instead of the measure-valued W-adjoint, for the numerical treatment of the optimization problem as in [10]. The results therein show that this approach leads to very satisfying results. The relation involving the gradient was found in many other contributions, see for example [24,25]. Another interesting fact is that the W-adjoint can be charaterized as first moment of the evolution equation for ν. See [13] for more details.
To summarize, passing to the mean-field limit and optimizing can be interchanged, if (P 2 , W 2 ) is chosen as metric space on the PDE level. Moreover, a convergence rate for the convergence of the controls in the limit N → ∞ can be shown [13]. Recent results can be also found in [26].