Research
Swarm Reservoirs
What’s In a Reservoir?
Reservoir computing has been discovered and rediscovered multiple times in multiple fields. For this article we will use a machine learning description of a reservoir computer: a Recurrent Neural Network where the hidden layers are somewhat randomly connected and initialized, and where only the output layer’s weights are trained. This turns out to be good enough for a large number of use cases and therefore saves quite a bit on compute and training.
In effect, there are a large number of systems, not just randomly connected artificial neurons, that give behavior good enough to perform reservoir computing with weights trained on the output. Roughly speaking, the idea is to have a nonlinear system of some kind where the input can be effectively “projected to higher dimensions” through the output, which a linear regression can then be trained upon.
As a result, it has become common to explore various substrates as candidate reservoirs: buckets of water, dead fish floating in a vortex, the tentacle of an octopus, mass spring systems, FPGAs, soft robots…all of these things can be used as a reservoir, though their performance varies according to their properties and the tasks that they would be trained upon.
There has been some debate in the past about other rules or even just heuristics concerning what is necessary for a reservoir or at least what makes a good one. A common concept cited is the “Echo State Principle” (ESP), which more or less means that without an input signal driving the system the behavior decays back into some sort of steady state, like the surface of a lake slowly settling after having been disturbed by a boat or rock. While not a hard-and-fast rule, it is worth considering when building a reservoir.
Swarms are Cool
Swarms, with their dance of collective behavior, represent an interesting alternative reservoir substrate. Flocks of birds or schools of fish exhibit an ability to process temporal data through their interactions that are just as complex as standard reservoirs such as Echo State Networks (ESNs) and Liquid State Machines (LSMs). One difference is that swarms do this by embodying a fluid, ever-changing reservoir of potential computational power, without the need for stability or structured connections between nodes.
By harnessing the natural computational capabilities of swarms—where each individual’s movement contributes to the collective computation—their application in RC lets us explore a bit more what collective decision making might be like.
Boid rules
Our boid flock of 200 agents in a 2D, toroidal space serves as a dynamic substrate, operating under three fundamental rules that guide their collective behavior. These rules are:
- Cohesion, which pulls boids towards the average position of their neighbors to maintain the group;
- Separation, which prevents overcrowding by maintaining a minimum distance between each boid; and
- Alignment, which steers boids towards the average heading of nearby flockmates.
Together, these rules enable the flock to simulate complex, coordinated movement.
How to use Boids as a Reservoir
The key to using any substrate as a reservoir is knowing how to input signals into the system and then interpret the resultant behavior as the output. In our framework, the input is modulated through a "temperature" parameter that affects the velocity of each boid at each timestep, altering their collective movement patterns, analogous to external stimuli affecting a natural system. Outputs are then derived from the average x,y velocity of the swarm and the mean local order of the system, which is calculated using the velocities of the boids relative to their neighbors. This setup captures the essence of the dynamic responses within the swarm, providing a rich state space. Ridge regression is trained on the outputs from the dataset, effectively “tuning the output weights, if you were to think of it like a neural network.
Initial Performance: Sinusoids and Mackey-Glass
So, does this work at all? Yes, at least a little bit. We have a proof of concept that a swarm is usable as a reservoir, even if it will take more work to know its specific strengths and weaknesses as a substrate. Part of this initial encouragement comes from mean squared error (MSE) results. On a time-series dataset generated by a Mackey-Glass function we can observe MSE=0.0303. This is not fantastic, but not terrible either as a first attempt. We can sanity check this result by testing against a sinusoid, as such a function, while relatively simple, can confuse systems since there are identical input values (x) that require a different output (y) depending on the slope of the wave. Here we see MSE=0.2704, roughly the same ballpark of values.
At first blush, it also appears that the swarm conforms to the ESP, given that when we perturb the system strongly for a bit and then stop, we see the behavior decay back towards the baseline behavior (or “attractor state”, if you will).
FORCE Learning
Interestingly, if we implement a different learning program that goes step-by-step called FORCE learning, we see some modest improvement. Sinusoid error drops to MSE=0.2319, while Mackey-Glass performance improves to MSE=0.2032. This suggests that while tuning the number of boids, the strength of the rules, or other parameters may give some modest improvement, it is likely that we’re approaching the fundamental limit of what this swarm can do for this problem.
What’s Next?
The first things to do are many more runs, averaged, to smooth out the noise as much as possible, and then to validate our assumptions about gains from further parameter tuning. Following that it’s probably time to consider a more rigorous and exhaustive characterization of the swarm as a substrate. The CHARC framework provides a method for exploring and evaluating the behavior space of reservoir substrates. Naturally, we’ll want to know where 2D swarms fit in this framework.
Beyond that and depending on what we find, the idea can be pushed in various directions. Do 3D swarms behave fundamentally differently? What is it like if we have multiple input methods so we can, for example, plug in the terms for Mackey-Glass directly instead of using a generated time-series as a single input? What happens when agents are more heterogeneous, or when interaction rules become more complex?
There is a line between this and more robust and representative multi-agent collaboration somewhere. The question is simply what it might take to find it.