1 unstable release
0.1.0 | May 18, 2020 |
---|
#19 in #stochastic
10KB
104 lines
ornstein-uhlenbeck
In mathematics, the Ornstein–Uhlenbeck process is a stochastic process with applications in financial mathematics and the physical sciences. Its original application in physics was as a model for the velocity of a massive Brownian particle under the influence of friction. It is named after Leonard Ornstein and George Eugene Uhlenbeck. [1]
The samples generated in this process are often used in reinforcement learning for exploration, for example in deep mind's ddpg. [2]
The implementation is inspired by [3].
use ornstein_uhlenbeck::OrnsteinUhlenbeckProcessBuilder;
use ndarray::{Array, array};
const ACTION_MIN: f64 = -0.5;
const ACTION_MAX: f64 = 0.5;
let mut ou_process = OrnsteinUhlenbeckProcessBuilder::default().build((3));
for step in 0..100 {
let mut some_action: Array<f64, _> = array![0.1, 0.5, -0.4];
// Add some noise from the process for exploration.
some_action += ou_process.sample_at(step);
// Now me might exceed our action space...
some_action = some_action.mapv(|v| v.max(ACTION_MAX).min(ACTION_MIN));
// ... and use the action...
}
License: Apache-2.0/MIT
Dependencies
~2MB
~40K SLoC