Evolutionary Algorithms implementations, for various (discrete & continuous) optimization problems, including for autonomous agent control.
Evolutionary Algorithms implementations, for various (discrete & continuous) optimization problems.
This repository contains implementations of the following:
Evolutionary Algorithms (EAs):
Optimization problems (of increasing difficulty):
| Optimization problem | Type | Parameters number |
|---|---|---|
| String | Discrete-number vector | 12 |
| Rastrigin function | Real-number vector | 100 |
| Policy Neural-Network for autonomous agent control | Real-number vector | 407 |
Note that the higher the dimensionality (parameters number) of the optimization problem,
the harder it is, and the slower the optimization process is.
A family of problem-independent, gradient-free, population-based, metaheuristic or stochastic,
global optimization algorithms.
Inspired by the biological concept of evolution by natural selection -
a population of candidate solution models follows an evolutionary process towards optimality
(optimization via evolution).
Employs a more biologically-accurate form of evolution - utilizes a loop of
parents selection, reproduction via crossover (AKA genetic recombination) and mutation.
Employs a more mathematical (and less biologically-accurate) form of evolution -
a single parent produces multiple variants of itself via cloning and mutation
(a small amount of random noise).
After the new population is evaluated, a single parent is created using a weighted-sum of
all the individuals (AKA population averaging).
Optimizing a fixed-length String (discrete-valued vector) towards a chosen target String
of 12 characters.
For generality purposes, the String (individual & target) is converted into a discrete-number
vector (and vice-versa), which is optimized.
Optimizing the Rastrigin function’s input parameters (x).
The Rastrigin function is a non-convex function (with multiple local minima),
and is used as a performance-test problem for optimization algorithms.
It is a typical example of non-linear multimodal function.
Optimizing the Policy Neural-Network’s parameters (layers’ weights and biases).
A Policy network receives the environment state as an input,
and outputs a probability distribution over all possible actions.
Any AI Gym environment can be chosen, as long as the relevant variables parameters
(env_name, input_dims, n_actions, optimal_fit) are changed accordingly.
Here, AI Gym’s CartPole environment is chosen as an example.
optimization process results:
Legend:
Environment: CartPole.
Neural-Network layers’ units number: (Input) 4,25,10,2 (Output).
Legend:
Notice how the optimization is better with a larger population size.
Notice how the optimization is better with a higher (and constant) mutation sigma.
Sigma = 1
Sigma = 0.5
Sigma: Init = 0.5, Decay = 0.9, Min = 0.01
Examples of how to use: