OpenAI Retro Contest – Everything I know about JERK agent

The first approach in the OpenAI Retro Contest which I started to implement, test and modify was the JERK approach. Jerk agent is one of the baseline scripts for this contest.

You can find it here:

I think it is the easiest algorithm to understand for programmers who doesn’t have any Machine Learning experience.

The pseudo-code for the JERK algorithm looks like this:

Why is this the easiest approach? Because this algorithm is based on rewards. But not the same kind of rewards like rainbow or ppo2. JERK algorithm has the moves already scripted before. It doesn’t learn the same way like the two others. Sonic runs forward and jumps and if it scores points or progresses on the level further it gets rewarded. It learns based on rewards and tries to not make the mistakes again, because making a mistake will cost him “reward points”. It’s somehow like with us, humans. We are motivated to do something if we get a possible reward at the end.

Read More

OpenAI Retro Contest – Docker Settings

Installing Docker on Windows 10 for Education

What we did before

We installed our gym retro and got our sonic game running with simple algorithm.

Last post:

PS: It’s actually computer playing sonic by running this script:

import gym_remote.exceptions as gre
import gym_remote.client as grc

from retro import make
def main():
    print('connecting to remote environment')
    env = make(game='SonicTheHedgehog-Genesis', state='GreenHillZone.Act1')
    print('starting episode')
    while True:
        action = env.action_space.sample()
        action[7] = 1
        ob, reward, done, _ = env.step(action)
        if done:
            print('episode complete')

if __name__ == '__main__':
    except gre.GymRemoteError as e:
        print('exception', e)

We want now to upload our agent to the server and we can do it through Docker.

PS2: The above example wont work on server because we use local enviroment!

from retro import make and env = make(game=’SonicTheHedgehog-Genesis’, state=’GreenHillZone.Act1′)


Follow the instructions on page:

Because of Personal Settings i would suggest to register account and log in first.

This way you get auto-fill of this form:

export DOCKER_REGISTRY=<docker registry url>
docker login >$DOCKER_REGISTRY \
    --username <docker registry username> \
    --password <docker registry password>

For test purposes i would first curl the files:

mkdir simple-agent
cd simple-agent
curl -O
curl -O
docker build -f simple-agent.docker -t $DOCKER_REGISTRY/simple-agent:v1 .


docker push $DOCKER_REGISTRY/simple-agent:v1

The next thing are local evaluations.

docker pull openai/retro-env
docker tag openai/retro-env remote-env

and running the agent:

retro-contest run --agent $DOCKER_REGISTRY/simple-agent:v1 \
    --results-dir results --no-nv --use-host-data \
    SonicTheHedgehog-Genesis GreenHillZone.Act1

If you run all this in git-bash, there is a possibility that you can’t close the agent.

I would than open new terminal and wrote:

docker ps # get the id of the running container
docker stop <container> # kill it (gracefully)

This way if you did something wrong you get an error message saying what went wrong.

You can see the scores and logs of the local evaluation if you open results folder and look at the log.csv and monitor.csv

If the local evaluation went good, we can upload the same agent on the page.

Go to “Jobs” on page and submit your agent: simple-agent:v1

Other Things:

If you want to upload your scenario.json with your jerk agent and include it in your agent i was told it is not possible. With the jerk agent from retro/baselines we can only edit the script. We can’t modify the scenario.json file.

To run the docker you need to have docker app installed.

Before configuration docker for windows i would suggest to create an account password. Account password is needed to configure docker app.

Other Blog posts on the same topic:

Jerk.agent configuration

View story at

Tensorflow PPO2 Algorithm

View story at

Configuring on Mac:

Building your own Images:

View story at


Making Objects Fall Random On The Screen in Unity

How to spawn object on the screen in Unity Randomly?

I am currently developing with a friend o mine a game. It’s called Badger Adventures. It’s a very simple game in which you are collecting Apples with an Badger.

So… how do you spawn apples randomly?

Actually, if you are an experienced user than you just need to check out the Random.range function which is built in Unity3d Library.


Otherwise check out my Script. It’s called AppleSpawn and well… it is spawning Apples.
using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class AppleSpawn : MonoBehaviour {

  public GameObject apple;			// Apple Object in Scene (Sprite)
  public GameObject badApple;			// Bad Apple Object in Scene (Sprite)

  public float spawnTime = 2f;            // How long between each spawn.
  public float fallSpeed = 40.0f;		//The speed of falling Apples

  private float timer = 0; //counting timer, reset after calling SpawnRandom() function
  private int randomNumber; 			//variable for storage of an random Number
  void Update () {

    timer += Time.deltaTime;   // Timer Counter
      SpawnRandom(); 			//Calling method SpawnRandom()
      timer = 0;				//Reseting timer to 0


  public void SpawnRandom()
      //Creating random Vector3 position
    Vector3 screenPosition = Camera.main.ScreenToWorldPoint(new Vector3(Random.Range(0,Screen.width), Random.Range(600 ,Screen.height), Camera.main.farClipPlane/2));

    //Instantiation of the Apple Object


In the update function there is an if statement because i wanted to restrict the number of spawned Apples. If i want to create more apples i can change the variable in the public function from 2 to 0.2. For the time counting i used Time.deltaTime because this function is frame rate independent.

SpawnRandom() method is creating an random Vector3 value on the top of the screen depending on the Screen Size (Camera) Width and Height.

If we wanted for example to place the spawned elements little lower on the screen we can change the number from 600 to 400. In your example the variable can be a little different depending on the screen size you are developing.

Instatiate just spawns the object on the scene.

That’s all. Thanks for Reading.