How to modify the agent in an openai gym environment? - reinforcement-learning

I want to modify the agents structure (for example the length of the legs) in the bipedal_walker environment.
Is there anyway to do that?

The easiest way would be to download and modify the OpenAI gym source code, and then use your locally modified version of gym in place of the pip package you installed.
For example, if you want to modify the length of the legs in the bipedal_walker environment, you'd probably want to play around with the LEG_H variable on this line of code.

Related

Is it possible to modify OpenAI environments?

There are some things that I would like to modify in the OpenAI environments. If we use the Cartpole example then we can edit things that are in the class init function but with environments that use Box2D it doesn't seem to be as straightforward.
For example, consider the BipedalWalker environment.
In this case, how would I edit things like the SPEED_HIP or SPEED_KNEE variables?
Yes, you can modify or create new environments in gym. The simplest (but not recommended) way is to modify the constants in your local gym installation directly, but of course that's not really nice.
A nicer way is to download the bipedal walker environment file (from here) and save it to a file (say, my_bipedal_walker.py)
Then you modify the constants in the my_bipedal_walker.py file, and then just import it in your code (assuming you put the file in a path that is importable, or the same folder as your other code files):
import gym
from my_bipedal_walker import BipedalWalker
env = BipedalWalker()
Then you have the env variable being an instance of the environment, with your defined constants for the physics computation, which you can use with any RL algorithm.
An even nicer way would be making your custom environment available in the OpenAI gym registry, which you can do by following the instructions here
You can edit the bipedal walker environment just like you can modify the cartpole environment.
All you have to do is modify the constants for SPEED_HIP and SPEED_KNEE.
If you want to change how those constants are used in the locomotion of the agent, you might also want to tweak the step method.
After making changes to the code, when you instantiate the environment, the new instance will be using the modifications you made.

TK GUI looks different on different linux machines but with same version

I have a Tk UI which when executed on different linux machine comes up with different UI. I checked tcl version is 8.6 for both platforms.
The UI looks like this.
invoke method : tclsh script name.tcl
I wanted to use 1st one as its more graphical and fonts are readable.
Can you point me to check what is the difference in the tcl version installed on machine B. I assume Tk package could be different, but i dont know how to check.
I think tk still uses good ol'resource databases for X11, which may be different on each platform. Check xrdb(1) for this. Also read ttk::intro(n) and ttk::style(n) to read about ttk widget styles.
System wide defaults go to
/usr/lib/X11/app-defaults/∗ Client resource specifications.
or similar (check /etc/X11 and /usr/share).
In your home directory there may be .Xdefaults for user specific defaults.
But the fonts are selected through fontconfig, which has other default directories. Check /etc/fonts.

OpenShift catridge using yum and augeas

What is the best way to create a custom OpenShift cartridge?
Looking at documentation and examples, I am seeing a lot of old-school compiling from source installation of the component that the cartridge needs to run.
Some examples https://www.openshift.com/blogs/lightweight-http-serving-using-nginx-on-openshift https://github.com/boekkooi/openshift-diy-nginx-php/blob/master/.openshift/action_hooks/build_nginx https://github.com/razorinc/redis-openshift-example/blob/master/.openshift/action_hooks/build and a ton of others are compiling from source..
I need to create some custom cartridges on my project, but doing it this way feels wrong.
Is there any reason I cant use yum and puppet/augeas to do the building, instead of curl, make and sed?
Or is this the best practice? In that case, why are we doing this 2000 style?
I'll do my best to explain this the best way I can. Feel free to let me know If I need to explain anything in more detail.
I'm assuming you're creating a custom binary cartridge (ie. you're creating a language cartridge such as ruby, python, etc.). Since none of the nodes have that binary installed on the system the custom cartridge you're creating will need to provide that binary and its libraries.
When you install a package with yum its going to install items in several different directories (/etc, /usr/, /var, etc). Since you're creating cartridge that will be copied over to several nodes you'll need to package all these items in a way that can be copied over to a node and then be executed without having to install them to the system.
As for doc's, I would suggest taking a look at these:
https://www.openshift.com/developers/download-cartridges
https://www.openshift.com/blogs/new-openshift-cartridge-format-part-1
https://www.openshift.com/blogs/new-openshift-cartridge-format-part-2

Cucumber examples reuse in different features/scenarios

I've been using cucumber for awhile and I've stumbled upon a problem:
Actual question:
Is there a solution to import the examples from a single file/db using cucumber specifically as examples?
Or alternatively is there a way to define a variable while already in-step to be an example?
Or alternatively again, is there an option to send the examples as variables when I launch the feature file/scenario?
The Problem:
I have a couple of scenarios where I would like to use exactly the same examples, over and over again.
It sounds rather easy, but the examples table is very large (more specifically it contains all the countries in the world and their appropriate continents). Thus repeating it would be very troublesome, especially if the table needs changing (I will need to change all the instances of the table separately)
Complication:
I have a rerun function that knows when a specific example failed and reruns it after the test is done.
Restrictions:
I do not want to edit my rerun file
Related:
I've noticed that there is already an open discussion about importing it from csv here:
Importing CSV as test data in Cucumber?
However that discussion is invalid to me because I have the rerun function that only knows to work only with examples, and the solution suggested there ruins that.
Thank you!
You can use CSV and other external file systems with QAF using different BDD syntax.
If you want to use cucumber steps or cucumber runner, you can use QAF-cucumber and BDD2 (preferred) or Gherkin syntax. QAF-cucumber will enable external test data and other qaf features with cucumber.
Below is the example feature file uses BDD2 syntax can be run using TestNG or Cucumber runner.
Feature: feature uses external data file
#datafie:resources/${env}/testdata.csv
#regression
Scenario: Another scenario exploring different combination using data-provider
Given a "${precondition}"
When an event occurs
Then the outcome should "${be-captured}"
testdata.csv file may look like:
TestcaseId,precondition,be-captured
123461,abc,be captured
123462,xyz,not be captured
You can run using TestNG or Cucumber runner. You can use any of inbuilt data provider or custom as well.

IPython, Sage, Dependencies, Anaconda, and Package Integration?

I've just started using IPython for interactive development and exploratory research, which I've found really exciting with all the cool features and possibilities. I am using the Anaconda package manager to manage dependencies, which includes IPython.
From what I've read, one goal of the IPython team is to eventually integrate Sage Math (CAS) into IPython, as a cell magic. Does anyone know if this is still under development? Or rather, if I wanted to use Sage now, is writing an extension the only way to do this [1]?
[1] https://github.com/ipython/ipython/wiki/Extensions-Index
Also, if I install additional packages for scientific development, not included in the Anaconda distribution, is that as easy as just pip or do I have to go through a Anaconda package build to handle dependencies and such? If I were only using IPython, I could understand just doing easy_install or pip as recommended in the docs, but I believe that overwrites existing dependencies within Anaconda. If I use pip, how does that affect Anaconda dependencies if I do not install in an Anaconda environment, which I take is the equivalent as virtualenv.py, and is this the way to also set up revision control (i.e. Mercurial)?
To clarify, I do not want to run IPython from within Sage, I want to run Sage, as a CAS, from within IPython. I'd rather go the Sage approach of integrating domain specific languages. Or in contrast, will IPython extensions replace Sage?
I'm a self taught programmer, not a professional software developer. As an engineer, I am used to Matlab, Mathematica, and commercial solutions, that allow me to abstract away the plumbing. I'm trying to wrap my mind around getting everything glued together, but it's like a mix of spaghetti soup and a dynamic link library, due to lack of knowledge. I'm probably using the wrong approach.
What I want is Anaconda/Enthought package management (IPython, pandas, etc..), custom rolled Sage through hooks/extensions or magics, extensions to packages not included in Anaconda (i.e. Matlab see [1] above), and revision control with Git and Mercurial. How would professional developers set this up on a Mac or Linux box?
Answering the first question:
Sage is a huge collection of mathematical software, including IPython. There's no way we'd integrate all of that into IPython.
Possibly what you've heard is that we're going to integrate Sage-style 'interacts' into IPython. That's where you have a slider to control the value of some input variable, and the output updates as you move it around, based on a calculation written in Python. That is still on our roadmap to add to IPython.
Another possibility is that you're thinking of SymPy, a Python-based CAS. SymPy works well within IPython, especially if you call sympy.init_printing() to get the fancy representations of expressions.
I wrote an IPython extension to load Sage customizations into the IPython notebook---in fact, that's how many of the customizations to IPython are done for the normal Sage interface. It basically turns the IPython notebook into an interface to Sage (e.g., preparsing is done, etc.).
You do need to run it from Sage's copy of IPython, though. Just start the IPython notebook:
sage -ipython notebook
and then load the sage extension in a cell:
%load_ext sage.misc.sage_extension
Pretty soon we'll upgrade to IPython 1.0 (I've made the changes necessary, and it needs to be reviewed). If you want to run IPython 1.0 already, email the sage-support mailing list and I'll post instructions.
To answer your other question, Sage includes many packages that are not available in Anaconda. Sage depends on these packages heavily for many features. I suppose there is a possibility getting Sage and its dependencies distributed with something like Anaconda, but no one is working on that as far as I know. There is some work on packaging Sage up for different linux distributions and replacing the package manager for Sage.
To clarify, I do not want to run Ipython from within Sage, I want to run Sage, as a CAS, from within Ipython. I'd rather go the Sage approach of integrating domain specific languages. Or in contrast, will Ipython extensions replace Sage?
If you want to run Sage within Ipython, the easiest thing to do is to use Sage's copy of Ipython:
$ sage -ipython
Python 2.7.5 (default, Aug 1 2013, 18:11:00)
Type "copyright", "credits" or "license" for more information.
IPython 0.13.1 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
In [1]: from sage.all import *
In [2]: integrate(x^2,x)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
<ipython-input-2-006357f5d9c0> in <module>()
----> 1 integrate(x^2,x)
NameError: name 'x' is not defined
In [3]: var('x')
Out[3]: x
In [4]: integrate(x^2,x)
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
<ipython-input-4-006357f5d9c0> in <module>()
----> 1 integrate(x^2,x)
/Users/.../sage-5.11.rc0/local/lib/python2.7/site-packages/sage/structure/element.so in sage.structure.element.Element.__xor__ (sage/structure/element.c:6754)()
RuntimeError: Use ** for exponentiation, not '^', which means xor
in Python, and has the wrong precedence.
In [5]: integrate(x**2,x)
Out[5]: 1/3*x^3
Note that there are some differences between this and Sage - for instance, there is no preparsing of the syntax. Presumably if you had Sage someplace your own installation of Ipython can find it, you could do this there too (though there is no easy_install and the Python versions might not match correctly!).
Please observe that Sage, as it currently is and should be in the near future, is a huge package that runs in POSIX (Linux-like) OSes only. If you intend it to run in Windows you must use VirtualBox with Sage running under a VM.
If you want a simple CAS that is simple to install and runs everywhere that Python does, you should consider taking a look at Sympy. It doesn't interact with all the software that Sage does, but is simple to use, lightweight (no external libraries needed) and is already included in the base Anaconda distribution.
Besides, it can be easily used inside other Python programs, since it's just like any other module and doesn't make changes to the Python syntax.