MountainCar openAI gym reset - reinforcement-learning

In the documentation of MountainCar, it is said that the start position should be between -0.6 and -0.4.
When printing the starting state for several trials, I found that the results is more between -0.6 and 0.4. The source code does mention the lower and upper bound to be -0.6 and -0.4, how come this is not reflected when calling reset?
Thanks for your help.

Related

Resume Training Caffe using Different GPU?

Please forgive what may be an insane question.
My setup: two machines, one with a GTX1080, one with an RX Vega 64. Retraining/fine tuning a bvlc_googlenet model on the GTX1080.
If I build Caffe for the Vega 64, then can I take a snapshot from the GTX1080 machine and restart training on the Vega 64? Would this work in the sense that the training would continue in a normal manner?
What if I moved the GTX1080 snapshot to a Volta V100 in AWS? Would this work?
I know Caffe will to some degree abstract the hardware, but I don't know how well it can do that. I need the GTX 1080 for something else...
Thanks in advance!
To my knowledge, this should work without a problem. Weight files and training snapshots are just blobs of numbers, that you should be able to resume on other hardware (e.g. CPU/GPU), a different machine with a different operating system, or between 32 and 64 bit processes.

Octave JIT compiler. Current state, and minimal example demonstrating effect

I hear very conflicting information about Octave's experimental JIT compiler feature, ranging from "it was a toy project but it basically doesn't work" to "I've used it and I get a significant speedup".
I'm aware that in order to use it successfully one needs to
Compile octave with the --enable-jit at configure time
Launch octave with the --jit-compiler option
Specify jit compilation preference at runtime using jit_enable and jit_startcnt commands
but I have been unable to reproduce the effects convincingly; not sure if this is because I've missed out any other steps I'm unaware of, or it simply doesn't have much of an effect on my machine.
Q: Can someone who has used the feature successfully provide a minimal working example demonstrating its proper use and the effect it has (if any) on their machine?
In short:
you don't need to do anything to use JIT, it should just work and speed up your code if it can;
it's mostly useless because it only works for simple loops;
it's little more than a proof of a concept;
there is no one currently working on improving it because:
it's a complicated problem;
it's mostly a fix for sloppy Octave code;
uses LLVM which is too unstable.
Q: Can someone who has used the feature successfully provide a minimal working example demonstrating its proper use and the effect it has (if any) on their machine?
There is nothing to show. If you build Octave with JIT support, Octave will automatically use faster code for some loops. The only difference is on the speed, and you don't have to change your code (although you can disable jit at runtime):
octave> jit_enable (1) # confirm JIT is enabled
octave> tic; x = 0; for i=1:100000, x += i; endfor, toc
Elapsed time is 0.00490594 seconds.
octave> jit_enable (0) # disable JIT
octave> tic; x = 0; for i=1:100000, x += i; endfor, toc
Elapsed time is 0.747599 seconds.
## but you should probably write it like this
octave> tic; x = sum (1:100000); toc
Elapsed time is 0.00327611 seconds.
## If Octave was built without JIT support, you will get the following
octave> jit_enable (1)
warning: jit_enable: support for JIT was unavailable or disabled when Octave was built
This is a simple example but you can see better examples and more details on the blog of the only person that worked on it, as well as his presentation at OctConf 2012. More details on the (outdated), Octave's JIT wiki page
Note that Octave's JIT works only for very simple loops. So simple loops that no one familiar with the language would write them in the first place. The feature is there as proof of concept, and a starting point for anyone that may want to extend it (I personally prefer to write vectorized code, that's what the language is designed for).
Octave's JIT has one other problem which is that it uses LLVM. Octave developers have found it too unreliable for this purpose because it keeps breaking backwards compatibility. Every minor release of LLVM has broken Octave builds so Octave developers stopped fixing when LLVM 3.5 got released and disabled it by default.

Xilinx UART simulation

Just started to use Vivado 2016.1
1. What is considered as simulation of a circuit?
2. what steps i follow to make a circuit simulation?
Thank you

CURAND_STATUS_DOUBLE_PRECISION_REQUIRED is undefined

While building my CUDA project I get the following error:
cutil_inline_runtime.h(328): error: identifier "CURAND_STATUS_DOUBLE_PRECISION_REQUIRED" is undefined
So I started googling. Since I couldn't find the solution (nor did I find the actual problem) I downloaded from nVIDIA CURAND guide pdf and started reading. In that pdf it says:
Enumerator:
...
**CURAND_STATUS_DOUBLE_PRECISION_REQUIRED** GPU does not have double precision required by MRG32k3a
...
This means I can't perform operations with double data type... right? Well that seems wrong because, I assure you, couple a days ago I made a project using double data type and it worked just fine. Does anyone have a suggestion? Thank you!
EDIT Since I was unable to find the "real" solution for the problem, eventually I commented out the lines with "CURAND_STATUS_DOUBLE_PRECISION_REQUIRED" in cutil_inline_runtime.h and now it works. I know it is "ugly" but it was the only thing that helped...
I also had this problem. The issue was that I was using different versions of the cuda SDK and the cuda toolkit. I was using a newer version of the SDK which referenced the definition of CURAND_STATUS_DOUBLE_PRECISION_REQUIRED, but that definition was undefined in curand.h in the older version of the cuda toolkit that I had installed.
Try installing the version of the toolkit that matches your cuda SDK version.
Try Googling "Compute Capability", it's how nvidia defines the various CUDA capabilities. In the CUDA C Programming guide, it's mentioned a couple of times that devices with compute capability 1.2 or lower do not support double precision FP math: CUDA C Programming, pages 140, and also check table F-1, namely the row about dpfp support.
Gefore 8800 GT is a G80 architecture chip, which is compute capability 1.0. You cannot do double computations on this card. Keep in mind much example code makes use of emulation modes, and the like, which can fool you into thinking it works.

Multiple GPUs in CUDA 3.2 and issues with Cuda 4.0

I am new to multiple GPUs. I have written a code for a single GPU and want to further speed up by use of multiple GPUs. I am working with two GTX 470 with MS VS 2008 and cuda toolkit 4.0
I am facing two problems.
First problem is my code somehow doesn't run fine with 4.0 build rules and works fine for 3.2 build rules. Also the SDK example of multiGPU doesn't build on VS2008 giving error
error C3861: 'cudaDeviceReset': identifier not found
My second problem is, if I have to work with 3.2 then according to the documentation, threads have to be launched separately and separate allocations to be made etc. What is the easiest library for launching threads for multiple gpus and can you please give some example for my setup for access to multiple GPUs.
The answer to the first question is that you are clearly linking an older version of the CUDA runtime library. cudaDeviceReset is a new addition to the API introduced in CUDA 4.0. So double check the build rules and make sure you really are pointing the linker at the CUDA 4.0 toolkit and not an earlier version
The second part of your question sounds like a "hai plz give me teh code" question, and that isn't really what this place is for. I will, however, give you a link to GPUWorker (code currently available here), which is a boost threads based multigpu framework that was originally part of the HOOMD molecular dynamics package. It should give you some hints on how to do a multithreaded, multigpu code, even if GPUWorker turns out to not be directly applicable to your needs.