I'm trying to alter this tensorflow js reinforcement example to let the AI learn the classic pong game in my own project. They both are very similar. Instead of moving left or right in the cart example I move the left paddle up or down while the right paddle is controlled by a simple algorithm following the ball's y coordinate.
My input state tensor consists of:
left paddle y coordinates normalized
ball's y coordinate normalized
ball's x velocity between -1 and 1
ball's y velocity between -1 and 1
Output is a single neuron with the probability of moving left (and inverse probability of moving right).
Each survived step gives a reward of 1 and after the ball moves out of bounds or after max 500 steps the game is reset. Rewards are discounted at the end.
But while the cart example already shows very nice results after around 50 iterations, in my example the paddle mostly stays on one side without moving after 200 iterations.
I tried already different hidden layer sizes like (64,32) or even (128, 256, 128), different activation functions (elu, sigmoid) and just using the paddle y and ball y corrdinates. But I don't see any improvements.
Any ideas how I can improve the learning rate?
Related
firstly, is bod2d friction like mu in high-school physics?
I have a compound body with two wheels on revolute joints one of which has a motor. When I try to drive it up an 30 degree slope my driven wheel spins and i don't get up.
I have set friction on all bodies as 0.9. If I remember my high school physics mu approx 1 should let me climb a slope of nearly 45 degrees but my wheels slip.
secondly as I increase friction towards 1 my model becomes increasingly unstable. i have some angular damping set but climbing the ramp the physics sometimes seems to enter some kind of positive feedback condition and the moving vehicle disappears up at a silly velocity,
Anyone seen anything like this?
thanks!
In my game I solved the low friction of wheels by setting the friction at way more than 1. So you should try that. Not sure about that instability that you are describing.
I have a camera with the coordinates x,y at height h that is looking onto the x-y-plane at a specific angle, with a specific field of view. I want to calculate the 4 corners the camera can see on the plane.
There is probably some kind of formula for that, but I can't seem to find it on google.
Edit: I should probably mention that I mean a camera in the 3D-Graphics sense. Specifically I'm using XNA.
I've had to do similar things for debugging graphics code in 3D games. I found the easiest way of thinking about it was creating the vectors representing the corners of the screen and then calculating the intersection with whatever relevant objects (in this case, a plane).
Take a look at your view-projection (or whatever your Camera's matrix stack looks like multiplied together) and realize the screen space they output to has corners with homogonized coordinates of (-1, -1), (-1, 1), (1, -1), (1, 1). Knowing this, you're left with one free variable and can solve for the vector representing the corner of the camera.
Although, this is a pain. It's much easier to construct a vector for each corner as if the camera isn't rotated or translated and then transform them by the view matrix to give you world space vectors. Then you can calculate the intersection between the vectors and the plane to get your four corners.
I have a day job, so I leave the math to you. Some links that may help with that, however:
Angle/Field of view calculations
Line plane intersection
ignoring lens distortions and assuming the lens is almost at the focal point then you just have a triangle formed by the sensor size and the lens, then the lens to the subject - similar trianlges gives you the size of the subject plane.
If you want a tilted object plane that's just a projection onto the perpendicular object plane
Im not sure how to use vectors correctly in game programming. I have been reading advanced game design with flash which shows you how to create a vector with a start point and endpoint and how to use that for games, for example the start point would be used for a characters position in a game and the x and y length would be used for velocity. But since I have started looking online I have found that vectors are usually just x and y with no start point or end point and a character would be moved by having a position vector and a velocity vector and acceleration vector. I have started creating my own vector class. And wondered what the reasons for and against each method are. Or is it completely not important?
Initially a vector means direction. Classical vector is used in physics to present a velocity so that the vector direction stands for the heading and the vector length is a speed.But in graphics vectors are used also to present position. So if you have some point, let's say, in 2d space noted by x ,y it remains point if you don't want to know in what direction it set relating to the origin which is usually a center of the coordinate system. In 2d graphics we deal with Cartesian coordinate system which has an origin in top left corner of the screen. But you can also have a direction of some vector relative to any other point in the space.That is why you have also vector operations like addition, subtracting ,dot product, cross product. All those help you to measure distances and angles between vectors. I would suggest you to buy a book on graphics programming. Most of them introduce an easy to grasp primer to vector math.And you don't need to write a vector class in AS 3.0 You have a generic one - Vector3d
In actionscript 3.0, I have two objects (a central red star and a orbiting blue planet). I want to use RK4 to plot the orbit. I'm running the simulation once per frame, and drawing once per frame. I have to relate the position of the blue planet in x,y to the central planet so I may be getting lost in the conversion somewhere. This is just for the 1,1 quadrant. I will be adjusting the gravity vector as the blue planet crosses from quadrant to quadrant.
PROBLEM: If I alter the time step, the orbit changes drastically. At small time steps, the orbit becomes a straight line. At large time steps, the orbit becomes tighter. The cooefficients for computing the acceleration for each "K" are not being scaled by dt (except for it being passed through the previous velocity vector).
Here is the RK4 code snip:
http://pastebin.com/Ee6HzBQ2
Velocity = length / time
so why a vector (x, y, z) is used to represent it?
Technically speaking, length divided by time gives you the speed, not velocity. Speed doesn't tell you which direction you are travelling in, while velocity does. In a three dimensional space, in order to describe where you are going and how fast, you need to supply three values: the direction AND speed you are going in each of the three fundamental directions (normally called axes and referred to by x, y, and z). But you could refer to them as forward/backward, sideways, and up/down if you want. For example, if you are travelling at 5km/hour upwards, the vector could be (0,0,5). Travelling 5km/hour downwards, your speed is just the same but the vector would be (0,0,-5). Travelling at 5km/hour at a 45 degree angle forward, the SPEED along each of the x and z axex would be the square root of 5, so the vector would be (approximately) (2.2,0,2.2). And so on.
Because velocity is not "length/time". It is the first derivative of position. Position is a vector, and so its derivatives are also vectors.
Most likely to measure the change in three dimensional space for the object.
Magnitude of the vector should be the speed you expect, and as the object changes direction, the vector components will most likely change.
You would use a vector because you can have velocity in 3 dimensions. In other words, the 3D velocity is the combination of distance/time in all 3 dimensions. It might be better to name the variables xPrime, yPrime, and zPrime, so that the vector more clearly represents velocity, rather than position.
Perhaps it is the speed that the object is moving in each of the directions in a 3D space, doing it this way means that you can extrapolate a direction of movement, after all velocity is movement with a direction.