I am trying to RL a simple game. Rather than processing the image of game, I decided to extract values from the game that structures the game.
Extraction was easy but the main problem is how to structure observation_space
My goal is to create my own python gym environment and do RL
(value and count gets changed per game but it remains static once game has been started)
type1, between 100 to 200 of 4 element array [x1, x2, y1, y2]
type2, between 0 to 10 of 3 element array [x, y1, y2]
type3, between 0 to 10 of 2 element array [x, y]
(value always exists but gets changed dynamically during the game)
between 0 to 50 of 4 element array [x1, x2, y1, y2]
10 stat values
1 reward values
All values are int16 (-32,768 to +32,767) except that reward value is unsigned int16 (0 to 65,535)
int16 is what the game is defined to hold value but in real game, it does not exceed -10000~10000 and rewards maximum is 5000
In this case, how am I supposed to structure observation_space?
Will it be effective if I just set with maximum possible size and fill it with 0s for non-existing values?
Or is there any best practices or example that I can reference?
Thank you!
Related
Does anyone know if there is a function in PyTorch that allows you to call a particular function for all the locations in a tensor where a condition is satisfied?
For example, for all the locations in a tensor where the value is equal to 100, a function has to be called without using any for loops. I want to modify other tensors for these locations. I know I can just get the list of locations and then call the function for each location but I want to avoid that. Thank you for your help.
Something like torch.where(a==100,function1())
Adding more details after the request in the comments:
function1(x,y): #(x,y) are co-ordinates
... some code based on (x,y)...
a[x,y] = 20
b[x,y] = 10
Let's assume we have 2 2D(single row) tensors a = [1 2 100 100 5 6 200 8 2] and b=[3 4 5 5 6 10 3 8 9].
Now I want to pass the coordinates of where a == 100 in our case (0,2) and (0,3) to function1 and modify the values of a and b at those locations. I hope this helps.
Background
I bought an Arduino magnetometer/compass with a QMC5883 chip from Amazon, however the bearing output I'm getting doesn't tally with the calculations I've found online. The serial output seems to be plausible (sinusoids with a phase difference of 90°), but the numbers I'm getting for the calculated bearing don't match what they are supposed to. I stored the serial output as a .csv file to graph the magnetometer response when turned through 360° in Excel:
Response was roughly as expected - Z remaining roughly steady (apart from a few wobbles caused by the cable!), X and Y varying sinusoidaly through 360°. (Bear in mind I couldn't turn the magnetometer at a constant speed with my hand which is why the curves are so unsteady).
However below is the graph of what heading was calculated; results were supposed to be between -180° and +180°:
As you can see it only varies around -60° to -160°, and each bearing reading is not unique as it is given by two different rotations of the magnetometer. The specific calculation in the code used (in full at the bottom) is:
bearing =180*atan2(y,x)/3.141592654; //values will range from +180 to -180°
bearing +=0-(19/60); //Adjust for local magnetic declination
Question
I can't figure out what's wrong with the calculation as it is used in a few different sources, and I would like to know how to convert the readings I'm getting to a usable range which is one to one instead of many to one, such as -180° to +180° or 0° to 360°.
Here is the code:
//There are several differences between the QMC5883L and the HMC5883L chips
//Differences in address: 0x0D for QMC5883L; 0x1E for HMC5883L
//Differences in register map (compare datasheets)
//Output data register differences include location of x,y,z and MSB and LSB for these parameters
//Control registers are also different (so location and values for settings change)
#include <Wire.h> //I2C Arduino Library
#define addr 0x0D //I2C Address for The QMC5883L (0x1E for HMC5883)
double scale=1.0;
void setup() {
// double scaleValues[9]={0.00,0.73,0.92,1.22,1.52,2.27,2.56,3.03,4.35};
// scale=scaleValues[2];
//initialize serial and I2C communications
Serial.begin(9600);
Wire.begin();
Wire.beginTransmission(addr); //start talking to slave
Wire.write(0x0B);
Wire.write(0x01);
Wire.endTransmission();
Wire.beginTransmission(addr); //start talking to slave
Wire.write(0x09);
Wire.write(0x1D);
Wire.endTransmission();
}
void loop() {
int x, y, z; //triple axis data
//Tell the QMC what regist to begin writing data into
Wire.beginTransmission(addr);
Wire.write(0x00); //start with register 00H for QMC5883L
Wire.endTransmission();
double bearing=0.00;
//Read the data.. 2, 8 bit bytes for each axis.. 6 total bytes
Wire.requestFrom(addr, 6);
//read 6 registers in order; register location (i.e.00H)indexes by one once read
if (6 <= Wire.available()) {
//note the order of following statements matters
//as each register will be read in sequence starting from data register 00H to 05H
//where order is xLSB,xMSB,yLSB,yMSB,zLSB,zMSB
//this is different from HMC5883L!
//data registers are 03 to 08
//where order is xMSB,xLSB,zMSB,zLSB,yMSB,yLSB
x = Wire.read(); //LSB x;
x |= Wire.read()<<8; //MSB x; bitshift left 8, then bitwise OR to make "x"
// x*=scale;
y = Wire.read(); //LSB y
y |= Wire.read()<<8; //MSB y;
// y*=scale;
z = Wire.read(); //LSB z; irrelevant for compass
z |= Wire.read()<<8; //MSB z;
// z*=scale;
bearing =180*atan2(y,x)/3.141592654;//values will range from +180 to -180 degrees
bearing +=0-(19/60);//Adjust for local magnetic declination
}
// Show Values
//Serial.print("X:");
Serial.print(x);
//Serial.print(" Y: ");
Serial.print(",");
Serial.print(y);
//Serial.print(" Z: ");
Serial.print(",");
Serial.print(z);
//Serial.print(" B: ");
Serial.print(",");
Serial.println(bearing);
delay(500);
}
For others reading this question:
The OP forgot to implement a x,y,z smoothing and an out of scope value removal. How this can be achieved and how its done look into the source code of this QMC5883 compass library:
QMC5883L Compass is an Arduino library for using QMC5583L series chip boards as a compass.
It supports:
Getting values of XYZ axis.
Calculating Azimuth.
Getting 16 point Azimuth bearing direction (0 - 15).
Getting 16 point Azimuth bearing Names (N, NNE, NE, ENE, E, ESE, SE, SSE, S, SSW, SW, WSW, W, WNW, NW, NNW)
Smoothing of XYZ readings via rolling averaging and min / max removal.
I have an acceleration data for X-axis and time vector for it. I determined the peaks more than threshold and now I should find the FFT for every peak.
As result I have this:
Peak Value 1 = 458, index 1988
Peak Value 2 = 456, index 1990
Peak Value 3 = 450, index 12081
....
Peak Value 9 = 432, index 12151
To find these peaks I used the peakfinder script.
The command [peakLoc, peakMag] = peakfinder(x0,...) gives me location and magnitude of peaks.
Also I have the Time (from time data vector) for each peak.
So what I suppose, that I should take every peak, find its width (or some data points around the peak) and make the FFT. Am I right? Could you help me in that?
I'm working in Octave and I'm new here :)
Code:
load ("C:\\..patch..\\peakfinder.m");
d =dlmread("C:\\..patch..\\acc2.csv", ";");
T=d(:,1);
Ax=d(:,2);
[peakInd peakVal]=peakfinder(Ax,10,430,1);
peakTime=T(peakInd);
[sortVal sortInd] = sort(peakVal, 'descend');
originInd = peakInd(sortInd);
for k = 1 : length(sortVal)
fprintf(1, 'Peak #%d = %d, index%d\n', k, sortVal(k), originInd (k));
end
plot(T,Ax,'b-',T(peakInd),Ax(peakInd),'rv');
and here you can download the data http://www.filedropper.com/acc2
FFT
d =dlmread("C:\\..path..\\acc2.csv", ";");
T=d(:,1);
Ax=d(:,2);
% sampling frequency
Fs_a=2000;
% length of FFT
Length_Ax=numel(Ax);
% number of lines of Fourier spectrum
fft_L= Fs_a*2;
% an array of time samples
T_Ax=0:1/Fs_a: Length_Ax;
fft_Ax=abs(fft(Ax,fft_L));
fft_Ax=2*fft_Ax./fft_L;
F=0:Fs_a/fft_L:Fs_a/2-1/fft_L;
subplot(3,1,1);
plot(T,Ax);
title('Ax axis');
xlabel('time (s)');
ylabel('amplitude)'); grid on;
subplot(3,1,2);
plot(F,fft_Ax(1:length(F)));
title('spectrum max Ax axis');
xlabel('frequency (Hz)');
ylabel('amplitude'); grid on;
It looks like you have two clusters of peaks, so I would plot the data over three plots: one of the whole timeseries, one zoomed in on the first cluster, and the last one zoomed in on the second cluster (note I have divided all your time values by 1e6 otherwise the tick labels get ugly):
figure
subplot(3,1,1)
plot(T/1e6,Ax,'b-',peakTime/1e6,peakVal,'rv');
subplot(3,1,2)
plot(T/1e6,Ax,'b-',peakTime(1:4)/1e6,peakVal(1:4),'rv');
axis([0.99*peakTime(1)/1e6 1.01*peakTime(4)/1e6 0.9*peakVal(1) 1.1*peakVal(4)])
subplot(3,1,3)
plot(T/1e6,Ax,'b-',peakTime(5:end)/1e6,peakVal(5:end),'rv');
axis([0.995*peakTime(5)/1e6 1.005*peakTime(end)/1e6 0.9*peakVal(5) 1.1*peakVal(end)])
I have set the axes around the extreme time and acceleration values, using some coefficients to have some "padding" around (the values of these coefficients were obtained through trial and error). This gives me the following plot, hopefully this is the sort of thing you are after. You can add x and y labels if you wish.
EDIT
Here's how I would do the FFT:
Fs = 2000;
L = length(Ax);
NFFT = 2^nextpow2(L); % Next power of 2 from length of Ax
Ax_FFT = fft(Ax,NFFT)/L;
f = Fs/2*linspace(0,1,NFFT/2+1);
% Plot single-sided amplitude spectrum.
figure
semilogx(f,2*abs(Ax_FFT(1:NFFT/2+1))) % using semilogx as huge DC component
title('Single-Sided Amplitude Spectrum of Ax')
xlabel('Frequency (Hz)')
ylabel('|Ax(f)|')
ylim([0 300])
giving the following result:
I need to find whether two lines overlap each other. I have the intersection code which returns 0, if two lines are parallel. But then I need to know if these two parallel lines overlap.
Edit:
A C B D
-----------------------------------------------
Line 1: A-B
Line 2: C-D
I need to find if Line 1 overlaps Line 2, but both lines can have a slope > 0.
You can compare to find if there is no over lap. you will have less comparisons in this way thus very efficient. Do following comparisons
D < A
B < C
If either case is true the lines are not overlapping. otherwise there must an overlap.
You will make least number of comparisons to determine if they are not overlapping. Otherwise there will be more comparisons to make.
Since you know they're both parallel, then just check whether line segment CD contains either of the endpoints of the first line (point A and point B).
For two co-linear line segments that are not necessarily axis-aligned:
Sort the vertices in clockwise order around the origin.
The lines overlap if the ordered vertices alternate between the two segments, e.g. Line1.Point1, Line2.Point1, Line1.Point2, Line2.Point2.
It is sufficient to calculate the areas of triangles ACB and CBD. If the area is 0, then the points are collinear, and if both areas are zero then the lines are overlapping.
You can calculate the area of a triangle ABC by this formula:
2*Area(ABC)= (bx – ax)(cy – ay) – (cx – ax)(by – ay);
Line equation is direction of line in infinite, by finding slope or intercept you wont be able do anything with them(even though horizontal line doesn't have slope), i suggest use point on line.
so AB is your line [(x,y),(x,y)] and C is on of the point (x,y) and then you need to check if point on the line.
Check Point On a Line
We are given two line segments
AB = line segment from (Ax,Ay) to (Bx,By)
CD = line segment from (Cx,Cy) to (Dx,Dy)
with the same slope.
Order the endpoints E1 < E2 < E3 < E4 such that Ei,x ≤ Ei+1,x and Ei,y ≤ Ei+1,y if Ei,x = Ei+1,x
If E1 and E2 are from different segments, the overlap is the segment from E2 to E3.
There are some degenerate cases:
A < B = C < D
A < C = D < B
A < B = C = D
A = B = C = D
These result in a single point of intersection. I'm not sure if any of those can occur in your system, but if so you'll have to decide whether or not you consider that "overlap" and add special case checks.
The characteristic of two segments of being in the same lines is called collinearity and can be tested calculating the area of the 2 triangles formed by one segment endpoints and, respectively, the endpoints of the other segment. If the area is zero or close to zero below a threshold the segments are collinear.
public static bool AreSegmentsCollinear(Segment a, Segment b, double epsilon)
{
return IsPointCollinear(a, b.Left, epsilon) && IsPointCollinear(a, b.Right, epsilon);
}
public static bool IsPointCollinear(Segment a, Vector2D p, double epsilon)
{
return Math.Abs(GetSignedTriangleArea2(a, p)) <= epsilon;
}
/// <returns>The squared signed area of the triangle formed with endpoints
/// of segment a and point p</returns>
public static double GetSignedTriangleArea2(Segment a, Vector2D p)
{
return (p - a.Left) ^ (a.Right - a.Left);
}
/// <summary>Cross product of vectors in 2D space. NB: it returns a
/// magnitude (scalar), not a vector</summary>
/// <returns>The area of the parallelogram formed with u, v as the edges</returns>
public static Double operator ^(Vector2D u, Vector2D v)
{
return u.X * v.Y - u.Y * v.X;
}
Just to be clear since there seems to be some confusion in the answers, the question being asked is as follows. Given two 2D line segments A and B how do I determine if both of the following are true:
A and B are colinear.
A and B intersect.
Note that there are tolerances involved in both questions i.e. how close to parallel and how near each other do A and B need to be to be considered colinear? How much do they need to overlap to be considered overlapping?
I think to handle such tolerances robustly the best algorithm is to treat the line segments as thin rectangles, where the thickness of the rectangles is a tolerance parameter t1. Let t2 be another tolerance parameter on slopes that are considered parallel. Then the algorithm becomes
If the slope of A and the slope of B are not within t2 of each other return false. To handle vertical lines cleanly, slope can be represented as a unit vector and the test can be on whether the Euclidean distance between the two unit vectors is smaller than t2.
Represent A and B as (non-axis-aligned) rectangles R1 and R2. Where R1 encloses A in the obvious way, i.e. it is length(A) + t1 units long and is t1 units wide with A centered inside it, and R2 analogously encloses B.
Determine if R1 and R2 intersect each other. This can be done relatively efficiently by treating each rectangle as the union of two triangles and testing for triangle-triangle intersections across all combinations of A triangles and B triangles. If there is an intersection return true; otherwise return false.
With lines l1 and l2 given in the following form [x1, y1, x2, y2] the following python code will give the intersection for collinear line segments with any slope.
intersection = line_intersect(l1, l2)
def line_intersect(l1, l2):
"""Find the intersection of two line segments"""
x1, y1, x2, y2 = l1
x3, y3, x4, y4 = l2
x_inter = component_intersect(x1, x2, x3, x4)
y_inter = component_intersect(y1, y2, y3, y4)
return math.sqrt(x_inter**2 + y_inter**2)
def component_intersect(c1, c2, c3, c4):
"""Calculate intersection in one dimension/component"""
# find left endpoints
cl1 = min(c1, c2)
cl2 = min(c3, c4)
# find right endpoints
cr1 = max(c1, c2)
cr2 = max(c3, c4)
# find endpoints of intersection
c_inter_l = max(cl1, cl2)
c_inter_r = min(cr1, cr2)
# calcuate intersection
c_inter = c_inter_r - c_inter_l
c_inter = max(c_inter, 0)
return c_inter
Suppose there are N points in a 2-D graph.Each point has some weight attached to it.I am required to draw a straight line such a way that the line divides the points into 2 groups such that total weight(sum of weight of points in that group) of part with smaller weight be as many as possible.My task is to find this value.How to go about it ?
Note:No three points lie on the same line.
This is not a homework or part of any contest.
You could just scan over all angles and offsets until you find the optimal solution.
For ease of computation, I would rotate all the points with a simple rotation matrix to align the points with the scanline, so that you only have to look at their x coordinates.
You only have to check half a circle before the scanline doubles up on itself, that's an angle of 0 to PI assuming that you're working with radians, not degrees. Also assuming that the points can be read from the data as some kind of objects with an x, y and weight value.
Pseudocode:
Initialize points from input data
Initialize bestDifference to sum(weights of points)
Initialize bestAngle to 0
Initialize bestOffset to 0
Initialize angleStepSize to an arbitrary small value (e.g. PI/100)
For angle = 0:angleStepSize:PI
Initialize rotatedpoints from points and rotationMatrix(angle)
For offset = (lowest x in rotatedpoints) to (highest x in rotatedpoints)
weightsLeft = sum of the weights of all nodes with x < offset
weightsRight = sum of the weights of all nodes with x > offset
difference = abs(weightsLeft - weightsRight)
If difference < bestDifference
bestAngle = angle
bestOffset = offset
bestDifference = difference
Increment angle by stepsize
Return bestAngle, bestOffset, bestDifference
Here's a crude Paint image to clarify my approach: