Given a dataset of samples in a multi dimensional space (in my case a 4D space) where the samples are present on all the corners of the 4D cube and a substantial amount of samples within this cube but not in a neatly grid. Each sample has an output value next to it's 4D coordinate. The cube has coordinates [0,0,0,0]..[1,1,1,1].
Given a new coordinate (4D) how can I come up with the best interpolated value given these samples? Eg how do I choose the samples to start with, how to interpolate.
As a first guess I would guess that this can be done with a two step process:
find the smallest convex pentachoron (4D equivalent of the 3D tetrahedron / the 2D triangle) around the coordinate we need to interpolate.
interpolate within this tetrahedron.
Especially step 1 seems quite complex and slow.
Here's the first approach I'd try.
Step 1
Find the point's 4 nearest neighbors by Euclidean distance. It's important that these 4 points are linearly independent because next they're used to create a Barycentric coordinate system. Those 4 points become the vertices of your pentachoron (aka 4-simplex).
If nearest-neighbor checks are too slow, try structuring your data into a spatial lookup tree that works in 4D.
Step 2
Now we need to associate a value with the interpolation point X. Start by deriving X's representation in this new Barycentric coordinate system. This Barycentric coordinate consists of 4 numbers, which collectively describe the relative distance between the interpolation point and each of the 4-simplex's vertices.
Normalize the Barycentric coordinate so its components sum to 1.
Each of those 4 simplex vertices are data points and have an output value. Combine those 4 output values into a vector.
Finally, interpolate by calculating the dot product of the normalized coordinate with the vector of output values.
Source: This idea is really just a 4D extension of this gem in middle of the Barycentric coordinate system page on Wikipedia.
Related
Given an RGB image of hand and 3d position of the keypoints of the hand as dataset, I want to do this as regression problem in DL. In this case input will be the RGB image, and output should be estimated 3d position of keypoints.
I have seen some info about regression but most of them are trying to estimate one single value. Is it possible to estimate multiple values(or output) all at once?
For now I have referred to this code. This guy is trying to estimate the age of a person in the image.
The output vector from a neural net can represent anything as long as you define loss function well. Say you want to detect (x,y,z) co-ordinates of 10 keypoints, then just have 30 element long output vector say (x1,y1,z1,x2,y2,z2..............,x10,y10,z10), where xi,yi,zi denote coordinates of ith keypoint, basically you can use any order you feel convenient with. Just be careful with your loss function. Say you want to calculate RMSE loss, you would have to extract tripes correctly and then calculate RMSE loss for each keypoint, or if you are fimiliar with linear algebra, just reshape it into a 3x10 matrix correctly and and have your results also as a 3x10 matrix and then just use
loss = tf.sqrt(tf.reduce_mean(tf.squared_difference(Y1, Y2)))
But once you have formulated your net you will have to stick to it.
I have fields where location data is in X- and Y columns in WGS84 meter-format. How can I convert these fields or create new fields with decimal degrees? Vector->Geometry Tools-> Export/Add geometry columns creates duplicate fields with the same meter-format. Similarly using field calculator with $x- and $y functions creates also fields with meter-formats.
I may be misinterpreting the question, but WGS84 is a geographic coordinate system, utilizing the WGS84 ellipsoid, its coordinate space is measured as lat long pairs and not meters. See unit of measurement here or here. As such WGS 84 is not represented as meters, see discussion here, here or here (comments). In short, WGS84 uses angular measurements to represent the locations within a three dimensional space, as a metered grid doesn't envelope the earth very well. WGS84 is always projected when displayed in GIS software (without changing the underlying data), it is projected to convert it from a 3 dimensional representation of the earth to a 2 dimensional.
Your data, if measured in meters, is projected. The WGS84 ellipsoid may be used as part of the basis of a projection, such as with UTM or WGS84 Antarctic Polar Stereographic. The projection you have and its parameters are critical to understanding how you determine the position of a point in degrees, as a point will essentially have to be unprojected to get its latitude and longitude.
Luckily this is relatively easy in GIS software.
In QGis you can change the coordinate reference system of your layer to WGS 84 (EPSG:4326) - which it could be already with the data coming from a different source or previous CRS - and then use the field calculator to calculate the geometry that you are looking for (assuming that your fields in meters represent something that can be calculated by the field calculator). This also requires your existing data to have a defined projection. If needed you can convert back after you have added the new data.
In Arc, the process is largely the same, using the "project" tool to reproject/unproject the data.
If your data layer does not have a defined projection, you will need to find it. If your data layer fields that are already in meters are not something easily calculated from the field calculator in qGIS, then it might get a little more involved (creating a layer from those fields, changing the CRS of that layer, calculating the fields in degrees...).
I have 3 arrays of X, Y and Z. Each have 8 elements. Now for each possible combination of (X,Y,Z) I have a V value.
I am looking to find a formula e.g. V=f(X,Y,Z). Any idea about how that can be done?
Thank you in advance,
Astry
You have a function sampled on a (possibly nonuniform) 3D grid, and want to evaluate the function at any arbitrary point within the volume. One way to approach this (some say the best) is as a multivariate spline evaluation. https://en.wikipedia.org/wiki/Multivariate_interpolation
First, you need to find which rectangular parallelepiped contains the (x,y,z) query point, then you need to interpolate the value from the nearest points. The easiest thing is to use trilinear interpolation from the nearest 8 points. If you want a smoother surface, you can use quadratic interpolation from 27 points or cubic interpolation from 64 points.
For repeated queries of a tricubic spline, your life would be a bit easier by preprocessing the spline to generate Hermite patches/volumes, where your sample points not only have the function value, but also its derivatives (∂/∂x, ∂/∂y, ∂/∂z). That way you don't need messy code for the boundaries at evaluation time.
What is the difference between quantization and simplification?
Is quantization another way of doing simplification?
Is it better to use quantization in certain situations?
Or should i be using a combination of both?
The total size of your geometry is controlled by two factors: the number of points and the number of digits (the precision) of each coordinate.
Say you have a large geometry with 1,000,000 points, where each two-dimensional point is represented as longitude in ±180° and latitude in ±90°:
[-90.07231180399987,29.501753271000098],[-90.06635619599979,29.499494248000133],…
Real numbers can have arbitrary precision (in JSON; in JavaScript they are limited by the precision of IEEE 754) and thus an infinite number of digits. But in practice the above is pretty typical, so say each coordinate has 18 digits. Including extra symbols ([, ] and ,), each point takes at most 1 + 18 + 1 + 18 + 1 = 39 bytes to encode in JSON, and the entire geometry is about 39 * 1,000,000 ≈ 39MB.
Now say we convert these real numbers to integers: both longitude and latitude are reduced to integers x and y where 0 ≤ x ≤ 99 and 0 ≤ y ≤ 99. A simple mapping between real-number points ⟨λ,φ⟩ and integer coordinates ⟨x,y⟩ is:
x = floor((λ + 180) / 360 * 100);
y = floor((φ + 90) / 180 * 100);
Since each coordinate now takes at most 2 digits to encode, each point takes at most 1 + 2 + 1 + 2 + 1 = 7 bytes to encode in JSON, and the entire geometry is about 7MB; we reduced the total size by 82%.
Of course, nothing comes for free: if you remove too much information, you will no longer be able to display the geometry accurately. The rule of thumb is that the size of your grid should be at least twice as big as the largest expected display size for the entire map. For example, if you’re displaying a world map in a 960×500 pixel space, then the default 10,000×10,000 (-q 1e4) is a reasonable choice.
So, quantization removes information by reducing the precision of each coordinate, effectively snapping each point to a regular grid. This reduces the size of the generated TopoJSON file because each coordinate is represented as an integer (such as between 0 and 9,999) with fewer digits.
In contrast, simplification removes information by removing points, applying a heuristic that tries to measure the visual salience of each point and removing the least-noticeable points. There are many different methods of simplification, but the Visvalingam method used by the TopoJSON reference implementation is described in my Line Simplification article so I won’t repeat myself here.
While quantization and simplification address these two different types of information mostly independently, there’s an additional complication: quantization is applied before the topology is constructed, whereas simplification is necessarily applied after to preserve the topology. Since quantization frequently introduces coincident points ([24,62],[24,62],[24,62]…), and coincident points are removed, quantization can also remove points.
The reason that quantization is applied before the topology is constructed is that geometric inputs are often not topologically valid. For example, if you takes a shapefile of Nevada counties and combine it with a shapefile of Nevada’s state border, the coordinates in one shapefile might not exactly match the coordinates in the other shapefile. By quantizing the coordinates before constructing the topology, you snap the coordinates to a regular grid and can get a cleaner topology with fewer arcs, hopefully correctly identifying all shared arcs. (Of course, if you over-quantize, then you can cause too many coincident points and get self-intersecting arcs, which causes other problems.)
In a future release, maybe 1.5.0, TopoJSON will allow you to control the quantization before the topology is constructed independently from the quantization of the output TopoJSON file. Thus, you could use a finer grid (or no grid at all!) to compute the topology, then simplify, then use a coarser grid appropriate for a low-resolution screen display. For now, these are tied together, so I recommend using a finer grid (e.g., -q 1e6) that produces a clean topology, at the expense of a slightly larger file. Since TopoJSON also uses delta-encoded coordinates, you rarely pay the full price for all the digits anyway!
The two are related, but have different purposes and results.
I believe quantization collapses nearby points based on the parameter (which you tune to the expected resolution of the view) - no point in having a resolution higher than the pixels that will be drawing the map. But it doesn't go out of the way to analyze the path to determine the optimal number of points needed to represent the shape.
Simplification is an algorithm that will analyze the polygon and reduce the number of points in an optimal manner such that the overall deformation of the polygon is minimized. Basically, it can be used to dramamatically reduce the number of points (and thus file size) without noticeable impact to the quality of the path.
As a parallel case study, consider a straight line made up of 10 points. Quantization will reduce the number of points (collapsing nearby or coincident points) based on the value you use. Simplification will analyze the line and realize that 8 out of the ten points can be removed without significantly changing the polygon's overall shape, and reduce the line to two points (because there is no deformation of the path by removing points on a line).
See also:
Topojson reference: https://github.com/mbostock/topojson/wiki/Command-Line-Reference
M. Bostock's Simplification article: http://bost.ocks.org/mike/simplify/
Both should be used in combination: quatization to reduce the map to a right sized grid, simplification to optimize the paths.
How should stereo (2 channel) audio data be represented for FFT? Do you
A. Take the average of the two channels and assign it to the real component of a number and leave the imaginary component 0.
B. Assign one channel to the real component and the other channel to the imag component.
Is there a reason to do one or the other? I searched the web but could not find any definite answers on this.
I'm doing some simple spectrum analysis and, not knowing any better, used option A). This gave me an unexpected result, whereas option B) went as expected. Here are some more details:
I have a WAV file of a piano "middle-C". By definition, middle-C is 260Hz, so I would expect the peak frequency to be at 260Hz and smaller peaks at harmonics. I confirmed this by viewing the spectrum via an audio editing software (Sound Forge). But when I took the FFT myself, with option A), the peak was at 520Hz. With option B), the peak was at 260Hz.
Am I missing something? The explanation that I came up with so far is that representing stereo data using a real and imag component implies that the two channels are independent, which, I suppose they're not, and hence the mess-up.
I don't think you're taking the average correctly. :-)
C. Process each channel separately, assigning the amplitude to the real component and leaving the imaginary component as 0.
Option B does not make sense. Option A, which amounts to convert the signal to mono, is OK (if you are interested in a global spectrum).
Your problem (double freq) is surely related to some misunderstanding in the use of your FFT routines.
Once you take the FFT you need to get the Magnitude of the complex frequency spectrum. To get the magnitude you take the absolute of the complex spectrum |X(w)|. If you want to look at the power spectrum you square the magnitude spectrum, |X(w)|^2.
In terms of your frequency shift I think it has to do with you setting the imaginary parts to zero.
If you imagine the complex Frequency spectrum as a series of complex vectors or position vectors in a cartesian space. If you took one discrete frequency bin X(w), there would be one real component representing its direction in the real axis (x -direction), and one imaginary component in the in the imaginary axis (y - direction). There are four important values about this discrete frequency, 1. real value, 2. imaginary value, 3. Magnitude and, 4. phase. If you just take the real value and set imaginary to 0, you are setting Magnitude = real and phase = 0deg or 90deg. You have hence forth modified the resulting spectrum, and applied a bias to every frequency bin. Take a look at the wiki on Magnitude of a vector, also called the Euclidean norm of a vector to brush up on your understanding. Leonbloy was correct, but I hope this was more informative.
Think of the FFT as a way to get information from a single signal. What you are asking is what is the best way to display data from two signals. My answer would be to treat each independently, and display an FFT for each.
If you want a really fast streaming FFT you can read about an algorithm I wrote here: www.depthcharged.us/?p=176