I want to split a 3D polyline with an equal distance interval. A lot of answers work well for a 2D case, e.g. Splitting MultiLine or LineString into equal segments of particular length using GeoPandas and shapely, Shapely was used to split a 2D polyline. However, Shapely is a planar geometry library and z, the height above or below the plane, is ignored in geometric analysis, which can not used to handle a 3D case.
Any suggestion will be appreciate, python example would be better. Thanks again.
The following code is a 3D points input, while getting a 2D distance result.
from shapely.geometry import LineString, Point, MultiPoint
import numpy as np
line = LineString([(0, 0, 0), (2, 1, 1), (3, 2, 2), (3.5, 1, 1), (5, 2, 2)])
distance_delta = 0.9
# generate the equidistant points
distances = np.arange(0, line.length, distance_delta)
points = MultiPoint([line.interpolate(distance) for distance in distances] + [line.boundary.geoms[1]])
Related
I've drawn a polygon which contains a point in Google Maps. But if I pass the coordinates to MySQL to calculate if the point is within the polygon, it returns false.
SELECT ST_Within(
ST_GeomFromText('POINT(8.34047 54.91320)', 4326),
ST_GeomFromText('POLYGON((62.144619879597 10.486242310988,54.622536815923 2.3124141859883,55.403637023919 23.977453248488,62.144619879597 10.486242310988))', 4326)
) AS is_point_within_polygon;
=> returns 0
But the point is obviously within the polygon:
I double-checked that using Python:
import numpy as np
from shapely.geometry import Point
from shapely.geometry.polygon import Polygon
if __name__ == '__main__':
v0 = [62.144619879597, 10.486242310988]
v1 = [54.622536815923, 2.3124141859883]
v2 = [55.403637023919, 23.977453248488]
lats_vect = np.array([v0[0], v1[0], v2[0]])
lons_vect = np.array([v0[1], v1[1], v2[1]])
lats_vect = np.append(lats_vect, lats_vect[0])
lons_vect = np.append(lons_vect, lons_vect[0])
lons_lats_vect = np.column_stack((lons_vect, lats_vect))
polygon = Polygon(lons_lats_vect)
point = Point(8.34047, 54.9132)
print(point.within(polygon))
=> prints True
What's wrong with the MySQL query?
I think there are two issues here:
First with the query. You list polygon in lat-lon order, but the point seems to be in lon-lat order. You probably want
SELECT ST_Within(
ST_GeomFromText('POINT(54.91320 8.34047)', 4326), -- NOTE CHANGE HERE
ST_GeomFromText('POLYGON((62.144619879597 10.486242310988,54.622536815923 2.3124141859883,55.403637023919 23.977453248488,62.144619879597 10.486242310988))', 4326)
) AS is_point_within_polygon;
Even this query returns FALSE, and this is expected in MySQL. 4326 is Geodesic coordinate system, meaning it operates on the spherical Earth, not on this flat map. With geodesic CRS, edges follow the geodesic shortest lines on Earth, not straight lines on flat map, and for really long lines like here and points close to the edge it matter:
Points slightly further North would be within the polygon, e.g. check out
SELECT ST_Within(
ST_GeomFromText('POINT(56 8.34047)', 4326),
ST_GeomFromText('POLYGON((62.144619879597 10.486242310988,54.622536815923 2.3124141859883,55.403637023919 23.977453248488,62.144619879597 10.486242310988))', 4326)
) AS is_point_within_polygon
I understand that the centroid of a polygon may be calculated from
from shapely.geometry import Polygon
coordinate_list = [[1,2], [2,3], [5,5]]
output = Polygon(coordinate_list).centroid
However, my coordinate_list is a multiple polygons, e.g. my coordinate_list = [[[1,2], [2,3], [5,5]], [[0,0], [0,1], [1,0]]]
Is there way to do this. Shapely appears to have a multipolygon class but it does not operate the same as the Polygon class.
You can use MultiPolygon().centroid, it's just that you can't pass that coordinate_list directly to MultiPolygon constructor as it:
/../ takes a sequence of exterior ring and hole list tuples /../
/../ also accepts an unordered sequence of Polygon instances /../
https://shapely.readthedocs.io/en/stable/manual.html#collections-of-polygons
# Based on Multipolygon sample,
# https://shapely.readthedocs.io/en/stable/code/multipolygon.py
from matplotlib import pyplot
from shapely.geometry import Polygon, MultiPolygon
from descartes.patch import PolygonPatch
# from https://github.com/shapely/shapely/blob/main/docs/code/figures.py
from figures import BLUE, BLACK, SIZE, set_limits, plot_coords, color_isvalid
fig = pyplot.figure(1, figsize=SIZE, dpi=90)
ax = fig.add_subplot(121)
set_limits(ax, -1, 6, -1, 6)
coordinate_list = [[[1,2], [2,3], [5,5]], [[0,0], [0,1], [1,0]]]
# "constructor takes a sequence of exterior ring and hole list tuples" -
# https://shapely.readthedocs.io/en/stable/manual.html#collections-of-polygons
multi = MultiPolygon([(coordinate_list[0], []), (coordinate_list[1], [])])
# "the constructor also accepts an unordered sequence of Polygon instances"
#multi = MultiPolygon([Polygon(coordinate_list[0]),Polygon(coordinate_list[1])])
plot_coords(ax, multi.centroid, color=BLACK)
for polygon in multi.geoms:
plot_coords(ax, polygon.exterior)
patch = PolygonPatch(polygon, facecolor=BLUE, edgecolor=BLUE, alpha=0.5, zorder=2)
ax.add_patch(patch)
I'm trying to use transformer to process some image data (not NLP data), e.g. 480 x 640 images with different sequence length, an example would be [6, 480, 640], [7, 480, 640], [8, 480, 640]. And I would like to put these three sequences into one batch.
However, most tutorials I saw use torchtext to deal with the non-fixed length problem. But since I run the transformer with my own dataset, torchtext is not applicable(is it?). After searching I find pad_sequence can be used to deal with this problem.
However I didn't find any tutorials about using pad_sequence with transformer. Is it applicable？Has anyone try it before?
Let's say we have 03 images with different dimensions. Applying pad_sequence function on them will result as follow:
import torch
from torch.nn.utils.rnn import pad_sequence
image_1 = torch.ones(25, 30)
image_2 = torch.ones(32, 30)
image_3 = torch.ones(29, 30)
images = pad_sequence([image_1, image_2, image_3])
print(images.size())
# torch.Size([32, 3, 30])
This remains the same if you are working with 3D images
import torch
from torch.nn.utils.rnn import pad_sequence
image_1 = torch.ones(25, 30, 50)
image_2 = torch.ones(32, 30, 50)
image_3 = torch.ones(29, 30, 50)
images = pad_sequence([image_1, image_2, image_3])
print(images.size())
# torch.Size([32, 3, 30, 50])
One thing you should be aware of with this function is that it only works when the images share the n - 1 dimensions. In other words, if you have something like this:
import torch
from torch.nn.utils.rnn import pad_sequence
image_1 = torch.ones(25, 30, 50)
image_2 = torch.ones(32, 50, 30)
image_3 = torch.ones(29, 31, 50)
images = pad_sequence([image_1, image_2, image_3])
# RuntimeError: The size of tensor a (50) must match the size of tensor b (30) at non-singleton dimension 2
print(images.size())
It won't work!
But anyways, since you're working with images, I suggest you to use the Pad transformation from torchvision. It works the same as the pad_sequence function but with more options. Just follow the doc.
It is very common tu use softmax function for converting an array of values in an array of probabilities. In general, the function amplifies the probability of the greater values of the array.
However, this function is not scale invariant. Let us consider an example:
If we take an input of [1, 2, 3, 4, 1, 2, 3], the softmax of that is [0.024, 0.064, 0.175, 0.475, 0.024, 0.064, 0.175]. The output has most of its weight where the '4' was in the original input. That is, softmax highlights the largest values and suppress values which are significantly below the maximum value. However, if the input were [0.1, 0.2, 0.3, 0.4, 0.1, 0.2, 0.3] (which sums to 1.6) the softmax would be [0.125, 0.138, 0.153, 0.169, 0.125, 0.138, 0.153]. This shows that for values between 0 and 1 softmax, in fact, de-emphasizes the maximum value (note that 0.169 is not only less than 0.475, it is also less than the initial proportion of 0.4/1.6=0.25).
I would need a function that amplifies differences between values in an array, emphasizing the greatest values and that is not so affected by the scale of the numbers in the array.
Can you suggest some function with these properties?
As Robert suggested in the comment, you can use temperature. Here is a toy realization in Python using numpy:
import numpy as np
def softmax(preds):
exp_preds = np.exp(preds)
sum_preds = np.sum(exp_preds)
return exp_preds / sum_preds
def softmax_with_temperature(preds, temperature=0.5):
preds = np.log(preds) / temperature
preds = np.exp(preds)
sum_preds = np.sum(preds)
return preds / sum_preds
def check_softmax_scalability():
base_preds = [1, 2, 3, 4, 1, 2, 3]
base_preds = np.asarray(base_preds).astype("float64")
for i in range(1,3):
print('logits: ', base_preds*i,
'\nsoftmax: ', softmax(base_preds*i),
'\nwith temperature: ', softmax_with_temperature(base_preds*i))
Calling check_softmax_scalability() would return:
logits: [1. 2. 3. 4. 1. 2. 3.]
softmax: [0.02364054 0.06426166 0.1746813 0.474833 0.02364054 0.06426166
0.1746813 ]
with temperature: [0.02272727 0.09090909 0.20454545 0.36363636 0.02272727 0.09090909
0.20454545]
logits: [2. 4. 6. 8. 2. 4. 6.]
softmax: [0.00188892 0.01395733 0.10313151 0.76204449 0.00188892 0.01395733
0.10313151]
with temperature: [0.02272727 0.09090909 0.20454545 0.36363636 0.02272727 0.09090909
0.20454545]
But the scale invariance comes with a cost: as you increase temperature, the output values will come closer to each other. Increase it too much, and you will have an output that looks like a uniform distribution. In your case, you should pick a low value for temperature to emphasize the maximum value.
You can read more about how temperature works here.
I want to use a Linear, Fully-Connected Layer as one of the input layers in my network. The input has shape (batch_size, in_channels, num_samples). It is based on the Tacotron paper: https://arxiv.org/pdf/1703.10135.pdf, the Enocder prenet part.
It feels to me as if Chainer and PyTorch have different implementations of the Linear layer - are they really performing the same operations or am I misunderstanding something?
In PyTorch, behavior of the Linear layer follows the documentations: https://pytorch.org/docs/0.3.1/nn.html#torch.nn.Linear
according to which, the shape of the input and output data are as follows:
Input: (N,∗,in_features) where * means any number of additional dimensions
Output: (N,∗,out_features) where all but the last dimension are the same shape as the input.
Now, let's try creating a linear layer in pytorch and performing the operation. I want an output with 8 channels, and the input data will have 3 channels.
import numpy as np
import torch
from torch import nn
linear_layer_pytorch = nn.Linear(3, 8)
Let's create some dummy input data of shape (1, 4, 3) - (batch_size, num_samples, in_channels:
data = np.array([1, 1, 1, 2, 2, 2, 3, 3, 3, 4, 4, 4], dtype=np.float32).reshape(1, 4, 3)
data_pytorch = torch.from_numpy(data)
and finally, perform the operation:
results_pytorch = linear_layer_pytorch(data_pytorch)
results_pytorch.shape
the shape of the output is as follows: Out[27]: torch.Size([1, 4, 8])
Taking a look at the source of the PyTorch implementation:
def linear(input, weight, bias=None):
# type: (Tensor, Tensor, Optional[Tensor]) -> Tensor
r"""
Applies a linear transformation to the incoming data: :math:`y = xA^T + b`.
Shape:
- Input: :math:`(N, *, in\_features)` where `*` means any number of
additional dimensions
- Weight: :math:`(out\_features, in\_features)`
- Bias: :math:`(out\_features)`
- Output: :math:`(N, *, out\_features)`
"""
if input.dim() == 2 and bias is not None:
# fused op is marginally faster
ret = torch.addmm(bias, input, weight.t())
else:
output = input.matmul(weight.t())
if bias is not None:
output += bias
ret = output
return ret
It transposes the weight matrix that is passed to it, broadcasts it along the batch_size axis and performs a matrix multiplications. Having in mind how a linear layer works, I imagine it as 8 nodes, connected through a synapse, holding a weight, with every channel in an input sample, thus in my case it has 3*8 weights. And that is exactly the shape I see in debugger (8, 3).
Now, let's jump to Chainer. The Chainer's linear layer documentation is available here: https://docs.chainer.org/en/stable/reference/generated/chainer.links.Linear.html#chainer.links.Linear. According to this documentation, the Linear layer wraps the function linear, which according to the docs, flattens the input along the non-batch dimensions and the shape of it's weight matrix is (output_size, flattend_input_size)
import chainer
linear_layer_chainer = chainer.links.Linear(8)
results_chainer = linear_layer_chainer(data)
results_chainer.shape
Out[21]: (1, 8)
Creating the layer as linear_layer_chainer = chainer.links.Linear(3, 8) and calling it causes a size mismatch. So in case of chainer, I have gotten a totally different results, because this time around I have a weight matrix that is of shape (8, 12) and my results have a shape of (1, 8). So now, here is my question : since the results are clearly different,both the weight matrices and the outputs have different shapes, how can I make them equivalent and what should be the desired output? In the PyTorch implementation of Tacotron it seems that the PyTorch approach is used as is (https://github.com/mozilla/TTS/blob/master/layers/tacotron.py) - Prenet. If that is the case, how can I make the Chainer produce the same results (I have to implement this in Chainer). I will be grateful for any inshight, sorry that the post has gotten this long.
Chainer Linear layer (a bit frustratingly) does not apply the transformation to the last axis. Chainer flattens the rest of the axes. Instead you need to provide how many batch axes there are, documentation which is 2 in your case:
# data.shape == (1, 4, 3)
results_chainer = linear_layer_chainer(data, n_batch_axes=2)
# 2 batch axes (1,4) means you apply linear to (..., 3)
# results_chainer.shape == (1, 4, 8)
You can also use l(data, n_batch_axes=len(data.shape)-1) to always apply to the last dimension which is the default behaviour in PyTorch, Keras etc.