Related
I have been trying to reproduce this plot using the code from here
My last attempts got me pretty close but I can't find the appropiate way to make the graph look like I want.
In my data, z is the numerical result of a simulation carried out under x and y conditions and I want to map that relationship, like you could do simply by a scatterplot but prettier.
My df is pretty large but it has this aspect:
x -> continous variable ranging from 120 - 300
y -> continous variable ranging from 0.2 - 1.8
z -> continous variable ranging from -0.0001 to 3000
This is my First attempt generated with this code
levelplot(z ~ x * y,
data,
panel = panel.levelplot.points,
cex = 0.7,
col.regions = rocket(25, alpha = .8, direction = -1),
colorkey = list(at = (breaks = c(1, 20, 50, 150, 500, Inf))),
scales = list(x=list(at = c(120, 140, 160, 180, 200, 220, 240, 260, 280, 299))),
xlab = "x",
ylab = "y"
) +
layer_(panel.2dsmoother(..., n = 100))
The problem is that, while points are a fair representation of my data, the 2d layer is not. I want to specially stress the difference between <1 and >=1 so my breaks start by 1, the rest would be the scale depending on z values for different experiments (this one reaches over 3000, other just up to 30)
I have also attempted to sepparate both graphs and you can take a look at the scatter plot and the 2dSmooth panel. Here is the code for both:
## Scatter
levelplot(z ~ x * y,
data,
panel = panel.levelplot.points,
cex = 0.7,
at = c(-Inf, 1, 30, 100, 500, Inf),
col.regions = rocket(25, alpha = .8, direction = -1),
colorkey = list(
labels = c("", "1", "30", "100", "500", "")
),
scales = list(x=list(at = c(120, 140, 160, 180, 200, 220, 240, 260, 280, 299))),
xlab = "x",
ylab = "y" )
## 2dSmooth
levelplot(z ~ x * y,
data,
panel = panel.2dsmoother,
n = 200,
cuts = 5,
col.regions = rocket(25, alpha = .8, direction = -1),
colorkey = T,
scales = list(x=list(at = c(120, 140, 160, 180, 200, 220, 240, 260, 280, 299))),
xlab = "x",
ylab = "y"
)
Note I am having troubles with CUT and AT.
When I pass the AT argument to my 2dSmooth graph, the result is absurd and I have no idea why.
However, the CUT result from the 2dSmooth plot are addequate, the problem is that I cannot label them as I don't know the limits the function has taken. If there would be a way of correctly labelling that graph, that would be it. Otherwise, I need to combine both.
¿Any idea where I am messing up?
¿Any possibility of having my 2dsmooth graph done with other libraries? So far I haven't found anything similar to levelplot() with panel.2dsmoother
Thank you for your help
Your problem seems very specific to your data, so it's difficult to answer without having access to it. But generally speaking,
I'm a bit surprised that you are getting a different set of cut points with your 2dSmooth graph, because when you don't specify at, the default values should only depend on the data (which remain unchanged) and not on the panel function. I'm not sure what's happening.
panel.2dsmoother() basically just fits a regression surface to your data, by default using loess(). If your goal is to remove the influence of "outliers" by tuning your at values to the fitted surface instead of the data, you would need to do that externally. You can mimic the code of panel.2dsmoother() to do this, e.g.,
n <- 100
mod <- loess(z ~ x * y, data)
xseq <- seq(120, 300, length = n)
yseq <- seq(0.2, 1.8, length = n)
grid <- expand.grid(x = xseq, y = yseq)
fit <- predict(mod, grid)
You could now visualize this surface using a standard levelplot() call using
(p <- levelplot(fit ~ x + y, grid))
and the corresponding at values should be roughly
do.breaks(range(fit), 5)
You can get the exact values using
trellis.panelArgs(p, 1)$at
When I try to train yolo3 with command
python mmdetection/tools/train.py config/yolo-darknet53.py
and everything goes ok until first epoch begin and error happens that
it says "ValueError: need at least one array to concatenate" I am searching for a long time on net. But no use. Please help or try to give some ideas how to achieve this.
here is the yolo3 configuration file
#!user/bin/env python3
# -*- coding: utf-8 -*-
_base_ = '../mmdetection/configs/_base_/default_runtime.py'
classes = ('hand',)
# model settings
model = dict(
type='YOLOV3',
pretrained='D:/workplace/srtp/handdetection/checkpoints/darknet53-a628ea1b.pth',
backbone=dict(type='Darknet', depth=53, out_indices=(3, 4, 5)),
neck=dict(
type='YOLOV3Neck',
num_scales=3,
in_channels=[1024, 512, 256],
out_channels=[512, 256, 128]),
bbox_head=dict(
type='YOLOV3Head',
num_classes=1,
in_channels=[512, 256, 128],
out_channels=[1024, 512, 256],
anchor_generator=dict(
type='YOLOAnchorGenerator',
base_sizes=[[(116, 90), (156, 198), (373, 326)],
[(30, 61), (62, 45), (59, 119)],
[(10, 13), (16, 30), (33, 23)]],
strides=[32, 16, 8]),
bbox_coder=dict(type='YOLOBBoxCoder'),
featmap_strides=[32, 16, 8],
loss_cls=dict(
type='CrossEntropyLoss',
use_sigmoid=True,
loss_weight=1.0,
reduction='sum'),
loss_conf=dict(
type='CrossEntropyLoss',
use_sigmoid=True,
loss_weight=1.0,
reduction='sum'),
loss_xy=dict(
type='CrossEntropyLoss',
use_sigmoid=True,
loss_weight=2.0,
reduction='sum'),
loss_wh=dict(type='MSELoss', loss_weight=2.0, reduction='sum')))
# training and testing settings
train_cfg = dict(
assigner=dict(
type='GridAssigner', pos_iou_thr=0.5, neg_iou_thr=0.5, min_pos_iou=0))
test_cfg = dict(
nms_pre=1000,
min_bbox_size=0,
score_thr=0.05,
conf_thr=0.005,
nms=dict(type='nms', iou_threshold=0.45),
max_per_img=100)
# dataset settings
dataset_type = 'VOCDataset'
dataset_root = 'D:/workplace/srtp/handdetection/VOC_HandDataSetVOC2007'
img_norm_cfg = dict(mean=[0, 0, 0], std=[255., 255., 255.], to_rgb=True)
train_pipeline = [
dict(type='LoadImageFromFile', to_float32=True),
dict(type='LoadAnnotations', with_bbox=True),
dict(type='PhotoMetricDistortion'),
dict(
type='Expand',
mean=img_norm_cfg['mean'],
to_rgb=img_norm_cfg['to_rgb'],
ratio_range=(1, 2)),
dict(
type='MinIoURandomCrop',
min_ious=(0.4, 0.5, 0.6, 0.7, 0.8, 0.9),
min_crop_size=0.3),
dict(type='Resize', img_scale=[(320, 320), (608, 608)], keep_ratio=True),
dict(type='RandomFlip', flip_ratio=0.5),
dict(type='Normalize', **img_norm_cfg),
dict(type='Pad', size_divisor=32),
dict(type='DefaultFormatBundle'),
dict(type='Collect', keys=['img', 'gt_bboxes', 'gt_labels'])
]
test_pipeline = [
dict(type='LoadImageFromFile'),
dict(
type='MultiScaleFlipAug',
img_scale=(608, 608),
flip=False,
transforms=[
dict(type='Resize', keep_ratio=True),
dict(type='RandomFlip'),
dict(type='Normalize', **img_norm_cfg),
dict(type='Pad', size_divisor=32),
dict(type='ImageToTensor', keys=['img']),
dict(type='Collect', keys=['img'])
])
]
data = dict(
samples_per_gpu=1,
workers_per_gpu=1,
train=dict(
type=dataset_type,
ann_file=f'{dataset_root}/ImageSets/Main/hand_train.txt',
img_prefix=f'{dataset_root}',
pipeline=train_pipeline),
val=dict(
type=dataset_type,
ann_file=f'{dataset_root}/ImageSets/Main/hand_val.txt',
img_prefix=f'{dataset_root}',
pipeline=test_pipeline),
test=dict(
type=dataset_type,
ann_file=f'{dataset_root}/ImageSets/Main/hand_test.txt',
img_prefix=f'{dataset_root}',
pipeline=test_pipeline))
# optimizer
optimizer = dict(type='SGD', lr=0.001, momentum=0.9, weight_decay=0.0005)
optimizer_config = dict(grad_clip=dict(max_norm=35, norm_type=2))
# learning policy
lr_config = dict(
policy='step',
warmup='linear',
warmup_iters=2000, # same as burn-in in darknet
warmup_ratio=0.1,
step=[218, 246])
# runtime settings
total_epochs = 273
evaluation = dict(interval=1, metric=['bbox'])
I'm trying to extract the features of images using VGG19 network (the output should be of dim : [1 , 7 , 7 , 512] per frame
here is the code I have used :
deep_net = models.vgg19(pretrained=True).cuda()
deep_net = nn.Sequential(*list(deep_net.children())[:-2])
deep_net.eval()
save_file_sample_path = '/media/data1/out.npy'
input_image = torch.zeros(1, 3, 224, 224)
output_feat = np.zeros(shape=[1, 49, 512])
with torch.no_grad():
im = default_loader('/media/data1/images/frame612.jpg')
im = transform(im)
input_image[0, :, :] = im
input_image = input_image.cuda()
output_feat = deep_net(input_image)
output_feat = output_feat.features[:-2].view(1, 512, 49).transpose(1, 2)
But I get the following error :
AttributeError: 'Sequential' object has no attribute 'features'
At the line :
output_feat = output_feat.features[:-2].view(1, 512, 49).transpose(1, 2)
Any idea why this does not work anymore? and how to fix?
Thanks!
It's because you are rebuilding deep_net with nn.Sequential so it loses the attribute features.
deep_net = models.vgg19(pretrained=True)
deep_net.features
Sequential(
(0): Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(1): ReLU(inplace=True)
...
(36): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
)
deep_net = nn.Sequential(*list(deep_net.children())[:-2])
deep_net.features
AttributeError: 'Sequential' object has no attribute 'features'
The equivalent you want now is this:
list(deep_net.children())[0][:-2]
I am looking for an in depth example of Latent Dirichlet Allocation (LDA) with seedwords specified for the topicmodels package in R.
The basic function takes on the form:
LDA(x, k, method = "Gibbs", control = NULL, model = NULL, ...)
And the documentation only states:
For method = "Gibbs" an additional argument seedwords can be specified
as a matrix or an object of class "simple_triplet_matrix"; the default
is NULL.
Can anyone point me to a complete example of how this would look and function?
Taken from this answer:
https://stats.stackexchange.com/questions/384183/seeded-lda-using-topicmodels-in-r
library("topicmodels")
data("AssociatedPress", package = "topicmodels")
## We fit 6 topics.
## We specify five seed words for five topics, the sixth topic has no
## seed words.
library("slam")
set.seed(123)
i <- rep(1:5, each = 5)
j <- sample(1:ncol(AssociatedPress), 25)
SeedWeight <- 500 - 0.1
deltaS <- simple_triplet_matrix(i, j, v = rep(SeedWeight, 25),
nrow = 6, ncol = ncol(AssociatedPress))
set.seed(1000)
ldaS <- LDA(AssociatedPress, k = 6, method = "Gibbs", seedwords = deltaS,
control = list(alpha = 0.1, best = TRUE,
verbose = 500, burnin = 500, iter = 100, thin = 100, prefix = character()))
apply(deltaS, 1, function(x) which(x == SeedWeight))
apply(posterior(ldaS)$terms, 1, function(x) order(x, decreasing = TRUE)[1:5])
I am currently using pycuda and scikits.cuda to solve linear equation A*x = b, where A is an upper/lower matrix. However the cublasStbsv routine requires a specific format.
To give an example: if a lower matrix A = [[1, 0, 0], [2, 3, 0], [4, 5, 6]], then the input required by cublasStbsv should be [[1, 3, 6], [2, 5, 0], [4, 0, 0]], where rows are diagonal, subdiagonal1, subdiagonal2, respectively. If using numpy, this can be easily done by stride_tricks.as_strided, but I dont know how to do similar things with pycuda.gpuarray. Any help would be appreciated, thanks. I found pycuda.compyte.array.as_strided, but it cannot be applied to gpuarray.
I got it done by using theano. First converted it to cudandarray, change stride and make a copy back to gpuarray. Just be careful about changes between Fortran and C order.
update:
finally got it done by using gpuarray.multi_take_put
def make_triangle(s_matrix, uplo = 'L'):
"""convert triangle matrix to the specific format
required by cublasStbsv, matrix should be in Fortran order,
s_matrix: gpuarray
"""
#make sure the dytpe is float32
if s_matrix.dtype != 'f':
s_matrix = s_matrix.astype('f')
dim = s_matrix.shape[0]
if uplo == 'L':
idx_tuple = np.tril_indices(dim)
gidx = gpuarray.to_gpu(idx_tuple[0] + idx_tuple[1] * dim)
gdst = gpuarray.to_gpu(idx_tuple[0] + idx_tuple[1] * (dim - 1))
return gpuarray.multi_take_put([s_matrix], gdst, gidx, (dim, dim))[0]
else:
idx_tuple = np.triu_indices(dim)
gidx = gpuarray.to_gpu(idx_tuple[0] + idx_tuple[1] * dim)
gdst = gpuarray.to_gpu(idx_tuple[0] + (idx_tuple[1] + 1) * (dim - 1))
return gpuarray.multi_take_put([s_matrix], gdst, gidx, (dim, dim))[0]