I have a model of LDA built from a vowpal wabbit. In tutorial I've seen that it is possible to print most valuables words foreach topic like this
Topic 1
0.997 printf
0.997 sizeof
0.996 characters
0.996 character
0.995 endl
0.995 stdio
0.994 iostream
0.993 cout
0.992 unsigned
0.991 malloc
0.991 typedef
0.991 cin
0.991 argc
0.989 size_t
0.988 len
0.988 std
0.986 unicode
0.986 ascii
0.986 fprintf
0.986 scanf
Topic 2
0.999 img
0.999 div
0.999 width
0.999 height
0.999 png
0.999 jquery
0.999 alt
0.999 imgur
0.999 css
0.999 border
0.999 margin
0.998 1px
0.998 color
0.998 jsfiddle
0.998 0px
0.998 getelementbyid
0.998 addsubview
0.998 jpg
0.998 alloc
0.998 cgrectmake
Topic 3
1.0 about
1.0 question
1.0 we
1.0 looking
1.0 best
0.999 good
0.999 since
0.999 better
0.999 say
0.999 their
0.999 wondering
0.999 most
0.999 computer
0.999 such
0.999 our
0.999 were
0.999 own
0.999 really
0.999 might
0.999 think
Topic 4
0.997 eventargs
0.996 mysql_query
0.996 linq
0.996 varchar
0.995 actionresult
0.995 ienumerable
0.995 lastname
0.995 firstname
0.994 tolist
0.994 entity
0.994 writeline
0.993 sqlcommand
0.993 dbo
0.993 user_id
0.993 binding
0.992 userid
0.992 datatable
0.992 databind
0.991 byval
0.991 connectionstring
Does anyone know how to produce such output when I only have file with a model?
Related
I want to use available several features to predict a variable. It does not seem to be related to vision or NLP. Although I believe there are good reasons that the variable to be predicted is a non linear function of these features. So I just use normal MLP like following:
class Net(nn.Module):
def __init__(self):
super(Net, self).__init__()
self.fc1 = nn.Linear(53, 200)
self.fc2 = nn.Linear(200, 100)
self.fc3 = nn.Linear(100, 36)
self.fc4 = nn.Linear(36, 1)
def forward(self, x):
x = F.leaky_relu(self.fc1(x))
x = F.leaky_relu(self.fc2(x))
x = F.leaky_relu(self.fc3(x))
x = self.fc4(x)
return x
net = Net().to(device)
loss_function = nn.MSELoss()
optimizer = optim.Adam(net.parameters(), lr=0.001, weight_decay= 1e-6)
def train_normal(model, device, train_loader, optimizer, epoch):
model.train ()
for batch_idx, (data, target) in enumerate (train_loader):
data = data.to (device)
target = target.to (device)
optimizer.zero_grad ()
output = model (data)
loss = loss_function (output, target)
loss.backward ()
torch.nn.utils.clip_grad_norm_(model.parameters(), 100)
optimizer.step ()
if batch_idx % 100 == 0:
print ('Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}'.format (
epoch, batch_idx * len (data), len (train_loader.dataset),
100. * batch_idx / len (train_loader), loss.item ()))
At first it seems to work and did learn something:
Train Epoch: 9 [268800/276316 (97%)] Loss: 0.217219
Train Epoch: 9 [275200/276316 (100%)] Loss: 0.234965
predicted actual diff
-1.18 -1.11 -0.08
0.15 -0.15 0.31
0.19 0.27 -0.08
-0.49 -0.48 -0.01
-0.05 0.08 -0.14
0.44 0.50 -0.06
-0.17 -0.05 -0.12
1.81 1.92 -0.12
1.55 0.76 0.79
-0.05 -0.30 0.26
But when it kept learning, I saw the results seemingly to be close to each other's average regardless the different input:
predicted actual diff
-0.16 -0.06 -0.10
-0.16 -0.55 0.39
-0.13 -0.26 0.14
-0.15 0.50 -0.66
-0.16 0.02 -0.18
-0.16 -0.12 -0.04
-0.16 -0.40 0.24
-0.01 1.20 -1.21
-0.07 0.33 -0.40
-0.09 0.02 -0.10
What technology / trick can prevent it? Also, how to increase the accuracy, shall I add more hidden layers or add more neurons of each layer?
One possible problem is that there is nothing to learn.
Check that your data is standardized and try different learning rates (maybe even cyclic learning rate). Something that can be happening is that the algorithm is not able to get inside the minima and keeps jumping around.
I am not sure, if you are using it but, use a standard implementation that works in another dataset and then change it to your problem, just to avoid small development mistakes. You can check either this tutorial How to apply Deep Learning on tabular data with FastAi but if you are really new I will totally recommend doing this MOOC https://course.fast.ai/. This should allow you to gain some level and understanding.
If you have all tabular data already you can try to use a machine learning algorithm like linear regression/gradient boosting. Just to check if your data has some info.
>>> import numpy as np
>>> from sklearn.linear_model import LinearRegression
>>> X = np.array([[1, 1], [1, 2], [2, 2], [2, 3]])
>>> # y = 1 * x_0 + 2 * x_1 + 3
>>> y = np.dot(X, np.array([1, 2])) + 3
>>> reg = LinearRegression().fit(X, y)
>>> reg.score(X, y)
1.0
>>> reg.coef_
array([1., 2.])
>>> reg.intercept_
3.0000...
>>> reg.predict(np.array([[3, 5]]))
array([16.])
Let me know if you find the solution to your problem!
Below is my last layer in training net:
layer {
name: "loss"
type: "SoftmaxWithLoss"
bottom: "final"
bottom: "label"
top: "loss"
loss_param {
ignore_label: 255
normalization: VALID
}
}
Note I adopt a softmax_loss layer. Since its calculation form is like: - log (probability), it's weird the loss can be negative, as shown below(iteration 80).
I0404 23:32:49.400624 6903 solver.cpp:228] Iteration 79, loss = 0.167006
I0404 23:32:49.400806 6903 solver.cpp:244] Train net output #0: loss = 0.167008 (* 1 = 0.167008 loss)
I0404 23:32:49.400825 6903 sgd_solver.cpp:106] Iteration 79, lr = 0.0001
I0404 23:33:25.660655 6903 solver.cpp:228] Iteration 80, loss = -1.54972e-06
I0404 23:33:25.660845 6903 solver.cpp:244] Train net output #0: loss = 0 (* 1 = 0 loss)
I0404 23:33:25.660862 6903 sgd_solver.cpp:106] Iteration 80, lr = 0.0001
I0404 23:34:00.451464 6903 solver.cpp:228] Iteration 81, loss = 1.89034
I0404 23:34:00.451661 6903 solver.cpp:244] Train net output #0: loss = 1.89034 (* 1 = 1.89034 loss)
Can anyone explain it for me? How can this happened?
Thank you very much!
PS:
The task I do here is semantic segmentation.
There are 20 object classes plus background in total(So 21 classes). The label range from 0-21. The extra label 225 is ignored which can be find in SoftmaxWithLoss definition at the beginning of this post.
Is caffe run on GPU or CPU ?
Print out prob_data that you get after softmax operation:
// find the next line in your cpu or gpu Forward function
softmax_layer_->Forward(softmax_bottom_vec_, softmax_top_vec_);
// make sure you have data in cpu
const Dtype* prob_data = prob_.cpu_data();
for (int i = 0; i < prob_.count(); i++) {
printf("%f ", prob_data[i]);
}
Consider the following code:
int i = 1;
System.out.println("1 binary: " + Long.toBinaryString(i));
long ri = Long.reverse(i);
System.out.println("1 reverse bit decimal: " + ri);
System.out.println("1 reverse bit binary: "+ Long.toBinaryString(ri));
BigInteger bil = new BigInteger(1, Longs.toByteArray(ri));
System.out.println("1 Sign-Magnitude BigInteger toString: " + bil.toString());
The output is:
1 binary: 1
1 reverse bit decimal: -9223372036854775808
1 reverse bit binary: 1000000000000000000000000000000000000000000000000000000000000000
1 Sign-Magnitude BigInteger toString: 9223372036854775808
Can anyone help to explain why the value of "1 Sign-Magnitude BigInteger toString:" is 9223372036854775808 (2^63)?
To get the sign-magnitude of a value, you simply take its absolute value as magnitude and remember the sign in a separate bit (or byte).
The sign-magnitude representation of, say, 722 is simply:
sign = 0
magnitude = 722
The sign magnitude of -722 is simply:
sign = 1
magnitude = 722
That is also what BigInteger uses.
Your code reverses a value, which means that, say, the 8 bit value 00000001 (1) is changed into 10000000 (128 or 2^7). That is not the same as inverting, which turns e.g. 00000001 (1) into 11111110 (254). That is what one's complement does. The generally used two's complement negates 00000001 (1) into 11111111 (255, i.e. 256 - 1). You should read up about two's complement, which takes some understanding. Sign-magnitude is, however, very easy to understand (but not always very practical -- addition, subtraction are different for signed and unsigned, etc. -- and that is why most processors use two's-complement)
So again: sign-magnitude works like this:
sign = (n < 0)
magnitude = abs(n)
I am practicing for midterm and apprently there's no answer key for it.
However, I practiced and got a result but not sure if this is correct since the solution is really long.
perfrom the following aritmetic operations using the 10 bit floating point standard(given on equation sheet)
-I need to convert to normalized
-standardize properly
-convert final answer to normalized 10 bit.
-convert the final answer to decimal.
-must indicate if there is any loss of precision
my solution is really long so I'm just gonna post only answer here.
-normalized
: 54.125 = 0 1100 10110
: 184 = 0 1110 01110
-standardized:
54.125 = 0 1110 0.01101
184 = 0 1110 1.01110
-result :
0 0010 0000
-denormalized: 0.03125
-precision lost: -129.90625
please help thanks!
In Lilypond I can change the color of one type of object with a line like
\override Staff.Clef #'color = #(rgb-color 0.4 0.5 0.6)
I'd like to have everything in the same (non-default) color, but I neither found a list of all the objects I could color nor did I find a command to change all the colors at once. Could anybody please point me to either?
The LilyPond Snippet Repository has a solution that iterates through the list of objects contained in all-grob-descriptions:
#(define (override-color-for-all-grobs color)
(lambda (context)
(let loop ((x all-grob-descriptions))
(if (not (null? x))
(let ((grob-name (caar x)))
(ly:context-pushpop-property context grob-name 'color color)
(loop (cdr x)))))))
% Example of usage:
\relative c' {
\applyContext #(override-color-for-all-grobs (x11-color 'blue))
c4\pp\< d e f
\grace { g16[( a g fis]) } g1\ff\!
}
Note that this will change the color of every graphical object only if you run it in the proper context (Score, I think, will generally suffice), so you may need to do the following if you're in the middle of, say, a Voice context:
\stopStaff
\context Score
\applyContext #(override-color-for-all-grobs (x11-color 'blue))
\startStaff
The list of graphical objects you need is at the bottom of this page. So a simple albeit tedious approach would be to iterate through all those objects you use, e.g.
\override Staff.Clef #'color = #(rgb-color 0.4 0.5 0.6)
\override Staff.NoteHead #'color = #(rgb-color 0.4 0.5 0.6)
\override Staff.Beam #'color = #(rgb-color 0.4 0.5 0.6)
\override Staff.Slur #'color = #(rgb-color 0.4 0.5 0.6)
etc.
There's probably a much better way but I can't figure it out. Alternatively, as has been suggested before you could consider doing some post-processing on the output from Lilypond, which may be simpler depending on the tools you have available.
I strongly recommend that you read the excellent documentation, in particular how to navigate the Internals Reference as covered by the Learning Manual and the Notation Reference
Also you may obtain a better answer from the lilypond-user mailing list.