I want to ask about linear mixed models.
The significance of the fixed variable is changed with a random structure.
For example, suppose there are 5 variables:
RT(response variable), covariate variable1(C.V.1), C.V.2, I.V.1, I.V.2.
all variables are within-subject variables excepting RT.
What I want to know is the interaction of I.V.2 and I.V.2.
In this situation, I set the two models using lmer().
First is that:
m1 <- lmer(rt ~ C.V.1 + C.V.2 + I.V.1*I.V.2 + (1+C.V.1 + C.V.2 + I.V.1*I.V.2|subject) + (1|word))
and second is that:
m2 <- lmer(rt ~ C.V.1 + C.V.2 + I.V.1*I.V.2 + (1+ I.V.1*I.V.2|subject) + (1|word))
when I analyzed this two models, the significance of fixed variable is different between the two models.
For example, the interaction of I.V.1 and I.V.2 is significant in m1, but not in m2.
I know setting the subject intercept means the responses would be different from each subject and setting the subject slope for I.V.1 means the effect of I.V.1 would be different from each subject.
But I don't know the relationship between fixed effects and random effects.
What is the meaning that considering random effects?
Can I interpret the result of the estimate of fixed variable as a coefficient when controlling the effect of other random effects like covariate variable?
And why the significance of fixed effect is changed with a random structure like the above two models?
Thank you for reading and I hope anyone would explain me these.
Related
What DataSetIterator should I use in order to create a DataSet object that contains miultiple features and labels? I have only seen examples similar to 'Iris example' where there is only one label and it is known how many different labels there are. In my problem there are four labels (position X, position Y, width and height of a shape) and many features (pixels values) and it's impossible to calculate how many different labels there could be.
I want something like this
RecordReader recordReader = new CSVRecordReader(0, ',');
recordReader.initialize(new FileSplit(new File(fileName)));
DataSetIterator iterator = new CustomDataSetIterator(recordReader, numRows, numFeatures, numLables);
DataSet allData = iterator.next();
Using data that looks like this
feature0;feature1;feature2;feature3;label0;label1;
I know that this question seems very basic and it is but I really had hard time finding any information about this topic in official tutorials or in documentation.
it seems like you are looking for an object detection kind of data with bounding boxes an multiple possible objects in your picture.
take a look at this example for that: https://github.com/eclipse/deeplearning4j-examples/blob/master/dl4j-examples/src/main/java/org/deeplearning4j/examples/convolution/objectdetection/HouseNumberDetection.java
in general there is a MultiDataSet that can take multiple inputs and can have multiple outputs.
I want to compare two smoothing methods for a bigram model:
Add-one smoothing
Interpolated Absolute Discounting
For the first method, I found some codes.
def calculate_bigram_probabilty(self, previous_word, word):
bigram_word_probability_numerator = self.bigram_frequencies.get((previous_word, word), 0)
bigram_word_probability_denominator = self.unigram_frequencies.get(previous_word, 0)
if self.smoothing:
bigram_word_probability_numerator += 1
bigram_word_probability_denominator += self.unique__bigram_words
return 0.0 if bigram_word_probability_numerator == 0 or bigram_word_probability_denominator == 0 else float(
bigram_word_probability_numerator) / float(bigram_word_probability_denominator)
However, I found nothing for the second method except for some references for 'KneserNeyProbDist'. However, this is for trigrams!
How can I change my code above to calculate it? The parameters of this method must be estimated from a development-set.
In this answer I just clear up a few things that I just found about your problem, but I can't provide a coded solution.
with KneserNeyProbDist you seem to refer to a python implementation of that problem: https://kite.com/python/docs/nltk.probability.KneserNeyProbDist
There exists an article about Kneser–Ney smoothing on wikipedia: https://en.wikipedia.org/wiki/Kneser%E2%80%93Ney_smoothing
The article above links this tutorial: https://nlp.stanford.edu/~wcmac/papers/20050421-smoothing-tutorial.pdf but this has a small fault on the most important page 29, the clear text is this:
Modified Kneser-Ney
Chen and Goodman introduced modified Kneser-Ney:
Interpolation is used instead of backoff. Uses a separate discount for one- and two-counts instead of a single discount for all counts. Estimates discounts on held-out data instead of using a formula
based on training counts.
Experiments show all three modifications improve performance.
Modified Kneser-Ney consistently had best performance.
Regrettable the modified Version is not explained in that document.
The original documentation by Chen & Goodman luckily is available, the Modified Kneser–Ney smoothing is explained on page 370 of this document: http://u.cs.biu.ac.il/~yogo/courses/mt2013/papers/chen-goodman-99.pdf.
I copy the most important text and formula here as screenshot:
So the Modified Kneser–Ney smoothing now is known and seems being the best solution, just translating the description beside formula in running code is still one step to do.
It might be helpful that below the shown text (above in screenshot) in the original linked document is still some explanation that might help to understand the raw description.
The assignment is to construct a two-column table that starts at x= -4 and ends with x= 5 with one unit increments between consecutive x values. It should have column headings ‘x’ and ‘f(x)’. I can't find anything helpful on html.table(), which is what we're supposed to use.
This what I have so far. I just have no idea what to put into the html.table function.
x = var('x')
f(x) = (5 * x^2) - (9 * x) + 4
html.table()
You might want to have a look at sage's reference documentation page on html.table
It contains the following valuable information :
table(x, header=False)
Print a nested list as a HTML table. Strings of html will be parsed for math inside dollar and double-dollar signs. 2D graphics will be displayed in the cells. Expressions will be latexed.
INPUT:
x – a list of lists (i.e., a list of table rows)
header – a row of headers. If True, then the first row of the table is taken to be the header.
There is also an example for sin (instead of f) with x in 0..3 instead of -4..5, that you can probably adapt pretty easily :
html.table([(x,sin(x)) for x in [0..3]], header = ["$x$", "$\sin(x)$"])
#Cimbali has a great answer. For completeness, I'll point out that you should be able to get this information with
html.table?
or, in fact,
table?
since I would say we want to advocate the more general table function, which has a lot of good potential for you.
That i want to do is to delete certain rows (or columns doesn't really mater...) from a given vector.
By going through Simulink's components found out that there is nothing performing such an operation,there are blocks help one add elements but nothing clearly for removing,so ended up trying to delete them by using a function block and following the online examples that demonstrate the usage of "[]".Lets say that i want to delete the second column of the vector u,i do u(:, 2) = [];.
That works absolutely fine in a separate m file or function but unfortunately not in a function block returning:
"Simulink does not have enough information to determine output sizes for
this block. If you think the errors below are inaccurate, try specifying
types for the block inputs and/or sizes for the block outputs."
and:
Size mismatch (size [4 x 4] ~= size [4 x 3]).
The size to the left is the size of the left-hand side of the assignment.
Function 'MATLAB Function' (#107.41.42), line 4, column 1:
"u"
Launch diagnostic report.
Is there any alternative you can suggest to remove several elements in a given vector in Simulink?
Thanks in advance
George
Finally,managed to do it without function block.There is a much easier way,by using Pad,and defining the output vector to be shorter than the input resulting in truncation.
Following up on a previous question... I've got my minimal horizon chart example much more minimaler than before ( minimal cubism.js horizon chart example (TypeError: callback is not a function) )
<body>
<div class="mag"></div>
<script type="text/javascript">
var myContext = cubism.context();
var myMetr = myContext.metric(function(start, stop, step, callback) {
d3.json("../json/600s.json.php?t0=" + start/1000 + "&t1=" + stop/1000 + "&ss=" + step/1000, function(er, dt) {
if (!dt) return callback(new Error("unable to load data, or has NaNs"));
callback(null, dt.val);
});
});
var myHoriz = myContext.horizon()
.metric(myMetr);
d3.select(".mag")
.call(myHoriz);
</script>
</body>
The d3.json() bit calls a server side .php that I've written that returns a .json version of my measurements. The .php takes the start, stop, step (which cubism's context.metric() uses) as the t0, t1, and ss items in its http query string and sends back a .json file. The divides by 1000 are because I made my .php expect parameters in s, not ms. And the dt.val is because the actual array of my measurements is in the "val" member of the json output, e.g.
{
"other":"unused members...",
"n":5,
"val":[
22292.078125,
22292.03515625,
22292.005859375,
22292.02734375,
22292.021484375
]
}
The problem is, now that I've got it pared down to (I think) the bare minimum, AND I actually understand all of it instead of just pasting from other examples and hoping for the best (in which scenario, most things I try to change just break things instead of improving them), I need to start adding parameters and functions back to make it visually more useful.
Two problems first of all are, this measurement hovers all day around 22,300, and only varies +/- 10 maybe all day, so the graph is just a solid green rectangle, AND the label just says constantly "22k".
I've fixed the label with .format(d3.format(".3f")) (versus the default .2s which uses SI metric prefixes, thus the "22k" above).
What I can't figure out is how to use either axis, scale, extent, or what, so that this only shows a range of numbers that are relevant to the viewer. I don't actually care about the positive-green and negative-blue and darkening colours aspects of the horizon chart. I just used it as proof-of-concept to get the constantly-shifting window of measurements from my .json data source, but the part I really need to keep is the serverDelay, step, size, and such features of cubism.js that intelligently grab the initial window of data, and incrementally grab more via the .json requests.
So how do I keep the cubism bits I need, but usefully change my all-22300s graph to show the important +/- 10 units?
update re Scott Cameron's suggestion of horizon.extent([22315, 22320])... yes I had tried that and it had zero effect. Other things I've changed so far from "minimal" above...
var myHoriz = myContext.horizon()
.metric(myMetr)
.format(d3.format(".2f"))
.height(100)
.title("base1 (m): ")
.colors(["#08519c", "#006d2c"])
// .extent([22315, 22320]) // no effect with or without this line
;
I was able to improve the graph by using metric.subtract inserting it above the myHoriz line like so: (but it made the numerical label useless now):
var myMetr2 = myMetr.subtract(22315);
var myHoriz = myContext.horizon()
.metric(myMetr2)
.format...(continue as above)
All the examples seem so concise and expressive and work fine verbatim but so many of the tweaks I try to make to them seem to backfire, I'm not sure why that is. And similarly when I refer to the API wiki... maybe 4 out of 5 things I use from the API work immediately, but then I always seem to hit one that seems to have no effect, or breaks the chart completely. I'm not sure I've wrapped my head around how so many of the parameters being passed around are actually functions, for one thing.
Next hurdles after this scale/extent question, will be getting the horizontal time axis back (after having chopped it out to make things more minimal and easier to understand), and switching this from an area-looking graph to more of a line graph.
Anyway, all direction and suggestion appreciated.
Here's the one with the better vertical scale, but now the numerical label isn't what I want:
Have you tried horizon.extent? It lets you specify the [min, max] value for the horizon chart. By default, a linear scale will be created to map values within the extent to the pixels within the chart's height (specified with `horizon.height or default to 30 pixels).