I have the following mixed effects model:
p1 <- lmer(log(price) ~ year*loca + (1|author), data = df)
'year' is continuous
'loca' is categorical variable with 2 levels
I am trying to plot the significant interaction from this model.
The following code (using the visreg package) plots the lines from each of the two 'loca' but it does not produce a 95% confidence band:
visreg(p1, "year", by = "loca", overlay = T,
line=list(lty = 1, col = c("grey", "black")), points=list(cex=1, pch=19,
col = c("grey", "black")), type="conditional", axes = T)
Then, I tried the following code which allows me to plot the lines, but with no data points on top and no CIs:
visreg(p1, "year", by = "loca", overlay = T,
line=list(lty = 1, col = c("grey60", "black")), points=list(cex=1,
pch=19, col = c("grey", "black")),
type="conditional", trans = exp, fill.par = list(col = c("grey80",
"grey70")))
I get CI bands when I use type = 'contrast' rather than 'conditional'. However, this doesn't work when I try to backtransform the price as above using trans = exp.
Overall I need to be able to plot the interaction with the following attributes:
Confidence bands
backtransformed points
predicted line (one for each level of 'loca')
More than happy to try other methods....but I can't seem to find any that work so far.
Help much appreciated!
one possibility is with the use of the effects package:
library(effects)
eff.p1 <- effect("year*loca", p1, KR=T)
then you could either directly plot it with what the package provides and customize it from there:
plot(eff.p1)
or take what effect produces and plot it with ggplot in a nicer plot:
eff.p1 <- as.data.frame(eff.p1)
ggplot(eff.p1, aes(year, linetype=factor(loca),
color = factor(loca))) +
geom_line(aes(y = fit, group=factor(loca)), size=1.2) +
geom_line(aes(y = lower,
group=factor(loca)), linetype =3) +
geom_line(aes(y = upper,
group=factor(loca)), linetype =3) +
xlab("year") +
ylab("Marginal Effects on Log Price") +
scale_colour_discrete("") +
scale_linetype_discrete("") +
labs(color='loca') + theme_minimal()
I can't really try the code without the data, but I think it should work.
This should do the trick:
install.packages(sjPlot)
library(sjPlot)
plot_model(p1, type = "int", terms = c(year,loca), ci.lvl = 0.95)
Although it comes out with some warnings about labels, testing on my data, it does the back transformation automatically and seems to work fine. Customising should be easy, because I believe sjPlot uses ggplot.
EDIT: #Daniel points out that alternative options which allow more customization would be plot_model(type = "pred", ...) or plot_model(type = "eff", ...)
Related
I am calculating in R an ANOVA with repeated measures in 2x2 mixed design. For this I use one of the following inputs in R:
(1)
res.aov <- anova_test(data = datac, dv = Stress, wid = REF,between = Gruppe, within = time )
get_anova_table(res.aov)
(2)
aov <- datac %>%
anova_test(dv = Stress, wid = REF, between = Gruppe, within = time, type = 3)
aov
Both lead to the same results. Now I want to add a covariate from the 1st measurement time point. So far I could not find a suitable R input for the ANCOVA for this repeated measures design.
Does anyone of you perhaps have an idea?
Many greetings
ANOVA <- aov(Stress~time+covariate, data = data)
summary(ANOVA)
For a simple ANCOVA, the R input of an ANOVA with the addition of the covariate applies. Unfortunately, I have no idea how this works in the repeated measures design.
I'm trying to figure out how sequence to sequence loss is calculated. I am using the huggingface transformers library in this case, but this might actually be relevant to other DL libraries.
So to get the required data we can do:
from transformers import EncoderDecoderModel, BertTokenizer
import torch
import torch.nn.functional as F
torch.manual_seed(42)
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
MAX_LEN = 128
tokenize = lambda x: tokenizer(x, max_length=MAX_LEN, truncation=True, padding=True, return_tensors="pt")
model = EncoderDecoderModel.from_encoder_decoder_pretrained('bert-base-uncased', 'bert-base-uncased') # initialize Bert2Bert from pre-trained checkpoints
input_seq = ["Hello, my dog is cute", "my cat cute"]
output_seq = ["Yes it is", "ok"]
input_tokens = tokenize(input_seq)
output_tokens = tokenize(output_seq)
outputs = model(
input_ids=input_tokens["input_ids"],
attention_mask=input_tokens["attention_mask"],
decoder_input_ids=output_tokens["input_ids"],
decoder_attention_mask=output_tokens["attention_mask"],
labels=output_tokens["input_ids"],
return_dict=True)
idx = output_tokens["input_ids"]
logits = F.log_softmax(outputs["logits"], dim=-1)
mask = output_tokens["attention_mask"]
Edit 1
Thanks to #cronoik I was able to replicate the loss calculated by huggingface as being:
output_logits = logits[:,:-1,:]
output_mask = mask[:,:-1]
label_tokens = output_tokens["input_ids"][:, 1:].unsqueeze(-1)
select_logits = torch.gather(output_logits, -1, label_tokens).squeeze()
huggingface_loss = -select_logits.mean()
However, since the last two tokens of the second input is just padding, shouldn't we calculate the loss to be:
seq_loss = (select_logits * output_mask).sum(dim=-1, keepdims=True) / output_mask.sum(dim=-1, keepdims=True)
seq_loss = -seq_loss.mean()
^This takes into account the length of the sequence of each row of outputs, and the padding by masking it out. Think this is especially useful when we have batches of varying length outputs.
ok I found out where I was making the mistakes. This is all thanks to this thread in the HuggingFace forum.
The output labels need to have -100 for the masked version. The transoformers library does not do it for you.
One silly mistake I made was with the mask. It should have been output_mask = mask[:, 1:] instead of :-1.
1. Using Model
We need to set the masks of output to -100. It is important to use clone as shown below:
labels = output_tokens["input_ids"].clone()
labels[output_tokens["attention_mask"]==0] = -100
outputs = model(
input_ids=input_tokens["input_ids"],
attention_mask=input_tokens["attention_mask"],
decoder_input_ids=output_tokens["input_ids"],
decoder_attention_mask=output_tokens["attention_mask"],
labels=labels,
return_dict=True)
2. Calculating Loss
So the final way to replicate it is as follows:
idx = output_tokens["input_ids"]
logits = F.log_softmax(outputs["logits"], dim=-1)
mask = output_tokens["attention_mask"]
# shift things
output_logits = logits[:,:-1,:]
label_tokens = idx[:, 1:].unsqueeze(-1)
output_mask = mask[:,1:]
# gather the logits and mask
select_logits = torch.gather(output_logits, -1, label_tokens).squeeze()
-select_logits[output_mask==1].mean(), outputs["loss"]
The above however ignores the fact that this comes from two different lines. So an alternate way of calculating loss could be:
seq_loss = (select_logits * output_mask).sum(dim=-1, keepdims=True) / output_mask.sum(dim=-1, keepdims=True)
seq_loss.mean()
thanks for sharing. However, the new version of transformers as of today actually does not "shift" anymore. The following is not needed.
#shift things
output_logits = logits[:,:-1,:]
label_tokens = idx[:, 1:].unsqueeze(-1)
output_mask = mask[:,1:
I am using a data set called sleep (found here: https://drive.google.com/file/d/15ZnsWtzbPpUBQN9qr-KZCnyX-0CYJHL5/view) to run a three way within subject ANOVA comparing Performance based on Stimulation, Deprivation, and Time. I have successfully done this before using anova_test from rstatix. I want to look at the sphericity output but it doesn't appear in the output. I have got it to come up with other three way within subject datasets, so I'm not sure why this is happening. Here is my code:
anova_test(data = sleep, dv = Performance, wid = Subject, within = c(Stimulation, Deprivation, Time))
I also tried to save it to an object and use get_anova_table, but that didn't look any different.
sleep_aov <- anova_test(data = sleep, dv = Performance, wid = Subject, within = c(Stimulation, Deprivation, Time))
get_anova_table(sleep_aov, correction = "GG")
This is an ideal dataset I pulled from the internet, so I'm starting to think the data had a W of 1 (perfect sphericity) and so rstatix is skipping this output. Is this something anova_test does?
Here also is my code using a dataset that does return Mauchly's:
weight_loss_long <- pivot_longer(data = weightloss, cols = c(t1, t2, t3), names_to = "time", values_to = "loss")
weight_loss_long$time <- factor(weight_loss_long$time)
anova_test(data = weight_loss_long, dv = loss, wid = id, within = c(diet, exercises, time))
Not an expert at all, but it might be because your factors have only two levels.
From anova_summary() help:
"Value
return an object of class anova_test a data frame containing the ANOVA table for independent measures ANOVA. However, for repeated/mixed measures ANOVA, it is a list containing the following components are returned:
ANOVA: a data frame containing ANOVA results
Mauchly's Test for Sphericity: If any within-Ss variables with more than 2 levels are present, a data frame containing the results of Mauchly's test for Sphericity. Only reported for effects that have more than 2 levels because sphericity necessarily holds for effects with only 2 levels.
Sphericity Corrections: If any within-Ss variables are present, a data frame containing the Greenhouse-Geisser and Huynh-Feldt epsilon values, and corresponding corrected p-values. "
After updating Openturns from 1.15 to 1.16rc1 I have the following issue with building the meta-model of the field function:
to reduce the computational burden:
ot.ResourceMap.SetAsUnsignedInteger("FittingTest-KolmogorovSamplingSize", 1)
algo = ot.FunctionalChaosAlgorithm(sample_X, outputSampleChaos)
algo.run()
metaModel = ot.PointToFieldConnection(postProcessing, algo.getResult().getMetaModel())
The "FittingTest-KolmogorovSamplingSize" was removed from OpenTurns 1.16rc1 and when I try to replace the fitting test with:
ot.ResourceMap.SetAsUnsignedInteger("FittingTest-LillieforsMaximumSamplingSize", 10)
Or with
ot.ResourceMap.SetAsUnsignedInteger("FittingTest-LillieforsMinimumSamplingSize", 1)
The code is freezing. Is there any solution for this?
The proposed solution is simply to use another distribution to model your data. You could have used any other multivariate continuous distribution of proper dimension. IMO it is not a valid answer as the distribution has no link to your data.
After inspection, it appears that the problem has nothing to do with Lilliefors's test. In OT 1.15 we were using this test (under the wrong name of Kolmogorov) to select automatically a distribution suited to the input sample, but we switched to a more sophisticated selection algorithm (see MetaModelAlgorithm::BuildDistribution). It is based on a first pass using the raw Kolomgorov test (thus ignoring the fact that parameters have been estimated) then an information-based criterion is used to select the most relevant model (AIC, AICC, BIC depending on the value of the "MetaModelAlgorithm-ModelSelectionCriterion" key in ResourceMap. The problem is caused by the TrapezoidalFactory class during the Kolmogorov phase. I will provide a fix ASAP in OpenTURNS master. In the mean time, I have adapted the proposed solution to something more adapted to your data:
degree = 6
dimension_xi_X = 3
dimension_xi_Y = 450
enumerateFunction = ot.HyperbolicAnisotropicEnumerateFunction(dimension_xi_X, 0.8)
basis = ot.OrthogonalProductPolynomialFactory(
[ot.StandardDistributionPolynomialFactory(ot.HistogramFactory().build(sample_X[:,i])) for i in range(dimension_xi_X)], enumerateFunction)
basisSize = enumerateFunction.getStrataCumulatedCardinal(degree)
#basis = ot.OrthogonalProductPolynomialFactory(
# [ot.HermiteFactory()] * dimension_xi_X, enumerateFunction)
#basisSize = 450#enumerateFunction.getStrataCumulatedCardinal(degree)
adaptive = ot.FixedStrategy(basis, basisSize)
projection = ot.LeastSquaresStrategy(
ot.LeastSquaresMetaModelSelectionFactory(ot.LARS(), ot.CorrectedLeaveOneOut()))
ot.ResourceMap.SetAsScalar("LeastSquaresMetaModelSelection-ErrorThreshold", 1.0e-7)
algo_chaos = ot.FunctionalChaosAlgorithm(sample_X,
outputSampleChaos,basis.getMeasure(), adaptive, projection)
algo_chaos.run()
result_chaos = algo_chaos.getResult()
meta_model = result_chaos.getMetaModel()
metaModel = ot.PointToFieldConnection(postProcessing,
algo_chaos.getResult().getMetaModel())
I also implemented a quick and dirty estimator of the L2-error:
# Meta_model validation
iMax = 5
# Input values
sample_X_validation = ot.Sample(np.array(month_1_parameters_MSE.iloc[:iMax,0:3]))
print("sample size=", sample_X_validation.getSize())
# sample_X = ot.Sample(month_1_parameters_MSE[['Rseries','Rsh','Isc']])
# output values
#month_1_simulated.iloc[0:1].transpose()
Field = ot.Field(mesh,np.array(month_1_simulated.iloc[0:1]).transpose())
sample_Y_validation = ot.ProcessSample(1,Field)
for k in range(1,iMax):
sample_Y_validation.add( np.array(month_1_simulated.iloc[k:k+1]).transpose() )
# In[18]:
graph = sample_Y_validation.drawMarginal(0)
graph.setColors(['red'])
drawables = graph.getDrawables()
graph2 = metaModel(sample_X_validation).drawMarginal(0)
graph2.setColors(['blue'])
drawables = graph2.getDrawables()
graph.add(graph2)
graph.setTitle('Model/Metamodel Validation')
graph.setXTitle(r'$t$')
graph.setYTitle(r'$z$')
drawables = graph.getDrawables()
L2_error = 0.0
for i in range(iMax):
L2_error = (drawables[i].getData()[:,1]-drawables[iMax+i].getData()[:,1]).computeRawMoment(2)[0]
print("L2_error=", L2_error)
You get an error of 79.488 with the previous answer and 1.3994 with the new proposal. Here is a graphical comparison.
Comparison between test data & previous answer
Comparison between test data & new proposal
The solution is to use:
degree = 1
dimension_xi_X = 3
dimension_xi_Y = 450
enumerateFunction = ot.LinearEnumerateFunction(dimension_xi_X)
basis = ot.OrthogonalProductPolynomialFactory(
[ot.HermiteFactory()] * dimension_xi_X, enumerateFunction)
basisSize =450 #enumerateFunction.getStrataCumulatedCardinal(degree)
adaptive = ot.FixedStrategy(basis, basisSize)
projection = ot.LeastSquaresStrategy(
ot.LeastSquaresMetaModelSelectionFactory(ot.LARS(), ot.CorrectedLeaveOneOut()))
ot.ResourceMap.SetAsScalar("LeastSquaresMetaModelSelection-ErrorThreshold", 1.0e-7)
algo_chaos = ot.FunctionalChaosAlgorithm(sample_X,
outputSampleChaos,basis.getMeasure(), adaptive, projection)
algo_chaos.run()
result_chaos = algo_chaos.getResult()
meta_model = result_chaos.getMetaModel()
metaModel1 = ot.PointToFieldConnection(postProcessing,
algo_chaos.getResult().getMetaModel())
I have plotted two graphs using plotly dash. But when the y-axis / x-axis tick size is more it gets cut off.
Y-axis :
Code :
data = [go.Scatter(x = df[df['S2PName-Category']==category]['S2BillDate'],
y = df[df['S2PName-Category']==category]['totSale'],
mode = 'markers+lines',
name = category) for category in df['S2PName-Category'].unique()]
layout = go.Layout(title='Category Trend',
xaxis = dict(title = 'Time Frame', tickformat = '%d-%b-%y'),
yaxis = dict(tickprefix= '₹', tickformat=',.2f',type='log'),
hovermode = 'closest',
plot_bgcolor = colors['background'],
paper_bgcolor = colors['background'],
font = dict(color = colors['text'])
)
X-Axis :
Code :
data = [go.Scatter(x = df[df['S2PName']==item]['S2BillDate'],
y = df[df['S2PName']==item]['totSale'],
mode = 'markers+lines',
name = item) for item in items]
layout = go.Layout(title='Category Trend',
xaxis = dict(title = 'Time Frame' , tickformat = '%d-%b'),
yaxis = dict(tickprefix= '₹', tickformat=',.2f',type='log',autorange = True),
hovermode = 'closest',
plot_bgcolor = colors['background'],
paper_bgcolor = colors['background'],
font = dict(color = colors['text'])
)
In the above 2 graphs , as the length of the tick value increases, it gets cut off . Is there a better way to handle this ?
Credit for #Flavia Giammarino in comments for the reference to the docs. I'm posting the answer for completeness.
https://plotly.com/python/setting-graph-size/
From that link the example below shows how to set margin:
fig.update_layout(
margin=dict(l=20, r=20, t=20, b=20),
)
Where l r t b correspond to left, right, top, bottom.
I had a similar problem with some Dash/Plotly charts and long y axis labels being truncated or hidden. There didn't seem to be much information or documentation on this issue, so it took a while to solve.
Solution: add this code to the layout settings to prevent truncation of the y axes labels:
fig.update_layout(
yaxis=dict(
automargin=True
)
)
or you can update the yaxes setting specifically:
fig.update_yaxes(automargin=True)
Update: I tried another version of Plotly (5.10 or above) which mentions setting the automargin setting to any combination of automargin=['left+top+right+bottom'] with similar results. This still seems a bit unstable and doesn't solve all possible scenarios or corner cases, but works fine in most cases, especially when the browser window is maximized.