I'm trying to fit a mixed effect model with a constrained parameter, and am struggling to make it work. Adding a small bit of complexity, is that one of the terms should be a polynomial.
Essentially what I'm looking for is something like the following, where var 1 is fixed at a certain value.
mod1 <- lmer(outcome ~ var1 + poly(var2,2) + (1 | Study), df)
It seems like it can be done using lmer with the Nelder-Mead option, but I can quite wrap my head around how to make it work.
I've also tried using the lavaan package, but I've never used it before and am getting hung up somewhere. Here is an example...
library(lavaan)
reprex_df <- structure(list(outcome = c(0.54, 5.06, 15.35, 5.4, 5.3, 1.57,
2.11, 2.71, 9.09, 7.96, 28.8, 4.4, 3.38, 15.43, 4.05), var1 = c(0.55,
3.42, 2.24, 2.24, 3.44, 1.82, 1.82, 2.23, 5.41, 2.61, 6.94, 3.98,
2.23, 5.29, 3.28), var2 = c(111, 235, 60, 197, 369, 342.78, 240.99,
406.5, 264, 263.8, 76, 679, 338, 116, 683), study = c("Study 1",
"Study 2", "Study 2", "Study 2", "Study 3",
"Study 4", "Study 4", "Study 6", "Study 5",
"Study 7", "Study 2", "Study 7", "Study 6",
"Study 5", "Study 2")), row.names = c(NA, -15L), class = c("tbl_df", "tbl", "data.frame"))
I think I can make a basic model (without the polynomial)
reprex_df
test.model <- ' outcome ~ var1 + var2 + study'
test.model <- sem(test.model,
data = reprex_df, cluster = "study")
coef(test.model)
But when I try and constrain var1 to a specific value I'm getting an error
test.model.constr <- ' outcome ~ var1 + var2 + study
var1 == 4.87
'
test.model.constr <- sem(test.model.constr,
data = reprex_df, cluster = "study")
Any help in constraining the parameter (using either lmer or lavaan) and/or adding a polynomial term in lavaan would be very much appreciated.
It's tricky because lmer "profiles out" the fixed-effect parameters, i.e. they're not explicitly fitted as part of the nonlinear optimization step.
Assuming var1 is numeric/continuous and we want to set a coefficient of b, how about
mod1 <- lmer(outcome ~ 1 + offset(b*var1) + poly(var2,2) + (1 | Study), df)
? This adds the term b*var1 directly to the model.
The glmmTMB package has a map argument that allows the user to fix any of the parameters explicitly to a particular value (or to constrain several parameters to have a common value).
Related
I have a heirarchy of tables in a MySQL 5.6 database that I need to query to a JSON format for use by a javascript tree structure.
Just as a test in my flask I did the following for just the top level
def get_all_customers():
response_object = {'status': 'success'}
cnx = mysql.connector.connect(user="", password="", database="", host="localhost", port=3306)
cursor = cnx.cursor()
cursor.execute('SELECT idx, name FROM listcustomers ORDER BY name')
data = []
for idx, name in cursor:
data.append({'id': idx, 'label':name, 'otherProp': "Customer"})
response_object['customers'] = data
return jsonify(response_object)
which returns
[
{ id: 1,
label: "customer 1",
otherProp: "Customer"
},
...
]
But each customer has locations, and each location has areas, and each area has assets, and each asset has projects, and I need to also query them into children of this json object. So, for example, just going one level deeper to locations, I would need something like this -
[
{ id: 1,
label: "customer 1",
otherProp: "Customer",
children: [
{
id: 5,
label: "location 5",
otherProp: "Location"
},
...
]
},
...
]
where in my database listlocatiosn who links to listcustomers via the it's parentCustomerId column. How can I manage this? Eventually this tree will have about 13,000 objects so I know just querying the data and then parsing it with python would be far more inefficient than if I am able to query properly to begin with.
VBA Noob here. Please excuse any gaps in terminology etc.
I am trying to parse a JSON file into a spreadsheet using VBA-JSON v2.2.3 (c) Tim Hall - https://github.com/VBA-tools/VBA-JSON.
The JSON file looks like this:
{
"site": "{5BEC7C29-FF95-4ECC-9314-064B52618EEE}",
"from": "2017-01-16",
"to": "2017-01-22",
"timeSheet": [
{
"date": "2017-01-16",
"person": "{E2A5FDE1-33F8-43CA-A01D-5DD4A3A5E23A}",
"personName": "James Smith",
"company": "{B03CF7B3-0BE9-44B4-8E55-47782FDD87C0}",
"companyName": "Acme Company Ltd",
"minutes": "510",
"activities": [
{
"name": "Training",
"code": "TR",
"minutes": "240"
},
{
"name": "Administration",
"code": "AD",
"minutes": "150"
},
{
"name": "Payroll",
"code": "PR",
"minutes": "60"
},
{
"name": "Meal break",
"code": "",
"minutes": "60"
}
]
}
]
}
There may be any number of 'timeSheet' records, as well as any number of 'Activities' within each timeSheet including zero.
I want a row in the spreadsheet for each activity, with the name and other data outputted next to that days activities. Essentially showing a log of all the activities done, for how long and by who. To complicate issues, I still need the name etc outputting even if no activities are recorded. I will then fill with 'unallocated time' or something similar.
Below is as far as I have got (abridged), with an updated count of the activities occurring every loop. This feels a little hacky and doesn't give me what I am looking for, often adding additional rows and sometimes missing activities entirely.
i = 2
j = 1
activCount = CStr(JSON("timeSheet")(1)("activities").Count)
If activCount = 0 Then activCount = 1
ws.Cells(i, 1) = JSON("site")
ws.Cells(i, 2) = JSON("from")
ws.Cells(i, 3) = JSON("to")
For Each item In JSON("timeSheet")
For j = 1 To activCount
On Error Resume Next
ws.Cells(i, 4) = item("date")
ws.Cells(i, 5) = item("personName")
ws.Cells(i, 6) = item("companyName")
ws.Cells(i, 7) = item("minutes")
ws.Cells(i, 9) = item("activities")(j)("name")
ws.Cells(i, 10) = item("activities")(j)("code")
ws.Cells(i, 11) = item("activities")(j)("minutes")
activCount = CStr(JSON("timeSheet")(i)("activities").Count)
If activCount = 0 Then activCount = 1
i = i + 1
Next
Next
Can someone help? I have run out of ideas and have been working it for some time! Thank you. :)
This worked fine for me:
Sub TestJson2()
Dim ts, act
Dim Json As Object, c As Range
'reading json from a worksheet cell...
Set Json = JsonConverter.ParseJson(Range("A3").Value)
Set c = ActiveSheet.Range("C5")
'loop over timesheets
For Each ts In Json("timeSheet")
'loop over timesheet activities
For Each act In ts("activities")
c.Resize(1, 11).Value = Array(Json("site"), Json("from"), Json("to"), _
ts("date"), ts("personName"), ts("companyName"), _
ts("minutes"), act("name"), act("code"), _
act("minutes"))
Set c = c.Offset(1, 0)
Next act
Next ts
End Sub
I'm trying to parse a json file with OpenStruct. Json file has an array for Skills. When I parse it I get some extra "garbage" returned. How do I get rid of it?
json
{
"Job": "My Job 1",
"Skills": [{ "Name": "Name 1", "ClusterName": "Cluster Name 1 Skills"},{ "Name": "Name 2", "ClusterName": "Cluster Name 2 Skills"}]
}
require 'ostruct'
require 'json'
json = File.read('1.json')
job = JSON.parse(json, object_class: OpenStruct)
puts job.Skills
#<OpenStruct Name="Name 1", ClusterName="Cluster Name 1 Skills">
#<OpenStruct Name="Name 2", ClusterName="Cluster Name 2 Skills">
If by garbage, you mean #<OpenStruct and ">, it is just the way Ruby represents objects when called with puts. It is useful for development and debugging, and it makes it easier to understand the difference between a String, an Array, an Hash and an OpenStruct.
If you just want to display the name and cluster name, and nothing else :
puts job.Job
job.Skills.each do |skill|
puts skill.Name
puts skill.ClusterName
end
It returns :
My Job 1
Name 1
Cluster Name 1 Skills
Name 2
Cluster Name 2 Skills
EDIT:
When you use job = JSON.parse(json, object_class: OpenStruct), your job variable becomes an OpenStruct Ruby object, which has been created from a json file.
It doesn't have anything to do with json though: it is not a json object anymore, so you cannot just write it back to a .json file and expect it to have the correct syntax.
OpenStruct doesn't seem to work well with to_json, so it might be better to remove object_class: OpenStruct, and just work with hashes and arrays.
This code reads 1.json, convert it to a Ruby object, adds a skill, modifies the job name, writes the object to 2.json, and reads it again as JSON to check that everything worked fine.
require 'json'
json = File.read('1.json')
job = JSON.parse(json)
job["Skills"] << {"Name" => "Name 3", "ClusterName" => "Cluster Name 3 Skills"}
job["Job"] += " (modified version)"
# job[:Fa] = 'blah'
File.open('2.json', 'w'){|out|
out.puts job.to_json
}
require 'pp'
pp JSON.parse(File.read('2.json'))
# {"Job"=>"My Job 1 (modified version)",
# "Skills"=>
# [{"Name"=>"Name 1", "ClusterName"=>"Cluster Name 1 Skills"},
# {"Name"=>"Name 2", "ClusterName"=>"Cluster Name 2 Skills"},
# {"Name"=>"Name 3", "ClusterName"=>"Cluster Name 3 Skills"}]}
I have imported a json file like this one:
library(rjson)
json_str <- '[{"id": 1, "code": 7909, "text": [{"col1": "a", "col2": "some text"}], "date": "2015-12-01"}, {"id": 2, "code": 7651, "text": [], "date": "2015-12-01"}, {"id": 3, "code": 4768, "text": [{"col1": "aaa", "col2": "Blah, blah"}, {"col1": "bbb", "col2": "Blah, blah, blah"}], "date": "2015-12-01"}]'
my.list <- fromJSON(json_str)
str(my.list)
Needless to say the real file is much longer.
As a result I get a nested list of 3 elements where each element is a list of 4, and then, the element $text is a list of variable length from nothing to any number of elements, in my case, usually no more than 3.
After some research I have found several answers about converting a list to data.frame, for example here and here. However, none of them work when one or more of the nested lists in '$text` is empty.
do.call(rbind, lapply(my.list, data.frame, stringsAsFactors=FALSE))
library(data.table)
rbindlist(my.list, fill=TRUE)
Both return an error.
I would like to either convert the list in $text to several columns of the data.frame or just one (pasting the content).
Another option would be to be able to skip some elements (say $text) and convert the rest of the list, then in a separate line convert those elements (say $text) to a different data.frame. I think I could somehow relate one data.frame to the other.
Can anyone give me any idea on how to do this.
Thanks
By the sounds of it, something like the following should work:
do.call(rbind.data.frame, lapply(my.list, function(x) {
x[["text"]] <- toString(unlist(x[["text"]]))
x
}))
## id code text date
## 2 1 7909 a, some text 2015-12-01
## 21 2 7651 2015-12-01
## 3 3 4768 aaa, Blah, blah, bbb, Blah, blah, blah 2015-12-01
This follows your idea of pasting the values together (here using toString) to form a single column in the data.frame.
I had asked the same question after editing 2 times of a previous question I had posted. I am sorry for the bad usage of this website. I have flagged it for deletion and I am posting a proper new question on the same here. Please look into this.
I am basically working on a recommender system code. The output has to be converted to sequence of JSON objects. I have a matrix that has a look up table for every item ID, with the list of the closest items it is related to and the the similarity scores associated with their combinations.
Let me explain through a example.
Suppose I have a matrix
In the below example, Item 1 is similar to Items 22 and 23 with similarity scores 0.8 and 0.5 respectively. And the remaining rows follow the same structure.
X1 X2 X3 X4 X5
1 22 23 0.8 0.5
34 4 87 0.4 0.4
23 7 92 0.6 0.5
I want a JSON structure for every item (every X1 for every row) along with the recommended items and the similarity scores for each combination as a separate JSON entity and this being done in sequence. I don't want an entire JSON object containing these individual ones.
Assume there is one more entity called "coid" that will be given as input to the code. I assume it is XYZ and it is same for all the rows.
{ "_id" : { "coid" : "XYZ", "iid" : "1"}, "items" : [ { "item" : "22", "score" : 0.8},{ "item": "23", "score" : 0.5}] }
{ "_id" : { "coid" : "XYZ", "iid" : "34"},"items" : [ { "item" : "4", "score" : 0.4},{ "item": "87", "score" : 0.4}] }
{ "_id" : { "coid" : "XYZ", "iid" : "23"},"items" : [ { "item" : "7", "score" : 0.6},{ "item": "92", "score" : 0.5}] }
As in the above, each entity is a valid JSON structure/object but they are not put together into a separate JSON object as a whole.
I appreciate all the help done for the previous question but somehow I feel this new alteration I have here is not related to them because in the end, if you do a toJSON(some entity), then it converts the entire thing to one JSON object. I don't want that.
I want individual ones like these to be written to a file.
I am very sorry for my ignorance and inconvenience. Please help.
Thanks.
library(rjson)
## Your matrix
mat <- matrix(c(1,34,23,
22, 4, 7,
23,87,92,
0.8, 0.4, 0.6,
0.5, 0.4, 0.5), byrow=FALSE, nrow=3)
I use a function (not very interesting name makejson) that takes a row of the matrix and returns a JSON object. It makes two list objects, _id and items, and combines them to a JSON object
makejson <- function(x, coid="ABC") {
`_id` <- list(coid = coid, iid=x[1])
nitem <- (length(x) - 1) / 2 # Number of items
items <- list()
for(i in seq(1, nitem)) {
items[[i]] <- list(item = x[i + 1], score = x[i + 1 + nitem])
}
toJSON(list(`_id`=`_id`, items=items))
}
Then using apply (or a for loop) I use the function for each row of the matrix.
res <- apply(mat, 1, makejson, coid="XYZ")
cat(res, sep = "\n")
## {"_id":{"coid":"XYZ","iid":1},"items":[{"item":22,"score":0.8},{"item":23,"score":0.5}]}
## {"_id":{"coid":"XYZ","iid":34},"items":[{"item":4,"score":0.4},{"item":87,"score":0.4}]}
## {"_id":{"coid":"XYZ","iid":23},"items":[{"item":7,"score":0.6},{"item":92,"score":0.5}]}
The result can be saved to a file with cat by specifying the file argument.
## cat(res, sep="\n", file="out.json")
There is a small difference in your output and mine, the numbers are in quotes ("). If you want to have it like that, mat has to be character.
## mat <- matrix(as.character(c(1,34,23, ...
Hope it helps,
alex