Assume a datascript db has a schema:
(def schema {:maker/cars {:db/cardinality :db.cardinality/many
:db/valueType :db.type/ref}})
(def conn (d/create-conn schema)
And has inserted some entities:
{:db/id 1
:car/name "x"}
{:db/id 2
:car/name "y"}
{:db/id 3
:car/name "z"}
{:db/id 4
:maker/name "Honda"
:maker/cars [1 2 3]}
Then the query
(d/q '[:find [(pull ?e [:maker/name {:maker/cars [:car/name]}])]
:where [?e :maker/name "Honda"]] #conn)
will get the result:
[#:maker{:name "Honda",
:cars [#:car{:name "x"} #:car{:name "y"} #:car{:name "z"}]}]
The :maker/cars is a vector.
How to write the pull query to get the result like below?
[#:maker{:name "Honda",
:cars {1 #:car{:name "x"}
2 #:car{:name "y"}
3 #:car{:name "z"}}]
Tried but got no lucky...
There’s no way. There aren’t many options to control the shape of data Pull API returns. Just do your own post-processing on a result that DataScript returns
Related
I have a table like this (with an jsonb column):
https://dbfiddle.uk/lGuHdHEJ
If I load this json with python into a dataframe:
import pandas as pd
import json
data={
"id": [1, 2],
"myyear": [2016, 2017],
"value": [5, 9]
}
data=json.dumps(data)
df=pd.read_json(data)
print (df)
I get this result:
id myyear value
0 1 2016 5
1 2 2017 9
How can a get this result directly from the json column via sql in a postgres view?
Note: This assumes that your id, my_year, and value array are consistent and have the same length.
This answer uses PostgresSQL's json_array_elements_text function to explode array elements to the rows.
select jsonb_array_elements_text(payload -> 'id') as "id",
jsonb_array_elements_text(payload -> 'bv_year') as "myyear",
jsonb_array_elements_text(payload -> 'value') as "value"
from main
And this gives the below output,
id myyear value
1 2016 5
2 2017 9
Although this is not the best design to store the properties in jsonb object and could lead to data inconsistencies later. If it's in your control I would suggest storing the data where each property's mapping is clear. Some suggestions,
You can instead have separate columns for each property.
If you want to store it as jsonb only then consider [{id: "", "year": "", "value": ""}]
This is my data:
>>> print(MyModel.objects.get(id=1).Fruits) #Fruits is JSONField
>>> print(favorites)
{"Title": ["Fruits"], "Name": ["Banana", "Cherry", "Apple", "Peach"], "Other":["Banana"]}
I define a query as follows:
>>> query = reduce(operator.or_, (Q(Fruits__Name__icontains=x) for x in favorites))
#or: query = reduce(operator.or_, (Q(Fruits__icontains={'Name':x}) for x in favorites))
>>> print(query)
(OR: ('Fruits__Name__icontains', 'Apple'), ('Fruits__Name__icontains', 'Banana'))
I want it return that item banana or an apple are in Name.
When I run this query (postgresql):
MyModel.objects.filter(query)
it doesn't match any item in the database.
With MyModel.Fruits as your JSONField, depending on the database used, it might be stored/accessed as string. You could try just accessing icontains directly on the JSON string value:
Fruits__icontains=x
So full query would be:
query = reduce(operator.or_, (Q(Fruits__icontains=x) for x in favorites))
I am using BehaviorSpace to run the model hundreds of times with different parameters. But I need to know the locations of all turtles as a result instead of only the number of turtles. How can I achieve it with BehaviorSpace?
Currently, I output the results in a csv file by this code:
to-report get-locations
report (list xcor ycor)
end
to generate-output
file-open "model_r_1.0_locations.csv"
file-print csv:to-row get-locations
file-close
end
but all results are popped into same csv file, so I can't tell the condition of each running.
Seth's suggestion of incorporating behaviorspace-run-number in the filename of your csv output is one alternative. It would allow you to associate that file with the summary data in your main BehaviorSpace output file.
Another option is to include list reporters as "measures" in your behavior space experiment definition. For example, in your case:
map [ t -> [ xcor ] of t ] sort turtles
map [ t -> [ ycor ] of t ] sort turtles
You can then parse the resulting list "manually" in your favourite data analysis language. I've used the following function for this before, in Julia:
parselist(strlist, T = Float64) = parse.(T, split(strlist[2:end-1]))
I'm sure you can easily write some equivalent code in Python or R or whatever language you're using.
In the example above, I've outputted separate lists for the xcor and the ycor of turtles. You could also output a single "list of lists", but the parsing would be trickier.
Edit: How to do this using the csv extension and R
Coincidentally, I had to do something similar today for a different project, and I realized that a combination of the csv extension and R can make this very easy.
The general idea is the following:
In NetLogo, use csv:to-string to encode list data into a string and then write that string directly in the BehaviorSpace output.
In R, use purrr::map and readr::read_csv, followed by tidyr::unnest, to unpack everything in a neat "one observation per row" dataframe.
In other words: we like CSV, so we put CSV in our CSV so we can parse while we parse.
Here is a full-fledged example. Let's say we have the following NetLogo model:
extensions [ csv ]
to setup
clear-all
create-turtles 2 [ move-to one-of patches ]
reset-ticks
end
to go
ask turtles [ forward 1 ]
tick
end
to-report positions
let coords [ (list who xcor ycor) ] of turtles
report csv:to-string fput ["who" "x" "y"] coords
end
We then define the following tiny BehaviorSpace experiment, with only two repetitions and a time limit of two, using our positions reporter as an output:
The R code to process this is pleasantly straightforward:
library(tidyverse)
df <- read_csv("experiment-table.csv", skip = 6) %>%
mutate(positions = map(positions, read_csv)) %>%
unnest()
Which results in the following dataframe, all neat and tidy:
> df
# A tibble: 12 x 5
`[run number]` `[step]` who x y
<int> <int> <int> <dbl> <dbl>
1 1 0 0 16 10
2 1 0 1 10 -2
3 1 1 1 9.03 -2.24
4 1 1 0 -16.0 10.1
5 1 2 1 8.06 -2.48
6 1 2 0 -15.0 10.3
7 2 0 1 -14 1
8 2 0 0 13 15
9 2 1 0 14.0 15.1
10 2 1 1 -13.7 0.0489
11 2 2 0 15.0 15.1
12 2 2 1 -13.4 -0.902
The same thing in Julia:
using CSV, DataFrames
df = CSV.read("experiment-table.csv", header = 7)
cols = filter(col -> col != :positions, names(df))
df = by(df -> CSV.read(IOBuffer(df[:positions][1])), df, cols)
I want to practice building models and I figured that I'd do it with something that I am familiar with: League of Legends. I'm having trouble replacing an integer in a dataframe with a value in a json.
The datasets I'm using come off of the kaggle. You can grab it and run it for yourself.
https://www.kaggle.com/datasnaek/league-of-legends
I have json file of the form: (it's actually must bigger, but I shortened it)
{
"type": "champion",
"version": "7.17.2",
"data": {
"1": {
"title": "the Dark Child",
"id": 1,
"key": "Annie",
"name": "Annie"
},
"2": {
"title": "the Berserker",
"id": 2,
"key": "Olaf",
"name": "Olaf"
}
}
}
and dataframe of the form
print df
gameDuration t1_champ1id
0 1949 1
1 1851 2
2 1493 1
3 1758 1
4 2094 2
I want to replace the ID in t1_champ1id with the lookup value in the json.
If both of these were dataframe, then I could use the merge option.
This is what I've tried. I don't know if this is the best way to read in the json file.
import pandas
df = pandas.read_csv("lol_file.csv",header=0)
champ = pandas.read_json("champion_info.json", typ='series')
for i in champ.data[0]:
for j in df:
if df.loc[j,('t1_champ1id')] == i:
df.loc[j,('t1_champ1id')] = champ[0][i]['name']
I get the below error:
the label [gameDuration] is not in the [index]'
I'm not sure that this is the most efficient way to do this, but I'm not sure how to do it at all either.
What do y'all think?
Thanks!
for j in df: iterates over the column names in df, which is unnecessary, since you're only looking to match against the column 't1_champ1id'. A better use of pandas functionality is to condense the id:name pairs from your JSON file into a dictionary, and then map it to df['t1_champ1id'].
player_names = {v['id']:v['name'] for v in json_file['data'].itervalues()}
df.loc[:, 't1_champ1id'] = df['t1_champ1id'].map(player_names)
# gameDuration t1_champ1id
# 0 1949 Annie
# 1 1851 Olaf
# 2 1493 Annie
# 3 1758 Annie
# 4 2094 Olaf
Created a dataframe from the 'data' in the json file (also transposed the resulting dataframe and then set the index to what you want to map, the id) then mapped that to the original df.
import json
with open('champion_info.json') as data_file:
champ_json = json.load(data_file)
champs = pd.DataFrame(champ_json['data']).T
champs.set_index('id',inplace=True)
df['champ_name'] = df.t1_champ1id.map(champs['name'])
I am facing problem how to check whether the list returned by the procedure consist of a single list or may have sub list inside.
#simple list
set a { 1 2 3 4}
# list consisting of sub list
set a { {1 2 3 4} {5 6 7 7} }
As above some times the variable a will have a list and sometime proc will return list consisting of sub list.
Update part
set a [mysqlsel $db "SELECT * FROM abc" -list]
I do not know weather query will return a single list or list consisting of sublist
You should really rethink your approach: since Tcl is typeless, you can't really tell if {{1 2 3 4} {5 6 7 8}} is a list of two lists or a list of two strings or a literal string {1 2 3 4} {5 6 7 8}, because all these propositions are true depending on how you make Tcl interpret this value.
Another thing, is that even if you were to try something like catch {lindex $element 0} or string is list $element on each top-level element to see if it can be interpreted as a list, that would qualify as being non-lists only strings that really can't be parsed as lists, like aaa { bbb. And string foo is also a proper list (of length 1, containing "foo" as its sole element).
One approach you can consider using is wrapping the returned value in another value which has some sort of "tag" attached to it--the trick routinely used in some other typeless languages like LISP and Erlang. That would look like this:
If you need to return 1 2 3 4, return {flat {1 2 3 4}} instead.
If you need to return {1 2 3 4} {5 6 7 8}, return {nested {{1 2 3 4} {1 2 3 4 5}}}.
Then in the client code do switch on the "tag" element and decapsulate the payload:
lassign [your_procedure ...] tag payload
switch -- $tag {
flat {
# do something with $payload
}
nested {
# do something with $payload
}
}