I have a MySQL database with a table containing a JSON field called things. The JSON looks like this
things = {"value1": "phil", "value2": "jill"}
I have collection of objects that I have pulled from the database via
my_things = Name_table.objects.values
Now, I'd like to filter the my_things collection by one of the JSON fields. I've tried this
my_things =
my_things.filter(things__contains={'value': 'phil'})
which returned an empty collection. I've also tried
my_things = my_things.filter(things={'value': 'phil'})
and
my_things = my_things.filter(things__exact={'value': 'phil'})
I'n using Django 1.10 and MySQL 5.7
Thoughts?
It depends on how exactly do you store JSON in field. If you use django-jsonfield, then your things will be string without spaces, with strings inside of quotation marks: '{"value1":"phil","value2":"jill"}'.
Then, via docs:
my_things = my_things.filter(things__contains='"value1":"phil"')
should return your filtered QuerySet, because
>>> tmp_str = '{"value1":"phil","value2":"jill"}'
>>> '"value1":"phil"' in tmp_str
True
Related
How can I select array "1" inside the "flavor" object from json code in mysql
Attribute name: settings
{"without":{"usd":{"new":"5","old":"8"},"weight":"5"},"color":{"2","3"},"flavor":{"1","2"}}
And how can I get a number inside the "usd" object inside "new" knowing these objects are inside the first object and they are variable, perhaps ["without" or "long" or ......]
Attribute name: settings
{"without":{"usd":{"new":"5","old":"8"},"weight":"5"},"color":{"2","3"},"flavor":{["1","2"}}
{"long":{"usd":{"new":"2","old":"3"},"weight":"2"},"medium":{"usd":{"new":"3","old":"4"},"weight":"3"},"short":{"usd":{"new":"4","old":"5"},"weight":"4"}}
{"short":{"usd":{"new":"4","old":"5"},"weight":"2"},"color":{"1","2"}}
LIKE
without = 5
long = 2
short = 4
I rebuilt the data format so that I could extract the required data
{"size":[{"id":1,"url":"without","weight":"5","price":{"usd":{"new":"5","old":"8"}}}],"color":[{"id":"2","url":"yellow"},{"id":"3","url":"green"}],"flavor":[{"id":"1","url":"berry"},{"id":"2","url":"strawberry"}]}
MYSQL
JSON_EXTRACT(details.settings, '$.color[*].url') LIKE '%yellow%'
I'm very new in Hadoop,
I'm using Spark with Java.
I have dynamic JSON, exmaple:
{
"sourceCode":"1234",
"uuid":"df123-....",
"title":"my title"
}{
"myMetaDataEvent": {
"date":"10/10/2010",
},
"myDataEvent": {
"field1": {
"field1Format":"fieldFormat",
"type":"Text",
"value":"field text"
}
}
}
Sometimes I can see only field1 and sometimes I can see field1...field50
And maybe the user can add fields/remove fields from this JSON.
I want to insert this dynamic JSON to hadoop (to hive table) from Spark Java code,
How can I do it?
I want that the user can after make HIVE query, i.e: select * from MyTable where type="Text
I have around 100B JSON records per day that I need to insert to Hadoop,
So what is the recommanded way to do that?
*I'm looked on the following: SO Question but this is known JSON scheme where it isnt my case.
Thanks
I had encountered kind of similar problem, I was able to resolve my problem using this. ( So this might help if you create the schema before you parse the json ).
For a field having a string data type you could create the schema :-
StructField field = DataTypes.createStructField(<name of the field>, DataTypes.StringType, true);
For a field having a int data type you could create the schema :-
StructField field = DataTypes.createStructField(<name of the field>, DataTypes.IntegerType, true);
After you have added all the fields in a List<StructField>,
Eg:-
List<StructField> innerField = new ArrayList<StructField>();
.... Field adding logic ....
Eg:-
innerField.add(field1);
innerField.add(field2);
// One instance can come, or multiple instance of value comes in an array, then it needs to be put in Array Type.
ArrayType getArrayInnerType = DataTypes.createArrayType(DataTypes.createStructType(innerField));
StructField getArrayField = DataTypes.createStructField(<name of field>, getArrayInnerType,true);
You can then create the schema :-
StructType structuredSchema = DataTypes.createStructType(getArrayField);
Then I read the json using the schema generated using the Dataset API.
Dataset<Row> dataRead = sqlContext.read().schema(structuredSchema).json(fileName);
I have a CSV file of data in the form
21.06.2016 23:00:00.349, 153.461, 153.427
21.06.2016 23:00:00.400, 153.460, 153.423
etc
The initial step of creating a frame involves the optional inclusion of a 'schema' to specify or rename column heads and specify types:
let df = Frame.ReadCsv(__SOURCE_DIRECTORY__ + "/data/GBPJPY.csv", hasHeaders=true, inferTypes=false, schema="TS (DateTimeOffset), Bid (float(3)), Ask (float(3))")
I would like to specify the first column of string values to be ParseExact'ed to DateTimeOffset of the format
"dd.mm.yyyy HH:mm:ss.fff"
(I'm assuming the use of the setting System.Globalization.CultureInfo.InvariantCulture).
How do I express the schema such that it will parse the datetime string in that first Frame.ReadCsv("file.csv", schema = ........ )? Or is this not possible to accomplish within the schema statement?
i tried to fetch data from mongodb using mongoengine with flask. query is work perfect the problem is when i convert query result into json its show only fields name.
here is my code
view.py
from model import Users
result = Users.objects()
print(dumps(result))
model.py
class Users(DynamicDocument):
meta = {'collection' : 'users'}
user_name = StringField()
phone = StringField()
output
[["id", "user_name", "phone"], ["id", "user_name", "phone"]]
why its show only fields name ?
Your query returns a queryset. Use the .to_json() method to convert it.
Depending on what you need from there, you may want to use something like json.loads() to get a python dictionary.
For example:
from model import Users
# This returns <class 'mongoengine.queryset.queryset.QuerySet'>
q_set = Users.objects()
json_data = q_set.to_json()
# You might also find it useful to create python dictionaries
import json
dicts = json.loads(json_data)
I would like to generate a simple json file from an database.
I am not an expert in parsing json files using python nor NDB database engine nor GQL.
What is the right query to search the data? see https://developers.google.com/appengine/docs/python/ndb/queries
How should I write the code to generate the JSON using the same schema as the json described here below?
Many thanks for your help
Model Class definition using NDB:
# coding=UTF-8
from google.appengine.ext import ndb
import logging
class Albums(ndb.Model):
"""Models an individual Event entry with content and date."""
SingerName = ndb.StringProperty()
albumName = ndb.StringProperty()
Expected output:
{
"Madonna": ["Madonna Album", "Like a Virgin", "True Blue", "Like a Prayer"],
"Lady Gaga": ["The Fame", "Born This Way"],
"Bruce Dickinson": ["Iron Maiden", "Killers", "The Number of the Beast", "Piece of Mind"]
}
For consistency, model names should by singular (Album not Albums), and property names should be lowercase_with_underscores:
class Album(ndb.Model):
singer_name = ndb.StringProperty()
album_name = ndb.StringProperty()
To generate the JSON as described in your question:
1) Query the Album entities from the datastore:
albums = Album.query().fetch(100)
2) Iterate over them to form a python data structure:
albums_dict = {}
for album in albums:
if not album.singer_name in albums_dict:
albums_dict[album.singer_name] = []
albums_dict[album.singer_name].append(album.album_name)
3) use json.dumps() method to encode to JSON.
albums_json = json.dumps(albums_dict)
Alternatively, you could use the built in to_dict() method:
albums = Album.query().fetch(100)
albums_json = json.dumps([a.to_dict() for a in albums])