I can't seem to find out if this is implemented in mysql? I can only find information relating to postgresql.
So, can you use JSONB in mysql or is it just JSON?
The main difference between the json and jsonb types in Postgres is that the latter is stored in a compressed binary format. From the MySQL documentation, it appears that MySQL's JSON type already has at least some of the behavior of Postgres' jsonb:
The JSON data type provides these advantages over storing JSON-format strings in a string column:
Optimized storage format. JSON documents stored in JSON columns are converted to an internal format that permits quick read access to document elements. When the server later must read a JSON value stored in this binary format, the value need not be parsed from a text representation. The binary format is structured to enable the server to look up subobjects or nested values directly by key or array index without reading all values before or after them in the document.
If I recall correctly, the MySQL JSON functions will still work correctly on JSON text (e.g. stored as varchar), so maybe MySQL's analogy to Postgres' json would just be storing JSON content as plain text.
Related
I have some json data which i am getting from a particular API. i am using postgresql as a db. What the best way to store the json data? Using row column format or saving the complete json data in a single field of jsonb type
From my experience with Django using postgresql: I am used to store the raw json in a single field.
Next, I parse it according to my needs.
I tried to add a json column to my database by using phpMyAdmin
but Unfortunately, phpMyAdmin converts the json column to Longtext type
So, I'm asking about the ability to use the JSON Where Clauses with this type
https://laravel.com/docs/5.7/queries#json-where-clauses
You cannot use those queries on non-JSON data types in MariaDB. And as of 10.2, it doesn't officially support it.
You can use the JSON helper functions to query against data (ie: where JSON_CONTAINS(...) and others.
You can also create columns that are extracted values from the JSON data using Virtual Columns
Here's a good post with much more detail.
I have a lot of data (JSON string) per day (around 150-200B).
I want to insert the JSON to Hadoop, what is the best way to do it (I need a fast insert and a fast query on JSON fields)?
Do I need to use hive and create Avro scheme to my JSON? Or do I need to insert the JSON as a string to a specific column?
If you want to make the data available in Hive to perform mostly aggregations on top of it, I would suggest 1 of the following method using spark.
If you have multiple-line json files
var df = spark.read.json(sc.wholeTextFiles("hdfs://ypur/hdfs/path/*.json").values)
df.write.format("parquet").mode("overwrite").saveAsTable("yourhivedb.tablename")
If you have single-line json files
val df = spark.read.json("hdfs://ypur/hdfs/path/*.json")
df.write.format("parquet").mode("overwrite").saveAsTable("yourhivedb.tablename")
Spark will automatically infer the table schema for you. If you are using cloudera distribution you will be able to read the data using impala (depending on your cloudera version it may not support complex structures)
I want to insert the JSON to Hadoop
You just put it in HDFS... Since you have data over a time period, you'll want to create partitions for Hive to read
jsondata/dt=20180619/foo.json
jsondata/dt=20180620/bar.json
Do I need to use hive and create Avro scheme to my JSON?
Nope. Not sure where you got mixed up between Avro and JSON. Now, if you could convert the JSON into defined Avro with a schema, then that would help improve Hive queries since querying structured binary is better than parsing JSON text.
do I need to insert the JSON as a string to a specific column?
Not recommended. You could, but then you cannot query it, via Hive's JSON Serde support
Don't forget with the above structure you'll need PARTITIONED BY (dt STRING). And in order for partitions to be created on the table for existing files, you'll need to manually (and daily) run an MSCK REPAIR TABLE command
i have JSON as string (from kafka)
Don't use Spark for that (at least, don't reinvent the wheel). My suggestion would be to use Confluent's HDFS Kafka Connect that comes with Hive table creation support.
I am new to PostgreSQL database(9.5.0), I need to store my json data in PostgreSQL database, so that I need to create a table for it, how can I create a table for it to store my json object on clicking of submit button from my front-end, so that it will be stored in created database table and my sample josn object is as follows(having key/value pairs, contains files also), Please help me.
{"key1":"value1","key2":"value2","key3":"value3","key4_file_name":"Test.txt"}
PostgreSQL has the json and jsonb (b for binary) data types. You can use these in your table definitions just like any other data type. If you plan to merely store the json data then the json data type is best. If, on the other hand, you plan to do a lot of analysis with the json data inside the PG server, then the jsonb data type is best.
My question is how to store a nested JSON as key value in mysql table. Earlier I thought of storing it as CSV but on deep diving I found it like it would be difficult to query those values and to manage as well.
Please help me in giving alternate solutions of how and where to store JSON.
The version of MySQL I am using does not support the JSON data type
If you are using MySQL 5.7, you can store ans query JSON objects. Look at this link for more details and examples.
Since MySQL 5.7 a native JSON data type is supported.
See also https://dev.mysql.com/doc/refman/5.7/en/json.html
If your MySQL version is lower the only option left is to store it as text. Otherwise you will not be able to store it in a simple key value pair manner.