Can anyone recommend a code snippet, script, tool for converting a Hive Map field to a Redshift JSON field?
I have a Hive table that has two Map fields and I need to move the data to Redshift. I can easily move the data to a string format but then lose some functionality with that. Would prefer to have Map ported to JSON to maintain key, value pairs.
Thanks.
You might want to try the to_json UDF
http://brickhouseconfessions.wordpress.com/2014/02/07/hive-and-json-made-simple/
Related
I need to import JSON data in a WP database. I found the correct table of the database but the example JSON data present in table is not in a normal JSON format.
I need to import:
{"nome": "Pippo","cognome": "Paperino"}
but the example data in the table is:
a:2:{s:4:"nome";s:5:"Pippo";s:7:"cognome";s:8:"Paperino";}
How i can convert my JSON to "WP JSON"?
The data is serialized, that's why it looks weird. You can use maybe_unserialize() in WordPress, this function will unserialize the data if it was serialized.
https://developer.wordpress.org/reference/functions/maybe_unserialize/
Some functions serialize data before saving it in wordpress, and some will also unserialize when pulling from the DB. So depending on how you save it, and how you later extract the data, you might end up with serialized data.
Sorry my english is not good.
I have problem with to display data format "geometry" from mysql database. I got the data from file extension .geojson (like json but but contains coordinate), I convert the .geojson data with cmd. and this data is formed.
for example my geojson data
and this is an example of converting geojson to database
i use laravel, to display this data what is the task of laravel or mysql (eg there is a special mysql function)? and how?
What are you looking to do with the geometry data and have you looked at the Geo JSON functions https://dev.mysql.com/doc/refman/8.0/en/spatial-geojson-functions.html such as St+GeomFromJson()??
Can I get help in creating a table on AWS Athena.
For a sample example of data :
[{"lts": 150}]
AWS Glue generate the schema as :
array (array<struct<lts:int>>)
When I try to use the created table by AWS Glue to preview the table, I had this error:
HIVE_BAD_DATA: Error parsing field value for field 0: org.openx.data.jsonserde.json.JSONObject cannot be cast to org.openx.data.jsonserde.json.JSONArray
The message error is clear, but I can't find the source of the problem!
Hive running under AWS Athena is using Hive-JSON-Serde to serialize/deserialize JSON. For some reason, they don't support just any standard JSON. They ask for one record per line, without an array. In their words:
The following example will work.
{ "key" : 10 }
{ "key" : 20 }
But this won't:
{
"key" : 20,
}
Nor this:
[{"key" : 20}]
You should create a JSON classifier to convert array into list of object instead of a single array object. Use JSON path $[*] in your classifier and then set up crawler to use it:
Edit crawler
Expand 'Description and classifiers'
Click 'Add' on the left pane to associate you classifier with crawler
After that remove previously created table and re-run the crawler. It will create a table with proper scheme but I think Athena will still be complaining when you will try to query it. However, now you can read from that table using Glue ETL job and process single record object instead of array-objects
This json - [{"lts": 150}] would work like a charm with below query:-
select n.lts from table_name
cross join UNNEST(table_name.array) as t (n)
The output would be as below:-
But I have faced a challenge with json like - [{"lts": 150},{"lts": 250},{"lts": 350}].
Even if there are 3 elements in the JSON, the query is returning only the first element. This may be because of the limitation listed by #artikas.
Definitely, we can change the json like below to make it work:-
{"lts": 150}
{"lts": 250}
{"lts": 350}
Please post if anyone is having a better solution to it.
I have a simple JSON file that I would like to insert into an HBase table.
My JSON file has the following format:
{
"word1":{
"doc_01":4,
"doc_02":7
},
"word2":{
"doc_06":1,
"doc_02":3,
"doc_12":8
}
}
The HBase table is called inverted_index, it has one column family matches.
I would like to use the keys word1,word2, etc as row keys and their values to be inserted in the column family matches.
I know that Hive supports JSON parsing and I've already tried it, but only when I know the keys in the JSON beforehand to access the records.
My problem is that I don't know what or How many words my JSON file contains, or How many matches each word will have (it can't be empty though).
My question: Is this even doable using hive only? If so, kindly provide some pointers on what hive queries/functions to use to achieve that.
If not, is there any alternative to tackle this? Thanks in advance.
I'm pretty new to spark streaming and scala. I have a Json data and some other random log data coming in from a kafka topic.I was able to filter out just the json data like this
val messages = KafkaUtils.createDirectStream[String, String, StringDecoder, StringDecoder](ssc, kafkaParams, topicsSet).map(_._2).filter (x => x.matches("^[{].*" ))
My json data looks like this.
{"time":"125573","randomcol":"abchdre","address":{"city":"somecity","zip":"123456"}}
Im trying to parse the json data and put it into hive table.
can someone please point me in the correct direction.
Thanks
There are multiple ways to do this.
create an external hive table with required columns and pointing to this data location.
When you create the table , you could use the default JSON serde and then use get_json_object hive function and load this raw data into a final table. Refer this for the function details
OR
You could try the avro serde and mention the avro schema as per your json message to create the hive table. Refer this for avro serde example
.
Hope it helps