I have one field in my data model that can be practically anything: a number, a string, a bool, or a complex object.
Can I store this in postgres as jsonb?
id | response (jsonb)
------------
1 | "hello"
2 | 3
3 | { "firstName": "bob", "lastName" : "wilson" }
4 | True
I'm currently getting this error when I try to save a numeric json value into that column:
column "response" is of type jsonb but expression is of type text
Is it only possible if I save it in an object structure, like this?
{ "value" : "hello" }
{ "value" : 3 }
{ "value" : ... }
I'd rather do it the first way if it's possible.
drop table if exists t;
create table t (j jsonb);
insert into t (j) values
(to_json(1)::jsonb),
(to_json('hello'::text)::jsonb),
(to_json(true)::jsonb)
;
select * from t;
j
---------
1
"hello"
true
Notice that the string must be cast before being converted to json.
It turns out that even though numbers, bools, and strings have valid JSON representations, the JSON standard requires that the top-level value be either an array or object.
So I shouldn't be attempting to do it this way. Instead, I should store it as an object.
Related
I have a huge json file of about 4500 lines.
I wish to extract the value of all the keys named "value".
The levels of json paths are not same.
JSON Sample :
{
k1:v1,
k2:v2,
k3:v3,
k4:{
k5:v5,
k6:{
k7:v7,
value:"value1"
}
}
k8:v8,
value:"value2"
}
There are multiple such "value" tags.
Is there a way using jq to get all the values ?
Use recursive descent.
.. | objects | if has("value") then .value else empty end
Here is a slightly shorter variation of oguz ismail's answer which uses Optional Object Identifier .foo? and Alternative operator // along with recursive descent.
.. | .value? // empty
Example output (using corrected JSON Sample)
"value2"
"value1"
Note that this shortcut won't produce the same output if the "value" key may be null as // can't distinguish between a null "value" and a null produced by the ? operator when the "value" key is missing. It that's a concern then testing for the presence of the "value" key with has is better.
Try it online!
Let's take a simple schema with two tables, one that describes an simple entity item (id, name)
id | name
------------
1 | foo
2 | bar
and another, lets call it collection, that references to an item, but inside a JSON Object in something like
{
items: [
{
id: 1,
quantity: 2
}
]
}
I'm looking for a way to eventually enrich this field (kind of like populate in Mongo) in the collection with the item element referenced, to retrieve something like
{
...
items: [
{
item: {
id: 1,
name: foo
},
quantity: 2
}
]
...
}
If you have a solution with PostgreSQL, I take it as well.
If I understood correctly, your requirement is to convert an Input JSON data into MySQL table so that you can work with JSON but leverage the power of SQL.
Mysql8 recently released JSONTABLE function. By using this function, you can store your JSON in the table directly and then query it like any other SQL query.
It should serve your immediate case, but this means that your table schema will have a JSON column instead of traditional MySQL columns. You will need to check if it serves your purpose.
This is a good tutorial for the same.
Is it possible to auto-increment inside PostgreSQL's new JSON type using just SQL (like serial) and not server code?
I can't really imagine why you'd want to, but sure.
CREATE SEQUENCE whywouldyou_jsoncol_seq;
CREATE TABLE whywouldyou (
jsoncol json not null default json_object(ARRAY['id'], ARRAY[nextval('whywouldyou_jsoncol_seq')::text]),
dummydata text;
);
ALTER SEQUENCE whywouldyou_jsoncol_seq OWNED BY whywouldyou.jsoncol;
insert into whywouldyou(dummydata) values('');
select * from whywouldyou;
jsoncol | dummydata
--------------+-----------
{"id" : "1"} |
(1 row)
Note that with this particular formulation it's the string "1" not the number 1 in the json. You might want to form the json object another way if you want to avoid that. This is just an example.
I am experimenting with the new JSON/JSONB objects in the latest(9.4) PostgreSQL. First, I will show you my test table:
CREATE TABLE "JSONtest" (
data jsonb
);
COPY "JSONtest" (data) FROM stdin;
{"id": 1, "age": 24, "male": false, "name": "Martha"}
{"id": 2, "age": 49, "male": true, "name": "Jim"}
\.
ALTER TABLE ONLY "JSONtest"
ADD CONSTRAINT "JSONtest_data_key" UNIQUE (data);
From this, I try to get the data and the type of certain columns:
SELECT "data"#>'{"male"}' FROM "JSONtest"; -- this returns true|false
SELECT jsonb_typeof("data"#>'{"male"}') FROM "JSONtest"; -- this returns 'boolean'
As read in the documentation, currently PostgreSQL can return data as json/jsonb types, if you use a single angle bracket in your query, or as text if you use double angle brackets:
SELECT '{"a":1}'::jsonb->'a' -- this will be of type jsonb
SELECT '{"a":1}'::jsonb->>'a' -- this will be of type string
But I need the actual type of the data in the JSON. What I tried was using the CAST function:
SELECT CAST('{"id":19}'::jsonb->>'a' AS integer) -- this will give back an int type
SELECT CAST('{"id":19}'::jsonb->>'a' AS json_typeof('{"id":19}'::jsonb->>'a')) -- this will obviously give an ERROR, because the type is given as a string
So my question is:
Can you cast with the target type specified as a string?
I could bypass this with a stored function, because there is only 6 options, what a json_typeof could return(object, array, boolean, string, number and null), but if there is a better way, then I would happily go for that.
Thank you all for the answers in advance!
--- edit #1 ---
Here is, what I came up with today as an experiment, but it caused an error with the following text: CASE types integer and text cannot be matched
SELECT
CASE jsonb_typeof("data"#>"query")
WHEN 'number' THEN ("data"#>>"query")::int
WHEN 'string' THEN "data"#>>"query"
WHEN 'boolean' THEN ("data"#>>"query")::boolean
END
FROM (
SELECT
'{"name" : "Jim", "dogs" : ["Barks", "Fluffy", "Spocky"], "id" : 4}'::jsonb AS "data",
'{"id"}'::text[] AS "query"
) AS "input"
Functions would cause the same trouble, because I need to specify the return type of it, which in this case cannot be determined.
json_populate_record() and json_to_record() also need type specified.
We are attempting to create a schema to load a massive JSON structure into Hive. We are having a problem, however, in that some fields have leading underscores for names--at the root level, this is fine, but we have not found a way to make this work for nested fields.
Sample JSON:
{
"_id" : "319FFE15FF908EDD86B7FDEADBEEFBD8D7284128841B14AA6A966923C268DF39",
"SomeThing" :
{
"_SomeField" : 22,
"AnotherField" : 2112,
"YetAnotherField": 1
}
. . . etc . . . .
Using a schema as follows:
create table testSample
(
id string,
something struct
<
somefield:int,
anotherfield:bigint,
yetanotherfield:int
>
)
row format serde 'org.openx.data.jsonserde.JsonSerDe'
with serdeproperties
(
"mapping.id" = "_id",
"mapping.somefield" = "_somefield"
);
This schema builds OK--however, after loading the in above sample, the value of "somefield" (the nested + leading underscore one) is always null (all the other values exist and are correct).
We've been trying a lot of syntax combinations, but to no avail.
Does anyone know the trick to hap a nested field with a leading underscore in its name?
Cheers!
Answering my own question here: there is no trick because you can't.
However, there's an easy work-around: you can tell Hive to treat the names as literals upon creating the schema. If you do this, you will also need to query using the same literal syntax. In the above example, it would look like:
`_something` struct<rest_of_definitions>
without any special serde properties for it.
Then use again in query:
select stuff.`_something` from sometable;
e.g., schema:
create table testSample
(
id string,
something struct
<
`_somefield`:int,
anotherfield:bigint,
yetanotherfield:int
>
)
row format serde 'org.openx.data.jsonserde.JsonSerDe'
with serdeproperties("mapping.id" = "_id");
for an input JSON like:
{
"_id": "someuid",
"something":
{
"_somefield": 1,
"anotherfield": 2,
"yetanotherfield": 3
}
}
with a query like:
select something.`_somefield`
from testSample
where something.anotherfield = 2;