how to use the table function file in clickhouse? - function

I am trying to use the table function file in clickhouse and below are what I have tried.
The background:
test sql:
create table test(a String,b Int32) ENGINE = Memory;--File(CSV)
insert into test (a,b) values ('world',22) ('quant',33);
insert into test (a,b) values ('hello',1);
select * from test;
SELECT a,b FROM test FORMAT Template SETTINGS format_template_resultset = '/home/resultset.format', format_template_row = '/home/row.format', format_template_rows_between_delimiter = '\n';
print:
head
the first:"world",the second:22;
the first:"quant",the second:33;
the first:"hello",the second:1;
end
the fail sql:
SELECT * FROM file("data",Template SETTINGS format_template_resultset = '/home/resultset.format', format_template_row = '/home/row.format', format_template_rows_between_delimiter = '\n', 'a String, b Int32');
print:
Syntax error: failed at position 36 ('SETTINGS'):
SELECT * FROM file("data",Template SETTINGS format_template_resultset = '/home/resultset.format', format_template_row = '/home/row.format', format_template_rows_between_delimiter = '\n', 'a String, b Int32');
Expected one of: DoubleColon, LIKE, GLOBAL NOT IN, end of query, AS, DIV, IS, UUID, OR, QuestionMark, BETWEEN, NOT LIKE, MOD, AND, Comma, alias, IN, ILIKE, Dot, NOT ILIKE, NOT, Arrow, token, NOT IN, GLOBAL IN
I have checked the documentation but it does not seems to be correct. How can I achieve this?

#vladimir s answer is correct,
SELECT * FROM file("data",Template, 'a String, b Int32') SETTINGS format_template_resultset = '/home/resultset.format', format_template_row = '/home/row.format', format_template_rows_between_delimiter = '\n';
thank you.

Related

placeholders for table names in python mysql

I am using python sql to edit a very simple table named students (whose columns are name and age), as shown below:
('Rachel', 22)
('Linckle', 33)
('Bob', 45)
('Amanda', 25)
('Jacob', 85)
('Avi', 65)
('Michelle', 45)
I am defining python functions to execute SQL code.
In my first function I want to update the age values in students table where the name matches something (e.g. Bob). If I define the following function:
def update_age(age, name):
c.execute("""UPDATE students SET age = %s
WHERE name = %s""", (age, name))
And then:
update_age(99, 'Bob')
I will get:
('Rachel', 22)
('Linckle', 33)
('Bob', 99)
('Amanda', 25)
('Jacob', 85)
('Avi', 65)
('Michelle', 45)
On a second function I would like to specify also the name of the table, with the following code:
def update_age_table(table, age, name):
c.execute("""UPDATE %s SET age = %s
WHERE name = %s""",
(table, age, name)) # note that here I am only replacing students by the placeholder %s
Then if I do:
update_age_table(table='students', age=95, name='Jacob')
I will get the following error message (it is long, I am only displaying the last sentence:
You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ''students' SET age = 95
WHERE name = 'Jacob'' at line 1
I guess that the error comes from the fact that I am assigning two of the placeholders to variables, namely age and name, which is not the case of the table name, where there is no variable assignment.
Does anyone know how I can use placeholders in SQL commands without assigning them to variables?
ThatÅ› because you cannot pass the table name as a parameter in the execute sentence. You should do it this way:
def update_age_table(table, age, name):
c.execute("UPDATE "+table+" SET age = %s
WHERE name = %s",
(table, age, name)) #
The prepared statement doesn't work for table names
EDIT
You have to remove the table parameter like this:
def update_age_table(table, age, name):
c.execute("UPDATE "+table+" SET age = %s WHERE name = %s",(age, name)) #
Sorry was a mistake
dt= datetime.datetime.now()
new_date=str(dt)
idname=input("Please enter Your Id. ")
bname= input("Please Enter name of book which you want to Issue: ")
idn=(idname,)
sql="insert into id%s (issuedbook,date)"%idn +"values (%s,%s)"
val=(bname,new_date)
cursor.execute(sql,val)
cnx.commit()
insert_data()```
Without having tested it, this should be a better coding style of the accepted answer. As the whole Q/A shows, the variables are passed only at cursor.execution() time to make it more secure, but the table statement of the execute() string is evaluated before the args are evaluated, that is why tables have to be plain text evaluated before execute() but the variables do not. See another example with similar challenge at Python - pass a list as params to SQL, plus more variables where the table is not passed either.
Therefore, just as an add-on for the rightly accepted query:
def update_age_table(UPDATE_QUERY, args):
c.execute(UPDATE_QUERY, args)
c.commit()
# example for string testing:
table, age, name = "table_x", 2, "name_y"
UPDATE_QUERY = f"""
UPDATE {table}
SET age = %s
WHERE name = %s
"""
# # UPDATE_QUERY Out:
# '\n UPDATE table_x\n SET age = %s\n WHERE name = %s\n'
args = [age, name]
update_age_table(UPDATE_QUERY, args)

What's wrong with this SQL query WHERE AND clause?

Previously, this was working:
$patient_story_set_photos = $wpdb->get_results('SELECT * FROM wp_before_after WHERE patientID = '.$post->ID.' AND patient_display = 1');
However, when I try to add another AND condition like this:
$patient_story_set_photos = $wpdb->get_results('SELECT * FROM wp_before_after WHERE patientID = '.$post->ID.' AND patient_display = 1 AND period_taken = '.$set->period_taken);
I get the following error on screen:
WordPress database error: [Unknown column '1hour' in 'where clause']
SELECT * FROM wp_before_after WHERE patientID = 8175 AND patient_display = 1 AND period_taken = 1hour
Can't see why there's a problem, are you not allowed to use multiple AND conditions in SQL?
The problem is not the AND, the problem is your 1hour, 1hour unquoted means a reference to an object (database, table) named 1hour, you need to quote '1hour'.
If you write
SELECT * FROM wp_before_after
WHERE patientID = 8175
AND patient_display = 1
AND period_taken = '1hour'
you will compare the field periodtaken to a string (CHAR,VARCHAR,TEXT) equal to '1hour'.
I assume period_taken is a field typed CHAR,VARCHAR or TEXT
Before anything, DO NOT CONCATENATE SQL STRINGS nowadays it is a MUST (see how to do it properly https://stackoverflow.com/a/60496/3771219)
The problem you are facing is because, I presume, that the period_taken field is some sort of Char/Varchar/String field and when you are filtering by a "Stringy" field you must sorround your literals values with single quotes:
SELECT *
FROM wp_before_after
WHERE patientID = 8175
AND patient_display = 1
AND period_taken = '1hour'
Hope this help

How do I modify fields inside the new PostgreSQL JSON datatype?

With postgresql 9.3 I can SELECT specific fields of a JSON data type, but how do you modify them using UPDATE? I can't find any examples of this in the postgresql documentation, or anywhere online. I have tried the obvious:
postgres=# create table test (data json);
CREATE TABLE
postgres=# insert into test (data) values ('{"a":1,"b":2}');
INSERT 0 1
postgres=# select data->'a' from test where data->>'b' = '2';
?column?
----------
1
(1 row)
postgres=# update test set data->'a' = to_json(5) where data->>'b' = '2';
ERROR: syntax error at or near "->"
LINE 1: update test set data->'a' = to_json(5) where data->>'b' = '2...
Update: With PostgreSQL 9.5, there are some jsonb manipulation functionality within PostgreSQL itself (but none for json; casts are required to manipulate json values).
Merging 2 (or more) JSON objects (or concatenating arrays):
SELECT jsonb '{"a":1}' || jsonb '{"b":2}', -- will yield jsonb '{"a":1,"b":2}'
jsonb '["a",1]' || jsonb '["b",2]' -- will yield jsonb '["a",1,"b",2]'
So, setting a simple key can be done using:
SELECT jsonb '{"a":1}' || jsonb_build_object('<key>', '<value>')
Where <key> should be string, and <value> can be whatever type to_jsonb() accepts.
For setting a value deep in a JSON hierarchy, the jsonb_set() function can be used:
SELECT jsonb_set('{"a":[null,{"b":[]}]}', '{a,1,b,0}', jsonb '{"c":3}')
-- will yield jsonb '{"a":[null,{"b":[{"c":3}]}]}'
Full parameter list of jsonb_set():
jsonb_set(target jsonb,
path text[],
new_value jsonb,
create_missing boolean default true)
path can contain JSON array indexes too & negative integers that appear there count from the end of JSON arrays. However, a non-existing, but positive JSON array index will append the element to the end of the array:
SELECT jsonb_set('{"a":[null,{"b":[1,2]}]}', '{a,1,b,1000}', jsonb '3', true)
-- will yield jsonb '{"a":[null,{"b":[1,2,3]}]}'
For inserting into JSON array (while preserving all of the original values), the jsonb_insert() function can be used (in 9.6+; this function only, in this section):
SELECT jsonb_insert('{"a":[null,{"b":[1]}]}', '{a,1,b,0}', jsonb '2')
-- will yield jsonb '{"a":[null,{"b":[2,1]}]}', and
SELECT jsonb_insert('{"a":[null,{"b":[1]}]}', '{a,1,b,0}', jsonb '2', true)
-- will yield jsonb '{"a":[null,{"b":[1,2]}]}'
Full parameter list of jsonb_insert():
jsonb_insert(target jsonb,
path text[],
new_value jsonb,
insert_after boolean default false)
Again, negative integers that appear in path count from the end of JSON arrays.
So, f.ex. appending to an end of a JSON array can be done with:
SELECT jsonb_insert('{"a":[null,{"b":[1,2]}]}', '{a,1,b,-1}', jsonb '3', true)
-- will yield jsonb '{"a":[null,{"b":[1,2,3]}]}', and
However, this function is working slightly differently (than jsonb_set()) when the path in target is a JSON object's key. In that case, it will only add a new key-value pair for the JSON object when the key is not used. If it's used, it will raise an error:
SELECT jsonb_insert('{"a":[null,{"b":[1]}]}', '{a,1,c}', jsonb '[2]')
-- will yield jsonb '{"a":[null,{"b":[1],"c":[2]}]}', but
SELECT jsonb_insert('{"a":[null,{"b":[1]}]}', '{a,1,b}', jsonb '[2]')
-- will raise SQLSTATE 22023 (invalid_parameter_value): cannot replace existing key
Deleting a key (or an index) from a JSON object (or, from an array) can be done with the - operator:
SELECT jsonb '{"a":1,"b":2}' - 'a', -- will yield jsonb '{"b":2}'
jsonb '["a",1,"b",2]' - 1 -- will yield jsonb '["a","b",2]'
Deleting, from deep in a JSON hierarchy can be done with the #- operator:
SELECT '{"a":[null,{"b":[3.14]}]}' #- '{a,1,b,0}'
-- will yield jsonb '{"a":[null,{"b":[]}]}'
For 9.4, you can use a modified version of the original answer (below), but instead of aggregating a JSON string, you can aggregate into a json object directly with json_object_agg().
Original answer: It is possible (without plpython or plv8) in pure SQL too (but needs 9.3+, will not work with 9.2)
CREATE OR REPLACE FUNCTION "json_object_set_key"(
"json" json,
"key_to_set" TEXT,
"value_to_set" anyelement
)
RETURNS json
LANGUAGE sql
IMMUTABLE
STRICT
AS $function$
SELECT concat('{', string_agg(to_json("key") || ':' || "value", ','), '}')::json
FROM (SELECT *
FROM json_each("json")
WHERE "key" <> "key_to_set"
UNION ALL
SELECT "key_to_set", to_json("value_to_set")) AS "fields"
$function$;
SQLFiddle
Edit:
A version, which sets multiple keys & values:
CREATE OR REPLACE FUNCTION "json_object_set_keys"(
"json" json,
"keys_to_set" TEXT[],
"values_to_set" anyarray
)
RETURNS json
LANGUAGE sql
IMMUTABLE
STRICT
AS $function$
SELECT concat('{', string_agg(to_json("key") || ':' || "value", ','), '}')::json
FROM (SELECT *
FROM json_each("json")
WHERE "key" <> ALL ("keys_to_set")
UNION ALL
SELECT DISTINCT ON ("keys_to_set"["index"])
"keys_to_set"["index"],
CASE
WHEN "values_to_set"["index"] IS NULL THEN 'null'::json
ELSE to_json("values_to_set"["index"])
END
FROM generate_subscripts("keys_to_set", 1) AS "keys"("index")
JOIN generate_subscripts("values_to_set", 1) AS "values"("index")
USING ("index")) AS "fields"
$function$;
Edit 2: as #ErwinBrandstetter noted these functions above works like a so-called UPSERT (updates a field if it exists, inserts if it does not exist). Here is a variant, which only UPDATE:
CREATE OR REPLACE FUNCTION "json_object_update_key"(
"json" json,
"key_to_set" TEXT,
"value_to_set" anyelement
)
RETURNS json
LANGUAGE sql
IMMUTABLE
STRICT
AS $function$
SELECT CASE
WHEN ("json" -> "key_to_set") IS NULL THEN "json"
ELSE (SELECT concat('{', string_agg(to_json("key") || ':' || "value", ','), '}')
FROM (SELECT *
FROM json_each("json")
WHERE "key" <> "key_to_set"
UNION ALL
SELECT "key_to_set", to_json("value_to_set")) AS "fields")::json
END
$function$;
Edit 3: Here is recursive variant, which can set (UPSERT) a leaf value (and uses the first function from this answer), located at a key-path (where keys can only refer to inner objects, inner arrays not supported):
CREATE OR REPLACE FUNCTION "json_object_set_path"(
"json" json,
"key_path" TEXT[],
"value_to_set" anyelement
)
RETURNS json
LANGUAGE sql
IMMUTABLE
STRICT
AS $function$
SELECT CASE COALESCE(array_length("key_path", 1), 0)
WHEN 0 THEN to_json("value_to_set")
WHEN 1 THEN "json_object_set_key"("json", "key_path"[l], "value_to_set")
ELSE "json_object_set_key"(
"json",
"key_path"[l],
"json_object_set_path"(
COALESCE(NULLIF(("json" -> "key_path"[l])::text, 'null'), '{}')::json,
"key_path"[l+1:u],
"value_to_set"
)
)
END
FROM array_lower("key_path", 1) l,
array_upper("key_path", 1) u
$function$;
Updated: Added function for replacing an existing json field's key by another given key. Can be in handy for updating data types in migrations or other scenarios like data structure amending.
CREATE OR REPLACE FUNCTION json_object_replace_key(
json_value json,
existing_key text,
desired_key text)
RETURNS json AS
$BODY$
SELECT COALESCE(
(
SELECT ('{' || string_agg(to_json(key) || ':' || value, ',') || '}')
FROM (
SELECT *
FROM json_each(json_value)
WHERE key <> existing_key
UNION ALL
SELECT desired_key, json_value -> existing_key
) AS "fields"
-- WHERE value IS NOT NULL (Actually not required as the string_agg with value's being null will "discard" that entry)
),
'{}'
)::json
$BODY$
LANGUAGE sql IMMUTABLE STRICT
COST 100;
Update: functions are compacted now.
With 9.5 use jsonb_set-
UPDATE objects
SET body = jsonb_set(body, '{name}', '"Mary"', true)
WHERE id = 1;
where body is a jsonb column type.
With Postgresql 9.5 it can be done by following-
UPDATE test
SET data = data - 'a' || '{"a":5}'
WHERE data->>'b' = '2';
OR
UPDATE test
SET data = jsonb_set(data, '{a}', '5'::jsonb);
Somebody asked how to update many fields in jsonb value at once. Suppose we create a table:
CREATE TABLE testjsonb ( id SERIAL PRIMARY KEY, object JSONB );
Then we INSERT a experimental row:
INSERT INTO testjsonb
VALUES (DEFAULT, '{"a":"one", "b":"two", "c":{"c1":"see1","c2":"see2","c3":"see3"}}');
Then we UPDATE the row:
UPDATE testjsonb SET object = object - 'b' || '{"a":1,"d":4}';
Which does the following:
Updates the a field
Removes the b field
Add the d field
Selecting the data:
SELECT jsonb_pretty(object) FROM testjsonb;
Will result in:
jsonb_pretty
-------------------------
{ +
"a": 1, +
"c": { +
"c1": "see1", +
"c2": "see2", +
"c3": "see3", +
}, +
"d": 4 +
}
(1 row)
To update field inside, Dont use the concat operator ||. Use jsonb_set instead. Which is not simple:
UPDATE testjsonb SET object =
jsonb_set(jsonb_set(object, '{c,c1}','"seeme"'),'{c,c2}','"seehim"');
Using the concat operator for {c,c1} for example:
UPDATE testjsonb SET object = object || '{"c":{"c1":"seedoctor"}}';
Will remove {c,c2} and {c,c3}.
For more power, seek power at postgresql json functions documentation. One might be interested in the #- operator, jsonb_set function and also jsonb_insert function.
If your field type is of json the following will work for you.
UPDATE
table_name
SET field_name = field_name::jsonb - 'key' || '{"key":new_val}'
WHERE field_name->>'key' = 'old_value'.
Operator '-' delete key/value pair or string element from left operand. Key/value pairs are matched based on their key value.
Operator '||' concatenate two jsonb values into a new jsonb value.
Since these are jsonb operators you just need to typecast to::jsonb
More info : JSON Functions and Operators
You can read my note here
UPDATE test
SET data = data::jsonb - 'a' || '{"a":5}'::jsonb
WHERE data->>'b' = '2'
This seems to be working on PostgreSQL 9.5
You can try updating as below:
Syntax: UPDATE table_name SET column_name = column_name::jsonb || '{"key":new_value}' WHERE column_name condition;
For your example:
UPDATE test SET data = data::jsonb || '{"a":new_value}' WHERE data->>'b' = '2';
This worked for me, when trying to update a string type field.
UPDATE table_name
SET body = jsonb_set(body, '{some_key}', to_json('value'::text)::jsonb);
Hope it helps someone else out!
Assuming the table table_name has a jsonb column named body and you want to change body.some_key = 'value'
I found previous answers more suitable for experienced PostgreSQL users. This one is for the beginners:
Assume you have the a table-column of type JSONB with the following value:
{
"key0": {
"key01": "2018-05-06T12:36:11.916761+00:00",
"key02": "DEFAULT_WEB_CONFIGURATION",
"key1": {
"key11": "Data System",
"key12": "<p>Health,<p>my address<p>USA",
"key13": "*Please refer to main screen labeling"
}
}
let's assume we want to set a new value in the row:
"key13": "*Please refer to main screen labeling"
and instead place the value:
"key13": "See main screen labeling"
we use the json_set() function to assign a new value to the key13
the parameters to jsonb_set()
jsonb_set(target jsonb, path text[], new_value jsonb[, create_missing boolean])
in "target" - I will place the jsonb column-name (this is the table column that is being modified)
"path"- is the "json keys path" leading-to (and including) the key that we are going to overwrite
"new_value" - this is the new value we assign
in our case we want to update the value of key13 which resides under key1 ( key1 -> key13 ) :
hence the path syntax is : '{key1,key13}'
(The path was the most tricky part to crack - because the tutorials are terrible)
jsonb_set(jsonb_column,'{key1,key13}','"See main screen labeling"')
To build upon #pozs's answers, here are a couple more PostgreSQL functions which may be useful to some. (Requires PostgreSQL 9.3+)
Delete By Key: Deletes a value from JSON structure by key.
CREATE OR REPLACE FUNCTION "json_object_del_key"(
"json" json,
"key_to_del" TEXT
)
RETURNS json
LANGUAGE sql
IMMUTABLE
STRICT
AS $function$
SELECT CASE
WHEN ("json" -> "key_to_del") IS NULL THEN "json"
ELSE (SELECT concat('{', string_agg(to_json("key") || ':' || "value", ','), '}')
FROM (SELECT *
FROM json_each("json")
WHERE "key" <> "key_to_del"
) AS "fields")::json
END
$function$;
Recursive Delete By Key: Deletes a value from JSON structure by key-path. (requires #pozs's json_object_set_key function)
CREATE OR REPLACE FUNCTION "json_object_del_path"(
"json" json,
"key_path" TEXT[]
)
RETURNS json
LANGUAGE sql
IMMUTABLE
STRICT
AS $function$
SELECT CASE
WHEN ("json" -> "key_path"[l] ) IS NULL THEN "json"
ELSE
CASE COALESCE(array_length("key_path", 1), 0)
WHEN 0 THEN "json"
WHEN 1 THEN "json_object_del_key"("json", "key_path"[l])
ELSE "json_object_set_key"(
"json",
"key_path"[l],
"json_object_del_path"(
COALESCE(NULLIF(("json" -> "key_path"[l])::text, 'null'), '{}')::json,
"key_path"[l+1:u]
)
)
END
END
FROM array_lower("key_path", 1) l,
array_upper("key_path", 1) u
$function$;
Usage examples:
s1=# SELECT json_object_del_key ('{"hello":[7,3,1],"foo":{"mofu":"fuwa", "moe":"kyun"}}',
'foo'),
json_object_del_path('{"hello":[7,3,1],"foo":{"mofu":"fuwa", "moe":"kyun"}}',
'{"foo","moe"}');
json_object_del_key | json_object_del_path
---------------------+-----------------------------------------
{"hello":[7,3,1]} | {"hello":[7,3,1],"foo":{"mofu":"fuwa"}}
Since PostgreSQL 14 you can use jsonb subscripting to access directly the elements of the JSON field and eventually update them.
UPDATE test SET data['a'] = '5' WHERE data['b'] = '2';
With PostgreSQL 9.4, we've implemented the following python function. It may also work with PostgreSQL 9.3.
create language plpython2u;
create or replace function json_set(jdata jsonb, jpaths jsonb, jvalue jsonb) returns jsonb as $$
import json
a = json.loads(jdata)
b = json.loads(jpaths)
if a.__class__.__name__ != 'dict' and a.__class__.__name__ != 'list':
raise plpy.Error("The json data must be an object or a string.")
if b.__class__.__name__ != 'list':
raise plpy.Error("The json path must be an array of paths to traverse.")
c = a
for i in range(0, len(b)):
p = b[i]
plpy.notice('p == ' + str(p))
if i == len(b) - 1:
c[p] = json.loads(jvalue)
else:
if p.__class__.__name__ == 'unicode':
plpy.notice("Traversing '" + p + "'")
if c.__class__.__name__ != 'dict':
raise plpy.Error(" The value here is not a dictionary.")
else:
c = c[p]
if p.__class__.__name__ == 'int':
plpy.notice("Traversing " + str(p))
if c.__class__.__name__ != 'list':
raise plpy.Error(" The value here is not a list.")
else:
c = c[p]
if c is None:
break
return json.dumps(a)
$$ language plpython2u ;
Example usage:
create table jsonb_table (jsonb_column jsonb);
insert into jsonb_table values
('{"cars":["Jaguar", {"type":"Unknown","partsList":[12, 34, 56]}, "Atom"]}');
select jsonb_column->'cars'->1->'partsList'->2, jsonb_column from jsonb_table;
update jsonb_table
set jsonb_column = json_set(jsonb_column, '["cars",1,"partsList",2]', '99');
select jsonb_column->'cars'->1->'partsList'->2, jsonb_column from jsonb_table;
Note that for a previous employer, I have written a set of C functions for manipulating JSON data as text (not as a json or jsonb type) for PostgreSQL 7, 8 and 9. For example, extracting data with json_path('{"obj":[12, 34, {"num":-45.67}]}', '$.obj[2]['num']'), setting data with json_path_set('{"obj":[12, 34, {"num":-45.67}]}', '$.obj[2]['num']', '99.87') and so on. It took about 3 days work, so if you need it to run on legacy systems and have the time to spare, it may be worth the effort. I imagine the C version is much faster than the python version.
Even though the following will not satisfy this request (the function json_object_agg is not available in PostgreSQL 9.3), the following can be useful for anyone looking for a || operator for PostgreSQL 9.4, as implemented in the upcoming PostgreSQL 9.5:
CREATE OR REPLACE FUNCTION jsonb_merge(left JSONB, right JSONB)
RETURNS JSONB
AS $$
SELECT
CASE WHEN jsonb_typeof($1) = 'object' AND jsonb_typeof($2) = 'object' THEN
(SELECT json_object_agg(COALESCE(o.key, n.key), CASE WHEN n.key IS NOT NULL THEN n.value ELSE o.value END)::jsonb
FROM jsonb_each($1) o
FULL JOIN jsonb_each($2) n ON (n.key = o.key))
ELSE
(CASE WHEN jsonb_typeof($1) = 'array' THEN LEFT($1::text, -1) ELSE '['||$1::text END ||', '||
CASE WHEN jsonb_typeof($2) = 'array' THEN RIGHT($2::text, -1) ELSE $2::text||']' END)::jsonb
END
$$ LANGUAGE sql IMMUTABLE STRICT;
GRANT EXECUTE ON FUNCTION jsonb_merge(jsonb, jsonb) TO public;
CREATE OPERATOR || ( LEFTARG = jsonb, RIGHTARG = jsonb, PROCEDURE = jsonb_merge );
You can also increment keys atomically within jsonb like this:
UPDATE users SET counters = counters || CONCAT('{"bar":', COALESCE(counters->>'bar','0')::int + 1, '}')::jsonb WHERE id = 1;
SELECT * FROM users;
id | counters
----+------------
1 | {"bar": 1}
Undefined key -> assumes starting value of 0.
For more detailed explanation, see my answer here: https://stackoverflow.com/a/39076637
I wrote small function for myself that works recursively in Postgres 9.4. Here is the function (I hope it works well for you):
CREATE OR REPLACE FUNCTION jsonb_update(val1 JSONB,val2 JSONB)
RETURNS JSONB AS $$
DECLARE
result JSONB;
v RECORD;
BEGIN
IF jsonb_typeof(val2) = 'null'
THEN
RETURN val1;
END IF;
result = val1;
FOR v IN SELECT key, value FROM jsonb_each(val2) LOOP
IF jsonb_typeof(val2->v.key) = 'object'
THEN
result = result || jsonb_build_object(v.key, jsonb_update(val1->v.key, val2->v.key));
ELSE
result = result || jsonb_build_object(v.key, v.value);
END IF;
END LOOP;
RETURN result;
END;
$$ LANGUAGE plpgsql;
Here is sample use:
select jsonb_update('{"a":{"b":{"c":{"d":5,"dd":6},"cc":1}},"aaa":5}'::jsonb, '{"a":{"b":{"c":{"d":15}}},"aa":9}'::jsonb);
jsonb_update
---------------------------------------------------------------------
{"a": {"b": {"c": {"d": 15, "dd": 6}, "cc": 1}}, "aa": 9, "aaa": 5}
(1 row)
As you can see it analyze deep down and update/add values where needed.
Sadly, I've not found anything in the documentation, but you can use some workaround, for example you could write some extended function.
For example, in Python:
CREATE or REPLACE FUNCTION json_update(data json, key text, value json)
returns json
as $$
from json import loads, dumps
if key is None: return data
js = loads(data)
js[key] = value
return dumps(js)
$$ language plpython3u
and then
update test set data=json_update(data, 'a', to_json(5)) where data->>'b' = '2';
The following plpython snippet might come in handy.
CREATE EXTENSION IF NOT EXISTS plpythonu;
CREATE LANGUAGE plpythonu;
CREATE OR REPLACE FUNCTION json_update(data json, key text, value text)
RETURNS json
AS $$
import json
json_data = json.loads(data)
json_data[key] = value
return json.dumps(json_data, indent=4)
$$ LANGUAGE plpythonu;
-- Check how JSON looks before updating
SELECT json_update(content::json, 'CFRDiagnosis.mod_nbs', '1')
FROM sc_server_centre_document WHERE record_id = 35 AND template = 'CFRDiagnosis';
-- Once satisfied update JSON inplace
UPDATE sc_server_centre_document SET content = json_update(content::json, 'CFRDiagnosis.mod_nbs', '1')
WHERE record_id = 35 AND template = 'CFRDiagnosis';
UPDATE table_name SET attrs = jsonb_set(cast(attrs as jsonb), '{key}', '"new_value"', true) WHERE id = 'some_id';
This what worked for me, attrs is a json type field. first cast to jsonb then update.
or
UPDATE table_name SET attrs = jsonb_set(cast(attrs as jsonb), '{key}', '"new_value"', true) WHERE attrs->>key = 'old_value';
what do you think about this solution ?
It will add the new value or update an existing one.
Edit: edited to make it work with null and empty object
Edit2: edited to make it work with object in the object...
create or replace function updateJsonb(object1 json, object2 json)
returns jsonb
language plpgsql
as
$$
declare
result jsonb;
tempObj1 text;
tempObj2 text;
begin
tempObj1 = substr(object1::text, 2, length(object1::text) - 2); --remove the first { and last }
tempObj2 = substr(object2::text, 2, length(object2::text) - 2); --remove the first { and last }
IF object1::text != '{}' and object1::text != 'null' and object1::text != '[]' THEN
result = ('{' || tempObj1 || ',' || tempObj2 || '}')::jsonb;
ELSE
result = ('{' || tempObj2 || '}')::jsonb;
END IF;
return result;
end;
$$;
usage:
update table_name
set data = updatejsonb(data, '{"test": "ok"}'::json)
For those who use mybatis, here is an example update statement:
<update id="saveAnswer">
update quiz_execution set answer_data = jsonb_set(answer_data, concat('{', #{qid}, '}')::text[], #{value}::jsonb), updated_at = #{updatedAt}
where id = #{id}
</update>
Params:
qid, the key for field.
value, is a valid json string, for field value,
e.g converted from object to json string via jackson,
If you want to use values from other columns in your JSON update command you can use string concatenation:
UPDATE table
SET column1 = column1::jsonb - 'key' || ('{"key": ' || column2::text || '}')::jsonb
where ...;
So, for example my string looks like this:
{"a1":{"a11":"x","a22":"y","a33":"z"}}
I update jsons by using temp table, which is good enough for rather small amount of data (<1.000.000). I found a different way, but then went on vacation and forgot it...
So. the query will be something like this:
with temp_table as (
select
a.id,
a->'a1'->>'a11' as 'a11',
a->'a1'->>'a22' as 'a22',
a->'a1'->>'a33' as 'a33',
u1.a11updated
from foo a
join table_with_updates u1 on u1.id = a.id)
update foo a
set a = ('{"a1": {"a11": "'|| t.a11updated ||'",
"a22":"'|| t.a22 ||'",
"a33":"'|| t.a33 ||'"}}')::jsonb
from temp_table t
where t.id = a.id;
It has more to do with string than json, but it works. Basically, it pulls all the data into temp table, creates a string while plugging concat holes with the data you backed up, and converts it into jsonb.
Json_set might be more efficient, but I'm still getting a hang of it. First time I tried to use it, I messed up the string completely...
If you want to add new fields as well you may try:
typeorm code
let keyName:string = '{key2}'
let valueName:string = '"new_value"'
emailLog: () => "jsonb_set(cast(email_log as jsonb), '" + keyNAme + "','" + valueName + "'," + "true" + ")"
This solution is an alternate to jsonb_set that works even if the column has NULL in it for the JSONB. The jsonb_set only works if the object exists.
In the example below, settings is a JSONB column on the Users Table.
UPDATE public."Users"
SET settings = coalesce("settings", '{}')::jsonb || '{ "KeyToSet" : "ValueToSet" }'
WHERE id=35877;
select * from pg_settings where name = 'deadlock_timeout';
begin;
create temp table a2(data jsonb);
insert into a2 values('{
"key0": {
"key01": "2018-05-06T12:36:11.916761+00:00",
"key02": "DEFAULT_WEB_CONFIGURATION",
"key1": {
"key11": "Data System",
"key12": "<p>Health,<p>my address<p>USA",
"key13": "*Please refer to main screen labeling"
}
}}'::jsonb);
commit;
nested jsonb structure update. can be applied to delete.
update a2 set data =
data::jsonb #- '{key0, key1, key13}'
|| '{"key13":"screen labeling"}'::jsonb
returning *;

SQL Server 2008: Error converting data type nvarchar to float

Presently troubleshooting a problem where running this SQL query:
UPDATE tblBenchmarkData
SET OriginalValue = DataValue, OriginalUnitID = DataUnitID,
DataValue = CAST(DataValue AS float) * 1.335
WHERE
FieldDataSetID = '6956beeb-a1e7-47f2-96db-0044746ad6d5'
AND ZEGCodeID IN
(SELECT ZEGCodeID FROM tblZEGCode
WHERE(ZEGCode = 'C004') OR
(LEFT(ZEGParentCode, 4) = 'C004'))
Results in the following error:
Msg 8114, Level 16, State 5, Line 1
Error converting data type nvarchar to float.
The really odd thing is, if I change the UPDATE to SELECT to inspect the values that are retrieved are numerical values:
SELECT DataValue
FROM tblBenchmarkData
WHERE FieldDataSetID = '6956beeb-a1e7-47f2-96db-0044746ad6d5'
AND ZEGCodeID IN
(SELECT ZEGCodeID
FROM tblZEGCode WHERE(ZEGCode = 'C004') OR
(LEFT(ZEGParentCode, 4) = 'C004'))
Here are the results:
DataValue
2285260
1205310
Would like to use TRY_PARSE or something like that; however, we are running on SQL Server 2008 rather than SQL Server 2012. Does anyone have any suggestions? TIA.
It would be helpful to see the schema definition of tblBenchmarkData, but you could try using ISNUMERIC in your query. Something like:
SET DataValue = CASE WHEN ISNUMERIC(DataValue)=1 THEN CAST(DataValue AS float) * 1.335
ELSE 0 END
Order of execution not always matches one's expectations.
If you set a where clause, it generally does not mean the calculations in the select list will only be applied to the rows that match that where. SQL Server may easily decide to do a bulk calculation and then filter out unwanted rows.
That said, you can easily write try_parse yourself:
create function dbo.try_parse(#v nvarchar(30))
returns float
with schemabinding, returns null on null input
as
begin
if isnumeric(#v) = 1
return cast(#v as float);
return null;
end;
So starting with your update query that's giving an error (please forgive me for rewriting it for my own clarity):
UPDATE B
SET
OriginalValue = DataValue,
OriginalUnitID = DataUnitID,
DataValue = CAST(DataValue AS float) * 1.335
FROM
dbo.tblBenchmarkData B
INNER JOIN dbo.tblZEGCode Z
ON B.ZEGCodeID = Z.ZEGCodeID
WHERE
B.FieldDataSetID = '6956beeb-a1e7-47f2-96db-0044746ad6d5'
AND (
Z.ZEGCode = 'C004' OR
Z.ZEGParentCode LIKE 'C004%'
)
I think you'll find that a SELECT statement with exactly the same expressions will give the same error:
SELECT
OriginalValue,
DataValue NewOriginalValue,
OriginalUnitID,
DataUnitID OriginalUnitID,
DataValue,
CAST(DataValue AS float) * 1.335 NewDataValue
FROM
dbo.tblBenchmarkData B
INNER JOIN dbo.tblZEGCode Z
ON B.ZEGCodeID = Z.ZEGCodeID
WHERE
B.FieldDataSetID = '6956beeb-a1e7-47f2-96db-0044746ad6d5'
AND (
Z.ZEGCode = 'C004' OR
Z.ZEGParentCode LIKE 'C004%'
)
This should show you the rows that can't convert:
SELECT
B.*
FROM
dbo.tblBenchmarkData B
INNER JOIN dbo.tblZEGCode Z
ON B.ZEGCodeID = Z.ZEGCodeID
WHERE
B.FieldDataSetID = '6956beeb-a1e7-47f2-96db-0044746ad6d5'
AND (
Z.ZEGCode = 'C004' OR
Z.ZEGParentCode LIKE 'C004%'
)
AND IsNumeric(DataValue) = 0
-- AND IsNumeric(DataValue + 'E0') = 0 -- try this if the prior doesn't work
The trick in the last commented line is to tack on things to the string to force only valid numbers to be numeric. For example, if you wanted only integers, IsNumeric(DataValue + '.0E0') = 0 would show you those that aren't.

Inserting images from file path -- Not getting value on the select statement

DECLARE #imgString varchar(800)
DECLARE #insertString varchar(3000)
DECLARE #imgNumber int
Declare #imgName varchar(100)
SET #imgNumber = 1
WHILE #imgNumber<> 101
BEGIN
SET #imgName = 'SELECT (items) FROM dbo.building_piclink'
SET #imgString = 'C:\Documents and Settings\Administrator\Desktop\photos\' + #imgName
SET #insertString = 'INSERT INTO dbo.building__ATTACH (DATA)
SELECT * FROM OPENROWSET(BULK N''' + #imgString + ''', SINGLE_BLOB) as tempImg'
SET #imgNumber = #imgNumber + 1
END
GO
I am having problems with the #imgName. I can't figure out how to get the value from the select statement not the (items) like below:
C:\Documents and Settings\Administrator\Desktop\photos\SELECT (items) FROM dbo.building_piclink
Thank you!
Your code has several problems:
1) You're selecting a file name from the view - but what if that view contains more than one entry?? Which filename are you selecting?? Your current code first of all doesn't work at all the way it is, and even if it were working - you're still potentially selecting hundreds of filenames into a single variable - which of course won't work....
So you'll need to fix this here first:
SET #imgName = 'SELECT (items) FROM dbo.building_piclink'
First of all - loose the single quotes:
SELECT #imgName = (items) FROM dbo.building_piclink
But now - do you have a unique ID that you can select for? Or do you want to get just the first entry (whatever that is) ??
So either you need:
SELECT #imgName = ImageFileName FROM dbo.building_piclink WHERE ..........
and fill in that WHERE clause with a condition that guarantees to return just a single row, or use TOP 1:
SELECT TOP (1) #imgName = ImageFileName FROM dbo.building_piclink
In that case - you'll just get exactly one filename - if you don't specify an ORDER BY, then there's no guarantee what you'll get - maybe you'll want to add a ORDER BY DueDate or something to prioritize which file names you get first.
2) Your code for loading the image data is non workable, either - what you need to do is build up the SQL statement as a string, and then execute it (called dynamic SQL) - something like this:
SET #imgString = 'C:\Documents and Settings\Administrator\Desktop\photos\' + #imgName
SET #insertString =
'INSERT INTO dbo.building__ATTACH (DATA)
SELECT * FROM OPENROWSET(BULK N''' + #imgString + ''', SINGLE_BLOB) as tempImg'
EXEC(#insertString) -- actually execute your SQL statement!
With these two fixes, you should be on the way to get this thing working