I m a beginner and trying to insert JSON values into the database using a tutorial
I have created the table using the following command
CREATE TABLE table_name( id character varying(50),
data json NOT NULL,
active boolean NOT NULL,
created_at timestamp with time zone NOT NULL,
updated_at timestamp with time zone NOT NULL,
CONSTRAINT table_name_pkey PRIMARY KEY (id)
);
The table is created with table_name.
Now I am trying to insert the values into the database:
INSERT INTO table_name
SELECT id,data,active,created_at,updated_at
FROM json_populate_record (NULL::table_name,
'{
"id": "1",
"data":{
"key":"value"
},
"active":true,
"created_at": SELECT NOW(),
"updated_at": SELECT NOW()
}'
);
It throws the following error
ERROR: Invalid input syntax for type JSON '{
Could anyone help me to resolve and insert the JSON values into the DB?
You can't include arbitrary SQL commands inside a JSON string. From a JSON "perspective" SELECT NOW() is an invalid value because it lacks the double quotes. But even if you used "select now()" that would be executed as a SQL query and replaced with the current timestamp) .
But I don't understand why you are wrapping this into a jsonb_populate_record. The better solution (at least in my opinion) would be:
INSERT INTO table_name (id, data, active, created_at, updated_dat)
VALUES ('1', '{"key": "value"}', true, now(), now();
If you really want to complicate things, you need to use string concatenation:
SELECT id,data,active,created_at,updated_at
FROM json_populate_record (NULL::table_name,
format('{
"id": "1",
"data":{
"key":"value"
},
"active":true,
"created_at": "%s",
"updated_at": "%s"
}', now(), now())::json
);
Related
I have a JSON file with data about employees and their skills. I need to model the data somehow in a PostgreSQL database (and the reason is related to the application we are developing).
The JSON file has a lot of data that I don't really need for my application (at least for now). I only need a few columns: Employee ID, Name, Qualifications. But the rest of the data should be stored in the table (only temporarily, as this is still a POC).
Data
{
"employee": {
"ID": 654534543,,
"Name": "Max Mustermann",
"Email": "max.mustermann#firma.de",
"skills": [
{"name": python, "level": 3},
{"name": c, "level": 2},
{"name": openCV, "level": 3}
],
},
"employee":{
"ID": 3213213,,
"Name": "Alex Mustermann",
"Email": "alex.mustermann#firma.de",
"skills":[
{"name": Jira, "level": 3},
{"name": Git, "level": 2},
{"name": Tensorflow, "level": 3}
],
}
};
I thought of creating a table with the columns: Employee ID as primary key, CHAR for the name, array for the skills and JSONB for the rest of the information about the employee.
TABLE
CREATE TABLE employee(
id INT PRIMARY KEY,
name VARCHAR(255) NOT NULL,
position VARCHAR(255) NOT NULL,
description VARCHAR (255),
skills TEXT [],
join_date DATE,
);
Some factors to keep in mind: the data should be periodically updated (lets say once a month), the application should use the database to query one (or more) employee ID(s) who are covering certain required skill set(and skill levels). And so far we are not sure if we are going to query json fields (but could be possible in near future)
also, the data is complicated and dense (what I attached below is merely a simplified sample), so I guess querying directly from a JSONB column would not be convenient (as mentioned in other similar questions)
My questions now are:
1- Would the proposed data model meet the required conditions, we have a huge json data file (fast search for employee skills, scalable, easy/fast query and retrieval of employee data (for e.g employee id)?
2- What should be considered when developing a relational database schema?
3- Would there be advantages to splitting the data into multiple tables? e.g. one table for employee personal data with employee ID as primary key, one table for skills with employee ID as foreign key and a text field for skills, one JSON table for the rest of the data.
I am using PostgreSQL 15.1 on windows 10. I am also still getting familiar with PostgreSQL databases.
much thanks
Here is what I would do:
create table employee (
id bigint not null primary key,
name text not null,
email text not null
);
create table skill (
id bigint generated always as identity primary key,
skill_name text not null unique
);
create table employee_skill (
id bigint generated always as identity primary key,
employee_id bigint not null references employee(id),
skill_id bigint not null references skill(id),
skill_level int not null,
unique (employee_id, skill_id)
);
Then, to populate the schema (after correcting the errors with the JSON):
with indata as (
select '[
{
"ID": 654534543,
"Name": "Max Mustermann",
"Email": "max.mustermann#firma.de",
"skills": [
{"name": "python", "level": 3},
{"name": "c", "level": 2},
{"name": "openCV", "level": 3}
]
},
{
"ID": 3213213,
"Name": "Alex Mustermann",
"Email": "alex.mustermann#firma.de",
"skills":[
{"name": "Jira", "level": 3},
{"name": "Git", "level": 2},
{"name": "Tensorflow", "level": 3}
]
}
]'::jsonb as j
), expand as (
select emp, skill
from indata
cross join lateral jsonb_array_elements(j) as el(emp)
cross join lateral jsonb_array_elements(emp->'skills') as sk(skill)
), insemp as (
insert into employee (id, name, email)
select distinct (emp->>'ID')::bigint, emp->>'Name', emp->>'Email'
from expand
on conflict (id) do update
set name = excluded.name, email = excluded.email
returning *
), insskill as (
insert into skill (skill_name)
select distinct skill->>'name'
from expand
on conflict (skill_name) do nothing
returning *
), allemp as (
select * from insemp union select * from employee
), allskill as (
select * from insskill union select * from insskill
), insempskill as (
insert into employee_skill (employee_id, skill_id, skill_level)
select e.id as employee_id, s.id as skill_id,
(i.skill->>'level')::int as skill_level
from expand i
join allemp e on e.id = (i.emp->>'ID')::bigint
join allskill s on s.skill_name = i.skill->>'name'
on conflict (employee_id, skill_id) do update
set skill_level = excluded.skill_level
returning *
)
delete from employee_skill
where (employee_id, skill_id) not in
(select employee_id, skill_id from insempskill
union
select employee_id, skill_id from employee_skill)
;
See working fiddle
I am looking for a way of find rows by given element of the json table that match the pattern.
Lets start with mysql table:
CREATE TABLE `person` (
`attributes` json DEFAULT NULL
);
INSERT INTO `person` (`attributes`)
VALUES ('[{"scores": 1, "name": "John"},{"scores": 1, "name": "Adam"}]');
INSERT INTO `person` (`attributes`)
VALUES ('[{"scores": 1, "name": "Johny"}]');
INSERT INTO `person` (`attributes`)
VALUES ('[{"scores": 1, "name": "Peter"}]');
How to find all records where attributes[*].name consists John* pattern?
In the John* case the query should return 2 records (with John and Johny).
SELECT DISTINCT person.*
FROM person
CROSS JOIN JSON_TABLE(person.attributes, '$[*]' COLUMNS (name TEXT PATH '$.name')) parsed
WHERE parsed.name LIKE 'John%';
https://sqlize.online/sql/mysql80/c9e4a3ffa159c4be8c761d696e06d946/
DB Fiddle
CREATE TABLE Days (
id int primary key,
date_fix VARCHAR(255),
date_calculated VARCHAR(255) GENERATED ALWAYS AS (DATE_ADD("date_fix", INTERVAL 1 DAY))
);
INSERT Days
(id, date_fix, date_calculated
)
VALUES
("1", "2019-01-01", ""),
("2", "2019-01-06", ""),
("3", "2019-05-01", ""),
("4", "2019-08-15", ""),
("5", "2019-10-03", "");
In the above table I want to insert a column called date_calculated which calculates the date as following:
date_fix + 1 day
Therefore, I tried to combine GENERATED ALWAYS with DATE_ADD("date_fix", INTERVAL 1 DAY) but I could not make it work so far.
I assume the issue is related to the INSERT statement since I currently use only "" for the column date_calculated but I do not have any clue how to replace this "" in order to achieve that the column is calculated as described.
Do you have any idea how to get the desired table with the calculated column?
You should add DATE before DATE_ADD in your query
CREATE TABLE Days (
id int primary key,
date_fix VARCHAR(255),
date_calculated VARCHAR(255)
GENERATED ALWAYS AS (DATE(DATE_ADD(date_fix, INTERVAL 1 DAY)))
);
Then you can insert your data
INSERT INTO Days (id, date_fix)
VALUES ("1", "2019-01-01"),
("2", "2019-01-06"),
("3", "2019-05-01"),
("4", "2019-08-15"),
("5", "2019-10-03");
Your code works fine with a couple of fixes:
CREATE TABLE Days (
id int primary key,
date_fix VARCHAR(255),
date_calculated VARCHAR(255) GENERATED ALWAYS AS (DATE_ADD(date_fix, INTERVAL 1 DAY))
);
INSERT Days (id, date_fix)
VALUES (1, '2019-01-01'),
(2, '2019-01-06'),
(3, '2019-05-01'),
(4, '2019-08-15'),
(5, '2019-10-03');
Here is a db<>fiddle.
Your major issue is that you are quite confused by quotes. When writing SQL, it is pretty simple:
Use SINGLE quotes for date and string constants.
Never use quotes at all for numbers.
Do not use DOUBLE quotes (unless you really understand why you are using them). MySQL extends SQL so this is a synonym for single quotes.
The second issue is that there is no need to insert a value into a generated column. The value is calculated when you query the column, not when you insert values.
Then the third issue is types. Do not store dates as strings. MySQL has a wonderful data type to store dates, called date. It was invented for a reason and you should use it.
The approach you are taking to create the columns is not the right approach and not recommended.
The right approach for this logic would be, create the table with Id & date_fix only and then using select statement generate your calculated results.
Why to use this method?
As, You will be creating a new column unnecessarily to store the information which you can get from select statement will not be scalable enough to work in a large enterprise environment. So always try to minimize the load.
Please find the below code:
CREATE TABLE Days (
id int primary key,
date_fix VARCHAR(255)
);
INSERT Days
(id, date_fix
)
VALUES
('1', '2019-01-01'),
('2', '2019-01-06'),
('3', '2019-05-01'),
('4', '2019-08-15'),
('5', '2019-10-03');
select id,date_fix, DATEADD(DAY,1,CAST(date_fix as DATETIME)) from Days;
I have save data in PostgreSQL as given below:
{"tags": "Tag 1,Tag 2,Tag 3"}
{"tags": "Tag 1,Tag 4,Tag 5"}
{"tags": "Tag 6,Tag 1,Tag 2"}
I want search records where 'Tag 2' or Tag 3 exists?
Table schema, create procedure is as below,
--create table
CREATE TABLE "tblIrsInputTagging" (
"IrsInputTaggId" serial NOT NULL,
"Irs_tags" json NOT NULL, CONSTRAINT "tblIrsInputTagging_pkey"
PRIMARY KEY ("IrsInputTaggId")
) WITH ( OIDS=FALSE );
ALTER TABLE "tblIrsInputTagging" OWNER TO "postgre";
--insert json record
INSERT INTO "tblIrsInputTagging" ("Irs_tags")
VALUES ( '{"tags": "Tag 1,Tag 2,Tag 3"}' );
INSERT INTO "tblIrsInputTagging" ("Irs_tags")
VALUES ( '{"tags": "Tag 1,Tag 4,Tag 5"}' );
INSERT INTO "tblIrsInputTagging" ("Irs_tags")
VALUES ( '{"tags": "Tag 6,Tag 1,Tag 2"}' );
i don't think there is some specific function to check per json items. But you can do so by casting the data type as text then check it with like or ilike operator.
SELECT col1, col2 FROM table_name WHERE CAST(col1 AS text) ilike '%value%'
notice that the col1 column is the json data type.
Let's say the column name is tags. In that case, you can query like below.
If there is something specific, let me know, so I might need to update below snippet.
SELECT col_1, col_2, .. col_n
FROM your_table
WHERE tags::json->'tags' ILIKE '%Tag2%'
OR
tags::json->'tags' ILIKE '%Tag3%'
I'm getting this error when trying to access data in a JSON object, does anybody know what it is causing it?
This is the query:
SELECT id, data FROM cities WHERE data->'location'->>'population' = '270816'
This is the JSON object:
location": {
"population": 270816,
"type": "city"
}
Any help would be really appreciate it. Thanks
I was able to get this SELECT to work in Postgres 9.3.1. Here's an sqlfiddle which illustrates that.
Here is the DDL and INSERT statement I used in the sqlfiddle:
create table cities
(
id serial,
data json
);
insert into cities (data) VALUES
('{
"location": {
"population": 270816,
"type": "city"
}
}'
);
What version of Postgres are you using? How are you inserting the JSON? What's the DDL for your cities table?
It suspect it may be an issue with the way you are inserting the JSON data. Try inserting it similar to the way I am doing in the sqlfiddle above and see if that works for you. i.e. as a pure SQL string, but one with valid JSON inside, into a column defined as json.
Just had what sounds like the same issue on Postgres 9.6.6. Improper string escaping caused mysterious JSONB behavior. Using pgAdmin4,
CREATE TABLE json_test (json_data JSONB);
INSERT INTO json_test (json_data) VALUES ('"{\"id\": \"test1\"}"');
INSERT INTO json_test (json_data) VALUES ('{"id": "test2"}');
SELECT json_data, json_data->>'id' as id FROM json_test;
returns the following pgAdmin4 output showing baffling failure to find id test2. Turns out the pgAdmin4 display is misleading. Situation becomes clear using text display from PSQL:
db=> CREATE TABLE json_test (json_data JSONB);
CREATE TABLE
db=> INSERT INTO json_test (json_data) VALUES ('"{\"id\": \"test1\"}"');
INSERT 0 1
db=> INSERT INTO json_test (json_data) VALUES ('{"id": "test2"}');
INSERT 0 1
db=> SELECT json_data, json_data->>'id' as id FROM json_test;
json_data | id
-----------------------+-------
"{\"id\": \"test1\"}" |
{"id": "test2"} | test2
(2 rows)
Where it is obvious that the first row was inserted as a string which just looks like JSON, not as a nested JSON object.