I want to fix or validate keys for JSON object in PostgreSQL(v10.7).
For instance, I have a JSON object called service_config which looks like;
{"con_type": "Foo", "capacity": 2, "capacity_unit": "gbps"}
And I have table:
id(serial) service_name(char) service_type(char) service_config(JSON)
-----------+---------------------+---------------------+---------------------
1 | com | ethernet | {"con_type": "ddc", "capacity": 2, "capacity_unit": "gbps"}
2 | res | gpon | {"con_type": "ftth", "capacity": 1, "capacity_unit": "gbps"}
Now, whenever I insert row into the table, I want to make sure or validate that the service_config column contains all the keys that are mentioned above, no more, no less. However, there could be null value for the keys.
Is this possible in Postgres and/or is there any better way to do this?
Possible solutions:
1- Validate service_config at the backend API and make sure all the keys are there. (currently in place and working)
2- Write a function in Postgres to validate service_config on insert and update. (doable but tedious)
Limitation: I cannot add any extension in Postgres.
I want to make sure or validate that the service_config column contains all the keys that are mentioned above, no more, no less. However, there could be null value for the keys.
Turn them into columns.
JSON is nice when you need to just dump some data into a row and you're not sure what it's going to be. Now that you are sure what it's going to be, and you want more constraints, that's what columns do best.
alter table whatever add column con_type text;
alter table whatever add column capacity integer;
alter table whatever add column capacity_unit text;
update whatever set
con_type = data->'con_type',
capacity = data->'capacity',
capacity_unit = data->'capacity_unit';
alter table whatever drop column data
The columns will always be there. Their values may be null. You can add per-column check constraints and indexes. No additional validations are necessary.
If you still need json, use jsonb_build_object.
select
jsonb_build_object(
'con_type', con_type,
'capacity', capacity,
'capacity_unit', capacity_unit
)
from whatever;
And, if you need it for compatibility purposes, you can make this a view.
create view whatever_as_json
select
*,
jsonb_build_object(
'con_type', con_type,
'capacity', capacity,
'capacity_unit', capacity_unit
) as data
from whatever;
Note that I use text, not char, because there is no advantage to char in Postgres. See the tip in 8.3. Character Types
There is no performance difference among these three types, apart from increased storage space when using the blank-padded type, and a few extra CPU cycles to check the length when storing into a length-constrained column. While character(n) has performance advantages in some other database systems, there is no such advantage in PostgreSQL; in fact character(n) is usually the slowest of the three because of its additional storage costs. In most situations text or character varying should be used instead.
Related
I'm trying to get results when both tables have the same machine number and there are entries that have the same number in both tables.
Here is what I've tried:
SELECT fehler.*,
'maschine.Maschinen-Typ',
maschine.Auftragsnummer,
maschine.Kunde,
maschine.Liefertermin_Soll
FROM fehler
JOIN maschine
ON ltrim(rtrim('maschine.Maschinen-Nr')) = ltrim(rtrim(fehler.Maschinen_Nr))
The field I'm joining on is a varchar in both cases. I tried without trims but still returns empty
I'm using MariaDB (if that's important).
ON ltrim(rtrim('maschine.Maschinen-Nr')) = ltrim(rtrim(fehler.Maschinen_Nr)) seems wrong...
Is fehler.Maschinen_Nr really the string 'maschine.Maschinen-Nr'?
SELECT fehler.*, `maschine.Maschinen-Typ`, maschine.Auftragsnummer, maschine.Kunde, maschine.Liefertermin_Soll
FROM fehler
JOIN maschine
ON ltrim(rtrim(`maschine.Maschinen-Nr`)) = ltrim(rtrim(`fehler.Maschinen_Nr`))
Last line compared a string to a number. This should be doing it.
Also, use the backtick to reference the column names.
The single quotes are string delimiters. You are comparing fehler.Maschinen_Nr with the string 'maschine.Maschinen-Nr'. In standard SQL you would use double quotes for names (and I think MariaDB allows this, too, certain settings provided). In MariaDB the commonly used name qualifier is the backtick:
SELECT fehler.*,
`maschine.Maschinen-Typ`,
maschine.Auftragsnummer,
maschine.Kunde,
maschine.Liefertermin_Soll
FROM fehler
JOIN maschine
ON trim(`maschine.Maschinen-Nr`) = trim(fehler.Maschinen_Nr)
(It would be better of course not to use names with a minus sign or other characters that force you to use name delimiters in the first place.)
As you see, you can use TRIM instead of LTRIM and RTRIM. It would be better, though, not to allow space at the beginning or end when inserting data. Then you wouldn't have to remove them in every query.
Moreover, it seems Maschinen_Nr should be primary key for the table maschine and naturally a foreign key then in table fehler. That would make sure fehler doesn't contain any Maschinen_Nr that not exists exactly so in maschine.
To avoid this problems in future, the convention for DB's is snake case(lowercase_lowercase).
Besides that, posting your DB schema would be really helpfull since i dont guess your data structures.
(For friendly development, is usefull that variables, tables and columns should be written in english)
So with this, what is the error that you get, because if table "maschine" has a column named "Maschinen-Nr" and table "fehler" has a column named "Maschinen_Nr" and the fields match each other, it should be correct
be careful with Maschinen-Nr and Maschinen_Nr. they have - and _ on purpose?
a very blind solution because you dont really tell what is your problem or even your schema is:
SELECT table1Alias.*, table2Alias.column_name, table2Alias.column_name
FROM table1 [table1Alias]
JOIN table2 [table2Alias]
ON ltrim(rtrim(table1Alias.matching_column)) = ltrim(rtrim(table2Alias.matching_column))
where matching_columns are respectively PK and FK or if the data matches both columns [] are optional and if not given, will be consider table_name
I want to efficiently store and efficiently manipulate bit flags for a record in MySQL. The SET datatype satisfies the first wish because up to 64 flags are stored as a single number. But what about the second? I have seen only awkward solutions like
UPDATE table_name SET set_col = (set_col | 4) WHERE condition;
UPDATE table_name SET set_col = (set_col & ~4) WHERE condition;
to respectively include and exclude a member into the value. I.e. I have to use numeric constants, which renders the code unmaintainable. Then I could have used INT datatype as well. If set_col definition gets changed (adding, removing or reordering the possible members), the code with hard-coded constants becomes a mess. I could try to enforce some discipline on coders to use only named variables in application language instead of numeric constants which would make maintenance easier, but not totally error-proof. Is there a solution where MySQL would resolve the symbolic names of set members to their correct numeric values? E.g. this does not work:
UPDATE person SET tag=tag | 'MGR'
To stem useless answers, I know about database normalization and a separate m-to-n relationship table, that is not the topic here. If you need a more concrete example, here you are:
CREATE TABLE `coder` (
`name` VARCHAR(50) NOT NULL,
`languages` SET('Perl','PHP','Java','Scala') NOT NULL
)
Changes to the set definition are unlikely but possible, maybe every other year, like splitting "Perl" into "Perl5" and "Perl6".
I found the answer here:
https://dev.mysql.com/doc/refman/5.7/en/set.html
Posted by John Kozura on April 12, 2011
Note that MySQL, at least
5.1+, seems to deal just fine with extra commas, so setting/deleting individual bits by name can be done very simply without creating a
"proper" list. So even something like SET flags=',,,foo,,bar,,' works
fine, if you don't care about a truncated data warning.
add bits:
UPDATE tbl SET flags=CONCAT_WS(',', flags, 'flagtoadd');
delete bits:
UPDATE tbl SET flags=REPLACE(flags, 'flagtoremove', '')
..or if you have a bit that's name is a subname of another bit like
"foo" and "foot", slightly more complicated:
UPDATE tbl SET flags=REPLACE(CONCAT(',', flags, ','), ',foo,', ',')
If the warnings do cause issues from you, then the solutions posted
above work:
add:
UPDATE tbl SET flags=TRIM(',' FROM CONCAT(flags, ',', 'flagtoadd'))
delete:
UPDATE tbl SET flags=TRIM(',' FROM REPLACE(CONCAT(',', flags, ','), ',flagtoremove,', ','))
This is my result
+----------------------------------------------------------------------------------------------------------------------------------------------+
| SUBSTRING(COLUMN_TYPE,5) |
+----------------------------------------------------------------------------------------------------------------------------------------------+
| ('Sedan','Hatch','Convertable','Van','Coupe','Light Truck','People Mover','SUV','Ute','Wagon','Cab Chassis','Sample Body','Body 4','BOdy 5') |
+----------------------------------------------------------------------------------------------------------------------------------------------+
This is my query
SELECT SUBSTRING(COLUMN_TYPE,5) FROM information_schema.COLUMNS WHERE TABLE_SCHEMA=DATABASE() AND TABLE_NAME='Ad_tbl' AND COLUMN_NAME='body_type'
I want the array to be ordered the other way.. So the Sedan would be in the last of the array instead of first.. Already Tried ORDER BY either ASC or DESC but no luck
You're querying the list of items in an ENUM definition, and you want to change the order? You can't do that without using ALTER TABLE to change your ENUM.
The order of items in an ENUM is related to the physical storage of the values. 'Sedan' is 1, 'Hatch' is 2, 'Convertable' (sic) is 3, etc. Changing the order of these strings requires changing the enumeration values.
Of course, you could change the order of displaying the strings in your application code. But this means parsing out the items from that list, splitting on comma, removing quotes and parens, etc.
But doing similar text-parsing in pure SQL will be an exercise in frustration, or at least, it'll be a huge waste of time.
This awkwardness of fetching the items in an ENUM definition is one of the reasons MySQL's ENUM data type is evil.
If you want to control the sort order without redefining the table,
you'll be better off using a lookup table instead of an ENUM.
There is a table with one column, named "info", with content like {"Twitter": 28, "Total": 28, "Facebook": 1}. When I write sql, I want to test whether "Total" is larger than 10 or not. Could someone help me write the query? (table name is landslides_7d)
(this is what I have)
SELECT * FROM landslides_7d WHERE info.Total > 10;
Thanks.
The data format seems to be JSON. If you have MySQL 5.7 you can use JSON_EXTRACT or the short form ->. Those functions don't exist in older versions.
SELECT * FROM landslides_7d WHERE JSON_EXTRACT(info, '$.total') > 10;
or
SELECT * FROM landslides_7d WHERE info->total > 10;
See http://dev.mysql.com/doc/refman/5.7/en/json-search-functions.html#function_json-extract
Mind that this is a full table scan. On a "larger" table you want to create an index.
If you're on an older version of MySQL you should create an extra column to your table and manually add the total value to that column.
You probably are storing the JSON in a single blob or string column. This is very inefficient, since you can't make use of indexes, and will need to parse the entire JSON structure on every where query. I'm not sure how much flexibility you need, but if the JSON attributes are relatively fixed, I recommend running a script (ruby, Python, etc.) on the table contents and storing "total" in a traditional columnar format. For example, you could add a new column "total" which contains the total attribute as an INT.
A side benefit of using a script is that you can catch any improperly formatted JSON - something you can't do in a single query.
You can also keep "total" column maintained with a trigger (on update/insert of "info"), using the JSON_EXTRACT function referenced in #johannes answer.
Trying to generate a unique code to prevent duplicates i use the following query using HAVING clause so i can use the alias but i get duplicate key errors:
SELECT
FLOOR(100 + RAND() * 899) AS random_code
FROM product_codes
HAVING random_code NOT IN (values)
LIMIT 1
The following code did not work and is what i need:
https://stackoverflow.com/a/4382586
Is there a better way to accomplish this or there is something wrong in my query?
If you want a unique code that is guaranteed to be unique, use the mySQL function UUID()
http://dev.mysql.com/doc/refman/5.0/en/miscellaneous-functions.html#function_uuid
"A UUID is designed as a number that is globally unique in space and time. Two calls to UUID() are expected to generate two different values, even if these calls are performed on two separate computers that are not connected to each other."
If a UUID is too long (e.g. it has to be exactly a certain number of digits), then hash the UUID (with md5 or sha-256 for example), take a certain number of bits and turn that into a decimal integer. The hashing is important since it's the whole UUID that guarantees uniqueness, not any one part of it. However, you will now be able to get hash collisions, which will be likely as soon as you have more than sqrt(2^bits) entries. E.g. if you use 10 bits for 0-1023, then after about 32 entries a hash collision becomes likely. If you use this few bits, consider an incrementing sequence instead.
Wanted to use a MYSQL query to get random numbers left between 0-999 as a code but tryed that query and also i ended filling the values condition from 0 to 999 and still got always duplicate codes, strange behaviour so i ended up using PHP.
The steps i use now:
Create an array populated with 0 to 999, in the future if i need more codes will use 0 to 9999.
$ary_1 = array(0, 1, 2, ...., 999)
Create other array populated with all codes in the table.
$ary_2 = array(4, 5, 985, 963, 589)
Get a resulting array using array_diff.
$ary_3 = array_diff($ary_1, $ary_2)
Get an array key using array_rand from the resulting array_diff.
$new_code_key = array_rand($ary_3, 1)
Use that key to get the code and create the MYSQL query.
$ary_3[$new_code_key]
Unset that key from the new codes array so i speed up the process and just have to get anoher array_rand key and unset later.
unset($ary_3[$new_code_key])