JSON_CONTAINS() in postgresql - mysql

Using MySQL 5.7, I want to know the PostgreSQL equivalent of
SELECT * FROM user_conversations WHERE JSON_CONTAINS(users, JSON_ARRAY(1))
How do you write JSON_CONTAINS(users, JSON_ARRAY(1)) in PostgreSQL
EDIT 1
there is my json it's just an array without son object :
[
"john",
"doe",
"does"
]
i want to get "doe" for exemple
EDIT 2
My table :
Column | Type | Modifiers | Storage | Statistics Target | Description
------------+--------------------------------+-------------------------------------------------------+----------+-----------------------+-------------
id | uuid | non NULL | plain | |
name | character varying(255) | non NULL | extended | |
image | character varying(255) | non NULL Par défaut, 'default.png'::character varying | extended | |
users | json | non NULL | extended | |
type | integer | non NULL | plain | |
emoji_id | integer | non NULL Par défaut, 0 | plain | |
created_at | timestamp(0) without time zone | | plain | |
updated_at | timestamp(0) without time zone |
EDIT 3:
I use laravel to execute queries :
DB::table('user_conversations')->whereRaw('JSON_CONTAINS(users, JSON_ARRAY(1))')->orderBy('created_at', 'desc')->paginate(5);
and it's work with mysql.

The two argument form of MySQL's JSON_CONTAINS() you cite has a signature of JSON_CONTAINS(json_doc, val) That sounds similar to the PostgreSQL JSON operator #> operator
-- returns true
SELECT '["john","bob","doe","dylan","mike","does"]'::jsonb #>
'["john","doe","does"]';
-- returns false
SELECT '["john","bob","doe","dylan","mike","does"]'::jsonb #>
'["john","mary"]';
If your type is json, that's fine just cast it to jsonb;
SELECT *
FROM user_conversations
WHERE users::jsonb #> json_text;

Related

Select where json contains

I have a postgres table:
Column | Type
------------------+------------------------+
id | bigint |
foo_id | integer |
foo_name | character varying(255) |
bar_id | json |
With values
id | foo_id | foo_name | bar_id
--------+----------------+-------------------------------+----------------------------
17 | 717639 | name 1 | [910, 5383774, 437, 5350]
18 | 8442028 | name 2 | [7274, 6241861, 437, 73669939]
19 | 77415 | name n | [7274, 5513, 249642574, 743181502]
How can I select from this table records ids where bar_id contains value 437?
You can use the operator #> to check if the array contains a specific value.
However that only works with jsonb values, not json values. So you need to cast your column.
select *
from the_table
where bar_id:jsonb #> '[437]';
It would be better to change the column's type to jsonb.

Querying a json field in mysql using JOOQ

I have a transaction table and it has a json type field called "request".
| Field | Type | Null | Key | Default |
| id | bigint | NO | PRI | NULL |
| request | json | NO | | NULL |
| response | json | YES | | NULL |
request has two attributes currencyCode and amount.
{"amount":100000,"currencyCode":"PHP"}
I can use following mysql query to fetch these values
select json_extract(request, "$.amount") as amount, json_extract(request, "$.currencyCode") as currency from transaction;
| amount | currency |
+--------+----------+
| 100000 | PHP |
| 100000 | PHP |
| 100000 | PHP |
I want to get these values using a jooq query something like this.
DSL.select(<Tables.TRANSACTION.REQUEST.amount>, <Tables.TRANSACTION.REQUEST.currencyCode>)
.from(Tables.TRANSACTION)
.fetch()
I really appreciate if someone can help me with this.
Using jOOQ 3.14's JSON_VALUE support
Starting with jOOQ 3.14, you will be able to use the new built-in standard JSON operator support, e.g. JSON_VALUE(). As per the docs:
This example using jOOQ:
jsonValue(val(JSON.json("[1,2]")), "$[*]")
Translates to the following dialect specific expressions:
...
-- MYSQL
json_extract('[1,2]', '$[*]')
Using plain SQL templating in jOOQ 3.13 and earlier
Whenever jOOQ doesn't support vendor specific functionality out of the box, you can resort to using plain SQL templating. Just write:
public static Field<String> jsonExtract(Field<?> field, String jsonPath) {
return DSL.field("json_extract({0}, {1})", String.class, field, DSL.inline(jsonPath));
}

How should I construct a database to store a lot of SHA1 data

I'm having trouble constructing a database to store a lot of SHA1 data and efficiently return results.
I will admit SQL is not my strongest skill but as an exercise I am trying to use the data from https://haveibeenpwned.com/Passwords which returns results pretty quickly
This is my data:
mysql> describe pwnd;
+----------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------+------------------+------+-----+---------+----------------+
| id | int(10) unsigned | NO | PRI | NULL | auto_increment |
| pwndpass | binary(20) | NO | | NULL | |
+----------+------------------+------+-----+---------+----------------+
mysql> select id, hex(pwndpass) from pwnd order by id desc limit 10;
+-----------+------------------------------------------+
| id | hex(pwndpass) |
+-----------+------------------------------------------+
| 306259512 | FFFFFFFEE791CBAC0F6305CAF0CEE06BBE131160 |
| 306259511 | FFFFFFF8A0382AA9C8D9536EFBA77F261815334D |
| 306259510 | FFFFFFF1A63ACC70BEA924C5DBABEE4B9B18C82D |
| 306259509 | FFFFFFE3C3C05FCB0B211FD0C23404F75E397E8F |
| 306259508 | FFFFFFD691D669D3364161E05538A6E81E80B7A3 |
| 306259507 | FFFFFFCC6BD39537AB7398B59CEC917C66A496EB |
| 306259506 | FFFFFFBFAD0B653BDAC698485C6D105F3C3682B2 |
| 306259505 | FFFFFFBBFC923A29A3B4931B63684CAAE48EAC4F |
| 306259504 | FFFFFFB58E389A0FB9A27D153798956187B1B786 |
| 306259503 | FFFFFFB54953F45EA030FF13619B930C96A9C0E3 |
+-----------+------------------------------------------+
10 rows in set (0.01 sec)
My question relates to quickly finding entries as it currently takes over 6 minutes
mysql> select hex(pwndpass) from pwnd where hex(pwndpass) = '0000000A1D4B746FAA3FD526FF6D5BC8052FDB38';
+------------------------------------------+
| hex(pwndpass) |
+------------------------------------------+
| 0000000A1D4B746FAA3FD526FF6D5BC8052FDB38 |
+------------------------------------------+
1 row in set (6 min 31.82 sec)
Do I have the correct data types? I search for storing sha1 data and a Binary(20) field is advised but not sure how to optimising it for searching the data.
My MySQL install is a clean turnkey VM https://www.turnkeylinux.org/mysql I have not adjusted any settings other than giving the VM more disk space
The two most obvious tips are:
Create an index on the column.
Don't convert every single row to hexadecimal on every search:
select hex(pwndpass)
from pwnd
where hex(pwndpass) = '0000000A1D4B746FAA3FD526FF6D5BC8052FDB38';
-- ^^^ This is forcing MySQL to convert every hash stored from binary to hexadecimal
-- so it can determine whether there's a match
In fact, you don't even need hexadecimal at all, save for display purposes:
select id, hex(pwndpass) -- This is fine, will just convert matching rows
from pwnd
where pwndpass = ?
... where ? is a placeholder that, in your client language, corresponds to a binary string.
If you need to run the query right in command-line, you can also use an hexadecimal literal:
select id, hex(pwndpass) -- This is fine, will just convert matching rows
from pwnd
where pwndpass = 0x0000000A1D4B746FAA3FD526FF6D5BC8052FDB38

Filter rows by JSON column that stores array of objects

I have the following table which its called products I omitted columns that I won't need:
+---------------------+----------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------------------+----------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| name | varchar(255) | NO | | NULL | |
| custom_fields | json | YES | | NULL | |
| parent_variation_id | int(11) | YES | MUL | NULL | |
| variation_fields | json | YES | | NULL | |
+---------------------+----------------+------+-----+---------+----------------+
I have two JSON columns which I want to use in the same query to filter products and its variations. Both columns have the same structure which is something like this to store custom fields a product might have:
[
{"name": "External use", "type": "checkbox", "value":"true"},
{"name": "Handmade", "type": "checkbox", "value":"true"},
....
]
The important attributes for the query to filter are name and value which is the name of the field and the value associated to that specific product respectively, in the example above we have a product which is handmade that can be used externally.
If the user wants to filter products he might send params like {"External use": "false", "Handmade":"true"} but right now I can only filter on one attribute, currently value so if I pass true it will bring all products that have value set to true in any field.
SELECT *
FROM `products`
WHERE json_search(custom_fields, 'all', 'true', NULL, '$[*].value') IS NOT NULL
I would like to know if its possible to apply an AND condition or something alike for the same JSON object inside these array of objects, so the param I pass is related to the field I need.
Yes you could. But to get performance out of such data I suggest creating a generated column (or two) and indexing it for faster queries.

Defining a webservice for usage analytics (dekstop application)

Current situation
I have a desktop application (C++ Win32), and I wish to track users' usage analytics anonymously (actions, clicks, usage time, etc.)
The tracking is done via designated web services for specific actions (install, uninstall, click) and everything is written by my team and stored on our DB.
The need
Now we're adding more usage types and events with a variety of data, so we need define the services.
Instead of having tons of different web services for each action, I want to have a single generic service for all usage types, that is capable of receiving different data types.
For example:
"button_A_click" event, has data with 1 field: {window_name (string)}
"show_notification" event, has data with 3 fields: {source_id (int), user_action (int), index (int)}
Question
I'm looking for an elegant & convenient way to store this sort of diverse data, so later I could query it easily.
The alternatives I can think of:
Storing the different data for each usage type as one field of JSON/XML object, but it would be extremely hard to pull data and write queries for those fields
Having extra N data fields for each record, but it seems very wasteful.
Any ideas for this sort of model? Maybe something like google analytics? please Advise...
Technical: The DB is MySQL running under phpMyAdmin.
Disclaimer:
There is a similar post, which brought to my attention services like DeskMetrics and Tracker bird, or try to embed google analytics to C++ native application, but I'd rather the service to by my own, and better understand how to design this sort of model.
Thanks!
This seems like a database normalization problem.
I am also going to assume that you also have a table named events where all events will be stored.
Additionally, I am going to assume you have to the following data attributes (for simplicity's sake): window_name, source_id, user_action, index
To achieve normalization, we will need the following tables:
events
data_attributes
attribute_types
This is how each of the tables should be structured:
mysql> describe events;
+------------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------+------------------+------+-----+---------+----------------+
| id | int(11) unsigned | NO | PRI | NULL | auto_increment |
| event_type | varchar(255) | YES | | NULL | |
+------------+------------------+------+-----+---------+----------------+
mysql> describe data_attributes;
+-----------------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------------+------------------+------+-----+---------+----------------+
| id | int(11) unsigned | NO | PRI | NULL | auto_increment |
| event_id | int(11) | YES | | NULL | |
| attribute_type | int(11) | YES | | NULL | |
| attribute_name | varchar(255) | YES | | NULL | |
| attribute_value | int(11) | YES | | NULL | |
+-----------------+------------------+------+-----+---------+----------------+
mysql> describe attribute_types;
+-------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------+------------------+------+-----+---------+----------------+
| id | int(11) unsigned | NO | PRI | NULL | auto_increment |
| type | varchar(255) | YES | | NULL | |
+-------+------------------+------+-----+---------+----------------+
The idea is that you will have to populate attribute_types with all possible types you can have. Then, for each new event, you will add an entry in the events table and corresponding entries in the data_attributes table to map that event to one or more attribute types with the appropriate values.
Example:
"button_A_click" event, has data with 1 field: {window_name "Dummy Window Name"}
"show_notification" event, has data with 3 fields: {source_id: 99, user_action: 44, index: 78}
would be represented as:
mysql> select * from attribute_types;
+----+-------------+
| id | type |
+----+-------------+
| 1 | window_name |
| 2 | source_id |
| 3 | user_action |
| 4 | index |
+----+-------------+
mysql> select * from events;
+----+-------------------+
| id | event_type |
+----+-------------------+
| 1 | button_A_click |
| 2 | show_notification |
+----+-------------------+
mysql> select * from data_attributes;
+----+----------+----------------+-------------------+-----------------+
| id | event_id | attribute_type | attribute_name | attribute_value |
+----+----------+----------------+-------------------+-----------------+
| 1 | 1 | 1 | Dummy Window Name | NULL |
| 2 | 2 | 2 | NULL | 99 |
| 3 | 2 | 3 | NULL | 44 |
| 4 | 2 | 4 | NULL | 78 |
+----+----------+----------------+-------------------+-----------------+
To write a query for this data, you can use the COALESCE function in MySQL to get the value for you without having to check which of the columns is NULL.
Here's a quick example I hacked up:
SELECT events.event_type as `event_type`,
attribute_types.type as `attribute_type`,
COALESCE(data_attributes.attribute_name, data_attributes.attribute_value) as `value`
FROM data_attributes,
events,
attribute_types
WHERE data_attributes.event_id = events.id
AND data_attributes.attribute_type = attribute_types.id
Which yields the following output:
+-------------------+----------------+-------------------+
| event_type | attribute_type | value |
+-------------------+----------------+-------------------+
| button_A_click | window_name | Dummy Window Name |
| show_notification | source_id | 99 |
| show_notification | user_action | 44 |
| show_notification | index | 78 |
+-------------------+----------------+-------------------+
EDIT: Bugger! I read C#, but I see you are using C++. Sorry about that. I leave the answer as-is as its principle could still be useful. Please regard the examples as pseudo-code.
You can define a custom class/structure that you use with an array. Then serialize this data and send to the WebService. For example:
[Serializable()]
public class ActionDefinition {
public string ID;
public ActionType Action; // define an Enum with possible actions
public List[] Fields; //Or a list of 'some class' if you need more complex fields
}
List AnalyticsCollection = new List(Of, Actiondefinition);
// ...
SendToWS(Serialize(AnalyticsCollection));
Now you can dynamically add as many events as you want with the needed flexibility.
on server side you can simply parse the data:
List[of, ActionDefinition] AnalyticsCollection = Deserialize(GetWS());
foreach (ActionDefinition ad in AnalyticsCollection) {
switch (ad.Action) {
//.. check for each action type
}
}
I would suggest adding security mechanisms such as checksum. I imagine the de/serializer would be pretty custom in C++ so perhaps as simple Base64 encoding can do the trick, and it can be transported as ascii text.
You could make a table for each event in wich you declare what param means what. Then you have a main table in wich you only input the events name and param1 etc. An admin tool would be very easy, you go through all events, and describe them using the table where each event is declared. E.g. for your event button_A_click you insert into the description table:
Name Param1
button_A_Click WindowTitle
So you can group your events or select only one event ..
This is how I would solve it.