Need some help with the MySQL query.
I have a table named custom_fields with attributes id, user_id, name, value etc.
name and values are of type string and it can store sometimes JSON also in the form of a string.
Requirement:
I need to list out all the unique user_id which satisfies below conditions
1) name must be starting with street_address_
2) value is a hash but stored as a string. if the country code is in ('us', 'in')
Here is the sample record.
id: 90489,
user_id: 30207,
name: "street_address_billing",
value:"{\"street_address1\":\"401 Lenora Street\",\"street_address2\":\"\",\"city\":\"Belltown\",\"locality\":\"Seattlel\",\"province\":\"WA\",\"postal_code\":\"111\",\"country_code\":\"us\"}",
deleted_at: nil,
active: true
Here is the query I'm trying. But it is not working. I have no error but it does not give any results either. instead of in if i use = and single value sometimes it gives the value.
SELECT id,user_id, #addr:= replace(replace(replace(replace(value, ':“', ':"'), '”,', '",'), '”}', '"}'), '{“' , '{"'), JSON_EXTRACT(#addr, '$.country_code') as billing_country
FROM `custom_fields` WHERE `custom_fields`.`active` = TRUE AND (name REGEXP '^street_address_')
AND JSON_EXTRACT(#addr, '$.country_code') in ('us', 'in');s
Despite the fact that SELECT syntax puts the select-list above the WHERE clause, WHERE conditions are evaluated first, to restrict the rows returned by the query. Then for those rows only, expressions in the select-list are evaluated. So your variables defined in the select-list cannot be used in the WHERE clause.
So you need to reference the column itself in your WHERE conditions:
SELECT id,user_id, JSON_UNQUOTE(JSON_EXTRACT(value, '$.country_code')) AS billing_country
FROM `custom_fields` WHERE `custom_fields`.`active` = TRUE AND (name REGEXP '^street_address_')
AND JSON_UNQUOTE(JSON_EXTRACT(value, '$.country_code')) IN ('us', 'in');
Also you need to use JSON_UNQUOTE() because the return value of JSON_EXTRACT() is not a plain string — it's a JSON document that happens to be a scalar. So it will return "us".
I've removed all the replace() stuff because you shouldn't need it. If you use JSON data, you should store it in a JSON column, and that will reject invalid JSON that contains smart-quote characters.
Related
I am storing event data in S3 and want to use Athena to query the data. One of the fields is a dynamic JSON field that I do not know the field names for. Therefore, I need to query the keys in the JSON and then use those keys to query for the first non-null for that field. Below is an example of the data stored in S3.
{
timestamp: 1558475434,
request_id: "83e21b28-7c12-11e9-8f9e-2a86e4085a59",
user_id: "example_user_id_1",
traits: {
this: "is",
dynamic: "json",
as: ["defined","by","the", "client"]
}
}
So, I need a query to extract the keys from the traits column (which is stored as JSON), and use those keys to get the first non-null value for each field.
The closest I could come was sampling a value using min_by, but this does not allow for me to add a where clause without returning null values. I will need to use presto's "first_value" option, but I cannot get this to work with the extracted JSON keys from the dynamic JSON field.
SELECT DISTINCT trait, min_by(json_extract(traits, concat('$.', cast(trait AS varchar))), received_at) AS value
FROM TABLE
CROSS JOIN UNNEST(regexp_extract_all(traits,'"([^"]+)"\s*:\s*("[^"]+"|[^,{}]+)', 1)) AS t(trait)
WHERE json_extract(traits, concat('$.', cast(trait AS varchar))) IS NOT NULL OR json_size(traits, concat('$.', cast(trait AS varchar))) <> 0
GROUP BY trait
It's not clear to me what you expect as result, and what you mean by "first non-null value". In your example you have both string and array values, and none of them is null. It would be helpful if you provided more examples and also expected output.
As a first step towards a solution, here's a way to filter out the null values from traits:
If you set the type of the traits column to map<string,string> you should be able to do something like this:
SELECT
request_id,
MAP_AGG(ARRAY_AGG(trait_key), ARRAY_AGG(trait_value)) AS trait
FROM (
SELECT
request_id,
trait_key,
trait_value
FROM some_table CROSS JOIN UNNEST (trait) AS t (trait_key, trait_value)
WHERE trait_value IS NOT NULL
)
However, if you want to also filter values that are arrays and pick out the first non-null value, that becomes more complex. It could probably be done with a combination of casts to JSON, the filter function, and COALESCE.
Got this QueryBuilder in one of my repositories.
$query = $em->createQueryBuilder('d')
->select('d, i, u, SUM(CASE WHEN t.user = :userId THEN 1 ELSE 0 END) as myTickets')
->leftJoin('d.item','i')
->leftJoin('d.users','u')
->leftJoin('d.tickets','t')
->where('d.active = 1')
->andWhere('d.state = 1')
->setParameter('userId',$user->getId())
->orderBy('d.dateFinish', 'ASC');
When i execute the code, MySQL throws me this error.
Key "premium" for array with keys "0, myTickets" does not exist
"premium" is a field of "d".
How can i recive the fields with the custom SUM?
Since you're using aggregate function in your query, you get the so called mixed result. Mixed result normally return your object fetched with your FROM clause as zero index [0]. The rest of your result is populated based on the aliases you set for your custom fields.
$result[0] will return the object you want to access.
$result['myTickets'] will return the result of your aggregate function. In this case, it's a SUM.
A quote from documentation:
SELECT u, UPPER(u.name) nameUpper FROM MyProject\Model\User u
This query makes use of the UPPER DQL function that returns a scalar value and because there is now a scalar value in the SELECT clause, we get a mixed result.
Conventions for mixed results are as follows:
The object fetched in the FROM clause is always positioned with the key ‘0’.
Every scalar without a name is numbered in the order given in the query, starting with 1.
Every aliased scalar is given with its alias-name as the key. The case of the name is kept.
If several objects are fetched from the FROM clause they alternate every row.
You can read more about this topic here.
Let's say I have a json column fields, like so:
{phone: 5555555555, address: "55 awesome street", hair_color: "green"}
What I would like to do is update all entries where the json key phone is present, and the result is of type number to be a string.
What I have is:
SELECT *
FROM parent_object
WHERE (fields->'phone') IS NOT NULL;
Unfortunately this still returns values where phone:null. I'm guessing that a JSON null is not equivalent to a SQL NULL.
How do I
1) How do I rule out JSON nulls
AND (fields->'phone') <> null produces
LINE 4: ...phone') IS NOT NULL AND (fields->'phone') <> 'null';
HINT: No operator matches the given name and argument type(s). You might need to add explicit type casts.
2) Check the type of the value at that key, this pseudocode (type_of (fields->'phone') == Integer) but in working PGSQL.
3) Modify this to update the column
UPDATE parent_object
SET fields.phone = to_char(fields.phone)
WHERE query defined above
As other folks have said, there is no reason to convert the variable to an integer just to them cast it to a string. Also, phone numbers are not numbers. :-)
You need to be using the ->> operator instead of ->. That alongside IS NOT NULL gets your SELECT query working.
Note the difference between the two tuple values after running this query:
SELECT fields->'phone', fields->>'phone'
FROM parent_object;
Your working query:
SELECT *
FROM parent_object
WHERE (fields->>'phone') IS NOT NULL;
Postgres does not currently natively support atomically updating individual keys within a JSON column. You can write wrapper UDFs to provide this capability to you: How do I modify fields inside the new PostgreSQL JSON datatype?
For checking the type of the value at key, postgres has the following in the documentation.
json_typeof ( json ) → text
jsonb_typeof ( jsonb ) → text
Returns the type of the top-level JSON value as a text string. Possible types are object, array, string, number, boolean, and null. (The null result should not be confused with a SQL NULL; see the examples.)
json_typeof('-123.4') → number
json_typeof('null'::json) → null
json_typeof(NULL::json) IS NULL → t
What I'm Using: The most recent MySQL on Ubuntu 12.
The Set Up: Suppose I have a table "EmployeePayment" with "Name" and "Hours" for each employee. Suppose I already have it populated with values.
The Question: When I use the command
select * from EmployeePayment where Name in ('');
I get the empty set, as I'd expect. But, when I use
select * from EmployeePayment where Name in ('' or '');
I get the entire table returned. Moreover, if I'm picky and put in the command
select Name, SUM(Hours) from EmployeePayment where Name in ('' or '');
then it only returns whatever is the top name from the table. What's happening with this "in" command?
First off, you need to get rid of the or, the proper syntax for the in clause uses commas to separate the possibilities, such as:
sql> select name from people where status in ('intelligent', 'good looking')
pax
1 row returned
What your current variant is doing is applying the or operator to give you a one-element in-list. See here for more detail.
The reason why you're only getting one row for the aggregated query is because you have no group by clause, so you're grouping all rows. Most DBMS' would then complain about having a non-aggregated column that isn't part of the grouping, but MySQL is a bit fancy-free and footloose with the rules in that regard.
It's obviously grouping over the whole table (as it should) but applying some default aggregating function to the name (which it probably shouldn't, but does according to its documentation).
This MySQL extension is covered here but heed the warning: MySQL can choose any of the myriad possible values for these non-aggregated, non-group-by columns, so it's more useful when you know that all the rows in a given group share the same value for the column.
You're effectively doing this:
select * from EmployeePayment where Name in (0);
The OR expression evaluates to 0, and WHERE Name IN (0); returns all rows. You have to use the proper IN syntax as suggested in the other answers:
SELECT * FROM EmployeePayment WHERE Name IN ('foo', 'bar');
IN uses comma separated values, for example: WHERE Name IN ('tim','beth')
So try WHERE Name IN ('','');
But more importantly, why would you want to check where a value is empty or empty? Or was that just to get the question across?
http://dev.mysql.com/doc/refman/5.0/en/comparison-operators.html#function_in
I had a query like
SELECT name, town FROM clients WHERE course = 1;
and it gave me some results.
But i noticed the query didn't output the latest entries. I look into the table clients with phpmyadmin and i saw more course fields with value 1.
After i changed the query to
SELECT name, town FROM clients WHERE course = '1';
i got the right output.
My question: why is this? and why did i get some results with the first query ?
ps: course is an enum field ( '0','1').
1 is either an integer value, or 'TRUE', or ... (ambiguous) while '1' is a string value.
As explained in the MySQL-documentation for the ENUM-type:
An ENUM is a string object with a value chosen from a list of
permitted values that are enumerated explicitly in the column
specification at table creation time.
[...]
If you wish to use a number as an enumeration value, you must enclose
it in quotation marks. If the quotation marks are omitted, the number
is regarded as an index. For this and other reasons—as explained later
in this section—we strongly recommend that you do not use numbers as
enumeration values.
So, ENUM is a string object, therefore you need to reference to it as a string.