Parsing json data in Mysql - mysql

I am using Mysql version as follows:
mysql> SHOW VARIABLES LIKE "%version%"
->
-> ;
+-------------------------+------------------------------+
| Variable_name | Value |
+-------------------------+------------------------------+
| innodb_version | 5.6.26 |
| protocol_version | 10 |
| slave_type_conversions | |
| version | 5.6.26 |
| version_comment | MySQL Community Server (GPL) |
| version_compile_machine | x86_64 |
| version_compile_os | osx10.8 |
+-------------------------+------------------------------+
7 rows in set (0.00 sec)
I have a table named tag and inside it, I have a text column namedtagconfig. This column has JSON data as stored as string as the following:
{ "option":{ "secure":"0" , "onetime":"0" , "rtmp":"1" , "hls":"0" , "http":"0" } , "video":[{ "format":"mp4" , "codec":"h264" , "bitrate":"800" , "width":"" , "height":"" , "fps":"" , "audiocodec":"mp3" , "audiobitrate":"96" }] }
I want to find the number of elements inside the video array in the json data.
I tried using common_schema like the following:
SELECT common_schema.get_option(be.tagconfig,'option') as t
from tag be LIMIT 200;
However, I get all the values as null. Preferable I want to create another column which has the number of elements in the video array in the json data. Is there a direct way in mysql to do this?

common_schema has another function called extract_json_value function that can achieve exactly what you need here. It it better used from complex json object like the one you have. Under the hood it uses the json_to_xml function to convert the json to xml format and allows the use of xpaths to traverse through the object
SELECT common_schema.extract_json_value(be.tagconfig, 'count(//video)') as COUNT
the above query should get you what you need

Related

Querying a json field in mysql using JOOQ

I have a transaction table and it has a json type field called "request".
| Field | Type | Null | Key | Default |
| id | bigint | NO | PRI | NULL |
| request | json | NO | | NULL |
| response | json | YES | | NULL |
request has two attributes currencyCode and amount.
{"amount":100000,"currencyCode":"PHP"}
I can use following mysql query to fetch these values
select json_extract(request, "$.amount") as amount, json_extract(request, "$.currencyCode") as currency from transaction;
| amount | currency |
+--------+----------+
| 100000 | PHP |
| 100000 | PHP |
| 100000 | PHP |
I want to get these values using a jooq query something like this.
DSL.select(<Tables.TRANSACTION.REQUEST.amount>, <Tables.TRANSACTION.REQUEST.currencyCode>)
.from(Tables.TRANSACTION)
.fetch()
I really appreciate if someone can help me with this.
Using jOOQ 3.14's JSON_VALUE support
Starting with jOOQ 3.14, you will be able to use the new built-in standard JSON operator support, e.g. JSON_VALUE(). As per the docs:
This example using jOOQ:
jsonValue(val(JSON.json("[1,2]")), "$[*]")
Translates to the following dialect specific expressions:
...
-- MYSQL
json_extract('[1,2]', '$[*]')
Using plain SQL templating in jOOQ 3.13 and earlier
Whenever jOOQ doesn't support vendor specific functionality out of the box, you can resort to using plain SQL templating. Just write:
public static Field<String> jsonExtract(Field<?> field, String jsonPath) {
return DSL.field("json_extract({0}, {1})", String.class, field, DSL.inline(jsonPath));
}

Can I query hbase tables in a namespace using Apache Drill?

I see the tables in hbase when I say show tables;
jdbc:drill:zk=<zkhost>> USE hbase;
+-------+------------------------------------+
| ok | summary |
+-------+------------------------------------+
| true | Default schema changed to [hbase] |
+-------+------------------------------------+
ns is the namespace
table1,2,3 are tables inside the namespace.
: jdbc:drill:zk=<zkhost>> show tables;
+---------------+----------------------------------+
| TABLE_SCHEMA | TABLE_NAME |
+---------------+----------------------------------+
| hbase | ATLAS_ENTITY_AUDIT_EVENTS |
| hbase | ns:table1 |
| hbase | ns:table2 |
| hbase | ns:table3 |
| hbase | atlas_janus |
+---------------+----------------------------------+
6 rows selected (30.111 seconds)
But when I try to query the table, drill doesnt identify the tables inside the namespace.
0: jdbc:drill:zk=vginthw496> select * from hbase.table2 limit 10;
Error: VALIDATION ERROR: From line 1, column 15 to line 1, column 19: Object 'table2' not found within 'hbase'
[Error Id: 02353729-39b2-4894-a21f-b6662a2a7a3c on <hostname>:31010] (state=,code=0)
How do I query hbase tables inside a namespace using apache drill?
using ``, like this:
select * from `namespace:table` limit 1;

mysql select count(*) results in Data too long for column

I am running a count(*) on a view, but I get the following error. The view has several joins and views. But this error should only apply when changing a value on a table field, right?
mysql> select count(*) from dm.vSscore24 ;
ERROR 1406 (22001): Data too long for column 'pbs' at row 1
mysql>
db info
mysql> show variables like '%version%';
+-------------------------+-----------------------------+
| Variable_name | Value |
+-------------------------+-----------------------------+
| innodb_version | 5.7.20 |
| protocol_version | 10 |
| slave_type_conversions | |
| tls_version | TLSv1,TLSv1.1 |
| version | 5.7.20-0ubuntu0.16.04.1-log |
| version_comment | (Ubuntu) |
| version_compile_machine | x86_64 |
| version_compile_os | Linux |
+-------------------------+-----------------------------+
thank you.
found the issue.
It was a function called inside a view that had a variable (named pbshape) too small for the maximum size of the column.
Luckily there were no columns named 'pbs' anywhere (in tables or views), and that made me wonder.
If the variable were named like bs I think I would never have found the answer :P
The word column in the error message 'Data too long for column' is extremely missleading.
thank you guys

Disable scientific notation in MySQL command-line client?

I have a MySQL table with many numeric columns (some INT, some FLOAT). I would like to query it with the MySQL command-line client (specifically, mysql Ver 14.14 Distrib 5.1.41, for debian-linux-gnu (x86_64) using readline 6.1), like so:
SELECT * FROM table WHERE foo;
Unfortunately, if the value of any numeric field exceeds 10^6, this client displays the result in scientific notation, which makes reading the results difficult.
I could correct the problem by FORMAT-ing each of the fields in my query, but there are many of them and many tables I would like to query. Instead I'm hoping to find a client variable or flag I can set to disable scientific notation for all queries.
I have not been able to find one in the --help or the man page, nor searching Google or this site. Instead all I find are discussions of preserving/removing scientific notation when using <insert-programming-language>'s MySQL API.
Thank you for any tips.
::edit::
Here's an example table ...
mysql> desc foo;
+--------------+-------------+------+-----+-------------------+
| Field | Type | Null | Key | Default |
+--------------+-------------+------+-----+-------------------+
| date | date | NO | PRI | NULL |
| name | varchar(20) | NO | PRI | NULL |
| val | float | NO | | NULL |
| last_updated | timestamp | NO | | CURRENT_TIMESTAMP |
+--------------+-------------+------+-----+-------------------+
and some example values ...
mysql> select * from foo where date='20120207';
+------------+--------+--------------+---------------------+
| date | name | val | last_updated |
+------------+--------+--------------+---------------------+
| 2012-02-07 | A | 88779.5 | 2012-02-07 13:38:14 |
| 2012-02-07 | B | 1.00254e+06 | 2012-02-07 13:38:14 |
| 2012-02-07 | C | 78706.5 | 2012-02-07 13:38:15 |
+------------+--------+--------------+---------------------+
Now, the actual values I loaded into the third field are:
88779.5, 1002539.25, 78706.5390625
and they can be seen exactly if I manipulate the value:
mysql> select date, name, ROUND(val, 10), last_updated from foo where ...
+------------+---+--------------------+---------------------+
| 2012-02-07 | A | 88779.5000000000 | 2012-02-07 13:38:14 |
| 2012-02-07 | B | 1002539.2500000000 | 2012-02-07 13:38:14 |
| 2012-02-07 | C | 78706.5390625000 | 2012-02-07 13:38:15 |
Something in the client seems to be enforcing that I only be allowed to see six significant figures, even though there are more in the table.
If a query such as
mysql> select ROUND(*, 2) from foo ...
were possible, that would be great! Otherwise I can't really take the time to individually wrap 100 column names in "ROUND()" whenever I need to inspect some data.
Interestingly, I occasionally use a phpMyAdmin interface to browse the contents of some of these tables, and that interface also has this 6 significant figure limitation. So it's not limited to just the CLI.
Well, after reading the documentation more thoroughly, I still can't see any reason why a client would limit itself to displaying only 6 sig figs from a FLOAT (especially when the table itself is definitely storing more).
Nonetheless, an acceptable solution (for this weary user) is to change all my tables to use DECIMAL(16,4) instead of FLOAT. Unfortunately, this makes all my numbers show up with 4 decimal places (even if they're all '0'). But at least all numbers have the same width now, and my client never displays them in scientific notation or limits the number of sig figs in its output.
Wouldn't the CAST function allow you to request that the values for a certain field are returned as DECIMAL ? Not an expert and haven't tried it, but that would be the first thing I try.
I know this is old but this helped me.. I used a view..
create view foo2 as select date, name, ROUND(val, 10) val, last_updated from foo
Then just do your queries on foo2. also works in phpmyadmin

MySQL: lower does not exist

I am trying to run a query like this:
SELECT lower (A.label) FROM addresses A;
and what I get in return is
ERROR 1305 (42000): FUNCTION db.lower does not exist
I am using
+-------------------------+-------------------------------+
| Variable_name | Value |
+-------------------------+-------------------------------+
| protocol_version | 10 |
| version | 5.0.92-community |
| version_comment | MySQL Community Edition (GPL) |
| version_compile_machine | x86_64 |
| version_compile_os | unknown-linux-gnu |
+-------------------------+-------------------------------+
Never had this issue before on any other system but this one... I was under the impression that lower is an internal function and is always there. What can be wrong?
Delete the space between lower and (
SELECT lower(A.label) FROM addresses A;
lower (A.label)
should be
lower(A.label)
MySQL doesn't like spaces between function name and argument: you are implying a column alias
(A.label) for a column lower otherwise
lower `(A.label)`
lower AS `(A.label)`