Extracting from Json in AWS Athena or Presto - json

My query below does not give me any result
WITH dataset AS (
SELECT responseelements FROM cloudtrail_logs
WHERE useridentity.type = 'Root'
AND eventname='CreateVpc'
ORDER BY eventsource, eventname;
AS blob
)
SELECT
json_extract(blob, '$.vpc.vpcId') AS name,
json_extract(blob, '$.ownerId') AS projects
FROM dataset
But if I run only the inner query
SELECT responseelements FROM cloudtrail_logs
WHERE useridentity.type = 'Root'
AND eventname='CreateVpc'
ORDER BY eventsource, eventname;
it gives me the correct response as a Json
{"requestId":"40aaffac-2c53-419e-a678-926decc48557","vpc":{"vpcId":"vpc-01eff2919c7c1da07","state":"pending","ownerId":"347612567792","cidrBlock":"10.0.0.0/26","cidrBlockAssociationSet":{"items":[{"cidrBlock":"10.0.0.0/26","associationId":"vpc-cidr-assoc-04136293a8ac73600","cidrBlockState":{"state":"associated"}}]},"ipv6CidrBlockAssociationSet":{},"dhcpOptionsId":"dopt-92df95e9","instanceTenancy":"default","tagSet":{},"isDefault":false}}
and if I pass this as data as below
WITH dataset AS (
SELECT '{"requestId":"40aaffac-2c53-419e-a678-926decc48557","vpc":{"vpcId":"vpc-01eff2919c7c1da07","state":"pending","ownerId":"347612567792","cidrBlock":"10.0.0.0/26","cidrBlockAssociationSet":{"items":[{"cidrBlock":"10.0.0.0/26","associationId":"vpc-cidr-assoc-04136293a8ac73600","cidrBlockState":{"state":"associated"}}]},"ipv6CidrBlockAssociationSet":{},"dhcpOptionsId":"dopt-92df95e9","instanceTenancy":"default","tagSet":{},"isDefault":false}}'
AS blob
)
SELECT
json_extract(blob, '$.vpc.vpcId') AS name,
json_extract(blob, '$.ownerId') AS projects
FROM dataset
it gives me result , what I am missing here ? So that I am able to make it run in one shot
Is it at all possible?

You're referencing the wrong column name in your query, it should be json_extract(responseelements, '$.vpc.vpcId') AS name instead of json_extract(blob, '$.vpc.vpcId') AS name. The AS blob part of this query does nothing since you can't alias an entire query, so take it out.
The AS blob works in your last example because you're selecting a value (the json string) into a column and the AS blob gives the column a name or alias of "blob". In your original query, you're selecting an existing column named responseelements so that's what you need to refer to in the json_extract function.

Related

How to use Oracle sql with json_object to generate json in NiFi

I am trying to use Oracle JSON_OBJECT to write a query to generate a specific json format.
Here is a sample query (the real query is more complex):
SELECT JSON_OBJECT('name' value name)
FROM table_a
WHERE name = 'John'
The query is working in Oracle. I used ExecuteSQL processor and put this query in it. It shows
illegal character in JSON_OBJECT('name' value name)
Any suggestions?
You most likely need to give the column an alias. Nifi probably doesn't like having spaces, single quotes, or parenthesis in the column name.
select json_object('name' value NAME) as json_with_name
from table_a
where NAME = 'John';

how to ORDER BY json object in MySQL

i'm trying to execute a query to give me 10 rows with the most biggest score ,column score in my table is a json object like :
{fa="7",en="7"}
how can i set my query to order by this json object ( it doesn't matter which of them (en or fa) used because they are always same )
Assuming your json is {"fa"="7","en"="7"} and assuming your json are in my_json_col column you could access using a -> operator and order by
SELECT *
from my_table
order by my_json_col->"fa"

Export DB2 select to CSV with headhers

I am trying to export DB2 select with headhers. But without any success, my actual code is:
db2 "EXPORT TO /tmp/result5.csv OF DEL MODIFIED BY NOCHARDEL
SELECT 1 as id, 'DEVICE_ID', 'USER_ID' from sysibm.sysdummy1
UNION ALL (SELECT 2 as id, DEVICE_ID, USER_ID FROM MOB_DEVICES) ORDER BY id"
which is not working (I suggest because USER_ID is INTEGER), when I change it for:
db2 "EXPORT TO /tmp/result5.csv OF DEL MODIFIED BY NOCHARDEL
SELECT 1 as id, 'DEVICE_ID', 'PUSH_ID' from sysibm.sysdummy1
UNION ALL (SELECT 2 as id, DEVICE_ID, PUSH_ID FROM MOB_DEVICES) ORDER BY id"
It works, DEVICE_ID and PUSH_ID are both VARCHAR.
MOB_DEVICE TABLE Any suggest how to solve this?
Thanks for advice.
DB2 will not export a CSV file with the headers, because the headers will be included as data. Normally, CSV file is for storage not viewing. If you want to view a file with its headers you have the following options:
Export to IXF file, but this file is not a flat file. You will need a spreadsheet to view it.
Export to a CSV file and include the headers by:
Select the columns names from the name, and then perform an extra step to add it to the file. You can use the describe command or perform a select on syscat.columns for this purpose, but this process is manual.
Perform a select union, in one part the data and in the other part the headers.
Perform a select and take the output to a file. Do not use export.
select * from myTable > myTable
Ignoring the EXPORT, thus just looking exclusively at the problematic UNION ALL query:
The DB2 SQL will want to conform the data of the mismatched data-types, into the numeric data-type; in this scenario, into the INTEGER data-type. Because conspicuously, the literal string value 'USER_ID' is not a valid representation of numeric value, that value can not be cast into an INTEGER value.
However, one can explicitly request to reverse that casting [whereby SQL wants to convert from string into numeric], to ensure that the SQL obeys the desired effect, to convert the INTEGER values from the column into VARCHAR values; i.e. explicit casting can ensure the data-types between the common columns of the UNION will be compatible, by forcing the values from the INTEGER column to match the data-type of the literal\constant character-string value of 'USER_ID':
with
mob_devices (DEVICE_ID, USER_ID, PUSH_ID) as
( values( varchar('dev', 1000 ), int( 1 ), varchar('pull', 1000) ) )
( SELECT 1 as id, 'DEVICE_ID', 'USER_ID'
from sysibm.sysdummy1
)
UNION ALL
( SELECT 2 as id, DEVICE_ID , cast( USER_ID as varchar(1000) )
FROM MOB_DEVICES
)
ORDER BY id

How can I get just the column names for a given mysql query?

I am creating a utility which lets users enter a SQL query for the purposes of importing data to my database.
The first step is to show a list of resulting fields so the user can route them to the destination fields.
When users import from MSSQL, I can use SET FMTONLY ON to fetch the list of output columns that the query would produce if ran (assuming the query is valid in the first place).
I haven't been able to find a way to do this for MySQL. EXPLAIN doesn't list the resulting fields.
Given the following query:
SELECT CONCATENATE(first_name, " ", last_name) AS name, age, foo
FROM customers
ORDER BY name ASC;
I ultimately need to get a list of output fields only, like this:
{ "name", "age", "foo" }
How can I do this in MySQL?
SET FMTONLY ON still requires you to get the column names and types manually, it just generates an empty result set.
For MySQL, add a WHERE FALSE somewhere
SELECT CONCATENATE(first_name, " ", last_name) AS name, age, foo
FROM customers
WHERE FALSE
ORDER BY name ASC;
You get this lovely execution plan
"id";"select_type";"table";"type";"possible_keys";"key";"key_len";"ref";"rows";"Extra"
"1";"SIMPLE";NULL;NULL;NULL;NULL;NULL;NULL;NULL;"Impossible WHERE"
Then parse the columns as you would set fmtonly on with MSSQL
For complex queries (nested, group by, limit-ed), wrap it in a subquery
select * from (
<your wonderful brilliant complex query>
) x where false
MSSQL would have complained if the inner query contains ORDER BY without TOP, MySQL is ok with it.
I think you need to look at the resultsetmetada. I carries the number of columns, column name, and a few more about the result set.
I think you're looking for DESC {table_name}

I need some help getting MySql to output some results using a subquery

I'm storing a list of numbers inside a table as a varchar(255) and want to use this list in another query's "IN() clause.
Here's what I mean:
Table Data:
CREATE TABLE IF NOT EXISTS `session_data` (
`visible_portf_ids` varchar(255) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
INSERT INTO `session_data` (`visible_portf_ids`) VALUES
('45,44,658,659,661,45,44,658,659,661')
I want to run a query like this to return a list of portfolio's "QUERY #1":
SELECT portfolio_hierarchy_id, account_id, name, leaf_node_portf_id
FROM portfolio_hierarchy
WHERE account_id = 1
AND leaf_node_portf_id IN
(
(SELECT visible_portf_ids
FROM session_data
WHERE username = 'ronedog')
)
ORDER BY name ASC
The result of the query above returns only 1 row, when there are a total of 3 that should have been returned.
If I run the subquery alone like this:
(SELECT visible_portf_ids
FROM session_data
WHERE username = 'ronedog')
it will return a list like this:
45,44,658,659,661,45,44,658,659,661
But, when I run Query #1 above, only one row of data, which is associated with the "visible_portf_ids" of "45" is returned.
If I replace the subquery with hard coded values like this:
SELECT portfolio_hierarchy_id, account_id, name, leaf_node_portf_id
FROM portfolio_hierarchy
WHERE account_id = 1
AND leaf_node_portf_id IN (45,44,658,659,661,45,44,658,659,661)
ORDER BY name ASC
then I get all 3 rows I'm expecting.
I'm guessing that MySql is returning the list as a string because its stored as a varchar() and so it stops processing after the first "visible_portf_ids" is found, which is "45", but I'm not really sure.
Anyone got any ideas how I can fix this?
Thanks in advance.
You should think about restructuring your tables storing each value in a new row, instead of concatenating them.
Until then, you can use the FIND_IN_SET() function:
AND FIND_IN_SET(leaf_node_portf_id,
(SELECT visible_portf_ids
FROM session_data
WHERE username = 'ronedog'
LIMIT 1)
) > 0
Unfortunately MySQL does not have a function to split a delimited string. Your IN argument is a single string with the result of your subquery. The reason it works when you hard-code it is that MySQL is parsing the values.
I suggest that you redesign your data base to store the visible ports list as separate rows in a separate table. Then you can retrieve them and use them in subqueries like you tried.