Suppose I have a table with three columns: id, name, state. The state column is of type jsonb and the structure would always have (as a minimum) a key called active e.g.
{"name": "some_name", active: true, ...}
{"name": "one_two", active: false, ...}
I've got everything to work with serializing and deserializing by writing my own #WritingConverter and #ReadingConverter (to a PGobject) and so I could write a query which then at the Java level performs the filter so only those which are 'active' remain.
But how can I, in Spring Data JDBC, write a query that would allow me to interrogate the json column at the level of the DB?
You'd use use the #Query annotation and whatever facilities your database offers to query json.
Just as an example this tutorial for working with JSON in Postgres suggests that something like should work.
#Query("select * from mytable where state ->> 'active' = :active ")
List<MyTable> findAllActive(String active)
You probably can convert state ->> 'active' to something suitable so you can use a boolean as the method argument, but I'd start with something simple.
In the database I have text field contains json with the structure:
"Limits":{
"fields":[
{
"key":"DAILY_LIMIT",
"value":"1559",
"lastModified":1543857829148,
},
{
"key":"MONTHLY_LIMIT",
"value":"25590",
"lastModified":1543857829148,
}
]
}
I need to check if daily_limit exists. It's easy to do with LIKE %DAILY_LIMIT% but performance is not so good and also I won't have access to value (right now I don't need it but maybe in future, it'll be needed). There is an option to check if this key exists without killing the db? I tried with 'Limits'->'fields'-> but I don't know what should be next... And it must be done by query, I cant pass object to backend and then check it
demo: db<>fiddle
If you want to do it the JSON way this could be a solution:
WITH data AS (
SELECT 'somedata' as somedata, '{"Limits":{"fields":[{"key":"DAILY_LIMITS","value":"1559","lastModified":1543857829148},{"key":"MONTHLY_LIMIT","value":"25590","lastModified":1543857829148}]}}'::jsonb as data
)
SELECT
d.*
FROM data d, jsonb_array_elements(data -> 'Limits' -> 'fields')
WHERE value ->> 'key' = 'DAILY_LIMITS'
jsonb_array_elements expands the array into one row each element. In the next step you are able to check the key's value.
But the demo shows, that a simple LIKE would be much faster as #404 mentioned correctly (have a look at the costs of both examples.)
I want to specify the return values for a specific update in sqlalchemy.
The documentation of the underlying update statement (sqlalchemy.sql.expression.update) says it accepts a "returning" argument and the docs for the query object state that query.update() accepts a dictionary "update_args" which will be passed as the arguments to the query statement.
Therefore my code looks like this:
session.query(
ItemClass
).update(
{ItemClass.value: value_a},
synchronize_session='fetch',
update_args={
'returning': (ItemClass.id,)
}
)
However, this does not seem to work. It just returns the regular integer.
My question is now: Am I doing something wrong or is this simply not possible with a query object and I need to manually construct statements or write raw sql?
The full solution that worked for me was to use the SQLAlchemy table object directly.
You can get that table object and the columns from your model easily by doing
table = Model.__table__
columns = table.columns
Then with this table object, I can replicate what you did in the question:
from your_settings import db
update_statement = table.update().returning(table.id)\
.where(columns.column_name=value_one)\
.values(column_name='New column name')
result = db.session.execute(update_statement)
tuple_of_results = result.fetchall()
db.session.commit()
The tuple_of_results variable would contain a tuple of the results.
Note that you would have to run db.session.commit() in order to persist the changes to the database as you it is currently running within a transaction.
You could perform an update based on the current value of a column by doing something like:
update_statement = table.update().returning(table.id)\
.where(columns.column_name=value_one)\
.values(like_count=table_columns.like_count+1)
This would increment our numeric like_count column by one.
Hope this was helpful.
Here's a snippet from the SQLAlchemy documentation:
# UPDATE..RETURNING
result = table.update().returning(table.c.col1, table.c.col2).\
where(table.c.name=='foo').values(name='bar')
print result.fetchall()
Am trying to Query inside an SQL table which has XML Column .
Table name: 'Purchase'
Column name: 'XML_COL'
Please find below xml data for column name 'XML_COL' under purchase table:
<ns1:Request xmlns:ns1="http://www.sample.com/hic/event/request"
xmlns:ns2="http://www.sample.com/hic/eventpayload/request">
<ns1:createeventRequest>
<ns1:eventPayLoad>
<ns2:eventPayLoad>
<Id>123456</Id>
</ns2:eventPayLoad>
</ns1:eventPayLoad>
</ns1:createeventRequest>
</ns1:Request>
I have written below query :
`select * from purchase,
XMLTABLE ('$d/Request/createeventRequest/eventPayLoad/eventPayLoad' PASSING XML_COL as "d"
COLUMNS
Id varchar(20) PATH 'Id') as a where(a.Id like '1234%');`
But this is returning me an empty column with no data.
But my requirement is it should fetch all the data for this particular Id.
Please help if any one faced this kind of issue.
Do we need to include namespaces as well while querying?? or am I missing any thing?
I think the expression PATH 'Id' is bit to simple...
I'm not familiar with MySQL's abilities to query XML... The Path Id would try to find an element "Id" from the current node (which is the root node in the first action). But there is no "Id"... You must either specify the full path, starting with a single / to start at the root node, or let the engine try a deep search, starting with two //
These paths should work:
SELECT ExtractValue(
'<ns1:Request xmlns:ns1="http://www.sample.com/hic/event/request" xmlns:ns2="http://www.sample.com/hic/eventpayload/request">
<ns1:createeventRequest>
<ns1:eventPayLoad>
<ns2:eventPayLoad>
<Id>123456</Id>
</ns2:eventPayLoad>
</ns1:eventPayLoad>
</ns1:createeventRequest>
</ns1:Request>',
'/ns1:Request[1]/ns1:createeventRequest[1]/ns1:eventPayLoad[1]/ns2:eventPayLoad[1]/Id[1]' ) AS result;
If there is only one element with a value (in your case "Id") you might use the simple deep search like this:
SELECT ExtractValue(
'<ns1:Request xmlns:ns1="http://www.sample.com/hic/event/request" xmlns:ns2="http://www.sample.com/hic/eventpayload/request">
<ns1:createeventRequest>
<ns1:eventPayLoad>
<ns2:eventPayLoad>
<Id>123456</Id>
</ns2:eventPayLoad>
</ns1:eventPayLoad>
</ns1:createeventRequest>
</ns1:Request>',
'//Id[1]' ) AS result;
But - in general - it is good advise to be as specific as possible...
Just cracked the query...When name spaces are being used in an XML, instead of the entire path, I found it's better to use '/*//' which traverses through the required element tag through XML.
Final Query:
select * from purchase,
XMLTABLE('$d' PASSING XML_COL as "d"
COLUMNS
Id varchar(20) PATH '/*//Id') as a where(a.Id like '1234%') with ur
Using 'with ur' helps to read the data that has not been committed in the database.
Please post comments if it is helpful.
This question is regarding Stream Analytics. I want to export a blob into a SQL.I know the process, my question is with the query I have to use.
{"performanceCounter":[{"available_bytes":{"value":994164736.0},"categoryName":"Memory","instanceName":""}],"internal":{"data":{"id":"459bf840-d259-11e5-a640-1df0b6342362","documentVersion":"1.61"}},"context":{"device":{"type":"PC","network":"Ethernet","screenResolution":{},"locale":"en-US","id":"RD0003FF73B748","roleName":"Sdm.MyGovId.Static.Web","roleInstance":"Sdm.MyGovId.Static.Web_IN_1","oemName":"Microsoft Corporation","deviceName":"Virtual Machine","deviceModel":"Virtual Machine"},"application":{"version":"R2.0_20160205.5"},"location":{"continent":"North America","country":"United States","clientip":"104.41.209.0","province":"Washington","city":"Redmond"},"data":{"isSynthetic":false,"samplingRate":100.0,"eventTime":"2016-02-13T13:53:44.2667669Z"},"user":{"isAuthenticated":false,"anonAcquisitionDate":"0001-01-01T00:00:00Z","authAcquisitionDate":"0001-01-01T00:00:00Z","accountAcquisitionDate":"0001-01-01T00:00:00Z"},"operation":{},"cloud":{},"serverDevice":{},"custom":{"dimensions":[],"metrics":[]},"session":{}}}
{"performanceCounter":[{"percentage_processor_total":{"value":0.0123466420918703},"categoryName":"Processor","instanceName":"_Total"}],"internal":{"data":{"id":"459bf841-d259-11e5-a640-1df0b6342362","documentVersion":"1.61"}},"context":{"device":{"type":"PC","network":"Ethernet","screenResolution":{},"locale":"en-US","id":"RD0003FF73B748","roleName":"Sdm.MyGovId.Static.Web","roleInstance":"Sdm.MyGovId.Static.Web_IN_1","oemName":"Microsoft Corporation","deviceName":"Virtual Machine","deviceModel":"Virtual Machine"},"application":{"version":"R2.0_20160205.5"},"location":{"continent":"North America","country":"United States","clientip":"104.41.209.0","province":"Washington","city":"Redmond"},"data":{"isSynthetic":false,"samplingRate":100.0,"eventTime":"2016-02-13T13:53:44.2668221Z"},"user":{"isAuthenticated":false,"anonAcquisitionDate":"0001-01-01T00:00:00Z","authAcquisitionDate":"0001-01-01T00:00:00Z","accountAcquisitionDate":"0001-01-01T00:00:00Z"},"operation":{},"cloud":{},"serverDevice":{},"custom":{"dimensions":[],"metrics":[]},"session":{}}}
{"performanceCounter":[{"percentage_processor_time":{"value":0.0},"categoryName":"Process","instanceName":"w3wp"}],"internal":{"data":{"id":"459bf842-d259-11e5-a640-1df0b6342362","documentVersion":"1.61"}},"context":{"device":{"type":"PC","network":"Ethernet","screenResolution":{},"locale":"en-US","id":"RD0003FF73B748","roleName":"Sdm.MyGovId.Static.Web","roleInstance":"Sdm.MyGovId.Static.Web_IN_1","oemName":"Microsoft Corporation","deviceName":"Virtual Machine","deviceModel":"Virtual Machine"},"application":{"version":"R2.0_20160205.5"},"location":{"continent":"North America","country":"United States","clientip":"104.41.209.0","province":"Washington","city":"Redmond"},"data":{"isSynthetic":false,"samplingRate":100.0,"eventTime":"2016-02-13T13:53:44.2668342Z"},"user":{"isAuthenticated":false,"anonAcquisitionDate":"0001-01-01T00:00:00Z","authAcquisitionDate":"0001-01-01T00:00:00Z","accountAcquisitionDate":"0001-01-01T00:00:00Z"},"operation":{},"cloud":{},"serverDevice":{},"custom":{"dimensions":[],"metrics":[]},"session":{}}}
Well you can see 3 json objects which of them have different fields for the objects in the array performanceCounter. Basically the first object of every object. in the first is available_bytes, 2nd is percentage_processor_total, and 3rd is percentage_processor_time.
Because I'm exporting this to a sql table called performaceCounter, I should have a different column for every different object, so I would like to save this into an string and then I will parse it in my app.
As starting point I have this query that reads an input(the blob) and write into an output(SQL)
Select GetArrayElement(A.performanceCounter,0) as a
INTO
PerformanceCounterOutput
FROM PerformanceCounterInput A
This GetArrayElement takes the index 0 of the array in performanceCounter but then writes a different column for each different field that find in every object. So I should have all different counters and create a column for each one, but my idea is more like a column call performanceCounterData and save string like
'"available_bytes":"value":994164736.0},"categoryName":"Memory","instanceName":""'
or this
"{"percentage_processor_total":{"value":0.0123466420918703},"categoryName":"Processor","instanceName":"_Total"}"
or
"{"percentage_processor_time":"value":0.0},"categoryName":"Process","instanceName":"w3wp"}"
How can I cast an array like a String?
I tried CAST(GetArrayElement(A.performanceCounter,0) as nvarchar(max)) but I can't.
Please some good help will be rewarded
With the following solution I get 2 columns with the name of the property and another with the value of the property, that it was my initial purpose
With pc as
(
Select
GetArrayElement(A.[performanceCounter],0) as counter
,A.context.data.eventTime as eventTime
,A.context.location.clientip as clientIp
,A.context.location.continent as continent
,A.context.location.country as country
,A.context.location.province as province
,A.context.location.city as city
FROM PerformanceCounterInput A
)
select
props.propertyName,
props.propertyValue,
pc.counter.categoryName,
pc.counter.instanceName,
pc.eventTime,
pc.clientIp,
pc.continent,
pc.country,
pc.province,
pc.city
from pc
cross apply GetRecordProperties(pc.counter) as props
where props.propertyname<>'categoryname' and props.propertyname<>'instancename'
Anyway if somebody finds how to write an object in plain text in analytics, still rewarded and appreciated will be
You can do something like below, this gives the counters as (propertyName, propertyValue) pairs.
with T1 as
(
select
GetArrayElement(iotInput.performanceCounter, 0) Counter,
System.Timestamp [EventTime]
from
iotInput timestamp by context.data.eventTime
)
select
[EventTime],
Counter.categoryName,
Counter.available_bytes [Value]
from
T1
where
Counter.categoryName = 'Memory'
union all
select
[EventTime],
Counter.categoryName,
Counter.percentage_processor_time [Value]
from
T1
where
Counter.categoryName = 'Process'
Query that gives one column per counter type can also be done, you will have to either do a join or a group by with 'case' statements for every counter.