I am unable to execute a query due to the following n1ql errors:
Repository Query:
public ClientMatrixFactorPopUp findBySupportedValuesAndRequestId(SortedMap supportedValues,Long requestId);
Expected N1QL:
SELECT * FROM b_factor
WHERE _class = "com.intervest.price.oxygen.model.ClientMatrixFactorPopUp" and requestId=21
and supportedValues= { "BAGGAGE_COVER": "0", "CANCELLATION_COVER": "0", "GADGET_COVER": "2", "TRIP_EXCESS": "250" }
StackTrace:
Unable to execute query due to the following n1ql errors:
{"msg":"Object member missing name or value: (`BAGGAGE_COVER` = 500) - at , \n Object member missing name or value: (`CANCELLATION_COVER` = 500) - at , \n Object member missing name or value: (`GADGET_COVER` = 1) - at , \n Object member missing name or value: (`TRIP_EXCESS` = 50) - at }","code":3000}
[
{
"b_factor": {
"_class": "com.intervest.price.oxygen.model.ClientMatrixFactorPopUp",
"requestId": 21,
results": [...],
"supportedValues":{
"BAGGAGE_COVER": "0",
"CANCELLATION_COVER":"0",
"GADGET_COVER":"2",
"TRIP_EXCESS":"250"
}
}
}
]
It looks like spring data generated wrong query. The error shows there is back ticks around BAGGAGE_COVER. It should have been constant object.
Check the query generation right.
The query provided above will work. Try in Web console and verify.
Unless really need exact match, Instead of object comparison should try required fields, So that document has extra fields it can match.
SELECT * FROM b_factor
WHERE _class = "com.intervest.price.oxygen.model.ClientMatrixFactorPopUp"
AND requestId = 21
AND supportedValues.BAGGAGE_COVER = "0"
AND supportedValues.CANCELLATION_COVER = "0"
AND supportedValues.GADGET_COVER = "2"
AND supportedValues.TRIP_EXCESS = "250";
Related
this is my test json file.
{
"item" : {
"fracData" : [ ],
"fractimeData" : [ {
"number" : "1232323232",
"timePeriods" : [ {
"validFrom" : "2021-08-03"
} ]
} ],
"Module" : [ ]
}
}
This is how I read the json file.
starhist_test_df = spark.read.json("/mapr/xxx/yyy/ttt/dev/rawdata/Test.json", multiLine=True)
starhist_test_df.createOrReplaceTempView("v_test_df")
This query works.
df_test_01 = spark.sql("""
select item.fractimeData.number from v_test_df""")
df_test_01.collect();
Result
[Row(number=['1232323232'])]
But this query doesn't work.
df_test_01 = spark.sql("""
select item.fractimeData.timePeriods.validFrom from v_test_df""")
df_test_01.collect();
Error
cannot resolve 'v_test_df.`item`.`fractimeData`.`timePeriods`['validFrom']' due to data type mismatch: argument 2 requires integral type, however, ''validFrom'' is of string type.; line 3 pos 0;
What do I have to change, to read the validFrom field?
dot notation to access values works with struct or array<struct> types.
The schema for field number in item.fractimeData is string and accessing it via dot notation returns an array<string> since fractimeData is an array.
Similarly, the schema for field timePeriods in item.fractimeData is <array<struct<validFrom>>, and accessing it via dot notation wraps it into another array, resulting in final schema of array<array<struct<validFrom>>>.
The error you get is because the dot notation can work on array<struct> but not on array<array>.
Hence, flatten the result from item.fractimeData.timePeriods to get back an array<struct<validFrom>> and then apply the dot notation.
df_test_01 = spark.sql("""
select flatten(item.fractimeData.timePeriods).validFrom as validFrom from v_test_df""")
df_test_01.collect()
"""
[Row(validFrom=['2021-08-03', '2021-08-03'])]
"""
Using Newtonsoft Json with VB.NET I am trying to read some nested keys/elements within a block of JSON.
The JSON looks like this and is held in string strSuppliedJSON:
{
"seller": {
"id": 123,
"name": "Seller Name",
"address1": "Seller address1",
"country": "Seller country"
},
"buyer": {
"id": 987,
"name": "Buyer name",
"address1": "Buyer address1",
"country": "Buyer country"
},
"interview": {
"call_id": 123,
"vin": "The vin from the machine section",
"call_date": "2019-12-31 23:59:59",
"questions": ["Question1", "Question2", "Question3", "Question5", "Question5"],
"triggers": [{
"question": "Question1",
"answers": ["Answer1", "Answer2"]
}]
}
}
Before reading the values I need to make sure some of the keys exist using ContainsKey.
The following works fine:
Imports Newtonsoft.Json
Imports Newtonsoft.Json.Linq
' Create a dictionary
Dim dictionary As IDictionary(Of String, JToken) = JObject.Parse(strSuppliedJSON)
' Check if key exists - interview:triggers
If JObject.Parse(dictionary("interview").ToString()).ContainsKey("triggers") = False Then
strAllChecksPassed = False
result = "ERROR: JSON element not found: interview:triggers"
End If
However, when trying to check or read the interview:triggers:question things are falling over.
Using this, fails:
' Check if key exists - interview:triggers
If JObject.Parse(dictionary("interview").ToString()).ContainsKey("triggers") = False Then
strAllChecksPassed = False
result = "ERROR: JSON element not found: interview:triggers"
Else
' interview:triggers DOES exist, now check if the question exists - interview:triggers:question
If JObject.Parse(dictionary("interview")("triggers").ToString()).ContainsKey("question") = False Then
strAllChecksPassed = False
result = "ERROR: JSON element not found: interview:triggers:question"
End If
End If
The line that throws the error is:
If JObject.Parse(dictionary("interview")("triggers").ToString()).ContainsKey("question") = False Then
And the error is:
Newtonsoft.Json.JsonReaderException: Error reading JObject from JsonReader. Current JsonReader item is not an object: StartArray. Path '', line 1, position 1
So I then tried to create a sub-dictionary of just the interview element.
I used this:
' Create a sub-dictionary of just the Interview element
Dim subDictionary As IDictionary(Of String, JToken) = JObject.Parse(dictionary("interview").ToString())
And if I then do Response.Write(subDictionary) I now see a smaller subset of my JSON, as expected:
{
"call_id": 123,
"vin": "The vin from the machine section",
"call_date": "2019-12-31 23:59:59",
"questions": ["Question1", "Question2", "Question3", "Question5", "Question5"],
"triggers": [{
"question": "Question1",
"answers": ["Answer1", "Answer2"]
}]
}
But then when trying to use my new subDictionary in the exact same way to see if the question key exists:
' Check if key exists - interview:triggers:question
If JObject.Parse(subDictionary("triggers").ToString()).ContainsKey("question") = False Then
strAllChecksPassed = False
result = "ERROR: JSON element not found: interview:triggers:question"
End If
I get the exact same error of:
Newtonsoft.Json.JsonReaderException: Error reading JObject from JsonReader. Current JsonReader item is not an object: StartArray. Path '', line 1, position 1
Even though the line of code is identical!
How do I check if the nested key interview:triggers:question exists in my JSON, and what its value is?
Look closely at "triggers" in the JSON: it's actually an array of objects, not an object. You need to index the array before you can access "question".
Also note that every time you call ToString, you're reserializing something that you just deserialized with Parse. You don't need to do that. Parse the JSON once into a JObject and then reuse that object.
My VB-fu isn't great; I wrote this in C# and then converted it but the critical part is Dim first As JToken = triggers(0). This gets the first array element, on which you can get the value associated with "question".
Dim suppliedObject As JObject = JObject.Parse(strSuppliedJSON)
Dim interview As JToken = suppliedObject("interview")
Dim triggers As JToken = If(interview IsNot Nothing, interview("triggers"), Nothing)
If triggers Is Nothing Then
strAllChecksPassed = False
result = "ERROR: JSON element not found: interview:triggers"
Else
Dim first As JToken = triggers(0)
Dim question As JToken = If(first IsNot Nothing, first("question"), Nothing)
If question Is Nothing Then
strAllChecksPassed = False
result = "ERROR: JSON element not found: interview:triggers:question"
End If
End If
It fails because your path is not proper.
[] is an Array. You write out the name for single items (within {}), for arrays you write index (number).
And why are you parsing multiple times? Use what you already have, for example:
Dim JsonResp As JObject = JObject.Parse(<JSON>)
'Now there's multiple ways to do the same, here's one
If JsonResp ("interview")("triggers")(0)("question") Is Nothing Then
strAllChecksPassed = False
result = "ERROR: JSON element not found: interview:triggers:question"
End If
I'm ingesting a large simple json dataset from Azure Blob and moving data into a "stage" called "cities_stage" with FILE_FORMAT = json like so.
(Here is the error steps are below "Error parsing JSON: unknown keyword "Hurzuf", pos 7.")
create or replace stage cities_stage
url='azure://XXXXXXX.blob.core.windows.net/xxxx/landing/cities'
credentials=(azure_sas_token='?st=XXXXX&se=XXX&sp=racwdl&sv=XX&sr=c&sig=XXX')
FILE_FORMAT = (type = json);
I then take this stage location and dump it into a table with a single variant column like so. The file I'm ingesting is larger than 16mb so I create individual rows for each object by using type = json strip_outer_array = true
create or replace table cities_raw_source (
src variant);
copy into cities_raw_source
from #cities_stage
file_format = (type = json strip_outer_array = true)
on_error = continue;
When I select * from cities_raw_source each row looks like the following.
{
"coord": {
"lat": 44.549999,
"lon": 34.283333
},
"country": "UA",
"id": 707860,
"name": "Hurzuf"
}
When I add a reference to "country" or "name" that's where the issues come in. Here is my query (I did not use country in this one but it produces the same result).
select parse_json(src:id),
parse_json(src:coord:lat),
parse_json(src:coord:lon),
parse_json(src:name)
from cities_raw_source;
ERROR:
Error parsing JSON: unknown keyword "Hurzuf", pos 7.
ID, Lat, and Lon all come back as expected if I remove "src:name"
Any help is appreciated!
It turns out I had everything correct except for the query itself.
When querying a VARIANT column you do not need to PARSE_JSON so the correct query would look like this.
select src:id,
src:coord:lat,
src:coord:lon,
src:name
from cities_raw_source;
Trying to pull first value from a parsed JSON string using the JsonConverter library.
What I currently have is:
result = objHTTP.responseText
Set parsedResult = JsonConverter.ParseJson(result)
i = 3
For Each Item In parsedResult("From")
wsSheet0.Cells(i, 12) = parsedResult("From")(Item)("Price")
i = i + 1
Next
With parsedResult("From")
wsSheet0.Cells(11, 12) = parsedResult("From")("Chocolate")("Price")("AsAtDate")
End With
The last line of this gets a type mismatch error, so I'm still trying to figure out how to just pull that one line item.
For the sake of clarity, the parsed JSON string looks like:
{
"From":{
"Chocolate":{
"Price":1.0,
"AsAtDate":"2018-05-04T00:00:00"
},
"Lime":{
"Price":1.35415115,
"AsAtDate":"2018-05-04T00:00:00"
},
"Strawberry":{
"Price":1.19517151,
"AsAtDate":"2018-05-04T00:00:00"
},
"Vanilla":{
"Price":0.77522986,
"AsAtDate":"2018-05-04T00:00:00"
},
"Blueberry":{
"Price":1.00084071,
"AsAtDate":"2018-05-04T00:00:00"
},
"Lemon":{
"Price":0.75030012,
"AsAtDate":"2018-05-04T00:00:00"
}
},
"To":"Chocolate",
"RequestedDate":"2018-05-22T08:26:16"
}
Use
parsedResult("From")("Chocolate")("AsAtDate")
Or more generally to get all:
parsedResult("From")(item)("AsAtDate")
I'm using the latest cassandra version and trying to save JSON like below and was successful,
INSERT INTO mytable JSON '{"username": "myname", "country": "mycountry", "userid": "1"}'
Above query saves the record like,
"rows": [
{
"[json]": "{\"userid\": \"1\", \"country\": \"india\", \"username\": \"sai\"}"
}
],
"rowLength": 1,
"columns": [
{
"name": "[json]",
"type": {
"code": 13,
"type": null
}
}
]
Now I would like to retrieve the record based on userid:
SELECT JSON * FROM mytable WHERE userid = fromJson("1") // but this query throws error
All this occurs in a node/express app and I'm using dse-driver as the client driver.
The CQL command worked like below,
SELECT JSON * FROM mytable WHERE userid="1";
However if it has to be executed via the dse-driver then the below snippet worked,
let query = 'SELECT JSON * FROM mytable WHERE userid = ?';
client.execute(query, ["1"], { prepare: true });
where client is,
const dse = require('dse-driver');
const client = new dse.Client({
contactPoints: ['h1', 'h2'],
authProvider: new dse.auth.DsePlainTextAuthProvider('username', 'pass')
});
If your Cassandra version is 2.1x and below, you can use the Python-based approach. Write a python script using Cassandra-Python API
Here you have to get your row first and then use python json's loads method, which will convert your json text column value into JSON object which will be dict in Python. Then you can play around with Python dictionaries and extract your required nested keys. See the below code snippet.
from cassandra.cluster import Cluster
from cassandra.auth import PlainTextAuthProvider
import json
if __name__ == '__main__':
auth_provider = PlainTextAuthProvider(username='xxxx', password='xxxx')
cluster = Cluster(['0.0.0.0'],
port=9042, auth_provider=auth_provider)
session = cluster.connect("keyspace_name")
print("session created successfully")
rows = session.execute('select * from user limit 10')
for user_row in rows:
#fetchign your json column
column_dict = json.loads(user_row.json_col)
print(column_dict().keys()
Assuming user-id is the partition key, and assuming you want to retrieve a JSON object corresponding to user of id 1, you should try:
SELECT JSON * FROM mytable WHERE userid=1;
If userid is of type text, you will need to add some quotes.