MariaDB: Accessing JSON array without JSON_TABLE - json

I have a JSON field which have the following data:
[{"low": 57.07, "rsi": 0.0, "date": 1675935000000, "high": 57.07, "open": 57.07, "close": 57.07, "ema_7": 0.0, "ema_21": 0.0, "symbol": "ACPL", "volume": 0, "SUPERT_10_1_0": 0.0, "SUPERTd_10_1_0": 1, "SUPERTl_10_1_0": 0.0, "SUPERTs_10_1_0": 0.0}, {"low": 57.0, "rsi": 0.0, "date": 1675935900000, "high": 58.49, "open": 57.07, "close": 58.4, "ema_7": 0.0, "ema_21": 0.0, "symbol": "ACPL", "volume": 2500, "SUPERT_10_1_0": 0.0, "SUPERTd_10_1_0": 1, "SUPERTl_10_1_0": 0.0, "SUPERTs_10_1_0": 0.0}, {"low": 57.7, "rsi": 0.0, "date": 1675936800000, "high": 58.5, "open": 58.4, "close": 58.49, "ema_7": 0.0, "ema_21": 0.0, "symbol": "ACPL", "volume": 27000, "SUPERT_10_1_0": 0.0, "SUPERTd_10_1_0": 1, "SUPERTl_10_1_0": 0.0, "SUPERTs_10_1_0": 0.0}, {"low": 58.15, "rsi": 0.0, "date": 1675937700000, "high": 59.5, "open": 58.5, "close": 59.5, "ema_7": 0.0, "ema_21": 0.0, "symbol": "ACPL", "volume": 41000, "SUPERT_10_1_0": 0.0, "SUPERTd_10_1_0": 1, "SUPERTl_10_1_0": 0.0, "SUPERTs_10_1_0": 0.0}, {"low": 59.0, "rsi": 0.0, "date": 1675938600000, "high": 59.5, "open": 59.5, "close": 59.0, "ema_7": 0.0, "ema_21": 0.0, "symbol": "ACPL", "volume": 2500, "SUPERT_10_1_0": 0.0, "SUPERTd_10_1_0": 1, "SUPERTl_10_1_0": 0.0, "SUPERTs_10_1_0": 0.0}]
The following query perfectly works for me:
SELECT indicators_15.symbol,indicators_15.open,indicators_15.close
FROM indicators_15,
JSON_TABLE(data, '$[*]' COLUMNS (
close DOUBLE PATH '$.close',
open DOUBLE PATH '$.open')
) indicators_15;
but my Hosting, Namecheap is using an older version of MariaDB hence it is failing. How can I come up with an equivalent non-JSON_TABLE version?
Below is the desired output:

To do this in an old version of MariaDB you need a table of numbers.
CREATE TABLE numbers ( number INT UNSIGNED PRIMARY KEY );
INSERT INTO numbers (number) VALUES (0), (1), (2), (3), (4), (5), (6), (7), (8), (9), (10);
Insert more numbers as needed, up to the maximum length of any of your JSON arrays.
Then you can use these numbers to extract the n-th entry from your JSON array.
SELECT i.symbol,
JSON_EXTRACT(i.data, CONCAT('$[', o.number, '].open')) AS open,
JSON_EXTRACT(i.data, CONCAT('$[', o.number, '].close')) AS close
FROM indicators_15 AS i
JOIN numbers AS o ON o.number < JSON_LENGTH(i.data);
Dbfiddle using MariaDB 10.5.
Let me know if this is not clear to you and I'll try to explain further.
This really demonstrates what a bad idea it is to use JSON in a relational database. There is no reason to use JSON in your example, because every array entry has the same fields. Use JSON if you can't predict the fields. Use normal rows and columns if the fields are the same in every record. Using JSON where it is not needed — when the developer can't understand how to use it or if you are constrained to use an old version of the software that doesn't have enough support for JSON functions — will only harm your software project by increasing time to develop and therefore increasing development costs.

Related

JSON_SEARCH return null for json array containing only integer values [duplicate]

I have a problem with a query. In my DB I have a field that contains a JSON with this format:
[[0, 16, 22, 37, 0, 0, 0, 71, 82],
[0, 18, 0, 36, 43, 0, 60, 0, 88],
[9, 10, 0, 0, 0, 58, 69, 77, 0]]
With this query
SELECT JSON_SEARCH(NumeriJSON, 'all', 77,null, '$[*]') AS Indice FROM Cartella WHERE JSON_CONTAINS(NumeriJSON->'$[*]', '77')
I want to get the position of the number in the JSON, but it returns null. Why? The structure of JSON is valid, because JSON_CONTAINS works well.
Thanks a lot.
https://dev.mysql.com/doc/refman/8.0/en/json-search-functions.html#function_json-search says:
JSON_SEARCH(json_doc, one_or_all, search_str[, escape_char[, path] ...])
Returns the path to the given string within a JSON document.
I emboldened the word string. JSON_SEARCH() doesn't work if the values in your JSON are integers.
Your example works if you make them strings:
insert into Cartella (NumeriJson) values (
'[["0", "16", "22", "37", "0", "0", "0", "71", "82"],
["0", "18", "0", "36", "43", "0", "60", "0", "88"],
["9", "10", "0", "0", "0", "58", "69", "77", "0"]]');
mysql> select json_search(numerijson, 'all', '77', null, '$[*]') as `index`
from cartella where json_contains(numerijson->'$[*]', '"77"');
+-----------+
| index |
+-----------+
| "$[2][7]" |
+-----------+
This issue was reported as a bug in 2015, but it was closed by simply documenting that it only supports string searches.
Supporting non-strings was then reported as a new feature request in 2018. So far, there is no solution.
In the meantime, you must store integers as strings if you want to use JSON_SEARCH().
I would also recommend to stop using JSON, and instead store integers in normal rows and columns. Then searches for integers work just fine.
P.S. Please don't use the back-formation "indice." There is no such word.

Azure Stream Analytics - JSON

I am trying to pass the below json through Azure Stream analytics to an Azure SQL server. The data is coming from the Azure IOT HUB and data is coming through happily.
nodes": {
"SN0013A20041E23697": {
"firmware_version": 5,
"transmission_count": 42,
"reserve_byte": 0,
"battery_level": 3.29406,
"type": 32,
"node_id": 0,
"rssi": 9,
"mass_concentration_pm_1_0": 0.88,
"mass_concentration_pm_2_5": 1.04,
"mass_concentration_pm_4_0": 1.13,
"mass_concentration_pm_10_0": 1.17,
"number_concentration_pm_0_5": 5.73,
"number_concentration_pm_1_0": 6.92,
"number_concentration_pm_2_5": 7.07,
"number_concentration_pm_4_0": 7.09,
"number_concentration_pm_10_0": 7.09,
"typical_particle_size": 0.48,
"humidity": 45.35,
"temperature": 20.84
},
"SN0013A20041E2367B": {
"firmware_version": 5,
"transmission_count": 43,
"reserve_byte": 0,
"battery_level": 2.99782,
"type": 32,
"node_id": 0,
"rssi": 16,
"mass_concentration_pm_1_0": 1.35,
"mass_concentration_pm_2_5": 1.43,
"mass_concentration_pm_4_0": 1.43,
"mass_concentration_pm_10_0": 1.43,
"number_concentration_pm_0_5": 9.13,
"number_concentration_pm_1_0": 10.77,
"number_concentration_pm_2_5": 10.83,
"number_concentration_pm_4_0": 10.83,
"number_concentration_pm_10_0": 10.83,
"typical_particle_size": 0.41,
"humidity": 45.72,
"temperature": 20.2
I can use a query like this and it will pass through one of the devices but not the other.
SELECT
"nodes"."SN0013A20041E23697"."temperature" as Temperature
, "nodes"."SN0013A20041E23697"."humidity" as Humidity
From input
Is there a way to pass through both devices in the same query?

Greater and less than operator in a URL

Hello everyone i have a db.json file which i run in http://localhost:3004
The format is like this:
[
{
"Symbol": "AAPL",
"prodType": "STK",
"Market": "Symbol Industries Inc",
"Country": "US",
"Quantity": 10,
"NominalValue": 123.45,
"AvgPrice": 131.16,
"PositionCost": 123.87,
"LastPrice": 123.567,
"PositionValue": 145.78,
"TotalValue": 123.347,
"PortfolioPercentage": 12,
"ProfitLoss": 1235.09,
"Results": "12 Dec, 2017",
"Dividend": "29 Apr, 2017"
},
{
"Symbol": "fsdfsd",
"prodType": "DER",
"Market": "Symbol Industries Inc",
"Country": "Greece",
"Quantity": 10,
"NominalValue": 123.45,
"AvgPrice": 131.16,
"PositionCost": 123.87,
"LastPrice": 123.567,
"PositionValue": 145.78,
"TotalValue": 123.347,
"PortfolioPercentage": 12,
"ProfitLoss": 1235.09,
"Results": "12 Dec, 2017",
"Dividend": "29 Apr, 2017"
}
]
i want to make a query to the URL to bring me the records which have total value greater than 100 and lesser than 130 for example but i can't. I use different approaches but no one is working properly. How can i implement such a query in this URL?

JSON Formatting error

I am getting this error while trying to import this JSON into google bigquery table
file-00000000: JSON table encountered too many errors, giving up. Rows: 1; errors: 1. (error code: invalid)
JSON parsing error in row starting at position 0 at file: file-00000000. Start of array encountered without start of object. (error code: invalid)
This is the JSON
[{'instrument_token': 11192834, 'average_price': 8463.45, 'last_price': 8471.1, 'last_quantity': 75, 'buy_quantity': 1065150, 'volume': 5545950, 'depth': {'buy': [{'price': 8471.1, 'quantity': 300, 'orders': 131072}, {'price': 8471.0, 'quantity': 300, 'orders': 65536}, {'price': 8470.95, 'quantity': 150, 'orders': 65536}, {'price': 8470.85, 'quantity': 75, 'orders': 65536}, {'price': 8470.7, 'quantity': 225, 'orders': 65536}], 'sell': [{'price': 8471.5, 'quantity': 150, 'orders': 131072}, {'price': 8471.55, 'quantity': 375, 'orders': 327680}, {'price': 8471.8, 'quantity': 1050, 'orders': 65536}, {'price': 8472.0, 'quantity': 1050, 'orders': 327680}, {'price': 8472.1, 'quantity': 150, 'orders': 65536}]}, 'ohlc': {'high': 8484.1, 'close': 8336.45, 'low': 8422.35, 'open': 8432.75}, 'mode': 'quote', 'sell_quantity': 998475, 'tradeable': True, 'change': 1.6151959167271395}]
http://jsonformatter.org/ also gives parse error for this JSON block. Need help understanding where the formatting is wrong - this is the JSON from a rest API
This is not valid JSON. JSON uses double quotes, not single quotes. Also, True should be true.
If I had to guess, I would guess that this is Python code being passed off as JSON. :-)
I suspect that even once this is made into correct JSON, it's not the format Google BigQuery is expecting. From https://cloud.google.com/bigquery/data-formats#json_format, it looks like you should have a text file with one JSON object per line. Try just this:
{"mode": "quote", "tradeable": true, "last_quantity": 75, "buy_quantity": 1065150, "depth": {"buy": [{"quantity": 300, "orders": 131072, "price": 8471.1}, {"quantity": 300, "orders": 65536, "price": 8471.0}, {"quantity": 150, "orders": 65536, "price": 8470.95}, {"quantity": 75, "orders": 65536, "price": 8470.85}, {"quantity": 225, "orders": 65536, "price": 8470.7}], "sell": [{"quantity": 150, "orders": 131072, "price": 8471.5}, {"quantity": 375, "orders": 327680, "price": 8471.55}, {"quantity": 1050, "orders": 65536, "price": 8471.8}, {"quantity": 1050, "orders": 327680, "price": 8472.0}, {"quantity": 150, "orders": 65536, "price": 8472.1}]}, "change": 1.6151959167271395, "average_price": 8463.45, "ohlc": {"close": 8336.45, "high": 8484.1, "open": 8432.75, "low": 8422.35}, "instrument_token": 11192834, "last_price": 8471.1, "sell_quantity": 998475, "volume": 5545950}
OP has a valid JSON record but that wouldn't work with Biq Query, and here's why:
Google Big Query supports, JSON objects {}, one object per line. Check this out.
This basically means that you cannot supply list [] as json records and expect Big Query to detect it. You must always have one json object per line.
Here's a quick reference to what I am saying.
and there are more.
at last,
I highly recommend you read up the below and check out the link for more information on different forms of JSON structures, read this from the json.org

yahoo finance stock quote api

Is there any way to get historical stock prices from the yahoo api in the json format? I'd like to use REST because it's more lightweight.
To follow up on user586050's answer with a specific example...
You can use the YQL yahoo.finance.historical data table for this request, and have the results come back in JSON format.
For example try this query (the link will take you to the YQL console where you can play with it):
select * from yahoo.finance.historicaldata where symbol = "YHOO" and startDate = "2009-09-11" and endDate = "2009-09-15"
Sample results in JSON format:
{
"query": {
"count": 3,
"created": "2011-12-31T19:44:20Z",
"lang": "en-US",
"results": {
"quote": [
{
"date": "2009-09-15",
"Date": "2009-09-15",
"Open": "16.01",
"High": "16.49",
"Low": "15.87",
"Close": "16.41",
"Volume": "64668200",
"Adj_Close": "16.41"
},
{
"date": "2009-09-14",
"Date": "2009-09-14",
"Open": "15.45",
"High": "15.58",
"Low": "15.28",
"Close": "15.57",
"Volume": "19451200",
"Adj_Close": "15.57"
},
{
"date": "2009-09-11",
"Date": "2009-09-11",
"Open": "15.53",
"High": "15.68",
"Low": "15.41",
"Close": "15.59",
"Volume": "26860700",
"Adj_Close": "15.59"
}
]
}
}
}
You can use YQL to fetch data. look at here the code in C#.
Or if you want current stock look at https://github.com/samkiller/stoxxbees
You can have stock data in both xml or json.
get json format you can use YQL to send request and get response, if you want to get history data in spreadsheet, you can use http://ichart.finance.yahoo.com/table.csv?s=company symbol e.g open http://ichart.finance.yahoo.com/table.csv?s=MD in your browser