I have a problem with google spreadsheets. I try to import a value from a link (which returns me a JSON) but it seems like it does not work.
I tried this:
https://medium.com/#paulgambill/how-to-import-json-data-into-google-spreadsheets-in-less-than-5-minutes-a3fede1a014a#.pb26xo98x
The link returns a json like this:
{
"data": [
{
"time": "2016-10-16T07:00:00+0000",
"value": "249.884067074"
}
],
"summary": {
"name": "Custom Events",
"period": "daily",
"since": "2016-10-17T00:00:00+0000",
"until": "2016-10-17T00:00:00+0000"
}
}
How can I extract the value from the data field?
I tried like this:
=ImportJSON(myUrl, "/data[0]/value", "noInherit,noTruncate,rawHeaders")
According to a comment on the project page there is a fix that should be manually applied:
Chris says:
November 4, 2014 at 11:35 pm (UTC -4)
Trevor,
I was able to fix this problem by making a minor change to the
ParseData_ function. I changed line 286 in version 1.2.1 to:
if (i >= 0 && data[state.rowIndex]) {
and it seems to have addressed the issue.
Thank you!
CR
Related
I have query which is traversing only in forward direction.
example:
{
"orderStatus": "SUBMITTED",
"orderNumber": "785654",
"orderLine": [
{
"lineNumber": "E1000",
**"trackingnumber": "12345,67890",**
"lineStatus": "IN-PROGRESS",
"lineStatusCode": 50
}
],
"accountNumber": 9076
}
find({'orderLine.trackingNumber' : { $regex: "^12345.*"} })**
When I use the above query I get the entire document. But I want to fetch the document when I search with 67890 value as well
At any part of time I will be always querying with single tracking number only.
12345 or 67890 Either with 12345 or 67890. There are chances tracking number value can extend it's value 12345,56789,01234,56678.
I need to pull the whole document no matter what the tracking number is in whatever position.
OUTPUT
should be whole document
{
"orderStatus": "SUBMITTED",
"orderNumber": "785654",
"orderLine": [
{
"lineNumber": "E1000",
"trackingnumber": "12345,67890",
"lineStatus": "IN-PROGRESS",
"lineStatusCode": 50
}
],
"accountNumber": 9076
}
Also I have done indexing for trackingNumber field. Need help here. Thanks in advance.
Following will search with either 12345 or 67890. It is similar to like condition
find({'orderLine.trackingNumber' : { $regex: /12345/} })
find({'orderLine.trackingNumber' : { $regex: /67890/} })
There's also an alternative way to do this
Create a text index
db.order.createIndex({'orderLine.trackingnumber':"text"})
You can make use of this index to search the value from trackingnumber field
db.order.find({$text:{$search:'12345'}})
--
db.order.find({$text:{$search:'67890'}})
--
//Do take note that you can't search using few in between characters
//like the following query won't give any result..
db.order.find({$text:{$search:'6789'}}) //have purposefully removed 0
To further understand how $text searches work, please go through the following link.
I'm getting Yahoo Finance data as a JSON file (via the YahooFinancials python API) and I would like to be able to parse the data in a smart way to feed my Google Sheet.
For this example, I'm interested in getting the "cash" variable under the "date" nested structure. But as you'll see, sometimes there is no "cash" variable under the first date, so I would like the script/formula to go and get the "cash" variable that's under the second date structure.
Here is sample 1 of JSON code:
{ "balanceSheetHistoryQuarterly": {
"ABBV": [
{
"2018-12-31": {
"totalStockholderEquity": -2921000000,
"netTangibleAssets": -45264000000
}
},
{
"2018-09-30": {
"intangibleAssets": 26625000000,
"capitalSurplus": 14680000000,
"totalLiab": 69085000000,
"totalStockholderEquity": -2921000000,
"otherCurrentLiab": 378000000,
"totalAssets": 66164000000,
"commonStock": 18000000,
"otherCurrentAssets": 112000000,
"retainedEarnings": 6789000000,
"otherLiab": 16511000000,
"goodWill": 15718000000,
"treasuryStock": -24408000000,
"otherAssets": 943000000,
"cash": 8015000000,
"totalCurrentLiabilities": 15387000000,
"shortLongTermDebt": 1026000000,
"otherStockholderEquity": -2559000000,
"propertyPlantEquipment": 2950000000,
"totalCurrentAssets": 18465000000,
"longTermInvestments": 1463000000,
"netTangibleAssets": -45264000000,
"shortTermInvestments": 770000000,
"netReceivables": 5780000000,
"longTermDebt": 37187000000,
"inventory": 1786000000,
"accountsPayable": 10981000000
}
},
{
"2018-06-30": {
"intangibleAssets": 26903000000,
"capitalSurplus": 14596000000,
"totalLiab": 65016000000,
"totalStockholderEquity": -3375000000,
"otherCurrentLiab": 350000000,
"totalAssets": 61641000000,
"commonStock": 18000000,
"otherCurrentAssets": 128000000,
"retainedEarnings": 5495000000,
"otherLiab": 16576000000,
"goodWill": 15692000000,
"treasuryStock": -23484000000,
"otherAssets": 909000000,
"cash": 3547000000,
"totalCurrentLiabilities": 17224000000,
"shortLongTermDebt": 3026000000,
"otherStockholderEquity": -2639000000,
"propertyPlantEquipment": 2787000000,
"totalCurrentAssets": 13845000000,
"longTermInvestments": 1505000000,
"netTangibleAssets": -45970000000,
"shortTermInvestments": 196000000,
"netReceivables": 5793000000,
"longTermDebt": 31216000000,
"inventory": 1580000000,
"accountsPayable": 10337000000
}
},
{
"2018-03-31": {
"intangibleAssets": 27230000000,
"capitalSurplus": 14519000000,
"totalLiab": 65789000000,
"totalStockholderEquity": 3553000000,
"otherCurrentLiab": 125000000,
"totalAssets": 69342000000,
"commonStock": 18000000,
"otherCurrentAssets": 17000000,
"retainedEarnings": 4977000000,
"otherLiab": 17250000000,
"goodWill": 15880000000,
"treasuryStock": -15961000000,
"otherAssets": 903000000,
"cash": 9007000000,
"totalCurrentLiabilities": 17058000000,
"shortLongTermDebt": 6024000000,
"otherStockholderEquity": -2630000000,
"propertyPlantEquipment": 2828000000,
"totalCurrentAssets": 20444000000,
"longTermInvestments": 2057000000,
"netTangibleAssets": -39557000000,
"shortTermInvestments": 467000000,
"netReceivables": 5841000000,
"longTermDebt": 31481000000,
"inventory": 1738000000,
"accountsPayable": 10542000000
}
}
]
}
}
The first date structure (under 2018-12-31) doesn't contain the cash variable. So I would like the Google sheet to go and search for the same data in 2018-09-30 and if not available go and search in 2018-06-30.
OR just scan the nested structure dates and fetch the first "cash" occurrence that will be found.
Basically, I would like to know how to skip the name of the date variable (i.e.2018-12-31) as it doesn't really matter, and just make the formula seek for the first available "cash" variable.
Main questions recap
How to skip mentioning an exact nested level name and scan what's
inside?
How to keep scanning until you find the desired variable with
a value that is not "null" (this can happen)?
What would be the entire formula to achieve the following logic: Scan the JSON file until you find the value > if no value found, fallback to this IMPORTXML function that calls an external API.
Let me know if you need more context about the issue and thanks in advance for your help :)
EDIT: this is the IMPORTJSON formula I use in the cell of the spreadsheet right now.
=ImportJSON("https://api.myjson.com/bins/8mxvi", "/financial/balanceSheetHistoryQuarterly/ABBV/2018-31-12/cash", "noHeaders")
Obviously, this one returns an error as there is nothing under that date. The JSON is also the valid link I use just now.
=REGEXEXTRACT(FILTER(
TRANSPOSE(SPLIT(SUBSTITUTE(A1, ","&CHAR(10), "×"), "×")),
ISNUMBER(SEARCH("*"&"cash"&"*",
TRANSPOSE(SPLIT(SUBSTITUTE(A1, ","&CHAR(10), "×"), "×"))))), ": (.+)")
=INDEX(ARRAYFORMULA(SUBSTITUTE(REGEXEXTRACT(FILTER(TRANSPOSE(SPLIT(SUBSTITUTE(
TRANSPOSE(IMPORTDATA("https://api.myjson.com/bins/8mxvi")), ","&CHAR(10), "×"), "×")),
ISNUMBER(SEARCH("*"&"cash"&"*", TRANSPOSE(SPLIT(SUBSTITUTE(
TRANSPOSE(IMPORTDATA("https://api.myjson.com/bins/8mxvi")), ","&CHAR(10), "×"), "×"))))),
":(.+)"), ",", "")), 1, 1)
I created a query for a elastic search watcher setup. The result set looks like this:
"_index": "transaction_broker-2017.09.15",
"_type": "transaction_broker",
"_id": "AV6Fn_UQ9KbnKce40avY",
"_score": 3.8539968,
"_source": {
"tbroker.workitem.sync_check.tbroker_value": 7000,
"source": "/logs/web/tomcat/tbroker.log.json",
"type": "transaction_broker",
"tbroker.job.instance_id": "lixporta-p00.xxxxxxx.15053054001381505305457198",
"tbroker.workitem.sync_check.backend_total_value": 6995,
"tbroker.appversion": "1.1.135-180",
"#version": 1,
"beat": {
"hostname": "lixporta-p00",
"name": "lixporta-p00",
"version": "5.1.1"
In the action section, I can access the fields by using:
"actions": {
"my-logging-action": {
"logging": {
"text": "There are {{ctx.payload.hits.hits.0._source.....
After the source tag, I use for example the "type" field from the list above. Other example is:
"ctx.payload.hits.hits.0._source.beat.hostname"
This works pretty fine...
But it is not possible to use a field like
"tbroker.workitem.sync_check.tbroker_value"
The parser thinks that this fields are nested, but this is only a fieldname with dots in it.
Is there any possiblity to "escape" this fieldname?
Anyone who also have had this problem ?
Many thanks & best regards
Claus
I think the following should work:
{{#ctx.payload.hits.hits.0._source}}{{tbroker.workitem.sync_check.tbroker_value}}{{/ctx.payload.hits.hits.0._source}}
It is a limitation of Mustache and this is a workaround.
Another example may help - when in a context looping through hits (I have added // comments purely for clarity - they aren't valid Mustache syntax & should be removed):
{{#ctx.payload.hits.hits}}
// This works fine
{{_source.foo}}
// Not working if each hit's _source contains "bar.baz", not nested "bar">"baz"
{{_source.bar.baz}}
{{/ctx.payload.hits.hits}}
Applying the same workaround by adding an extra context/section:
{{#ctx.payload.hits.hits}}
// Put us in the context of [the current hit] > _source
{{#_source}}
// Now both of these work...
{{foo}}
// ...including this one containing a dot (yay!)
{{bar.baz}}
{{/_source}}
{{/ctx.payload.hits.hits}}
There is no way to directly access source fields that have dots in them, but if you apply a transform like this:
"transform": {
"script": {
"inline": "return [ 'host' : ctx.payload.hits.hits[0]._source.host, 'tbroker_value' : ctx.payload.hits.hits[0]._source['tbroker.workitem.sync_check.tbroker_value']]",
"lang": "painless"
}
}
and then you can use {{ctx.payload.host}} and {{ctx.payload.tbroker_value}} in your action.
I have been at this for hours. How do you get filtering to work?
None of the solutions online works.
All I want to do is grab the comments(highlighted by these <<<>>>) from the json below from the name “Tori Smith”. I’ve been using this app to test http://jsonpath.curiousconcept.com/.
This is as far as I have gotten: ‘comments.data..from.id’
Data:
{
"id":"12029930209393029_10100748134340048",
"from":{
"id":"12029930209393029",
"name":"Tori Smith"
},
"message":"Buy this now",
"picture":"https:\/\/fbexternal-a.akamaihd.net\/app_full_proxy.php?app=141861192518680&v=1&size=z&cksum=a0471c1f5895cd22c74474fabc989c7e&src=http%3A%2F%2Fmedia3.policymic.com%2FYTM2OWUwN2Q0MSMvRnlpUTkxZU9DMWtGWFZ6TUNiYWh3RkxveXRjPS8yeDE6MTI4Nng2MjIvMTI4MHg2MjAvZmlsdGVyczpxdWFsaXR5KDcwKS9odHRwOi8vczMuYW1hem9uYXdzLmNvbS9wb2xpY3ltaWMtaW1hZ2VzL2JqNTdvbTZxZGd1N3ZpaGtvcWVrNnlzaTI5bW55dGZqanEwMWhuc3FqYjgxc3dkeGcyN2F6czV0eXV0bWJzZTguanBn.jpg",
"link":"http:\/\/mic.com\/articles\/101252\/this-tiny-box-will-let-you-stay-anonymous-on-the-internet",
"name":"This Tiny Box Will Let You Stay Anonymous on the Internet",
"caption":"Mic",
"description":"A simple and elegant solution to a major technology problem.",
"icon":"https:\/\/fbcdn-photos-d-a.akamaihd.net\/hphotos-ak-xpf1\/t39.2081-0\/10333103_752719651432828_1597152122_n.png",
"actions":[
{
"name":"Comment",
"link":"https:\/\/www.facebook.com\/12029930209393029\/posts\/10100748134340048"
},
{
"name":"Like",
"link":"https:\/\/www.facebook.com\/12029930209393029\/posts\/10100748134340048"
}
],
"privacy":{
"description":"Your friends",
"value":"ALL_FRIENDS",
"friends":"",
"networks":"",
"allow":"",
"deny":""
},
"type":"link",
"status_type":"app_created_story",
"application":{
"name":"Mic",
"namespace":"micmediaapp",
"id":"141861192518680"
},
"created_time":"2014-10-14T14:54:54+0000",
"updated_time":"2014-10-15T03:55:19+0000",
"comments":{
"data":[
{
"id":"10100748134340048_10100748984636048",
"from":{
"id":"123094958239849866",
"name":"Don Draper"
},
"message":"Even if I use Tor",
"can_remove":true,
"created_time":"2014-10-15T03:03:29+0000",
"like_count":0,
"user_likes":false
},
{
"id":"10100748134340048_10100749036726658",
"from":{
"id":"12029930209393029",
"name":"Tori Smith"
},
<<< "message":"Yes this can go with you and I think it works for all apps outside of TOR browser", >>>
"can_remove":true,
"created_time":"2014-10-15T03:55:19+0000",
"like_count":0,
"user_likes":false
},
{
"id":"10100748134340048_1010074901234658",
"from":{
"id":"12029930209393029",
"name":"Tori Smith"
},
<<< "message":"Second Text", >>>
"can_remove":true,
"created_time":"2014-10-15T03:55:19+0000",
"like_count":0,
"user_likes":false
}
],
"paging":{
"cursors":{
"after":"WTI5dGJXVnVkRjlqZFhKemIzSTZNVEF4TURBM05Ea3dNelkzTWpZMk5UZzZNVFF4TXpNME5UTXhPVG95",
"before":"WTI5dGJXVnVkRjlqZFhKemIzSTZNVEF4TURBM05EZzVPRFEyTXpZd05EZzZNVFF4TXpNME1qSXdPVG94"
}
}
}
}
To get at the comments you need to introduce a filter expression for filtering to comments by "Tori Smith" and then select the comment field.
Taking it step by step, to get all the comments, you need this:
$.comments.data
Then to filter to only the comments by "Tori Smith" add a filter like this:
$.comments.data[?(#.from.name == 'Tori Smith')]
Finally, to select only the message expand the query like this:
$.comments.data[?(#.from.name == 'Tori Smith')].message
I have tested this using the online JSON query tool here:
http://www.jsonquerytool.com/sample/jsonpathwhereselectcomments
I'm trying to track daily stats for an individual.
I'm having a hard time adding a new day inside "history" and can also use a pointer on updating "walkingSteps" as new data comes in.
My schema looks like:
{
"_id": {
"$oid": "50db246ce4b0fe4923f08e48"
},
"history": [
{
"_id": {
"$oid": "50db2316e4b0fe4923f08e12"
},
"date": {
"$date": "2012-12-24T15:26:15.321Z"
},
"walkingSteps": 10,
"goalStatus": 1
},
{
"_id": {
"$oid": "50db2316e4b0fe4923f08e13"
},
"date": {
"$date": "2012-12-25T15:26:15.321Z"
},
"walkingSteps": 5,
"goalStatus": 0
},
{
"_id": {
"$oid": "50db2316e4b0fe4923f08e14"
},
"date": {
"$date": "2012-12-26T15:26:15.321Z"
},
"walkingSteps": 8,
"goalStatus": 0
}
]
}
db.history.update( ? )
I've been browsing (and attempting) the mongodb documentation but they don't quite break it all the way down to dummies like myself... I couldn't quite translate their examples to my setup.
Thanks for any help.
E = noob trying to learn programming
Adding a day:
user = {_id: ObjectId("50db246ce4b0fe4923f08e48")}
day = {_id: ObjectId(), date: ISODate("2013-01-07"), walkingSteps:0, goalStatus: 0}
db.users.update(user, {$addToSet: {history:day}})
Updating walkingSteps:
user = ObjectId("50db246ce4b0fe4923f08e48")
day = ObjectId("50db2316e4b0fe4923f08e13") // second day in your example
query = {_id: user, 'history._id': day}
db.users.update(query, {$set: {"history.$.walkingSteps": 6}})
This uses the $ positional operator.
It might be easier to have a separate history collection though.
[Edit] On the separate collections:
Adding days grows the document in size and it might need to be relocated on the disk. This can lead to performance issues and fragmentation.
Deleting days won't shrink the document size on disk.
It makes querying easier/straightforward (e.g. searching for a period of time)
Even though #Justin Case puts the right answer he doesn't explain a few things in it extremely well.
You will notice first of all that he gets rid of the resolution on dates and moves their format to merely the date instead of date and time like so:
day = {_id: ObjectId(), date: ISODate("2013-01-07"), walkingSteps:0, goalStatus: 0}
This means that all your dates will have 00:00:00 for their time instead of the exact time you are using atm. This increases the ease of querying per day so you can do something like:
db.col.update(
{"_id": ObjectId("50db246ce4b0fe4923f08e48"),
"history.date": ISODate("2013-01-07")},
{$inc: {"history.$.walkingSteps":0}}
)
and other similar queries.
This also makes $addToSet actually enforce its rules, however since the data in this sub document could change, i.e. walkingSteps will increment $addToSet will not work well here anyway.
This is something I would change from the ticked answer. I would probably use $push or something else instead since $addToSet is heavier and won't really do anything useful here.
The reason for a separate history collection in my view would be what you said earlier with:
Yes, the amount of history items for that day.
So this array contains a set of days, which is fine but it sounds like the figure that you wish to get walkingSteps from, a set of history items, should be in another collection and you set walkingSteps according to the count of the amount of items in that other collection for today:
db.history_items.find({date: ISODate("2013-01-07")}).count();
Referring to MongoDB Manual, $ is the positional operator which identifies an element in an array field to update without explicitly specifying the position of the element in the array. The positional $ operator, when used with the update() method and acts as a placeholder for the first match of the update query selector.
So, if you issue a command to update your collection like this:
db.history.update(
{someCriterion: someValue },
{ $push: { "history":
{"_id": {
"$oid": "50db2316e4b0fe4923f08e12"
},
"date": {
"$date": "2012-12-24T15:26:15.321Z"
},
"walkingSteps": 10,
"goalStatus": 1
}
}
)
Mongodb might try to identify $oid and $date as some positional parameters. $ also is part of the atomic operators like $set and $push. So, it is better to avoid use this special character in Mongodb.