I'm trying to get result from BigQuery view in a google sheets.
To do that I'm using a script from Ido Green that he posted on your blog. https://greenido.wordpress.com/2013/12/16/big-query-and-google-spreadsheet-intergration/
It works well for Tables, but I need to extend to query Views.
When I'm using SQL with View I get this error:
GoogleJsonResponseException: Invalid table name: `my_project:Data_Set.1_0_View_all_users_with_domain_names` [Try using standard SQL (https://cloud.google.com/bigquery/docs/reference/standard-sql/enabling-standard-sql)].
If you don't specify otherwise the BigQuery API will default to LegacySQL and you're using StandardSQL notation. You can try setting useLegacySql to false by replacing line 31 to something like:
queryRequest.setQuery(sql).setTimeoutMs(100000).setUseLegacySql(false);
Related
I am trying to create a new field by creating a calculated field and parsing some json in Google DataStudio. However I keep getting an error saying Failed to fetch data from the underlying data set if I use REGEX_EXTRACT
formula I am using:
REGEXP_EXTRACT(json,'(?<="received_at":")(.*?)(?=")')
json string:
{"end_device_ids":{"device_id":"eui-a81758fffe0600cc","application_ids":{"application_id":"3cc54cfd-a809-42e3-89bb-4497ec67fe42"},"dev_eui":"A81758FFFE0600CC","join_eui":"0000000000000000","dev_addr":"260B1B1E"},"correlation_ids":["as:up:01FG833XJCCNBGDDNDYDRV9SRR","gs:conn:01FG16JYQBRKZC3K8HXE6CWK89","gs:up:host:01FG16JZSSC6GWA8TNY733T5GK","gs:uplink:01FG833XBKQGR83P9MTXSEQQHE","ns:uplink:01FG833XBR0NHHAM079T00ZC76","rpc:/ttn.lorawan.v3.GsNs/HandleUplink:01FG833XBR8SJYB9BMX8ZC61YY","rpc:/ttn.lorawan.v3.NsAs/HandleUplink:01FG833XJBN6BQX9HP4MYK51YR"],"received_at":"2021-09-23T01:08:02.252783252Z","uplink_message":{"session_key_id":"AXvkNZkkbAdO+pD5hVSLXg==","f_port":5,"f_cnt":1230,"frm_payload":"AQDoAjwHDjY=","decoded_payload":{"humidity":60,"temperature":23.2,"vdd":3638},"rx_metadata":[{"gateway_ids":{"gateway_id":"iot-stadslab-1","eui":"A84041FFFF1F9951"},"time":"2021-09-23T01:08:06.175212Z","timestamp":2063422380,"rssi":-125,"channel_rssi":-125,"snr":-8.2,"location":{"latitude":51.69522207692182,"longitude":5.293667793175701,"altitude":15,"source":"SOURCE_REGISTRY"},"uplink_token":"ChwKGgoOaW90LXN0YWRzbGFiLTESCKhAQf//H5lREKyn9dcHGgsI8qavigYQroDmECDgr7vshpo1","channel_index":4}],"settings":{"data_rate":{"lora":{"bandwidth":125000,"spreading_factor":10}},"data_rate_index":2,"coding_rate":"4/5","frequency":"867300000","timestamp":2063422380,"time":"2021-09-23T01:08:06.175212Z"},"received_at":"2021-09-23T01:08:02.040299926Z","consumed_airtime":"0.370688s","network_ids":{"net_id":"000013","tenant_id":"ttn","cluster_id":"ttn-eu1"}}}
My regular expression is working as expected inside Regex101. Why is my formula not working? If I try to manually add a column in google sheets using the REGEXEXTRACT(A2, "(?<=\"received_at\":\").*?(?=\")") function I am also getting an error that tells me nothing. Any solution to my problem is appreciated.
Thank you in advance.
Your regex is
=regexextract(A2,"""received_at"":"".*?""")
I'm using a spreadsheet as a database to store client information indexed by case number. I search and filter the data using regular expression. E.g., Search for case number H123456 and/or H987654, use regex H123456|H987654. A series of scripts does the following:
Copies and pastes the filtered data to another sheet where it can be edited.
Queries my Gmail and Drive for results containing the case numbers and returns corresponding content. Here's the problem.
Everything works perfectly when I only search for one case number. But when I search for multiple case numbers and the | is inserted, the copy and paste scripts fail to run. The Gmail query and drive query runs fine. Why would that be? The only thing I see that's different is that the search term includes |. FYI - I don't get an error. The scripts finish without error message.
Manipulating Pivot table in Google Adwords Script is not yet supported. Is there any script or project alternative for this?
You can generate a new tab, and insert in any cell formula with a Google Query like that:
=QUERY(IMPORTRANGE("1fSdx7f3rg_Vp_11yFuqKZmHraqFit8A", "Headline1!A1:J"), "select Col1, sum(Col4), sum(Col5), sum(Col7), sum(Col8) group by Col1", 1)
, where you specify the source of the data, the query and a headers parameter.
Note, that the columns should be named like this: Col1,.Col2 etc. So the columns order should be strict (if you want to generate a report regularly).
Google Visualization API Query Language (used in the "query" parameter) is almost the same as SQL, but with some limitations on commands that can be used.
I have enabled the Advanced Drive Service in an apps script. For a file in Google Drive, I need to set metadata properties that may already exist, so I am trying to use the update method of Drive.Properties. This method is supposed to "Update a property or add it if it doesn't exist." (See Properties: update.)
The following code fails silently (with nothing logged to the Logger and no properties added to the file). However, if I step through these same lines, I do see the catch block executed every time.
var fileId = '1jgoihblahblahblah' /* an existing file id */;
var property = {key: 'TestKey', value: 'TestValue', visibility: 'PUBLIC'}
try {
Drive.Properties.update(property, fileId, "TestKey");
} catch (e) {
Logger.log(e);
}
If I replace the call to update with this line:
Drive.Properties.insert(property, fileId);
... then the code works every time. The properties are written and are returned by calling Drive.Properties.list(fileId).
Testing the same values using the API explorer (with real values or the "TestKey" and "TestValue" examples above) always works correctly, so I don't think it's simply a matter of providing bad data. UPDATE: The API explorer is now returning an error "Property keys may only contain letters, numbers, and the characters .!#$%^&*()-_/" no matter what values I pass in. However, there are obviously no invalid characters in key names like "TestKey" and "Link" (which is one of the actual keys I am using).
1) How can I troubleshoot the error from within the Script editor? I can't yet figure out how to retrieve the error text, since update fails silently even without the try/catch.
2) I am using the syntax specified by the autocomplete prompt, since there's not actually any documentation for Apps Script syntax for advanced service.
Is that syntax correct? If so, am I doing something wrong that I'm not seeing?
This problem is irrelevant because despite its name, Drive.Properties.insert does the same thing.
See https://issuetracker.google.com/issues/36759642
When developing a Data Flow I don't always want to output the results to a destination - but I would like to see the data.
Is there a way to attach a Data viewer to an output without having to have a destination?
The file and raw destination have limitations on the data type they accept - and I don't want to attach conversions just to test/build code.
Is there some kind of output to null ? i could then get a data view on the result set
There is a (free) custom "trash" destination available from a third party:
http://www.sqlis.com/post/Trash-Destination-Adapter.aspx
I usually use Export column transform - if left with default configuration, it does nothing, so it is equivalent to the custom "Trash" destination mentioned by Ed, but you don't have to install anything.
When debugging and wanting to view the data in the buffer, I usually throw in a Union All and connect it below the component who's output I want to see. Then add a Data Viewer on the connector and voila, there it is!
There isn't really. You can use a RecordSet Destination, or a Row Count transform instead.