I have slate dashboard which uses fusion sheet for backend data and fusion service api/fusion queries to retrieve data from fusion sheets.
I want write data into fusion sheet with one of the column having array type of data.
Does anyone knows how to write back array data type into fusion sheet using fusion query?
What I tried
giving this payload to fusion query result in error Invalid argument
data = {
rows: [
"columnID" : {
"type": "untypedString",
"untypedString": ['a','b']
}
]}
giving this payload to fusion query it writes data as it is (I thought it will be interpreted as array in fusion sheet cell)
data = {
rows: [
"columnID" : {
"type": "untypedString",
"untypedString": "=array('a','b')"
}
]}
WHAT I WANT
to write array data in fusion sheet
Could you try with this :
{
"rows": [
{
"columnID": {
"type": "cellValue",
"cellValue": {
"type": "stringArray",
"stringArray": [
"A", "B"
]
}
}
}
]
}
Instead of this :
{
"rows": [
{
"columnID": {
"type": "untypedString",
"untypedString": "test"
}
}
]
}
For reference, this is possible to find it in the dev docs, by looking at the specific query you are using and the objects it can take as arguments.
Related
I've got a process that reads a mapping configuration
stored in smartsheet.
This is where the process admins can control the data flow.
Ultimately this will be stored in a snowflake table to be used by other Talend flows.
I've brought it to this point which includes getting updated names formapped columns and sheets.
My objective now is to create JSON to add the new rows to Smartsheet and also update existing rows.
The only difference in the two calls is the inclusion of the row id in Smartsheet.
For this example, I am focusing on new rows.
I'm confident I can adapt any solution that addresses new rows to updating existing rows as it only involves one more JSON attribute.
I'm having a little trouble wrapping my head around Smartsheet's unique way of storing rows and columns in JSON responses.
Each Row is a collection of cells.
Each cell is a collection of attributes which includes the column id the cell belongs to.
Here is what I have at this point
Data to be converted to JSON in a table read from cache memory (tHashOutput and tHashInput via several steps addressing other requirements)
LOADED_DATE_TIME_STR
SSHEET_NAME
SSHEET_ID
SSHEET_ROW_ID
SSHEET_COL_ID
SSHEET_COL_NAME
DB_TBL_NAME
DB_COL_NAME
20220221232059
sheet_name_1_str
sheet_id_1_int
null
xxxxxxxxxxxxxxx1
sheet_1_col_name_a_str
null
null
20220221232059
sheet_name_2_str
sheet_id_2_int
null
xxxxxxxxxxxxxxx2
sheet_2_col_name_b_str
null
null
20220221232059
sheet_name_2_str
sheet_id_2_int
null
xxxxxxxxxxxxxxx3
sheet_2_col_name_c_str
null
null
The mapping configuration sheet has 5 important columns
that I have not yet incorported their mapping and ids into the process.
I will do this once I have an idea of where this part of the flow is headed
LOADED_DATE_TIME_STR = col_id_1_int
SSHEET_NAME = col_id_2_int
SSHEET_ID = col_id_3_int
SSHEET_COL_ID = col_id_4_int
SSHEET_COL_NAME = col_id_6_int
Output json format: (this will be a sub-element of a larger JSON tree).
Specifically, each array of cells defines a row and will be a sub-element
of the row within the Smartsheet API structure
{
"cells": [
{
"columnId": col_id_1_int,
"value": "20220221232059"
},
{
"columnId": col_id_2_int,
"value": "sheet_name_1_str"
},
{
"columnId": col_id_3_int,
"value": "sheet_id_1_int"
}
{
"columnId": col_id_4_int,
"value": "xxxxxxxxxxxxxxx1"
}
{
"columnId": col_id_6_int,
"value": "sheet_1_col_name_a_str"
}
]
},
{
"cells": [
{
"columnId": col_id_1_int,
"value": "20220221232059"
},
{
"columnId": col_id_2_int,
"value": "sheet_name_2_str"
},
{
"columnId": col_id_3_int,
"value": "sheet_id_2_int"
}
{
"columnId": col_id_4_int,
"value": "xxxxxxxxxxxxxxx2"
}
{
"columnId": col_id_6_int,
"value": "sheet_2_col_name_b_str"
}
]
},
...
I'm currently trying to extract data out of Log Analytics through its REST API. I have been successful at using a Copy Data activity to store the response in an Azure Data Lake Gen 2 account.
The format is roughly similar to the example from the Log Analytics API Reference Page.
{
"tables": [
{
"name": "PrimaryResult",
"columns": [
{
"name": "Category",
"type": "string"
},
{
"name": "count_",
"type": "long"
}
],
"rows": [
[
"Administrative",
20839
],
[
"Recommendation",
122
],
[
"Alert",
64
],
[
"ServiceHealth",
11
]
]
}
] }
My dataset is much larger with more columns more values etc but the principals are the same.
What I am trying to do is generate a new JSON file that would hold the table but multiple documents in the same file e.g.
[{
"Category": "Administrative",
"count_": 20839
},
{
"Category": "Recommendation",
"count_": 122
},
{
"Category": "Alert",
"count_": 64
},
{
"Category": "ServiceHealth",
"count_": 11
}]
The output of this would be stored back into the data lake and then ideally could be used as a source for a copy activity to go into an Azure SQL Database.
I have tried accomplishing this using Data Flows Flattening but haven't been successful with this up until this point as when trying to map the column name it doesn't see individual column names just that level of the document where the column names are defined.
How would I go about flattening the dataset so it appears as desired? Is this an unrealistic expectation of Data flows or is this task more suitable for something like Azure Databricks?
I am reverse engineering an app that sends queries to
SOMESERVERNAME.analysis.windows.net/public/reports/querydata via an HTTP POST of an JSON-structured query.
Some initial lines of a sample query are at the end of this message.
I can't find any documentation on this anywhere. I don't know if this is some secret API or what. I ultimately would like to just ignore the aggregations altogether and just dump the raw data, which seems to sit in some flat-file type container on the back-end, but without some API documentation I'm stuck with just re-running the super basic handful of queries I've been able to intercept.
Note: this app is an embedded analytics page created with PowerBI, but the only REST API I can find for PowerBI has nothing to do with querying, but just basic object management.
Thanks!
{
"version": "1.0.0",
"queries": [
{
"Query": {
"Commands": [
{
"SemanticQueryDataShapeCommand": {
"Query": {
"Version": 2,
"From": [
{
"Name": "s",
"Entity": "Sheet1"
}
],
"Select": [
{
"Aggregation": {
"Expression": {
"Column": {
"Expression": {
"SourceRef": {
"Source": "s"
}
},
"Property": "Total"
}
},
"Function": 0
},
"Name": "Sum(Sheet1.Total)"
}
],
"Where": [
{
"Condition": {
"In": {
"Expressions": [
{
"Column": {
"Expression": {
"SourceRef": {
"Source": "s"
}
},
"Property": "Year"
}
}
],
"Values": [
[
{
"Literal": {
"Value": "'2018'"
}
}
]
]
}
}
},
............
I have built a client that scrapes data off a specific Power BI report using the same API, but probably you'll be able to adapt it to your use case. Maybe we can even abstract the code into a more generalized Power BI client!
Having tinkered with the API for two days, I realised that there are many ways the data can be formatted:
"nested"/multidimensional data can be unflattened, flattened by 1 degree, etc.
a primary "table" of a result dataset (in data.PH) can reference others (in data.SH)
The basics are as follows:
A dataset is structured like a multidimensional table, with cells containing values.
In a set of cells, the first always has a field S that contains the schema of its and all subsequent cells.
The schema maps a field of each cell's object with a selection from your query, e.g. the G0 field with the queried column age.
My client seems to work only with a specific type of query (SemanticQueryDataShapeCommand), a specific nr of dimensions and a specific column marked as primary (via Binding.Primary). But maybe that helps! https://github.com/derhuerst/fetch-bvg-occupancy/blob/1ebb864b1ff7130f9d2f0ab031c6d78bcabdd633/lib/parse-dataset.js
The only documented way to use this API is through the ADOMD.NET or OleDb provider.
If you want to send a DAX/MDX query and retrieve data programmatically, there's a sample of how to front-end the service with a simple REST API here.
Similarly to the bq command-line, can the bigquery googleSheetsOptions range be used when defining bigquery queries in Apps Script (externalDataConfiguration.googleSheetsOptions.range)?
https://cloud.google.com/bigquery/docs/reference/rest/v2/tables
For example, given a table definition like google_sheets_tabeledef.json below, can I pass this table definition as a bigquery method in Apps Script?
{
"autodetect": false,
"sourceFormat": "GOOGLE_SHEETS",
"sourceUris": [
"https://docs.google.com/spreadsheets/d/xxxxxxxxxxxxxx"
],
"maxBadRecords": 1,
"googleSheetsOptions":
{
"range": "test_sheet!A1:B20",
"skipLeadingRows": 0
},
"schema" : {
"fields": [
{
"name": "col1",
"type": "string"
},
{
"name": "col2",
"type": "int64"
},
]
}
}
Related question:
bigQuery Google Drive query multiple sheets with googleSheetsOptions range
I believe that this has now (answer written in Aug-2019) been added to the table definition in bigQuery. So when defining the table via the webpage, there is now a range section as shown in the screenshot below:
specify range in BigQuery Google Sheets table connector
I'm Working on AngularJS.
In this part of the project my goal is to obtain a JSON structure after filling a form with some particulars values.
Here's the fiddle of my simple form: Fiddle
With the form I will do a query to KairosDB, that is my NoSql Database, I will query data from it by a JSON object. The form is structured in this way:
a Name
a certain Number of Tags, with Tag Id ("ch" for example) and tag value ("932" for example)
a certain Number of Aggregators to manipulate data coming from DB
Start Timestamp and End Timestamp (now they are static and only included in the final JSON Object)
After filling this form, with my code I'll obtain for example this JSON object:
{
"metrics": [
{
"tags": [
{
"id": "ch",
"value": "932"
},
{
"id": "ch",
"value": "931"
}
],
"aggregators": {
"name": "sum",
"sampling": [
{
"value": "1",
"unit": "milliseconds",
"type": "SUM"
}
]
}
}
],
"cache_time": 0,
"start_absolute": 123,
"end_absolute": 1234
}
Unfortunately, KairosDB accepts a different structure, and as you could see, Tag id "ch" doesn't hase an "id" string before, or for example, Tag values coming from the same tag id are grouped together
{
"metrics": [
{
"tags": {
"ch": [
"932",
"931"
]
},
"name": "AIENR",
"aggregators": [
{
"name": "sum",
"sampling": {
"value": "1",
"unit": "milliseconds"
}
}
]
}
],
"cache_time": 0,
"start_absolute": 1367359200000,
"end_absolute": 1386025200000
}
My question is: Is there a way to obtain the JSON structure like the one accepted by Kairos DB with an Angular JS form?. Thanks to everyone.
I've seen this topic as the one more similar to mine but it isn't in AngularJS.
Personally, I'd do the refactoring work in the backend - Have what ever server interfaces sends and receives data do the manipulation - Otherwise you'll end up needing to refactor your data inside Angular anywhere you want to use that dataset.
Where as doing it in the backend would put it in a single access point.
Of course, you could do it in Angular, just replace userString in the submitData method with a copy of the array and replace the tags section with data in the new format, and likewise refactor the returned result to the correct format when you get a reply.