In data factory I'm using a Copy Data activity where the Source is a REST API, that's returning data in JSON format, with a Sink thats a JSON file type in ADLS Storage.
The problem I'm having is that only the first record in the "Domestic Title Nos" array appears in the JSON file. The remaining records, however many, are not to be seen.
Also, when I "Preview" my data in the Source page, it shows all the nested data in the array.
API Call Returns this JSON data
Mapping in my Copy Data activity
File contents after sink to JSON file type
As shown in below screenshot, you have used items as Collection reference.
You need to use “Domestic Title Nos” as Collection reference.
Note: For records where the array marked as collection reference is
empty and the check box is selected, the entire record is skipped.
Refer - Schema and data type mapping in copy activity
In the end it worked out to be easier to download the entire JSON document using
this mapping and then used "Flatten" in a Data Flow to extract the parts that I needed.
Related
The values for data.CustomerNumber do not get written to the file, the other elements do.
The CustomerNumber is Pascal case because it is a custom value in the system I am pulling the data out of.
I have tried the map complex values option, advanced editor, deleted, started over. I originally was trying to save this to a Azure Sql table but gave up , and just tried to write to a file. I have been able to do this type of mapping in other pipelines so I am really at a loss for why it doesn't work in this case.
The actual json content for the mapping
I am a beginner in angular. My question is - Based on the excel file which user uploads, api returns some data in JSON format. So the column headers and data are dynamic. I want to display the json data in HTML table.
I want to know how should my json look like and its respective TS and HTML code.
User upload an excel file like this
file example
My output should look like this :
output
First column and row is dynamic.
As you are new to Angular, I will show you one of the procedure you can use
Angular part:
Every time the server sends data, you can get your data by HTTP methods or web-socket by "service" from angular.
Service:
in services, you can import HTTP to get data and after you get data you can emit it by using "behavior-subject" to any component you want which contains your tables and save data values there. (behavior-subject is from rxjs library).
Component:
Now your component in angular has the json data every time and you can analyze
your JSON file, based on your JSON shape you can use *ngfor in the HTML part of the component table and it will change the table online.
I'm currently working with JSON API data which updated daily by my data team. I only need to extract certain information from the JSON file using pandas. So I managed to transform the data by pulling 1 sample JSON file from the API provided. However, I also need to update the transformed data in pandas with other JSON files from the API and I'm stuck here. FYI the JSON files from the API contain in folders name by according to date pulled
The following is the code I used to pull 1 sample JSON data for the transformation process.
#pull 1 sample data from API
df = requests.get('http://phpweb.123.co/atz-pull/output/2019-07-30/abc123.json').json()
So I managed to transform the data into the form i required. However I need to update the transformed data with other json files from different date folders.
Let say the output folders from the api also consist of the following (note that the json file name is unique ;
/output/2019-07-30/abc456.json
/output/2019-07-30/def678.json
/output/2019-07-29/wef678.json
Appreciate helps
I would like to know if I could filter a report before download it's data inside the csv file, I'm using this url:
https://secure.gooddata.com/gdc/app/projects/project_id/execute/raw/
I couldn't found any information at gooddata api about this, the body of the post request accept this json:
{'report_req': {'reportDefinition': '/gdc/md/{project_id}/obj/{obj_id}'} }
There are other parameters for filtering the data?
If don't, there is another way to download filtered data?
If there is another way, can I change from .csv to .json? Or maybe get the json data inside the response body?
Definition of filters is a part of "reportDefinition". The solution is to use or create report definition which contains required filters and perform the mentioned "execute/raw" call. Result of "execute/raw" call is always in csv format.
For Example,
I have a document in my CouchDB database that holds two fields
"password" and "username".
This is the URL for my document:http://127.0.0.1:5984/_utils/document.html?sgram/fdcfc14940fbaa0d86674046ce005107
I want to retrieve the value of these fields from that specific document in CouchDB and parse it in Xcode.
I tried using http-get but it just returns the entire page source.
You want to make a request to this endpoint
http://127.0.0.1:5984/sgram/fdcfc14940fbaa0d86674046ce005107
For a json response from couchdb just don't include _utils/document.html in between. If you do it will give you back the futon html page.