So, I'm doing some testing of google cloud storage access. I am trying to manually upload a json file (via the console) that I will programmatically change. But Google doesn't seem to like my json file. The beginning of the file looks like:
{
'mlsid': "MD-BRIGHT",
'mlsno': "1000000300",
'maxres': "2x",
'modified': "20180201133459",
'count': 28,
'photos': {
'org': [
{
'uri': "https://storage.googleapis.com/idx-photos-gs.ihouseprd.com/MD-BRIGHT/1000000300/org/000.jpg",
'width': 625,
'height': 768
},
. . .
Which looks correct to me, but when I try to open the file in a window from the GCS control panel, it displays this error:
Error: Parse error on line 1:
{ 'mlsid': "MD-BRIGHT"
-----^
Expecting 'STRING', '}'
I mean, isn't 'mlsid' a string? I have tried it with both single and double quotes. Quite baffled.
Valid JSON requires double-quotes and not single-quotes for keys and strings.
{
"mlsid": "MD-BRIGHT",
Related
I am using Mini forge and have PDAL installed in my computer.
I cannot get the PDAL to output the output.las file .
If I create an empty las file for the output , nothing happens and if I don't create it in the directory there is an error.
this is my line in the mini forge command prompt:
pdal pipeline autzen.laz shape-clip.json.
The input laz file is fine , I have viewed it using cloud compare.
Here is the code for the shape-clip:
[
{
"type":"readers.las",
"filename":"C:\\Users\\502011\\OneDrive - Amey plc\\Documents\\Programming\\Python\\Point Cloud Processig Project\\PDAL tutorials\\Point_cloud_Data.las"
},
{
"type":"filters.overlay",
"dimension":"Classification",
"datasource":"attributes.vrt",
"layer":"OGRGeoJSON",
"column":"CLS"
},
{
"type":"filters.range",
"limits":"Classification[5:5]"
},
{
"type":"writers.las",
"filename":"C:\\Users\\502011\\OneDrive - Amey plc\\Documents\\Programming\\Python\\Point Cloud Processig Project\\PDAL tutorials\\output.las"
}
bearing in mind I added the type recently before I had this:
[ input.laz
{----},
{======},
etc
,
output.las
]
i've also tried to no avail:
[ input.las
{----},
{======},
etc
,
output.las
]
whats the issue here ?
I was expecting the input las to be transformed and clipped with a section of its point cloud saved to the output.las file.
I want to get some data from a json file. I can access everything with the code below.
import json
with open('C:\\Users\\me\\Documents\\stdin.json', 'r', encoding='utf8', errors='ignore') as json_file:
data = json.load(json_file)
print("Type: ", type(data))
print("VM: ", data["Datacenter"])
The .json file looks like this:
{
"Datacenter":[
{
"Folder":[
{
"Folder":[
{
"VirtualMachine":[
{
"moid":"vm-239566",
"name":"DEV CentOS 6",
},
{
"moid":"vm-239464",
"name":"DEV Sles 12",
},
],
"moid":"group-v239127",
"name":"DEV-VMs"
},
],
"moid":"group-v78",
"name":"Test and Dev"
},
{
"VirtualMachine":[
{
"moid":"vm-66130",
"name":"Hyv16-clone",
}
],
"moid":"group-v77",
"name":"Templates"
}
],
"moid":"datacenter-21",
"name":"Datencenter"
}
],
"vSphereHost":"srv01",
"vSphereProductLine":"vpx",
"vSphereServer":"VMware vCenter Server",
"vSphereVersion":"xxx",
"version":"1.0",
"viewType":"VMs and Templates"
}
Note that the original json file was much bigger as I deleted lines for readabilty. Also note that I run everything from Command Line, as my IDE always gives me the error UnicodeEncodeError: 'charmap' codec can't encode characters in position 22910-22912: character maps to <undefined>
I tried to use data["VirtualMachine"] instead of data["Datacenter"] but then I get an error... TypeError: 'VirtualMachine' is an invalid keyword argument for this function.
So how can I get/print the moid and name of a VM? I am really new to coding and don´t know how to deal with nested dictionarys
However your question does not seems clear but from whatever you have mentioned this can be help you deriving the value in nested json if you have already have dataframe created. You can go ahead and add get() till you reach what you require. Below is the sample that you can use
import json
data = data.apply(lambda x: json.loads(json.loads(x).get("Folder","{}")).get("moid") if x else None)
In my pipeline I reach through REST API using GET request to a 3rd party database. As an output I receive a bunch of JSON files. The number of JSON files I have to download (same as number of iterations I will have to use) is in one of the fields in JSON file. The problem is that the field's name is 'page-count' which contains "-".
#activity('Lookup1').output.firstRow.meta.page.page-count
Data Factory considers dash in field's name as a minus sign, so I get an error instead of value from that field.
{"code":"BadRequest","message":"ErrorCode=InvalidTemplate, ErrorMessage=Unable to parse expression 'activity('Lookup1').output.firstRow.meta.page.['page-count']'","target":"pipeline/Product_pull/runid/f615-4aa0-8fcb-5c0a144","details":null,"error":null}
This is how the structure of JSON file looks like:
"firstRow": {
"meta": {
"page": {
"number": 1,
"size": 1,
"page-count": 7300,
"record-count": 7300
},
"non-compliant-record-count": 7267
}
},
"effectiveIntegrationRuntime": "intergrationRuntimeTest1",
"billingReference": {
"activityType": "PipelineActivity",
"billableDuration": [
{
"meterType": "SelfhostedIR",
"duration": 0.016666666666666666,
"unit": "Hours"
}
]
},
"durationInQueue": {
"integrationRuntimeQueue": 1
}
}
How to solve this problem?
The below syntax works when retrieving the value for a json element with a hyphen. It is otherwise treated as a minus sign by the parser. It does not seem to be documented by Microsoft however I managed to get this to work through trial and error on a project of mine.
#activity('Lookup1').output.firstRow.meta.page['page-count']
This worked for us too. We had the same issue where we couldn't reference an output field that contained a dash(-). We referenced this post and used the square brackets and single quote and it worked!
Example below.
#activity('get_token').output.ADFWebActivityResponseHeaders['Set-Cookie']
I am trying to figure out how to plot data from a local '.JSON' file using angular2-highcharts example.
I followed the example in 'https://www.npmjs.com/package/angular2-highcharts' to first understand how to plot .JSON data and it worked. I took the data available for the example and created a local .JSON file (copied the content from 'https://www.highcharts.com/samples/data/jsonp.php?filename=aapl-c.json&callback=JSONP_CALLBACK' in notepad and saved it with UTF-8 encoding as a .JSON file), and replaced the file path for the JSON request to this. When I do this though, I get an error - response with status 200.
constructor(jsonp : Jsonp) {
//jsonp.get('https://www.highcharts.com/samples/data/jsonp.php?filename=aapl-c.json&callback=JSONP_CALLBACK').subscribe(res => {
jsonp.get('./data.json').subscribe(res => {
this.options = {
title : { text : 'AAPL Stock Price' },
series : [{
name : 'AAPL',
data : res.json(),
tooltip: {
valueDecimals: 2
}
}]
};
});
}
options: Object;
};
Since I am not super familiar with json data/ Javascript or angular2 I am not sure if I am missing something very basic here. Any help is appreciated.
as far as I know, Response Status 200 specifies that request was successful. i.e. your request was successfully handled. perhaps you want to try checking response data.
check your callback for response data.
Using http instead of json helped. I made use of the suggestion in this answer https://stackoverflow.com/a/36305814/4567096.
I have been trying for a few days to get a parameter sent from the API Gateway in AWS to a Lambda function and I am having no success.
I decided to start from the beginning so I followed their walkthrough (http://docs.aws.amazon.com/apigateway/latest/developerguide/getting-started.html#getting-started-new-lambda)
I have checked this walkthrough twice and I have followed the steps to the letter.
Problem
When I test the API from Postman or in Swift I am getting the error:
{"message": "Could not parse request body into json: Unexpected character (\'-\' (code 45)) in numeric value: expected digit (0-9) to follow minus sign, for valid numeric value\n at [Source: [B#c036d15; line: 1, column: 3]"}
In postman, When I view the result as JSON I just get
Bad String
Lambda Function
The function is the basic example from the Walkthrough:
console.log('Loading event');
exports.handler = function(event, context) {
var name = (event.name === undefined ? 'No-Name' : event.name);
console.log('"Hello":"' + name + '"');
context.done(null, {"Hello":name}); // SUCCESS with message
};
When Tested from the Lambda Console and with the Test data I get the result:
{
"Hello": "TestUser123"
}
When Tested from the API Gateway Test, The result is also:
{
"Hello": "TestUser123"
}
Can anyone see why both test consoles are allowing this work but when tested with POSTMAN or used within a Swift Script it does not work ?
Edit 1
In postman, I have set the content-type to application/json
The script returns the default:
{
"Hello": "user"
}
However, When I add in the parameters name and TestUser123 in POSTMAN, this is when it returns the error.
Update 1
Ok, so I changed the mapping template to one that I found on another answer:
{ "name": "$input.params('name')" }
Now the result is:
{
"Hello": ""
}
Any Ideas why it is not getting the name?
I just got stuck with this today.
your mapping template is:
{ "name": "$input.params('name')" }
AWS uses AWS Velocity templates, which; even though looks like JSON, is different.
if you use
{ "name": $input.params('name') } // notice no quotes
for the mapping template right at the integration request step, then it should work as expected.
Read the error message very carefully, it actually tells you the problem. For example, I got
Could not parse request body into json: Unexpected character (\'\"\' (code 34)): was expecting comma to separate Object entries
So the problem is that I'm missing a comma. I check my Lambda's Integration Request - Body Mapping Template:
{
"age" : $input.json('$.persondata.age'),
"income" : $input.json('$.persondata.income')
"height" : $input.json('$.persondata.height')
}
Can you spot the problem? I am missing a comma after the income line.
Here is another example.
Could not parse request body into json: Unexpected character (\'}\' (code 125)): expected a value
When I look at the Integration Request - Body Mapping Template:
#set($inputRoot = $input.path('$'))
{
"age" : $inputRoot.age,
"height" : $inputRoot.height,
"income" : $inputRootincome
}
Can you spot the problem? I am missing a dot in $inputRootincome.
Error Message :
Could not parse request body into json: Could not parse payload into json: Unrecognized token \' \': was expecting (\'true\', \'false\' or \'null\')
Cause of the error : When string values inside the json are not assigned using double quotations in the aws mapping template the error occurs.
Solution : (Please Note : This example is for application/json type request template)
Actually the solution for the problem is, if you are using a value of type string in json then its value should be assigned inside a ("" - double quotation marks) in the mapping template.
The below shown example has the following attributes :
customerId - string (Please note : this value comes from a query parameter)
customerName - string
customerAge - integer
isPermanentEmployee - boolean
customerAddress - string (Please note this is an optional parameter)
And the mapping template should be defined like the example shown below
Refer the example below :
#set($inputRoot = $input.path('$'))
{
"CustomerId": "$input.params('customerId')",
"CustomerName": "$inputRoot.customerName",
"CustomerAge": $inputRoot.customerAge,
"IsPermanentEmployee": $inputRoot.isPermanentEmployee
"CustomerAddress ": #if($inputRoot.customerAddress == "") "" #elseif($inputRoot.customerAddress != "") "$inputRoot.customerAddress" #end
}
If you note the above mapping template, I would have given string values inside double quotation marks("") which will solve the error
Also this example contains how to handle optional parameters in aws mapping templates using #if#else statements.
It is likely that you had copy-pasted multiple lines in your "Integration Request" in the API gateway.
When copying a line and pasting it below, you might have copied the hidden character '\n' at the end of that line. This is probably causing issues at the lambda function.
Example: Copying the line containing age and pasting it twice and modifying them to have height and income respectively.
#set($inputRoot = $input.path('$'))
{
"age" : $inputRoot.age,
"height": $inputRoot.height,
"income": $inputRoot.income
}
Instead of copy-pasting, just type the line out for height and income.