I am getting this error message while I am trying to insert my data into line table, using coordinates for points between road segments.
ERROR: parse error - invalid geometry
HINT: "SRID=27700;LINESTRING((5" <-- parse error at position 24 within geometry
SQL state: XX000
Part of my code:
NSERT INTO public."RoadSegments"("no", "seg_ID", "description", "location", "length", "the_geom")
VALUES
(1,'Seg_1','Shephards Bush to Royal Crescent','Shephards Bush','540',ST_GeomFromEWKT('SRID=27700;LINESTRING((51.504593 -0.220437),(51.505233 -0.214105))')),
(2,'Seg_2','Royal Crescent to Norland Square','Notting Hill','306',ST_GeomFromEWKT('SRID=27700;LINESTRING((51.505233 -0.214105),(51.506053 -0.209956))')),
(3,'Seg_3','Norland Square to Holland Park','Notting Hill','383',ST_GeomFromEWKT('SRID=27700;LINESTRING((51.506053 -0.209956),(51.507575 -0.204795))')),
(4,'Seg_4','Holland Park to Notting Hill Gate','Notting Hill','477',ST_GeomFromEWKT('SRID=27700;LINESTRING((51.507575 -0.204795),(51
You seem to have too much parentheses in your LINESTRING parameters. Try the following:
INSERT INTO public."RoadSegments"("no", "seg_ID", "description", "location", "length", "the_geom")
VALUES
(1,'Seg_1','Shephards Bush to Royal Crescent', 'Shephards Bush','540',ST_GeomFromEWKT('SRID=27700;LINESTRING(51.504593 -0.220437,51.505233 -0.214105)')),
(2,'Seg_2','Royal Crescent to Norland Square', 'Notting Hill', '306',ST_GeomFromEWKT('SRID=27700;LINESTRING(51.505233 -0.214105,51.506053 -0.209956)')),
(3,'Seg_3','Norland Square to Holland Park', 'Notting Hill', '383',ST_GeomFromEWKT('SRID=27700;LINESTRING(51.506053 -0.209956,51.507575 -0.204795)')),
(4,'Seg_4','Holland Park to Notting Hill Gate','Notting Hill', '477',ST_GeomFromEWKT('SRID=27700;LINESTRING(51.507575 -0.204795,51 ... '))
This should be the correct syntax (no extra parentheses around the coordinates)
Related
I am new to Data Scraping. I am reading the data from a file having JSON objects as one row
{"name": "Soul Sweet \u2018Taters (Step-by-Step!)", "ingredients": "4 whole Medium Sweet Potatoes\n1 cup Sugar\n1 cup Milk\n2 whole Eggs\n1 teaspoon Vanilla Extract\n1 teaspoon Salt\n1 cup Brown Sugar\n1 cup Pecans\n1/2 cup Flour\n3/4 sticks Butter", "cookTime": "PT30M","prepTime": "PT45M"}
{"name": "Cranberry-Pomegranate Sauce", "ingredients": "1 bag (about 12 To 16 Oz) Fresh Cranberries\n16 ounces, fluid Pomegranate Juice\n3/4 cups Sugar, More Or Less To Taste", "cookTime": "PT15M","prepTime": "PT2M"}
{"name": "Whiskey Maple Cream Sauce", "ingredients": "1-1/2 cup Heavy Cream\n5 Tablespoons Pure Maple Syrup\n3 Tablespoons Light Corn Syrup\n1 Tablespoon Whiskey (can Add More If Desired)","cookTime": "PT15M","prepTime": "PT5M"}
I am looking for assistance on below:
Filter rows that contain text sugar in the Ingredients column.
convert ISO prep time and ISO cook time to human-readable format to add a new column total time(prep time+cook time) for filtered rows.The ISO time is in minutes and even in Hours
Expected Output
name
total_cook_time
Soul Sweet \u2018Taters (Step-by-Step!)
75M
Cranberry-Pomegranate Sauce
17M
My sample code
import json
from datetime import datetime
currentdate=datetime.today().strftime('%Y/%m/%d')
absolutepath='project/sniper/'+'/'+currentdate+'/*.json'
new_data = []
totaltime = {}
data = [json.loads(line) for line in open('absolutepath', 'r')]
for d in data:
if 'sugar' in d.get('ingredients').lower() # case senstitive
new_data.append(d.get('name'))
total_time = int(d['prepTime'].replace('PT', '').replace('M','')) + int(d['cookTime'].replace('PT', '').replace('M',''))
totaltime[d['name']] = total_time
This is one way to do it, as long as the durations are the same format. Otherwise more logic and/or regex could be friendlier. Assumes you are working with a list of recipes called 'data'.
The new_data list holds recipes with Sugar and totaltime dictionary is the recipe name total time.
new_data = []
totaltime = {}
for d in data:
# print(d.get('ingredients')
if 'Sugar' in d.get('ingredients'): # case senstitive
new_data.append(d.get('ingredients'))
total_time = int(d['prepTime'].replace('PT', '').replace('M','')) + int(d['cookTime'].replace('PT', '').replace('M',''))
totaltime[d['name']] = total_time
totaltime
{'Soul Sweet ‘Taters (Step-by-Step!)': 75, 'Cranberry-Pomegranate Sauce': 17}
I retrieved a dataset from a news API in JSON format. I want to extract the news description from the JSON data.
This is my code:-
import requests
import json
url = ('http://newsapi.org/v2/top-headlines?'
'country=us&'
'apiKey=608bf565c67f4d99994c08d74db82f54')
response = requests.get(url)
di=response.json()
di = json.dumps(di)
for di['articles'] in di:
print(article['title'])
The dataset looks like this:-
{'status': 'ok',
'totalResults': 38,
'articles': [
{'source':
{'id': 'the-washington-post',
'name': 'The Washington Post'},
'author': 'Derek Hawkins, Marisa Iati',
'title': 'Coronavirus updates: Texas, Florida and Arizona officials say early reopenings fueled an explosion of cases - The Washington Post',
'description': 'Local officials in states with surging coronavirus cases issued dire warnings Sunday about the spread of infections, saying the virus was rapidly outpacing containment efforts.',
'url': 'https://www.washingtonpost.com/nation/2020/07/05/coronavirus-update-us/',
'urlToImage': 'https://www.washingtonpost.com/wp-apps/imrs.php?src=https://arc-anglerfish-washpost-prod-washpost.s3.amazonaws.com/public/K3UMAKF6OMI6VF6BNTYRN77CNQ.jpg&w=1440',
'publishedAt': '2020-07-05T18:32:44Z',
'content': 'Here are some significant developments:\r\n<ul><li>The rolling seven-day average for daily new cases in the United States reached a record high for the 27th day in a row, climbing to 48,606 on Sunday, … [+5333 chars]'}])
Please guide me with this!
There are few corrections needed in your code.. below code should work and i have removed API KEY in answer make sure that you add one before testing
import requests
import json
url = ('http://newsapi.org/v2/top-headlines?'
'country=us&'
'apiKey=<API KEY>')
di=response.json()
#You don't need to dump json that is already in json format
#di = json.dumps(di)
#your loop is not correctly defined, below is correct way to do it
for article in di['articles']:
print(article['title'])
response.json
{'status': 'ok',
'totalResults': 38,
'articles': [
{'source':
{'id': 'the-washington-post',
'name': 'The Washington Post'},
'author': 'Derek Hawkins, Marisa Iati',
'title': 'Coronavirus updates: Texas, Florida and Arizona officials say early reopenings fueled an explosion of cases - The Washington Post',
'description': 'Local officials in states with surging coronavirus cases issued dire warnings Sunday about the spread of infections, saying the virus was rapidly outpacing containment efforts.',
'url': 'https://www.washingtonpost.com/nation/2020/07/05/coronavirus-update-us/',
'urlToImage': 'https://www.washingtonpost.com/wp-apps/imrs.php?src=https://arc-anglerfish-washpost-prod-washpost.s3.amazonaws.com/public/K3UMAKF6OMI6VF6BNTYRN77CNQ.jpg&w=1440',
'publishedAt': '2020-07-05T18:32:44Z',
'content': 'Here are some significant developments:\r\n<ul><li>The rolling seven-day average for daily new cases in the United States reached a record high for the 27th day in a row, climbing to 48,606 on Sunday, … [+5333 chars]'}]}
Code:
di = response.json() # Understand that 'di' is of type 'dictionary', key-value pair
for i in di["articles"]:
print(i["description"])
"articles" is one of the keys of dictionary di, It's corresponding value is of type list. "description" , which you are looking is part of this list (value of "articles"). Further list contains the dictionary (key-value pair).You can access from key - description
Hi I´m in need of help to extract a field from a JSON Helper return
Using the following syntax
tell application "JSON Helper"
set Resultado to (fetch JSON from Request)
end tell
I get the following result from a Google Geocoding API request.
{results:{{formatted_address:"Rua Dr. Carmelo D'Agostino, 628 -
Jardim Rincão, São Paulo - SP, 02991-040, Brazil", partial_match:true,
address_components:{{short_name:"628", long_name:"628",
types:{"street_number"}}, {short_name:"Rua Dr. Carmelo D'Agostino",
long_name:"Rua Doutor Carmelo D'Agostino", types:{"route"}},
{short_name:"Jardim Rincão", long_name:"Jardim Rincão",
types:{"political", "sublocality", "sublocality_level_1"}},
{short_name:"São Paulo", long_name:"São Paulo",
types:{"administrative_area_level_2", "political"}}, {short_name:"SP",
long_name:"São Paulo", types:{"administrative_area_level_1",
"political"}}, {short_name:"BR", long_name:"Brazil", types:{"country",
"political"}}, {short_name:"02991-040", long_name:"02991-040",
types:{"postal_code"}}},
geometry:{viewport:{northeast:{lat:-23.432391219708,
lng:-46.725645019708}, southwest:{lat:-23.435089180292,
lng:-46.728342980292}}, location:{lat:-23.4337402, lng:-46.726994},
location_type:"ROOFTOP"}, place_id:"ChIJgVGzRqL7zpQRTQPNqsmBVLY",
types:{"street_address"}}}, status:"OK"}
I would like to extract just
02991-040
which appears three times on the returning message,
once as part of the formatted address:
formatted_address:\"Rua Dr. Carmelo D'Agostino, 628 - Jardim Rincão,
São Paulo - SP, 02991-040, Brazil
and twice as the postal code
{short_name:"02991-040", long_name:"02991-040", types:{"postal_code"}
I have spent quite some time trying to find it by myself but may be I haven't tried hard / smart enough so would really appreciate any help.
Best
The script has three variables firstResult, secondResult and thirdResult which will contain the three values you want to extract.
if Resultado's status is "OK" then
set theData to item 1 of results of Resultado
set formattedAddress to formatted_address of theData
set {TID, text item delimiters} to {text item delimiters, ", "}
set firstResult to text item 4 of formattedAddress
set text item delimiters to TID
set components to address_components of theData
tell last item of components
set secondResult to its long_name
set thirdResult to its short_name
end tell
end if
I'm using the following syntax:
tell application "JSON Helper"
set Resultado to (fetch JSON from Request)
end tell
and I get:
{results:{{formatted_address:\"Rua Dr. Carmelo D'Agostino, 628 - Jardim Rincão, São Paulo - SP, 02991-040, Brazil\", partial_match:true, address_components:{{short_name:\"628\", long_name:\"628\", types:{\"street_number\"}}, {short_name:\"Rua Dr. Carmelo D'Agostino\", long_name:\"Rua Doutor Carmelo D'Agostino\", types:{\"route\"}}, {short_name:\"Jardim Rincão\", long_name:\"Jardim Rincão\", types:{\"political\", \"sublocality\", \"sublocality_level_1\"}}, {short_name:\"São Paulo\", long_name:\"São Paulo\", types:{\"administrative_area_level_2\", \"political\"}}, {short_name:\"SP\", long_name:\"São Paulo\", types:{\"administrative_area_level_1\", \"political\"}}, {short_name:\"BR\", long_name:\"Brazil\", types:{\"country\", \"political\"}}, {short_name:\"02991-040\", long_name:\"02991-040\", types:{\"postal_code\"}}}, geometry:{viewport:{northeast:{lat:-23.432391219708, lng:-46.725645019708}, southwest:{lat:-23.435089180292, lng:-46.728342980292}}, location:{lat:-23.4337402, lng:-46.726994}, location_type:\"ROOFTOP\"}, place_id:\"ChIJgVGzRqL7zpQRTQPNqsmBVLY\", types:{\"street_address\"}}}, status:\"OK\"}." number -1728 from item 1 of {results:{{formatted_address:"Rua Dr. Carmelo D'Agostino, 628 - Jardim Rincão, São Paulo - SP, 02991-040, Brazil", partial_match:true, address_components:{{short_name:"628", long_name:"628", types:{"street_number"}}, {short_name:"Rua Dr. Carmelo D'Agostino", long_name:"Rua Doutor Carmelo D'Agostino", types:{"route"}}, {short_name:"Jardim Rincão", long_name:"Jardim Rincão", types:{"political", "sublocality", "sublocality_level_1"}}, {short_name:"São Paulo", long_name:"São Paulo", types:{"administrative_area_level_2", "political"}}, {short_name:"SP", long_name:"São Paulo", types:{"administrative_area_level_1", "political"}}, {short_name:"BR", long_name:"Brazil", types:{"country", "political"}}, {short_name:"02991-040", long_name:"02991-040", types:{"postal_code"}}}, geometry:{viewport:{northeast:{lat:-23.432391219708, lng:-46.725645019708}, southwest:{lat:-23.435089180292, lng:-46.728342980292}}, location:{lat:-23.4337402, lng:-46.726994}, location_type:"ROOFTOP"}, place_id:"ChIJgVGzRqL7zpQRTQPNqsmBVLY", types:{"street_address"}}}, status:"OK"}
now I want to get just the "location" in other words I just want these two values
lat:-23.4952611, lng:-46.6724604
It should be no brainer but I wasn't able to do it myself after several hours trying so anyhelp will be much appreciated!
#vanian taking advantage of your skills (and thanking you very much for it) can you please tell me now how to get:
"02991-040" that appears on the "formatted_address" and also here
{short_name:\"02991-040\", long_name:\"02991-040\", types:{\"postal_code\"}
Best!
The JSON is converted to AppleScript lists and records
if Resultado's status is "OK" then
tell location of geometry of item 1 of results of Resultado
set latitude to its lat
set longitude to its lng
end tell
end if
I have a web service returning following json:
[
{
"id": "9469",
"title": "A person grabbed by police because being Nigerian he was carrying a Ghanaian passport!",
"introtext": "A person has grabbed by police because being Nigerian he was having a Ghanaian passport!
An individual has gotten by police on the grounds that being Nigerian he was having a Ghanaian visa!
A 29-year-old Nigerian has been captured for endeavoring to get a visa with a falsely acquired Ghanaian travel permit."
}
]
JSONLint is giving following error:
Parse error on line 5:
... "introtext": "A person has grabbe
-----------------------^
Expecting 'STRING', 'NUMBER', 'NULL', 'TRUE', 'FALSE', '{', '['
I am really unable to understand what is making json invalid here? Is it newline or what? What can I do to make it work? Thanks.
New line present in introtex, this is valid json check it
[
{
"id": "9469",
"title": "A person grabbed by police because being Nigerian he was carrying a Ghanaian passport!",
"introtext": "A person has grabbed by police because being Nigerian he was having a Ghanaian passport! An individual has gotten by police on the grounds that being Nigerian he was having a Ghanaian visa! A 29-year-old Nigerian has been captured for endeavoring to get a visa with a falsely acquired Ghanaian travel permit."
}
]
Put your whole string in one line.
example
[
{
"id": "9469",
"title": "A person grabbed by police because being Nigerian he was carrying a Ghanaian passport!",
"introtext": "A person has grabbed by police because being Nigerian he was having aGhanaian passport! An individual has gotten by police on the grounds that being Nigerian he was having a Ghanaian visa! A 29-year-old Nigerian has been captured for endeavoring to get a visa with a falsely acquired Ghanaian travel permit."
}
]
In case you want line break use
"introtext": "A person has grabbed by police because \n being Nigerian he was having aGhanaian passport! An individual has gotten by police on the grounds that being Nigerian he was having a Ghanaian visa! A 29-year-old Nigerian has been captured for endeavoring to get a visa with a falsely acquired Ghanaian travel permit."
Thanks to #MONTYHS and #AvinashGarg for pointing out the mistake. Newline is causing invalid JSON. To have break lines in my json, I am replacing all newline tags <br/> with a special character, e.g. |~ (bar and a tilde). At client side, after parsing json, I replace occurrences of |~ with <br/> back to display properly.
Hope it helps someone. Reference is here.