I am starting a project to import a CSV, transform it, and re-export back to a new CSV. Step one has been to get Node CSV up and running.
I have installed the node-csv-npm package to my Meteor app directory and confirmed in terminal that I am "already using" node-csv-npm.
Yet, for some reason, when I run the basic example to test my set-up (see code), I get back "Uncaught ReferenceError: csv is not defined"
Simple Coffee Code I am using:
if Meteor.isClient
console.log "client started."
csv().from.string("#Welcome\n\"1\",\"2\",\"3\",\"4\"\n\"a\",\"b\",\"c\",\"d\"", comment: "#").to.array (data) ->
console.log data
# [ [ '1', '2', '3', '4' ], [ 'a', 'b', 'c', 'd' ] ]
if Meteor.isServer
console.log "Server Started"
Thanks for any insight!
As requested in comments, posting entire app code (.coffee file)
if Meteor.isClient
console.log "Client Started"
if Meteor.isServer
console.log "Server Started"
csv().from.string("#Welcome\n\"1\",\"2\",\"3\",\"4\"\n\"a\",\"b\",\"c\",\"d\"", comment: "#").to.array (data) ->
console.log data
# [ [ '1', '2', '3', '4' ], [ 'a', 'b', 'c', 'd' ] ]
Related
I'm having a problem with my code, no errors occur but for some reason I'm not getting the desired
outcome
This is the 'json' data that the user will receive
books = [
{'id': 0,
'title': 'A fire Upon the Deep',
'author': 'Vernor Vinge',
'first_sentence': 'The coldsleep itself was dreamless.',
'year_published': '1992'},
{'id': 1,
'title': 'The Ones Who Walk Away From Omelas',
'author': 'Ursula K. Le Guin',
'first_sentence': 'With a clamor of bells that set the swallows soaring, the \
Festival of Summer came to the city Omelas, bright-towered by the sea.',
'published': '1973'},
{'id': 2,
'title': 'Dhalgren',
'author': 'Samuel R. Delany',
'first_sentence': 'to wound the autumnal city.',
'published': '1975'}
]
If no id's exist then it returns an error message which it's not supposed to seeing as each book has an id
#app.route('/api/v1/resource/books', methods=['GET'])
def api_id():
if 'id' in request.args:
id = int(request.args['id'])
else:
return "Error: ID not provided. Please specify an ID"
results = []
for book in books:
if book['id'] == id:
results.append(book)
return jsonify(results)
Perhaps, the request url will be something like http://localhost:5000/api/v1/resource/books/?id=1
For multiple book ids, it will be something like http://localhost:5000/api/v1/resource/books/?id=1&id=2
from flask import Flask, request, jsonify
app = Flask(__name__)
books = [
{'id': 0,
'title': 'A fire Upon the Deep',
'author': 'Vernor Vinge',
'first_sentence': 'The coldsleep itself was dreamless.',
'year_published': '1992'},
{'id': 1,
'title': 'The Ones Who Walk Away From Omelas',
'author': 'Ursula K. Le Guin',
'first_sentence': 'With a clamor of bells that set the swallows soaring, the \
Festival of Summer came to the city Omelas, bright-towered by the sea.',
'published': '1973'},
{'id': 2,
'title': 'Dhalgren',
'author': 'Samuel R. Delany',
'first_sentence': 'to wound the autumnal city.',
'published': '1975'}
]
#app.route("/api/v1/resource/books/", methods=["GET"])
def api_id():
# This will look for 'id' in the url. If there's no id/ids, it will take None as default value
bids = request.args.getlist('id')
# if you are expecting to serve only one id at a time, use >> request.args.get('id')
if bids:
results = []
for book in books:
for bid in bids: # this for loop won't be required for strictly single id
if book['id'] == int(bid):
results.append(book)
return jsonify(results)
else:
return "Error: ID not provided. Please specify an ID"
if __name__ == "__main__":
app.run()
This code is working for me. I hope it does for you as well.
Request Url:
http://localhost:5000/api/v1/resource/books/?id=0&id=2
Output:
[{"author":"Vernor Vinge","first_sentence":"The coldsleep itself was dreamless.","id":0,"title":"A fire Upon the Deep","year_published":"1992"},{"author":"Samuel R. Delany","first_sentence":"to wound the autumnal city.","id":2,"published":"1975","title":"Dhalgren"}]
Request Url:
http://localhost:5000/api/v1/resource/books/?id=0
Output:
[{"author":"Vernor Vinge","first_sentence":"The coldsleep itself was dreamless.","id":0,"title":"A fire Upon the Deep","year_published":"1992"}]
Request Url:
http://localhost:5000/api/v1/resource/books/
Output:
Error: ID not provided. Please specify an ID
I have a series of files that look similar to this:
[
{
'id':1,
'transactions': [
{
'date': '2019-01-01',
'amount': 50.50
},
{
'date': '2019-01-02',
'amount': 10.20
},
]
},
{
'id':2,
'transactions': [
{
'date': '2019-01-01',
'amount': 10.20
},
{
'date': '2019-01-02',
'amount': 0.50
},
]
}
]
I load these files to Spark using the following code
users= spark.read.option("multiline", "true").json(file_location)
The result is a SparkData Frame with two columns id and transactions where transactions is a StructType.
I want to be able to "map" the transactions per user to aggregate them.
Currently I am using rdd and a function that looks like this:
users.rdd.map(lambda a: summarize_transactions(a.transactions))
The summarize function can be of two types:
a) Turn the list of objects into a Pandas Dataframe to summarize it.
b) Iterate over the list of objects to summarize it.
However I find out that a.transactions is a list of pyspark.sql.types.Row. Instead of actual dictionaries.
1) Is this the best way to accomplish my goal?
2) How can I turn the list of Spark Rows into the original list of Dictionaries?
I found a way to solve my own problem:
STEP 1: LOAD DATA AS TEXTFILE:
step1= sc.textFile(file_location)
STEP 2: READ AS JSON AND FLATMAP
import json
step2 = step1.map(lambda a: json.loads(a)).flatMap(lambda a: a)
STEP 3: KEY MAP REDUCE
setp3 = (
step2
.map(lambda line: [line['id'], line['transactions']])
.reduceByKey(lambda a, b: a + b)
.mapValues(lambda a: summarize_transactions(a))
)
I am using the UK Bus API to collect bus arrival times etc.
In Python 3 I have been using....
try:
connection = http.client.HTTPSConnection("transportapi.com")
connection.request("GET", "/v3/uk/bus/stop/xxxxxxxxx/live.json?app_id=xxxxxxxxxxxxxxx&app_key=xxxxxxxxxxxxxxxxxxxxxxxxxxx&group=route&nextbuses=yes")
res = connection.getresponse()
data = res.read()
connection.close()
from types import SimpleNamespace as Namespace
x = json.loads(data, object_hook=lambda d: Namespace(**d))
print("Stop Name : " + x.stop_name)
Which is all reasonably simple, however the JSON data returned looks like this...
{
"atcocode":"xxxxxxxx",
"smscode":"xxxxxxxx",
"request_time":"2020-03-10T15:42:22+00:00",
"name":"Hospital",
"stop_name":"Hospital",
"bearing":"SE",
"indicator":"adj",
"locality":"Here",
"location":{
"type":"Point",
"coordinates":[
-1.xxxxx,
50.xxxxx
]
},
"departures":{
"8":[
{
"mode":"bus",
"line":"8",
"line_name":"8",
"direction":"North",
"operator":"CBLE",
"date":"2020-03-10",
Under "departures" the key name changes due to the bus number / code.
Using Python 3 how do I extract the key name and all subsequent values below/within it?
Many thanks for any help!
You can do this:
for k,v in x["departures"].items():
print(k,v) #or whatever else you wanted to do.
Which returns:
8 [{'mode': 'bus', 'line': '8', 'line_name': '8', 'direction': 'North', 'operator': 'CBLE', 'date': '2020-03-10'}]
So k is equal to 8 and v is the value.
I am new to Azure Cognitive Services. I am using python 3.5 and the Azure Face service, specifically the "Face - Find Similar" API. I am getting an error formatting the Post JSON body. The documentation says that I should either provide a single faceId or a list of faces in a face list, e.g., 'extended_family'. In my case, I want to use the face list. I do not know how to format the faceID parameter in the JSON so that call processes the face list. I have tried different alternatives to setting faceId to '{}' or 'Null', '', 'False', and this results in errors.
Below is my JSON body for my POST:
{'faceId': '97522b8b-02b6-4115-99e0-6dc1f5d45f51', 'faceIds': ['97522b8b- 02b6-4115-99e0-6dc1f5d45f51', '0ca8f3e4-edf1-4c14-b926-3b47eae7e29c', '2fadbb12-b10b-4761-aaaa-c50f1dc765c3', '56f464d5-b388-4fc7-9051-6991cf5f1d0d', '29931480-632e-40b6-aa0c-9e03e36e95f9', '7a8085b2-2013-4742-a51a-a5543a0347e8'], 'faceListId': 'extended_family', 'maxNumOfCandidatesReturned': 20, 'mode': 'matchPerson'}
Since faceId is populated, I get:
error: 'code': 'BadArgument', 'message': 'Argument faceListId and faceIds cannot be provided at the same time.'
If I leave faceId blank, such as:
{'faceId': '', 'faceIds': ['824e3d83-a94f-4ef2-949e-55a55b2ef256', '51f3c1a5-4e16-4b14-89fa-f1342a2c46ec', '0480d2e0-ff05-44de-b3d8-94408277b1c5', 'c7d767fb-0fbe-46c8-b7af-2c8f675bfd8d', 'ca7e82a7-cd3f-417b-bffa-77c9d47c1439', 'f7130e90-9e1f-428a-a773-93c87932a420'], 'faceListId': 'extended_family', 'maxNumOfCandidatesReturned': 20, 'mode': 'matchPerson'}
I get the following:
{'error': {'code': 'BadArgument', 'message': 'Request body is invalid.'}}
If I remove the faceId term from the JSON:
{'faceIds': ['690feffd-5c86-47d7-ac3c-224b0eafa90f', '936564e0-31aa-43e3-916e-c7b236bea8e0', '614c04cb-4375-44c8-b393-89d64b4c1ebd', 'a29f8e5c-50ba-4bb8-8bf8-98e356a9125a', '073d7865-2aaf-4806-9bef-ddca478137ea', '7e416e83-5973-4aa1-b1fc-3a25b5174bb3'], 'faceListId': 'andersen_extended_family', 'maxNumOfCandidatesReturned': 20, 'mode': 'matchPerson'}
{'error': {'code': 'BadArgument', 'message': 'Face ID is invalid.'}}
My code is as follows:
facelist = list() #This is populated upstream using the Face Detect API
facelistid = 'extended_family'
faceid = ''
payload = {'faceId': faceid, 'faceIds':facelist,'faceListId':facelistid, 'maxNumOfCandidatesReturned':20,'mode': "matchPerson"}
req = requests.post(serviceurlpersongroup, data = json.dumps(payload) , headers = {'Ocp-Apim-Subscription-Key': key})
jinfo = req.json()
Just as the error message explained, faceListId and faceIds (not faceId) should not be provided at the same time. So the payload you use should be
payload = {'faceId': faceid, 'faceListId': facelistid, 'maxNumOfCandidatesReturned': 20, 'mode': "matchPerson"}
or
payload = {'faceId': faceid, 'faceIds': facelist, 'maxNumOfCandidatesReturned': 20, 'mode': "matchPerson"}
Feel free to update if you have any further questions.
Ive just started learning python and Im having a go at using a google api. But I hit a brick wall trying to parse the JSON with simplejson.
How do I go about pulling single values (ie product or brand fields) out of this mess below
{'currentItemCount': 25, 'etag': '"izYJutfqR9tRDg1H4X3fGx1UiCI/hqqZ6pMwV1-CEu5NSqfJO0Ix-gs"', 'id': 'tag:google.com,2010:shopping/products', 'items': [{'id': 'tag:google.com,2010:shopping/products/1196682/8186421160532506003',
'kind': 'shopping#product',
'product': {'author': {'accountId': '1196682',
'name': "Dillard's"},
'brand': 'Merrell',
'condition': 'new',
'country': 'US',
'creationTime': '2011-03-10T08:11:08.000Z',
'description': u'Merrell\'s "Trail Glove" barefoot running shoe lets your feet follow their natural i$
'googleId': '8186421160532506003',
'gtin': '00797240569847',
'images': [{'link': 'http://dimg.dillards.com/is/image/DillardsZoom/03528718_zi_amazon?$product$'}],
'inventories': [{'availability': 'inStock',
'channel': 'online',
'currency': 'USD',
'price': 110.0}],
'language': 'en',
'link': 'http://www.dillards.com/product/Merrell-Mens-Trail-Glove-Barefoot-Running-Shoes_301_-1_301_5$
'modificationTime': '2011-05-25T07:42:51.000Z',
'title': 'Merrell Men\'s "Trail Glove" Barefoot Running Shoes'},
'selfLink': 'https://www.googleapis.com/shopping/search/v1/public/products/1196682/gid/8186421160532506003?alt=js$
The JSON you've pasted in the question is not valid. But when you fixed that here's how to use simplejson:
import simplejson as json
your_response_body = '["foo", {"bar":["baz", null, 1.0, 2]}]'
obj = json.loads(your_response_body)
print(obj[1]['bar'])
And a link to the documentation.