I currently have a program that scrapes the text and author and time stamp from the news websites I visit and pushes it to a firebase database in .json format. I'm now pulling that data into r to perform all the fun sentiment & nlp analysis. When i first started I had the data public and, using curl, I could easily pull the .json data into r. Code:
library(RJSONIO)
library(RCurl)
# grab the data
raw_data <- getURL("https://[PROJECT_ID].firebaseio.com/.json)
# covert from JSON into a list in R
dataJSON <- fromJSON(raw_data)
I have since switched the firebase database from public to private and now when i run the above code I receive a "Permission Denied" error. The question I have is how can I access my json data now that I have switched from public to private?
I created a Service admin account in firebase and generated a private key after reading that would give me an auth token that I could use in something like this format:
curl 'https://[PROJECT_ID].firebaseio-demo.com/users/jack/name.json?access_token=<TOKEN>'
But in the private key there is no "token".
If anyone has any insight or recommendations it'd be hugely appreciated!
Thanks,
Ben
Related
I'm a CoreML and CreateML newbie and have been playing around with CreateML for the past few days and started to wonder if there are any ways to keep the training model up to date without creating a new model with an updated csv model.
Is there a way to get a JSON file from online and train it?
I've tried let dataTable = try MLDataTable(contentsOf: URL(string: "the API in JSON format I am using"))
and it gave me an error: "MLDataTable can only load from a file URL."
The basic gist: Is there a way to train a model without having to download the updated version of the CSV or JSON file everyday?
Thanks :D
How should the Chatbot conversation raw data excel csv look like?
Now I am building a Chatbot with IBM Watsons. The client would want the chatbot conversation raw data export in excel/csv format. As the conversation is very long, How should we get the most appropriate format for the Chatbot conversation raw data csv?
Thanks!
Based on your question, I guess you should use Logs/ examples on the API Reference for base what you CSV file gonna have, for example: inputText, outputText, conversation_id, dateConversation, intents, entities, context, workspace_id, etc.
In this example using Python that will return these data (you can also use another language that has HTTP request support):
List logs:
list_logs(self, workspace_id, sort=None, filter=None, page_limit=None, cursor=None)
Request:
import json
import watson_developer_cloud
conversation = watson_developer_cloud.ConversationV1(
username='{username}',
password='{password}',
version='2018-02-16'
)
response = conversation.list_logs(
workspace_id = '9978a49e-ea89-4493-b33d-82298d3db20d'
)
print(json.dumps(response, indent=2))
Obs.: Take a look at this example saving inside one txt file the conversation Logs, you can also check this answer to use the same logic to save in one csv file.
Obs.: Nowadays Watson conversation changed the name for Watson Assistant.
Official Watson Conversation API Reference (cURL, Node, Java, Python examples)
The NZ companies register offers a json file containing all publicly available business info. This file comes in at a whopping 40gb, but there is also a smaller json file (~250mb) containing data on unincorporated entities (sole traders etc). As a warm up excercise I thought i'd have a go importing it into R to get an idea of size, scalability and computational reqs.
I'm having alot of trouble importing the smaller json file into R. I've tried jsonlite, RJSONIO, rjson but it appears that the file is written in an 'unorthodox' json format, hence the standard 'fromJSON' commands are falling over. Below is a portion of the file (2 entities) which i've been trying to import into R: test.json
library(jsonlite)
json <- fromJSON("test.json", flatten=TRUE)
Error in parse_con(txt, bigint_as_char) :
parse error: invalid object key (must be a string)
zbn": [{ "entity": [{ { "australianBusinessNumbe
(right here) ------^
NB: JSONlint doesn't seem to think the file is a valied JSON file
My thought is that I may need to use stream_in() or readLines() but I am no very proficient with these functions. Any help or insight greatly appreciated. Cheers
I'm trying to use the New York City Bus API in order to create an app. The last API I worked with, was while learning Python. I have no idea where to begin.
This is the link to the documentation
http://bustime.mta.info/wiki/Developers/SIRIIntro
If you guys could point me towards how I can get the data from the API. I already got the Key.
Thank You so Much
You can use requests to fetch data from api.
import requests
url = "http://api.prod.obanyc.com/api/siri/vehicle-monitoring.json?key={}".format(key)
# fetch data
data = requests.get(url).json()
# do something with data.
I'm doing a small task that requires the usage of GitHub's APIs.
Most of the data can be retrieved from the main API, but some specific information is an API itself.
Here's how the main API is read by R, using jsonlite package:
The next task is to tell R to sum the number of labels that are 0 in each of those rows of that column "url". For that, R needs to read each API separately.
I'm trying to do something like this:
library(jsonlite)
issues <- fromJSON("THE API LINK")
jsonapis <- issues$url
apis <- fromJSON(jsonapis)
That retrieves the following error message:
Error: Argument 'txt' must be a JSON string, URL or file.
To use with URL, I'd have to copy paste each one manually - I think - and all suggestions I found only in regard to 'file' do not fit.
I have another task similar that involves reading from many APIs simultaneously, but I presume I'll be able to do finish that if I can understand this one.
Can someone help me?
I apologize if this is not very clear, but this is the best I can do. Unfortunately, I'm not too comfortable in these topics and APIs is really a pain to me.
Thanks in advance!