Add JSON data to MySQL database - mysql

I'm developing a Grails concert application where I am acquiring the actual events from bandsintown.com using AngularJS.
$.getJSON("http://api.bandsintown.com/events/search.json?&api_version=2.0&app_id=FYP&location=Dublin,Ireland", function(result)
I am able to retrieve the events and I now want to turn each event into an object and add them to my database, so I can have a unique page for each event with their subsequent details and whatnot.
Is anyone able to provide me guidance on how to do such a thing?
Thank you!

I wrote something a while ago as a plugin which may help you get your head around what you need to do :
private String viewTicketField(String jiraserver, String jirauser,String jirapass, String url, String customField) {
if (url.endsWith('undefined')) {
return
}
String output=''
try {
HBuilder hBuilder=new HBuilder()
RESTClient http = hBuilder.httpConn(jiraserver, jirauser, jirapass,httpConnTimeOut,httpSockTimeOut)
http.request(Method.GET ) { req ->
uri.path = url
response.success = { resp, json ->
log.info "Process URL Success! ${resp.status} "
def result=JSON.parse(json.toString())
if (result.fields."customfield_${customField}") {
output=result.fields."customfield_${customField}"
}
}
response.failure = { resp ->
log.error "Process URL failed with status ${resp.status}"
}
}
}
catch (HttpResponseException e) {
log.error "Failed error: $e.statusCode"
}
return output
}
HBuilder class can be found in src
After you have connected and got back your JSON response you then parse each line
You create a new domain class
String url
String datetime
String app_id
String ticket_url
... and so on this is an entire entry from above url so for each key that you actually require put in domain class as strings then parse each iteration with above code
{"id":13926481,"url":"http://www.bandsintown.com/event/13926481?app_id=FYP","datetime":"2017-02-23T17:00:00","ticket_url":"http://www.bandsintown.com/event/13926481/buy_tickets?app_id=FYP\u0026came_from=233","artists":[{"name":"Rackhouse Pilfer","url":"http://www.bandsintown.com/RackhousePilfer","mbid":null}],"venue":{"id":3334537,"url":"http://www.bandsintown.com/venue/3334537","name":"Convention Centre","city":"Dublin","region":"07","country":"Ireland","latitude":53.3500292,"longitude":-6.2385286},"ticket_status":"unavailable","on_sale_datetime":null

Related

java.io.IOException: stream is closed

I am getting below error when i am trying to print response from http connection. I think i am using the stream twice and for this reason it says stream is closed but not sure.
The error is coming on the writeInstFile() method where i am trying to print response using getInputStream().getText()
I just wanted to create new string variable and put the connection stream response in that variable and print it so that i can see what response i am getting when i try to do http request:
private void writeInstFile(File outFile) {
Properties config = workflowEnvironment.getConfig()
//http connection to get Authentication token
HttpURLConnection connection = connection("https://XXX/RequestToken")
if (connection.responseCode == HTTP_OK) {
String authToken = authTokenFromJson(jsonFromDssRequestTokenConnection(connection))
log.info("Authentication token: $authToken")
println(connection.getInputStream().getText());
if (connection.responseCode == HTTP_OK) {
println("Got http response code: ${connection.responseCode}, message: ${connection.responseMessage}")
log.info("Successful DSS request to ${connection.getURL()}")
LazyMap json = jsonFromDssExtractionConnection(connection)
.....
}
private static String authTokenFromJson(LazyMap json) {
json.value
}
private static LazyMap jsonFromDssRequestTokenConnection(HttpURLConnection connection) {
connection.inputStream.withCloseable { inStream ->
new JsonSlurper().parse(inStream as InputStream)
} as LazyMap
}

Angular 2+ - Handle JAVA Rest 401 response

Angular service.ts:
getExcel() {
let link: string = (this.url);
return link;
}
component.ts:
public getExcel() {
//checks if string is empty, undefined or not null
if (this.langCode) {
return window.open(this.avrs.getExcel());
}
}
Java rest.java
#GET
#Path("/excel")
#Produces("application/vnd.ms-excel")
public Response getTransportCostsExcel(
#Context HttpServletRequest request,
) {
byte[] excelInBytes = excelGen.getExcel();
if(excelInBytes == null){
return Response.status(Response.Status.NOT_FOUND).entity("No data").build();
}
//Header details
String contentType = "application/vnd.ms-excel";
Response.ResponseBuilder responseBuilder = javax.ws.rs.core.Response.ok((Object) excelInBytes);
responseBuilder.type(contentType);
responseBuilder.header("Content-Disposition", "attachment; filename=" + fileName);
//Returns Excel
return responseBuilder.build();
}
When I try calling my api from postman i get "No data" and status is 401 not found. So the rest method works fine.
I get my excel file if data is found. But I can't seem to handle the 401 response. Angular opens a new window and says
site not avaliable: ERR_INVALID_RESPONSE
As you can see im not using http.get cause I want the user to start downloading the excel if the file is found.

Post a list and iterate it using Dart

I'm trying to post some data from a dart project to another and store them in a mongoDB
Post code:
import 'dart:io';
void main() {
List example = [
{"source": "today", "target": "tomorrow"},
{"source": "yesterday", "target": "tomorrow"},
{"source": "today", "target": "yesterday"}
];
new HttpClient().post('localhost', 4040, '')
.then((HttpClientRequest request) {
request.headers.contentType = ContentType.JSON;
request.write(example);
return request.close();
});
}
Code that receives it, inside another file
void start() {
HttpServer.bind(address, port)
.then((HttpServer server) {
// Log in console to show that server is listening
print('Server listening on ${address}:${server.port}');
server.listen((HttpRequest request) {
request.transform(UTF8.decoder).listen(sendToDatastore);
});
});
}
void sendToDatastore(String contents) {
var dbproxy = new dbProxy("myDb");
dbproxy.write("rawdata", contents);
index++;
// non related to the problem code
}
bool write(collectionName, document)
{
Db connection = connect();
DbCollection collection = connection.collection(collectionName);
connection.open().then((_){
print('writing $document to db');
collection.insert(document);
}).then((_) {
print('closing db');
connection.close();
});
return true;
}
What I'm struggling with is that I'm using
request.transform(UTF8.decoder).listen(sendToDatastore);
so I'm converting the request stream to a string as I couldn't find the way to send it as Json.
And then in sendToDatastore I'm not able to parse it properly in order to store it. As far as I understand I'd need to get every Json object as a Map to store it as I'm getting this error
Uncaught Error: type 'String' is not a subtype of type 'Map' of 'document'.
Thanks,
UPDATE
If I try to do something like this in sendToDatastore
void sendToDatastore(String contents) {
var dbproxy = new dbProxy("myDb");
var contentToPass = JSON.decode(contents);
contentToPass.forEach((element) => dbproxy.write("rawdata", element));
index++;
// non related to the problem code
}
It raises this error
Uncaught Error: FormatException: Unexpected character (at character 3)
[{source: today, target: tomorrow}, {source: yesterday, target: tomorrow}, ...
^
In the use of JSON.decode
UPDATE2
The error was that I wasn't sending actual Json from the "post code". I used
// ...
request.write(JSON.encode(example));
// ...
and everything worked fine
Thanks
You should be able to use the dart:convert package.
You can then use:
String str = JSON.encode(obj)
and
var obj = JSON.decode(str)
to convert string/json.

POSTing JSON with HttpsURLConnection & OutputStreamWriter fails

I'm using Processing 2.0.2 and importing java's java.net,javax.net, and java.io classes.
I'm trying to send a JSONObject to "https://www.bitstamp.net/api/balance/".
The JSONObject contains login info. I create it with this function:
JSONObject BitstampLoginJSON() {
JSONObject loginJSON = new JSONObject();
loginJSON.setString("password", "mypassword");
loginJSON.setString("user", "username");
return loginJSON;
}
I setup my connection successfully and then pass the connection to this function to send the request.
void SendBitstampBalanceRequestFromConnection(HttpsURLConnection givenCon) {
try {
givenCon.setDoOutput(true);
givenCon.setDoInput(true);
givenCon.setRequestProperty("Content-Type", "application/json");
givenCon.setRequestProperty("Acccept", "application/json");
givenCon.setRequestMethod("POST");
givenCon.connect();
OutputStreamWriter requestSender = new OutputStreamWriter(givenCon.getOutputStream());
String JSONString = BitstampLoginJSON().toString();
requestSender.write(JSONString);
requestSender.flush();
requestSender.close();
}
catch(IOException e) {
e.printStackTrace();
}
}
I get a response code 200 and returned message of "OK". Bitstamp returns a JSON Object that says this:
"error": "Missing user and/or password POST parameters"}
I'm quite confused as to why Bitstamp is not receiving or accepting my JSONObject. I have read about a half dozen posts on SO about JSON and POSTing but I can't seem to get it.

Jersey calling a get with object as parameter does not work

I am new to Jersey, I am trying to develop a GET for search results. For this I need to send a object with the search criteria and data. I wonder what I am doing wrong. I am getting the following exception on my Junit test case
com.sun.jersey.api.client.UniformInterfaceException: GET http://localhost:8081/mCruiseOnCarPool4All/carpool4all/Search/Request/com.mcruiseon.carpool.concrete.SearchConcrete#676e3f returned a response status of 404 Not Found
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:686)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:507)
at test.carpool4all.SingleSearchTest.testPost(SingleSearchTest.java:89)
My Server side GET
#GET
#Path ("Request/{search}")
#Consumes({ MediaType.APPLICATION_JSON })
#Produces({ MediaType.APPLICATION_JSON })
public Response search(#PathParam("search") SearchConcrete searchConcrete) {
SearchJourneyRequest request = new SearchJourneyRequest(searchConcrete) ;
SearchJourneyResponse response ;
clientSession = sessionManager.getClientSession(searchConcrete.getIdentityHash()) ;
clientSession.getSendQueue().sendRequest(request) ;
try {
response = (SearchJourneyResponse)clientSession.waitAndGetResponse(request) ;
} catch (WaitedLongEnoughException e) {
return Response.serverError().build() ;
} catch (UnableToResolveResponseException e) {
return Response.serverError().build() ;
}
return Response.ok(response.getSearchResults()).build();
}
Client Side Junit test
SearchConcrete searchProvider = new SearchConcrete(Globals.SearchCriteria.FlexiTime,
identityHash,
// more parameters
);
service = client.resource(UriBuilder.fromUri("http://localhost:8081/mCruiseOnCarPool4All/carpool4all/Search/Request/"+searchProvider).build());
Object[] searchResults = service.type(MediaType.APPLICATION_JSON).get(Object[].class);
Edit : Thanks to #eugen, to solve this, I added a concrete class with my Object[] as a private member. Instead of a GET, I used a POST here is the fixed code. Now my carpool search results are coming :).
service = client.resource(UriBuilder.fromUri(
"http://localhost:8081/mCruiseOnCarPool4All/carpool4all/Search/Request").build());
SearchResultsConcrete searchResults = service.type(MediaType.APPLICATION_JSON).post(SearchResultsConcrete.class, searchProvider);
assertNotNull(searchResults);
assertNotNull(searchResults.getSearchResults()) ;
assertTrue(searchResults.getSearchResults().length == 3) ;
assertTrue(searchResults.getSearchResults()[SearchJourneyResponse.FLEXI_POSITION].length > 0) ;
assertTrue(searchResults.getSearchResults()[SearchJourneyResponse.FLEXIENDTIME_POSITION].length > 0) ;
#POST
#Path ("Request")
#Consumes({ MediaType.APPLICATION_JSON })
#Produces({ MediaType.APPLICATION_JSON })
public Response search(JAXBElement<SearchConcrete> element) {
SearchJourneyRequest request = new SearchJourneyRequest((SearchConcrete)element.getValue()) ;
SearchJourneyResponse response ;
clientSession = sessionManager.getClientSession(((SearchConcrete)element.getValue()).getIdentityHash()) ;
clientSession.getSendQueue().sendRequest(request) ;
try {
response = (SearchJourneyResponse)clientSession.waitAndGetResponse(request) ;
} catch (WaitedLongEnoughException e) {
return Response.serverError().build() ;
} catch (UnableToResolveResponseException e) {
return Response.serverError().build() ;
}
return Response.ok(response.getSearchResults()).build();
}
You are doing things wrong.
First lets analyze your error :
com.sun.jersey.api.client.UniformInterfaceException: GET http://localhost:8081/mCruiseOnCarPool4All/carpool4all/Search/Request/com.mcruiseon.carpool.concrete.SearchConcrete#676e3f returned a response status of 404 Not Found.
We can see that com.mcruiseon.carpool.concrete.SearchConcrete#676e3f is appended at the end of the url of your request (confirmed by your test case "http://localhost:8081/mCruiseOnCarPool4All/carpool4all/Search/Request/"+searchProvider). It does not make sense.
I assume you want to send a json with your GET request? You can't do that! GET has no body but only the url (and the headers), and appending json to your url does not make sense. If you want to send json you must do a POST.
You are trying to do something similar to this post What is the maximum length of JSON object received by JAX-RS web service using GET?, have a look at my answer.
Here is how to do what you want with Genson library http://code.google.com/p/genson/.
Download genson with maven, it will automaticaly enable json support when it is in your classpath and handle all the databinding.
Then change your server side method:
#POST
#Consumes({ MediaType.APPLICATION_JSON })
#Produces({ MediaType.APPLICATION_JSON })
public Response search(SearchConcrete searchConcrete) {
...
}
Change your test to:
// configure your client to use genson
ClientConfig config = new DefaultClientConfig();
config.getClasses().add(GensonJsonConverter.class);
cli = Client.create(config);
// then the test code
TypeOfTheResponse response = cli.resource("http://localhost:8081/mCruiseOnCarPool4All/carpool4all/Search/Request")
.accept(MediaType.APPLICATION_JSON)
.type(MediaType.APPLICATION_JSON)
.post(TypeOfTheResponse.class, searchProvider);
Remarks: do not use Object[] as the response, use a concrete java class like MyResponseItem[]
You are missing a slash / here:
"...Request"+searchProvider
This must be
"...Request/" + searchProvider
Edit
You can't just add an Object to an URL. This
"http://localhost:8081/mCruiseOnCarPool4All/carpool4all/Search/Request/"+searchProvider
will cause the toString() method of SearchConcrete to be called. The result is
com.mcruiseon.carpool.concrete.SearchConcrete#676e3f
The server has no way to reconstruct the SearchConcrete from this.