If you view the source of a Google+ profile page, it appears rather complex. It seems most of the data is kept in a huge JSON-like objects. However, they don't seem to be really JSON, since they don't get recognized when I try to decode them. I am hoping the format is more clear to other people here. How would you go about parsing it? It seems it would fairly trivial, if you know where to start.
Here is a sample profile, for example: http://plus.google.com/104560124403688998123
Here's a PHP API I'm working on. It can download and parse the data for a profile page and people's public relationships.
https://github.com/jmstriegel/php.googleplusapi
The JSON piece is a bit mangled. To generate valid JSON, you basically have to remove the first 5 characters that prevent XSRF attacks and then add in all the nulls that have been removed. Here's the code specific to handling parsing the weird Google Plus JSON responses:
https://github.com/jmstriegel/php.googleplusapi/blob/master/lib/GooglePlus/GoogleUtil.php
Call GoogleUtil::FetchGoogleJSON( $url ) and you'll get back a giant array that you can then pull data from. Using this, it should be trivial to make a proxy service to translate stuff into valid json(p) for you to use in your own apps.
I don't have access to Google+ yet, so I'll just answer the general question - that is, how to parse JSON.
JSON is just JavaScript, so parsing it is as simple as evaluating the script. To do this, use the eval() JavaScript function.
var obj = eval('{"JSON":"goes here"}');
Another option is to leverage a console tool. Popular modern browsers pretty much all have them. I recommend Firebug for Firefox in particular.
Using Firefox, log into Google+, then open the Firebug console. You can use the console's dir() command to create a browseable representation of the data. Ex:
console.dir(eval('{"JSON":"goes here"}'));
Sorry I can't be more specific about how to get a handle on Google+'s JSON in particular; without access to the service, this is about the best I can do blind. Good luck!
Thanks to Jason for the excellent php class which reads a profile page into an array.
I've used this class as a base and then parsed it, based upon Russell Beattie's python code from the original appspot rss feed application.
Code here
A few notes:
I use this to merge G+ and WP feeds, hence writing posts into an intermediate array ($items).
I have a convention of creating a pseudo title in Google Plus posts, by emboldening a line and adding two newlines before writing the post. The function getTitle strips this out as a better formatted title in my website and getSummary produces the rest of the post with duplicating the title.
It's made up of a number of parts, an object describing your picasa images, one describing the fields on your profile, one describing your friends.
Most of the long numbers are the internal IDs of people, posts and photos. For instance, my ID is 105249724614922381234. Other than that, it could be parsed if you needed to.
Related
I found WP REST API very interesting in making custom functionalities in WordPress websites. However, I find it hard to read my JSON endpoints' results.
The normal output of JSON endpoint is wrapped in html and pre tags. T result appears in one long line of compressed string.
I need to integrate my website to a mobile app to be done by another developer and I would like to display the API endpoints (e.g. link) to appear as a regular JSON Object like:
I'm trying to find a workaround like a hook or a filter to make the JSON results appear as I desired. Or equivalent AJAX related code would be nice.
I use a Chrome extension of JSON Formatter to view the results which prints out with readability in mind.
https://github.com/callumlocke/json-formatter
I have an API that produces JSON like this:
)]}',
{
//JSON DATA
}
The //JSON DATA is valid JSON, but the )]}', up top is not.
When I try to GET this data via a Logic App, I get:
BadRequest. Http request failed: the content was not a valid JSON.
So, a few related questions:
1) Can I tell the logic app to return the invalid JSON anyway?
2) How can debug the issue better? I happen to know that the response is invalid, but what if I didn't? Can I see the raw data somewhere?
3) This is all done via the Azure web portal. Are there better tools? Visual Studio?
I should also mention that if I call a route on the same API that returns XML instead of JSON, then the Logic App works fine. So it definitely doesn't like the JSON response in particular.
Thanks!
First of all, please do not post three questions as a single question.
Question 1). The best thing you can do is make the API return a valid JSON object. This is good for million reasons. Here're a few:
it's pretty much a standard (either valid JSON or XML -- yeah, old school way);
therefore, no users of this API (including you) will need to struggle and guess what's going on and why;
your Logic App's step will just work without adding extra complexity;
you will make this world and your karma better.
If API-side changes are not within your reach, I don't think you can do much. If you're lucky and the HTTP action is successful (Status Code 2xx), you can try to use a Query Action with a function that truncates the first characters. It will look something like this (I don't know the exact syntax): #Substring(body('myHttpGet'), 4, length(body('myHttpGet')) - 4) where myHttpGet is the id of the Http Get action.
However, once again, if possible, I strongly recommend fixing up the API which is the root cause of the problem, instead of dealing with garbage response after that.
UPDATE Another thing you can do is wrap the dirty API. For example, you could create a trivial Azure Function that invokes the API you don't directly control, and sanitizes the response for you consumption requirements. This Azure Function function should be easy to call from the Logic App. It costs almost nothing (unless we're talking millions of requests/month). The only drawback here is the increasing latency, which may be not an issue at all -- test it and see whether it adds less than 100ms or so... Oh, and don't forget to file a ticket with the API owner, they make our world a bad place!
Question 2) In Azure Logic App web UI you can Look into the execution details and the error will definitely be there.
Question 3) You're asking for a tool recommendation which is by definition a highly subjective thing and is off-topic on StackOverflow.
TL/DR: The other app is not producing valid JSON.
Meaning, this is not a problem for you to solve. The other app has to return valid JSON if the owner claims it should.
If they cannot or will not produce valid JSON, then the first thing you need to do is inform your management that you will have to spend a lot of extra time accommodating their non-standard format.
after years of absence from web programming I now start to write a new web application from scratch. I learned the ropes of REST and found some helfful presentations and webinars about good and bad "URI styles". But all these tutorials seems to assume that there are only ressources that can be mapped to entitities that persist in a database. If one needs to GET a HTML (user-friendly) version of something, then the prevalent answer is to use content negiotation via HTTP header fields. The ultimate goal seems to have only URIs that follow the REST paradigm.
But what is the "correct" way to display web pages that cannot be mapped to an entity but are required anyway?
Perhaps I make an example to clarify what I mean. Assume we have a collection Persons and the entity Person.
Then we have
POST /Persons/ to create a new person in the collection
DELETE /Person/{id}/ to delete the person
PUT /Person/{id}/ to modifiy the person (yes, I know it can also mean to create one)
GET /Persons/ to get the list of all persons
GET /Person/{id}/ to get an individual person
and so on
With respect to the GET operations I generally found the advice to use the Accept and X-Requested-With header fields of the request to create a response that either returns the "bare" person object in a computer-readable respresentation (i.e. JSON, XML, etc.) or that returns a fully-fledged web page for a browser.
But what is about the PUT operation. Ultimately, this operation will send a bare person object (i.e. JSON, XLM) that is going to be created, but before this step I need to collect the data from the user. This means I need some empty web form that is just "eye-candy" for the human user. Of course, I could habe something like GET /newPersonForm/ but it seems that this contradict the REST philosophy, because /newPersonForm/ is an URI that only points to some user interface element.
At the moment I see to options:
Use the same name space for both kind of URIs:
POST /Persons/ --> REST api
DELETE /Person/{id}/ --> REST api
PUT /Persons/{id}/ --> REST api
GET /Persons/ --> REST api or UI (after content negiotation)
GET /Person/{id}/ --> REST api or UI (after content negiotation)
GET /Person/creationForm --> non-REST, pure UI
GET /aboutus --> non-REST, pure UI, additional company information
Make separate name spaces:
/api/... --> contains everything for REST
/ui/... --> contains html web pages
With the first approach I feel that it is somebit "unclean". Although the second approach seems cleaner, I see two problems. Firstly, if one follows this approach cleanly, one gets much double URIs, because one dispense with content negiotation and has an UI web page for every REST function. I have GET /api/Person/{id}/ to return a JSON object and GET /ui/Person/{id} to return a browser version. Secondly, I feel that this approach contradict REST philosophy because search egines and web crawlers cannot understand the structure of the site.
Any recommendations what the best practice is?
First of all, let's get a few misconceptions out of the way.
Anything for which you have semantics identifiable by an URI is a resource.
The HTTP methods don't map to CRUD operations like that. Read this answer for more on that. I recommend you read it before continuing reading this answer. This one is probably going to be helpful too.
There's no such thing as an URI that follows the REST paradigm. The only constraints REST imposes on URIs is that they must identify one and only one resource, and they must be treated as atomic identifiers. The semantics of the URI is irrelevant, although obviously you should design URIs that make sense for the developers and users.
As you already figured out, the correct way to return an user-friendly representation of something is through negotiation via the Accept header. It doesn't matter if it's not something that maps to a database. That's an implementation detail only the server knows, and that's what REST is about. When you retrieve something from a REST API, it doesn't matter if it's coming from the application server, from a cache somewhere, from a static file served by Amazon S3, or even an FTP link. The client should simply follow links, like when you click a link on a webpage and you don't care where the result comes from.
Both options you present are valid, but that has nothing to do with REST. Separating them in api and ui is a nice way to organize things for you, but what really matters to REST is how the client obtains those links. If they are implementing clients by reading those URIs in documentation and filling up values, that's not REST.
Think about it from a web browsing perspective. How do you reach that /newPersonForm html page? You followed a link somewhere else that had a label telling you to click on it to create a new Person. If you are the one clicking, it doesn't matter if it's /newPersonForm or /forms/newperson or simply /persons. REST works in the exact same way.
OK so I have authenticated fine and understand JSON including how to process it with PHP...
http://php.net/manual/en/function.json-decode.php
But my issues are that the JSON data that the API returns after hitting up FB...me/home end point is very varied. I understand that so is the data on a FB news stream but...
My end game is to show this info in a mash-up with other data but there seems no clear way of formatting it and each item in the JSON response is hugely different with different values. No consistent "item_type" : "photo / status update / message" that I could use to template it up into nicely formatted content blocks on the front end?
Any suggestions on the best way to process / style this type of vary varied JSON?
Thanks.
I would suggest checking out Google's gson library for parsing through JSON documents. Assuming that you are familiar with Java it makes it relatively easy to develop a custom parser that can adapt to documents stored under a flexible schema.
The library page is here. You can find the JsonReader documentation here.
Edit: It also might help to post some example documents so we can get a better idea of what kinds of results you're getting.
Recently came across the term JSONC in a YouTube API. I browsed the Web, but found nothing much about it. Can someone explain whether these two are the same or different?
There is also jsonc aka "JSON with comments", created by Microsoft and used by Visual Studio Code. The logic for it can be found here, alas without exhaustive specification (though I'd like to be proven wrong on this).
On top of that there is this project with an actual specification which is also called jsonc, but also does far more than just adding comments.
While there definitely is a use for these technologies, some critical thinking is advised. JSON containing comments is not JSON.
JSON-C seems to just be a variation of JSON mainly targeted at C development. I.e., from the open source docs, "JSON-C implements a reference counting object model that allows you to easily construct JSON objects in C, output them as JSON formatted strings and parse JSON formatted strings back into the C representation of JSON objects."ref^1
From the YouTube API perspective (specifically, version 2, not the new version 3), The JSON-C response is just a condensed version of the JSON response (removing "duplicate, irrelevant or easily calculated values").ref^2
Why would the JSON response have "duplicate, irrelevant or easily calculated values" values anyway? Because it is converting the original ATOM XML format directly to JSON in a loseless conversion. You can find out more details here.
However, I would suggest using version 3 of the YouTube Data API. It is much easier to use. =)
JSONC is an open-source, Javascript API created by Tomás Corral Casas for reducing the size of the amount of JSON data that is transported between clients and servers. It uses two different approaches to achieve this, JSONC.compress and JSONC.pack. More information can be found on the JSONC Github page:
https://github.com/tcorral/JSONC