I want to make a line chart that automatically updates with data from this page: http://www2.nve.no/h/hd/plotreal/Q/0027.00025.000/knekkpunkt.csv
The .csv is updated once per hour, and contains date and two values for amount of water flowing in a river. How can I set up a Highcharts, or similar, to get data from this file, and render to a graph?
I don't have server access to nve.no, where the data is stored.
Appreciate any ideas!
As a part of the ZingChart team, we've received several questions similar to this one. You're likely to initially run in to cross-domain issues when using a resource that is hosted on another domain, especially since you do not have any access to that server.
There are ways around this, however. One method involves using JSONP and YQL (Yahoo! Query Language). Using YQL, you can pull from the URL that you've provided and have the data returned as JSONP.
Here is a JSbin demo that shows this off: http://jsbin.com/hidel/1/edit?html,output
At the beginning I recommend you to visit the working with data article.
Related
I've been wondering how to fetch the PlayStation server status. They display it on this page:
https://status.playstation.com/en-us/
But PlayStation is known to use APIs instead of PHP database fetches. After looking around in the source code of the site, I found that they have a separate file called /data.json.
https://status.playstation.com/en-us/data.json
The content of this file is the same as the index file (for some reason). They use stuff like {{endDateTitle}} and {{message}}, but I can't find where it's defined, if it's pulled using a separate file or just pulled from a database using PHP.
How can I "reverse" this site and see if there's a API I can use to display the status on my site?
Maybe I did not get the question right, but it seems pretty straightforward.
If using firefox, open Developer tools, Network. Reload the page.
You can clearly see the requested URL
https://status.playstation.com/data/statuses/region/SCEA.json
It seems that an empty list as a status means "No problems" (since there are no problems I cannot verify this assumption. That's all
The parenthesis {{}} are used by various HTML templating languages, like angular, so you'd have to go through the js code to understand where they get updated.
I am at this website -
http://www.zoominfo.com/s/#!search/company/1.64.eyJjb21wYW55TmFtZSI6xIB2YWx1xIw6ImEiLCJpc1VzZWTEjXRyxJN9fQ%3D%3D
If you see the company name - Agilent Technologies Inc.
Its neither there in page source, nor in any json format.
But it does show in the Dom of Chrome Developer tool.
I have looked and analysed almost every requests that it sent, but still couldn't find where this data is saved.
By where the data is saved - I am looking to find where I can scrape that data from?
If by using python-requests and BeautifulSoup
I do see an XMLHTTPREQUEST made, not sure what that means, or if that is the clue to my answer.
I am still learning python, and it would be a very useful information if someone helps me with this.
Thanks in advance.
After the HTML is loaded, js requests for the data through an XMLHTTPREQUEST which is loaded right after the request is received on your client. That's why you see the DOM element right there using element inspector.
You didn't mention what goal you want to achieve or what tool you are using. Please be specific on your question. If you do not have any idea about this kind of pattern, google out angularjs, see some example.
do see an XMLHTTPREQUEST made, not sure what that means, or if that is the clue to my answer.
It means that javascript embedded in the page is sending an extra HHTP request to the web server. It is likely that the "Agilent Technologies Inc." text is being returned in the server's response to that request, and the javascript in the page is then injecting the text into the DOM in the appropriate place.
Where is the Data stored on Website
That is a completely different question ...
(You have already noted that the data (e.g. the company name) gets injected into the page displayed by your browser.)
On the server side, the data could be stored in the web server (or its back-end systems) in a variety of ways. Or it might not be stored at all. There is no way of knowing ... without looking at the server-side code and configurations.
I have a drupal site that is being used strictly as a CMS that produces JSON feeds using services and services_views, which are consumed by a separate site. What I would like to do (and I have a working proof of concept of this) is allow for a "live preview" on the real site, by intercepting the node form preview / submit, encoding the node as JSON, and loading a special page on the live site that consumes that JSON and displays the page accordingly.
The problem with this JSONized node is, it's different from the JSON being produced by my view (using services_views). My end goal is to produce JSON that is identical for both previewed and non-previewed objects, without having to maintain separate output methods (I could easily hand-customize the json but then when my view for the public api changes I have to make the same changes to the preview json. Trying to avoid this).
I'm looking for feedback on this approach. Is what I'm attempting even possible? The ideas I've been able to come up with so far are:
being able to (conditionally) drive my view with data from a non-databse source
sneakily inserting data into the view object during one of the stages of execution? Kludgy but I'm not above that :)
saving a "clone" node (or revision?) of the node being previewed and let the view use that to display the preview JSON?
Maybe this is the wrong approach altogether and there's something better? (Trying to intercept and format the services output in my module... maybe avoid services_views altogether?)
If anyone can offer some advice, insight or opinions on how to best proceed here, I'd be really grateful.
in a custom module, you could set up a page that grabs the json output from the view page.
$JSON = file_get_contents($url);
that way the preview stays bound to the view, even if the view changes.
First I think it's not an easy task what you are trying to achieve. So before all, good luck.
I think you could intercept the node submission data, then create a node programatically, then render that node, and then export the rendered node to JSON. Inmediately after you get the JSON, delete this node, because the programmatically created node is only for preview.
This task could be more CPU demanding but think that previewing content exactly as the content will look is difficult.
Your rss feeds that your site reads could be filtered with some parameter to avoid programmatically created nodes (prewiew nodes), despite these nodes will be available for a very short time.
I'm trying to add some functionality to an internal Access database that will automatically create tasks on Asana.
My VBA experience is somewhat limited but by examining various code samples online and tinkering I've been able to POST tasks with all the necessary data and GET info back.
But I'm now struggling with being able to upload file attachments to a task. I'm not sure how to go about it.
Leaving the content type as application/x-www-form-urlencoded which works for the normal POST statements when creating tasks and just pointing the send command to a file location using doesn't work and results in an error of "file is not an object", I'm guessing this is because all that's contained in the send command is a file=path pair.
Do I need to encode the file at all, if so how?
I'm hoping someone can point me in the right direction.
Thanks.
You can check out how curl does it, but I believe it needs to be multipart/form-data - I would strongly recommend using a library rather than doing the encoding manually, since there are often subtle gotchas.
Basically, it works a lot like a standard form upload from a web browser.
Hope that helps!
Using the excellent Wordpress plugin, JSON API, I've created an application that retrieves fresh data by visiting a URL along the following lines, doing a Unix curl dump.
http://www.website.net/?json=get_recent_posts&post_type=tree&count=200
The problem is this: Because the site delivers cache versions using W3 Total Cache, the stuff that gets curled is inconsistent. Sometimes I get the JSON that I need. Other times I just get a cache showing me "website.net"'s front page, without the effects of the query string.
I'm hoping to do one of two things:
1) Find a way for my request to bypass w3 Total Cache. But how? Is there something I can put into the query string that will tell Wordpress to give me a fresh page read and not a cache?
If that can't work,
2) Get the plugin data through PHP rather than through the RESTful Web request. But I only know how to use JSON API for Web URL calls and not for using the JSON data internally.
Any help much appreciated. I also thought that maybe W3TC's function to cache query string results might work, but I can't figure it out.
If you are using page cache using disk enhanced in w3tc settings, the URL with query strings will not cached. I think this solves your problem.