How can I generate a file like this for Bing Heat Map data? - heatmap

I am working on a fairly simple Heat Map application where the longitude and latitude of the points will be stored in a SQL Server database. I have been looking at an example that uses an array of objects as follows (eliminated a lot of data for brevity):
/* Sample data to demonstrate Bing Maps Heatmap */
/* http://alastair.wordpress.com */
var CrimeData = [
new Microsoft.Maps.Location(52.67280, 0.94392),
new Microsoft.Maps.Location(52.62423, 1.29493),
new Microsoft.Maps.Location(52.62187, 1.29080),
new Microsoft.Maps.Location(52.58962, 1.72228),
new Microsoft.Maps.Location(52.69915, 0.24332),
new Microsoft.Maps.Location(52.51161, 0.99350),
new Microsoft.Maps.Location(52.59573, 1.17067),
new Microsoft.Maps.Location(52.94351, 0.49153),
new Microsoft.Maps.Location(52.64585, 1.73145),
new Microsoft.Maps.Location(52.75424, 1.30079),
new Microsoft.Maps.Location(52.63566, 1.27176),
new Microsoft.Maps.Location(52.63882, 1.23121)
];
What I want to do is present the user with a list of some sort that displays all the data sets that exist in the database (they each have a name associated with them) and then allow the user to check all or only a select few. I will then need to generate an array like the above to create the heat map. Any ideas on a good approach to this?

What you trying to achieve is more related to a web developement rather than only related to Bing Maps.
To summarize, you have multiple ways to do this but it really depends on what you are capable to do and what you need in the interface.
What process/technology?
First, you need to determine what process you want to follow to display the data and it will set the technology that you will use. The questions that you need to ask yourself are:
Do you want to be able to change the data sets dynamically without refreshing the whole page?
If yes, it means that you will have to use asynchronous data loading through a dedicated web service in order to avoid loading all the information at the initial load of the page.
Do you have lots of data to load?
If so, it might comfort you with asynchronous loading to avoid loading all data.
If not loading every elements in multiple arrays might be the simplest solution.
Implementation
So now, you want to create a web service to load the data asynchronously, you can take a look at the following websites :
http://www.asp.net/get-started
http://www.stefanprodan.com/2011/04/async-operations-with-jquery-ajax-and-asp-net-mvc/
There might be interesting other website, you will be able to find them. If needed, add comment and I'm sure the community will help you.
If you want to generate the data directly in the script, it could be simple as you can compose the JavaScript directly in your dynamically created HTML page (in your ASP.Net markup code or whatever technology you're using).

Related

How to restrict fields returned by stackexchange api, and turn off paging?

I'd like to have a list of just the current titles for all questions in one of the smaller (less than 10,000 questions) stackexchange site. I tried the interactive utility here: https://api.stackexchange.com/docs/questions and it both reports the result as a json at the bottom, and produces the requesting url at the top. For example:
https://api.stackexchange.com/2.2/questions?order=desc&sort=activity&tagged=apples&site=cooking
returns this JSON in my browser:
{"items":[{"tags":["apples","crumble"],"owner":{ ...
...
...],"has_more":true,"quota_max":300,"quota_remaining":252}
What is quota? It was 10,000 on one search on one site, but suddenly it's only 300 here.
I won't be doing this very often, what I'd like is the quickest way to edit that (or similar of course) url so I can get a list of all of the titles on a small site. I don't understand how to use paging, and I don't need any of the other fields. I don't care if I get them, but I'm thinking if I exclude them I can have more at once.
If I need to script it, python (2.7) is my preferred (only) language.
quota_max is the number of requests your application is allowed per day. 300 is the default for an unregistered application. This used to be mentioned directly on the page describing throttles, but seems to have been removed. Here is historical information describing the default.
To increase this to 10,000, you need to register an application and then authenticate by passing an access token in your script.
To get all titles on a site, you can use a Python library to help:
StackAPI. The answer below will use this library. DISCLAIMER: I wrote this library
Py-StackExchange
SEAPI
StackPy
Assuming you have registered your application and authenticated we can proceed.
First, install StackAPI (documentation):
pip install stackapi
This code will then grab the 10,000 most recent questions (max_pages * page_size) for the site hardwarerecs. Each page costs you one API hit, so the more items per page, the few API calls.
from stackapi import StackAPI
SITE = StackAPI('hardwarerecs')
SITE.page_size = 100
SITE.max_pages = 100
# Filter to only get question title and link
filter = '!BHMIbze0EQ*ved8LyoO6rNjkuLgHPR'
questions = SITE.fetch('questions', filter=filter)
In the questions variable is a dictionary that looks very similar to the API output, except that the library did all the paging for you. Your data is in questions['data'] and, in this case, contains a list of dictionaries that look like this:
[
...
{u'link': u'http://hardwarerecs.stackexchange.com/questions/29/sound-board-to-replace-a-gl2200-in-a-house-of-worship-foh-setting',
u'title': u'Sound board to replace a GL2200 in a house-of-worship FOH setting?'},
{ u'link': u'http://hardwarerecs.stackexchange.com/questions/31/passive-gps-tracker-logger',
u'title': u'Passive GPS tracker/logger'}
...
]
This result set is limited to only the title and the link because of the filter we applied. You can find the appropriate filter by adjusting what fields you want in the web UI and copying the filter field.
The hardwarerecs parameter that is passed when creating the SITE parameter is the first part of the site's domain URL. Alternatively, you can find it by looking at the api_site_parameter for your site when looking at the /sites end point.

Scraping data after filling out form?

I'm doing a little project for my class and I'm just a beginner, so please forgive me if I mix up some of my terminology.
Basically, I'm creating an interactive journey planner for my city's public transit system. Unfortunately, they haven't made all the data I need publicly available. So instead of putting all my time into gathering the data for personal use, I've opted to do some screen scraping - letting their servers calculate the journey info from a START and STOP variable and then displaying the selected info on my page.
So is it possible to fill out a form's fields remotely, and then scrape the data on the page that subsequently loads? And if so, what would be the quickest, most convenient way? This happens to be a case where the data can't be manipulated via the URL, so it has to access the data by filling out the form first.
The website in question:
http://jp.translink.com.au/travel-information/journey-planner
Here is what you can do:
1.) Send a POST Request to the journey-planner with some data like that (be aware that CORS might jump in, then you could use cURL via PHP or whatsoever):
Start:Wickham Tce, Spring Hill
End:Upper Edward St, Spring Hill
SearchDate:10/05/2013 12:00:00 AM
TimeSearchMode:LeaveAfter
SearchHour:7
SearchMinute:40
TimeMeridiem:AM
TransportModes:Bus
TransportModes:Train
TransportModes:Ferry
MaximumWalkingDistance:1500
WalkingSpeed:Normal
ServiceTypes:Regular
ServiceTypes:Express
ServiceTypes:NightLink
FareTypes:Standard
FareTypes:Prepaid
FareTypes:Free
2.) You will get a new response location. This seems to be a REST link. Important for you is the id at the end. You will have to call that page and parse the HTML and look for a div with the HTML-id option-summaries, where you will find more information within the divs travel-option-1 to travel-option-n. You have to look at it carefully in order to find out which information is stored whee and how you will be able to use it.
In order to find such things you should learn how to use Firebug or Chrome's development tools.
This is one way to solve your problem. Probably not the best but still better than "screen-scraping" anything. But it will ask you for a lot of skills and effort. Furthermore if the data provider is going to change just a bit your solution will not work anymore. Additionally they might prevent your access by CORS or anything else (blocking your IP etc.)

How do I save geolocation data for an HTML5 or trigger.io app for path mapping later?

Im creating an app that needs to track the location of the user (with their knowledge, like a running app) so that I can show them their route later. Should I use HTML5 with some timeout interval to save the coordinates every N seconds and if so, how often should I save the data and how should I save it (locally using local storage or post it to the server?)
Also, what is the easiest way to display the map of where the user has been later?
Has anyone done anything like this before?
The timeout interval for forge.geolocation is up to you and the balance of responsiveness of your application. Also, network traffic is expensive. So maybe you can buffer... say the last 10 geopositions... and then Http post (or whatever... see Parse below) in bulk? And since the geo data sounds like temporary device data why would there be a need to persist using forge.prefs? Unless maybe you need to the app to work "offline"?
For permanent storage I would look at Parse (generous free plan) and their Parse.GeoPoint class via their Javascript or REST Api as one possible solution. They have some nifty methods like (kilometersTo, milesTo, radiansTo) - https://parse.com/docs/js/symbols/Parse.GeoPoint.html

Tool(s) to + column & rows on HTML5 tables

I was about to fork a compatibility guide & when I looked at the HTML, I saw it was a table.
Tables are good to show data like this, but not fun to edit when you have to edit every few weeks.
What tools can edit an existing HTML table very quickly & cleanly add/remove/clone columns & rows quickly & painlessly please?
Can be online, GUI or WYSIWYG (prefered) or command-line driven, on or off-line (though online with JavaScript engine that is Node.js compatible is preferred).
Or perhaps another way is better to show tabular data that changes every few weeks, like AJAX presenting JSON data, but still need some hand-holding for the best scenario here also.
Table & suggestion in question:
https://github.com/kangax/es5-compat-table/blob/gh-pages/es6/index.html
https://github.com/kangax/es5-compat-table/issues/29
Different people has different needs and attitudes!
1).You can use a spreadsheet program to edit the table and export to html.Google spreadsheet provides functionality to embed a preadsheet to a web page or to download(export) the spreadsheet in simple HTML, you can use CSS to style it to fit your needs.
2).Google's API can grab data from a spreadsheet and show like a table.What you have to do is just edit the Google spreadsheet!
I checked the API and could build a quick example,Here is the fiddle!
Here is a little summery of what I did,
var query = new google.visualization.Query(
'http://spreadsheets.google.com/tq?key=pCQbetd-CptGXxxQIG7VFIQ&pub=1');
The spread sheet I refer here is http://spreadsheets.google.com/ccc?key=pCQbetd-CptGXxxQIG7VFIQ
// Apply query language.
//I'm selecting all the cols!
query.setQuery('SELECT * ');
// Send the query with a callback function.
query.send(handleQueryResponse);
in my callback,
var data = response.getDataTable();
//init the table
visualization =
new google.visualization.Table(document.getElementById('table'));
visualization.draw(data, null);

Having trouble binding JSON data to a mobile list in Adobe Flash builder

Hi, i have been having some problems using JSON data in flash builder lately and I was hoping someone could help me out here.
I have spent the past month working solidly on this issue, so I HAVE been looking around, HAVE been trying out everything I can find or think of. I am simply stuck.
I have been working on a flex mobile application for the Blackberry Playbook tablet with Adobe Flash Builder 4.6. It is a reddit app, designed to give users the main reddit feed, subreddits, search function, hopefully log in stuff etc. Of course I need the aid of the reddit API to access this information, of which the documentation can be found here: https://github.com/reddit/reddit/wiki/API/ The api uses either XML or JSON formatted data.
Now onto my problem- Like mentioned above, i want to display the reddit feed inside the app. I want to be able to use a item renderer to customize the data that is shown within each entry of the list.
One entry would consist of:
1) a thumbnail of the image in the post
2) The title of the post
3) a 'like/dislike' button, but this is unimportant at the moment.
Of course to start out, i placed a spark List component onto the design view. Then i configured a new HTTP data service using the Data/Services panel. I specified http://www.reddit.com/r/all.json for the url. I configured the return type, and the did a Test Operation. Everything connected just fine. All the data came through as normal. I will give you an idea of what the data comes back as so that you may understand my issue later on.
Test Operation Results (json data structure):
NoName1
data
after
before
children
[0]
data
media_embed
score
id
title
thumbnail
url
(etc etc...)
kind
[1]
data
media_embed
score
title
thumbnail
(etc etc...)
kind
[2] (array continues)
modhash
kind
As you can see, to get to the thumnail for example, you would have to go through data.children[].data.thumnail. When I tried to bind this data to the spark List component, I specified the data service to be from the one above. Then I specified the Data provider option to be Children[], as this value is typically set to the array. Now this is where the trouble begins. The final option, Label Field, only gave me one value to choose from: 'kind'. So as you can tell, it wasnt expecting the data to go any further nested. It stops at the two value just within each array item, which would be Data and Kind, though it only offers me Kind. I need to go one level further to access Title and Thumbnail. This is my problem.
Now, I have analyzed the code for the binding, and I have tried altering it to accomodate the further nested value. No success what so ever. The following is the code that the binding generates:
<s:List
id="myList" width="100%" height="100%" change="myList_changeHandler(event)"
creationComplete="myList_creationCompleteHandler(event)" labelField="kind">
<s:AsyncListViewlist="{TypeUtility.convertToCollectionredditFeedJSONResult.lastResult.data.children)}"/>
<s:List>
Obviously i would want to have something like along the lines of:
"TypeUtility.convertToCollection(redditFeedJSONResult.lastResult.data.children.data" and then set the labelField="title" or "thumbnail".
I certainly hope someone can help me with this. I am out of my mind as to what to do. If you need any further clarification I would be happy to provide it. I tried to explain the situation above as clearly as possible. Thank you so much.
Ted
I have this situation often: get an XML or JSON data from the server, then trying to use it as a dataProvider for a spark.components.List or for a mx.controls.Menu and then they just wouldn't display the data as I want them, because something in the data is different from what they expect. And then they display wrong XML-children or [Object,Object,etc.]
And in such cases I just create an empty ArrayCollection() which serves as dataProvider instead (and can be sorted and/or filtered too). And when data comes from the server, I push() new Objects into it:
[Bindable]
private var _data:ArrayCollection = new ArrayCollection();
public function update(xlist:XMLList):void {
_data.length = 0;
for each (var xml:XML in xlist)
_data.push({label: xml, event: xml.#event});
}
This always works. And if you get your next problem - flickering of the List, then that is solvable by merging data too.
Good luck with your Playbook development, which is a cool piece of hardware :-)