I'm trying to fetch "current / max Players" from this site:
http://rust-servers.net/server/64099/
I would like to display just the player numbers, eg. "70 / 175" on another website and have it update every time someone visits my .html page (I can change that one to .php if needed.)
How would I go about doing that in the most simple and efficient way?
I've googled the issue for some hours without any luck, I'm no closer to understanding what I would want to use to do this as everyone seems to suggest widely different methods and most seem way too verbose for the simple thing I'm trying to do - many examples fetch the data as JSON (?) with some JS/jQuery (?) and use a bit of code to find specific items in that data, define it as a variable or array and then display it later.
I've figured that the information I want can be referred to using XPath "/html/body/div[4]/div/div/div[9]/div[1]/table/tbody/tr[4]/td[2]/strong" but that's about it. How do I proceed from there?
Thank you.
It depends on which platform do you want to use.
If you use C#, you can use HtmlAgilityPack library.
It is basically like this:
var webClient.DownloadString("http://rust-servers.net/server/64099/");
var doc = new HtmlAgilityPack.HtmlDocument();
doc.LoadHtml(htmlCode);
var node = doc.DocumentNode.SelectSingleNode("/html[1]/body[1]/div[4]/div[1]/div[1]/div[9]/div[1]/table[1]/tbody[1]/tr[4]/td[2]");
var nodeValue = node.InnerText;
Related
I am new to symfony and open-source in general. Im trying to create a simple product submition page which would store product's information in mysql. So far I've only been able to hard-code all the information and successfully store it (this was my first task to achieve). This is the code for it:
public function createAction()
{
$product = new Product();
$product->setName('Keyboard');
$product->setPrice(15.50);
$product->setDescription('Logitech keyboard v2.00');
$em = $this->getDoctrine()->getManager();
$em->persist($product);
$em->flush();
return new Response("Saved new product with: " .$product->getId() ."Id");
}
Now I want to have a page with couple text boxes and to be able to take this information, validate it and then store it. What would be a clever way to do it? As I'm learning for my internship, I want to be able to do it in a best possible way. Are twig templates the best solution here? What about fetching user's input? I would be really thankful if someone could write step-by-step bullet points on how to achieve my goal in abstract sentences. Thank you and have a great remaining Friday!
1) One of the best and yet simple ways for you to start your mission would be by reading Symfony book chapter about Forms. It contains basic examples about form rendering, validation and submission using twig.
2) If you need to know more about how to save everything to database and would like to use Doctrine (most common way) then this chapter covers it in detail and is also not too deep to make it easy enough.
3) If you would like to deepen your understanding of Validation then you should definitely read about it too
http://symfony.com/doc/current/book/forms.html
http://symfony.com/doc/current/book/doctrine.html
http://symfony.com/doc/current/book/validation.html
I am working on a fairly simple Heat Map application where the longitude and latitude of the points will be stored in a SQL Server database. I have been looking at an example that uses an array of objects as follows (eliminated a lot of data for brevity):
/* Sample data to demonstrate Bing Maps Heatmap */
/* http://alastair.wordpress.com */
var CrimeData = [
new Microsoft.Maps.Location(52.67280, 0.94392),
new Microsoft.Maps.Location(52.62423, 1.29493),
new Microsoft.Maps.Location(52.62187, 1.29080),
new Microsoft.Maps.Location(52.58962, 1.72228),
new Microsoft.Maps.Location(52.69915, 0.24332),
new Microsoft.Maps.Location(52.51161, 0.99350),
new Microsoft.Maps.Location(52.59573, 1.17067),
new Microsoft.Maps.Location(52.94351, 0.49153),
new Microsoft.Maps.Location(52.64585, 1.73145),
new Microsoft.Maps.Location(52.75424, 1.30079),
new Microsoft.Maps.Location(52.63566, 1.27176),
new Microsoft.Maps.Location(52.63882, 1.23121)
];
What I want to do is present the user with a list of some sort that displays all the data sets that exist in the database (they each have a name associated with them) and then allow the user to check all or only a select few. I will then need to generate an array like the above to create the heat map. Any ideas on a good approach to this?
What you trying to achieve is more related to a web developement rather than only related to Bing Maps.
To summarize, you have multiple ways to do this but it really depends on what you are capable to do and what you need in the interface.
What process/technology?
First, you need to determine what process you want to follow to display the data and it will set the technology that you will use. The questions that you need to ask yourself are:
Do you want to be able to change the data sets dynamically without refreshing the whole page?
If yes, it means that you will have to use asynchronous data loading through a dedicated web service in order to avoid loading all the information at the initial load of the page.
Do you have lots of data to load?
If so, it might comfort you with asynchronous loading to avoid loading all data.
If not loading every elements in multiple arrays might be the simplest solution.
Implementation
So now, you want to create a web service to load the data asynchronously, you can take a look at the following websites :
http://www.asp.net/get-started
http://www.stefanprodan.com/2011/04/async-operations-with-jquery-ajax-and-asp-net-mvc/
There might be interesting other website, you will be able to find them. If needed, add comment and I'm sure the community will help you.
If you want to generate the data directly in the script, it could be simple as you can compose the JavaScript directly in your dynamically created HTML page (in your ASP.Net markup code or whatever technology you're using).
I'm trying to extract elements from a table, I have successfully used get and HTML:TableExtract to get elements of the table. The problem is the table is multi page and navigated with an arrow button to show additional pages. How would i extract these other pages as they are not new links but I think generated with JS or such?
Specifically I am trying to extract the table under Data for this Data Range at:
http://ycharts.com/companies/GOOG/pe_ratio#series=type:company,id:GOOG,calc:pe_ratio,,id:AAPL,type:company,calc:pe_ratio,,id:AMZN,type:company,calc:pe_ratio&zoom=3&startDate=&endDate=&format=real&recessions=false
See how there is the Viewing x of 45 and the First, Previous, Next, Last button.
The rest of the table elements can be viewed with next, how would i extract these in perl?
Update::
Hi Simbabque, Thanks for the response.
So I see if you click on next it calls:
ng-click="getHistoricalData(historicalData.currentPage+1)"
Is there a way I can call this method? I tried to use click,but it is not bound a name. (JS?)
I was trying to use Mechanize::Firefox now but I feel like their must be an easy way to use regular Mech and call the function and re-read the page?
The website builds up the tables using AJAX requests. Those are a little harder to parse. You can use WWW::Mechanize to fetch the initial page and then hit the AJAX calls for the table. It helps you keep track of cookies and stuff automatically.
use strict; use warnings;
use WWW::Mechanize;
my $mech = WWW::Mechanize->new;
$mech->get('http://ycharts.com/companies/GOOG/pe_ratio#series=type:company,id:GOOG,calc:pe_ratio,,id:AAPL,type:company,calc:pe_ratio,,id:AMZN,type:company,calc:pe_ratio&zoom=3&startDate=&endDate=&format=real&recessions=false');
my $response = $mech->post(
'http://ycharts.com/companies/GOOG/pe_ratio/data_ajax',
{
startDate => '1/1/1962',
endDate => '12/3/2013',
pageNum => 4,
}
);
if ( $response->is_success ) {
print $response->decoded_content; # or whatever
} else {
die $response->status_line;
}
This is just a basic example and will not work. It gives a 403 Forbidden. Probably there is more data required. Use Firebug or a similar tool to inspect what is happening. For example, there's another call to http://ping.chartbeat.net/ping?h=ycharts.com&p=%2Fcompanies%2FGOOG%2Fpe_ratio&u=o3m6snxteynby1b8&d=ycharts.com&g=20054&n=1&f=00001&c=10.81&x=200&y=1812&o=1663&w=658&j=30&R=0&W=1&I=0&E=109&e=6&b=1903&t=usmc0fjfd1j0h87g&V=16&_ happening automatically every now and again, with varying parameters. That is most likely required to keep the session going.
This page is pretty sophisticated. This might not be the best approach.
You could also try to use WWW::Mechanize::Firefox or even Selenium to remote-operate a browser. That will be better suited as it takes care of all the AJAX stuff that is happening.
Or you could look for a public API that just hands over that data voluntarily. I bet there is one around... or just pay for a ycharts pro account and hit the download button. ;-)
My problem lies in 2 parts however I'm hoping solving 1 will fix the other. I've been trying to parse through a page and get all the comments found within a forum thread.
The comments are found using a RegEx pattern and the idea is that whatever lies in the comment will be read into an array until there aren't any more comments left. Each comment div follows this format
<div id="post_message_480683" style="margin-right:2px;"> something </div>
I'm trying to locate up to "post_message_[some number]" since each number seems to be generated randomly and then get whatever is between that particular div. My 1st problem is my RegEx just doesn't seem to be working I've tried a few but none yielded any results (except for when I insert the post message no. in manually), Here's the code so far:
function GetPosts() {
var posts = new Array(60);
var url = "http://forums.blackmesasource.com/showthread.php?p=480683";
var geturl = UrlFetchApp.fetch(url).getContentText().toString();
var post_match = geturl.match(/<div id="post_message_(.+)" style="margin-right:2px;">(\w.+)<\/div>/m);
Logger.log(post_match);
}
Edit: I initially tried getting this info via GAS's Xml.Parse() class but after grabbing the URL I just didn't know what to do since suffixing
.getElement().getElement('div') (I also tried .getElements('div') and other variations with 'body' & 'html')
would cause an error. Here is the last code attempt I tried before trying the RegEx route:
function TestArea() {
var url = "http://forums.blackmesasource.com/showthread.php?p=480683";
var geturl = UrlFetchApp.fetch(url).getContentText().toString();
//after this point things stop making sense
var parseurl = Xml.parse(geturl, true);
Logger.log(geturl);
//None of this makes sense because I don't know HOW!
//The idea: Store each cleaned up Message Div in an Array called posts
//(usually it's no more than 50 per page)
//use a for loop to write each message into a row in GoogleSpreasheet
for (var i = 0; i <= parseurl - 1; i++) {
var display = parseurl[i];
Logger.log(parseurl); }
}
Thanks for reading!
In general like the comment points out - be aware of parsing HTML with RegEx.
In my past personal experience, I've used Yahoo's YQL platform to run the HTML through and using XPath on their service. Seems to work decently well for simple reliable markup. You can then turn that into a JSON or XML REST service that you can grab via UrlFetch and work on that simplified response. No endorsement here, but this might be easier than to bring down the full raw HTML into Google Apps Script. See below for the YQL Console. I also don't know what their quotas are - you should review that.
Of course, the best course is to convince the site owner to provide an RSS feed or an API.
Hi, i have been having some problems using JSON data in flash builder lately and I was hoping someone could help me out here.
I have spent the past month working solidly on this issue, so I HAVE been looking around, HAVE been trying out everything I can find or think of. I am simply stuck.
I have been working on a flex mobile application for the Blackberry Playbook tablet with Adobe Flash Builder 4.6. It is a reddit app, designed to give users the main reddit feed, subreddits, search function, hopefully log in stuff etc. Of course I need the aid of the reddit API to access this information, of which the documentation can be found here: https://github.com/reddit/reddit/wiki/API/ The api uses either XML or JSON formatted data.
Now onto my problem- Like mentioned above, i want to display the reddit feed inside the app. I want to be able to use a item renderer to customize the data that is shown within each entry of the list.
One entry would consist of:
1) a thumbnail of the image in the post
2) The title of the post
3) a 'like/dislike' button, but this is unimportant at the moment.
Of course to start out, i placed a spark List component onto the design view. Then i configured a new HTTP data service using the Data/Services panel. I specified http://www.reddit.com/r/all.json for the url. I configured the return type, and the did a Test Operation. Everything connected just fine. All the data came through as normal. I will give you an idea of what the data comes back as so that you may understand my issue later on.
Test Operation Results (json data structure):
NoName1
data
after
before
children
[0]
data
media_embed
score
id
title
thumbnail
url
(etc etc...)
kind
[1]
data
media_embed
score
title
thumbnail
(etc etc...)
kind
[2] (array continues)
modhash
kind
As you can see, to get to the thumnail for example, you would have to go through data.children[].data.thumnail. When I tried to bind this data to the spark List component, I specified the data service to be from the one above. Then I specified the Data provider option to be Children[], as this value is typically set to the array. Now this is where the trouble begins. The final option, Label Field, only gave me one value to choose from: 'kind'. So as you can tell, it wasnt expecting the data to go any further nested. It stops at the two value just within each array item, which would be Data and Kind, though it only offers me Kind. I need to go one level further to access Title and Thumbnail. This is my problem.
Now, I have analyzed the code for the binding, and I have tried altering it to accomodate the further nested value. No success what so ever. The following is the code that the binding generates:
<s:List
id="myList" width="100%" height="100%" change="myList_changeHandler(event)"
creationComplete="myList_creationCompleteHandler(event)" labelField="kind">
<s:AsyncListViewlist="{TypeUtility.convertToCollectionredditFeedJSONResult.lastResult.data.children)}"/>
<s:List>
Obviously i would want to have something like along the lines of:
"TypeUtility.convertToCollection(redditFeedJSONResult.lastResult.data.children.data" and then set the labelField="title" or "thumbnail".
I certainly hope someone can help me with this. I am out of my mind as to what to do. If you need any further clarification I would be happy to provide it. I tried to explain the situation above as clearly as possible. Thank you so much.
Ted
I have this situation often: get an XML or JSON data from the server, then trying to use it as a dataProvider for a spark.components.List or for a mx.controls.Menu and then they just wouldn't display the data as I want them, because something in the data is different from what they expect. And then they display wrong XML-children or [Object,Object,etc.]
And in such cases I just create an empty ArrayCollection() which serves as dataProvider instead (and can be sorted and/or filtered too). And when data comes from the server, I push() new Objects into it:
[Bindable]
private var _data:ArrayCollection = new ArrayCollection();
public function update(xlist:XMLList):void {
_data.length = 0;
for each (var xml:XML in xlist)
_data.push({label: xml, event: xml.#event});
}
This always works. And if you get your next problem - flickering of the List, then that is solvable by merging data too.
Good luck with your Playbook development, which is a cool piece of hardware :-)