ive webscraped before using beautiful soup by making a html request.
but now i dont want data from say wikipedia,i want data from something after it has been modified by the user. eg teslas website has this feature https://www.tesla.com/en_gb/models/design#battery where you personalize your car. suppose i wanted to open this page, change the settings from long range to high performance and THEN grab the inner html from
<span class="specs--value">
(how long it takes to go from 0 to 60)
how could i do this. write a script in python/c# which can grab data from the browser i currently have open, using what tools can i gather data from my current browser
ideally this will be for google chrome
edit:
alternate idea. perhaps i could make a post request which somehow recieves html data as if i did click on the high performance button. but how could i do that?
You could use selenium, It allows to simulate user navegation and extract data during the process.
Related
I'm trying to write a very simple client application for iOS to go to a website with three buttons. Each of these buttons load a different webpage and run a command in an application on the server, takes the results from that command and displays them on a map on the very same page that the button loaded.
What I'm trying to achieve is to be able to do the following:
Click on one of the three buttons.
Have it then run the query.
JUST PULL THE DIV WITH THE MAP TO SHOW THE DATA instead of the entire page, which is what I currently have happening.
I was going to go about this by parsing the html with "libxml2" and "hpple" but I thought that I would ask here before I got started instead of spending a few days on this to realise that I am going about it the wrong way.
So with all of that said I guess my main questions are:
Is this possible?
Is the way I'm going about it correct?
2a. If the way I am going about it is wrong, how best should I go about it?
Normally on the webpage the map, that I am trying to pull, updates dynamically every second or so. In order to make the map dynamic in my application will I have to poll the site every time I want the map to update? or will it automatically update?
I am trying to call a page in my customers webapplication (Exact Synergy Enterprise)
This is the link: http://someserveridontdisclose/Synergy/docs/CSCANEduCourseCard.aspx?ProjectNr=ACPGINTV
Within this page is an Ajax TabContainer with several TabPanels. One of them is called 'Doelgroepen'
I dont have the source for this application, as i am not the developer of it. We only develop custom extentions to it.
Here's the question: Is it possible to focus on one of the tabs USING ONLY AN URL? If so How?
Thank you very much for your thoughts about this.
try to set with javascript. you'll have to write your own js to get index number you want from url, then set like this
$find('<%=TabContainer1.ClientID%>').set_activeTabIndex(2);
http://forums.asp.net/t/1127834.aspx
http://www.aspforums.net/Threads/420684/ASPNet-AJAX-TabContainer-Set-Active-Tab-Client-side-using-JavaScript/
If you do not have access to the code and if this is not part of the requirement / design specification for the application you are using (ie: what you asked the developer to do), then the answer is No.
The control does not have "native" support for URL tab selection. There needs to be specific code in the application in order to handle this.
It is however very easy to implement, if you absolutely need it, it shouldn't take much time (about 15-30 lines of code, depending on how many tabs/urls combination you need).
You can find a running sample of the AjaxControlToolkit Tabs control at the following link (the available functionnalities are described in there):
http://www.asp.net/ajaxLibrary/AjaxControlToolkitSampleSite/Tabs/Tabs.aspx
If the TAB has an ID you could make it visible by adding '#tabid' to the URL.
I just started working with html, and I am creating balloon description templates for Google Earth. I am creating site logs for gps locations around the world, and these sites get updated every now and then with a new receiver or antenna.
I have a table set up with all the information about the equipment we have at these sites, and I want to be able to change values in the table when the equipment is changed.
Right now the only way to do that is to go into the html code and find the specific value you need to change and then save the new html code. This works, but it would be nice if I could edit the values without having to go into the code. I know you can do contenteditable=true and then have some text you can edit, but when you refresh the page it goes back to the orignal text.
My question is if there is a way to edit the html code permanently without actually going through all the lines of code to find the one thing you want to change. If this cannot be done, or if my question makes no sense let me know. I probably just need to work with this stuff for more than a week.
I am trying to make a push notification about Internet page update, but downloading the full page (700k) again every minute is quite troublesome for users. Is there is a way to download only a specific part of page?
As far as I have read there isn't any way to get delta information about the page. Is there a method which allows that? I haven't found one for a day (if there is, how can I locate the certain byte where my information is placed in the first place?
What you want to do is learn some basics in AJAX calls.
You set an auto timer to reload div contents on whatever needs to be updated.
You could use a Range header like this:
Range: bytes=0-1000
This will obviously get you the first 1000 bytes.
I'm creating a video embed page for a real estate site, where a user can go to watch a video tour of a given home. There is no other reason to visit that particular page, so I figured that I could use a simple MySQL Update to a "video view tally" column for that homes's row, which will update views=views+1 each time the page is loaded.
That's easy enough, but I want to give as realistic a "view" count as possible, so I'm trying to come up with a way to have that view tallied ONLY once the page has been loaded for a set number of seconds (say, 30).
Any thoughts on a good way to handle the timing aspect? I'd like to avoid javascript, if possible, but I'm open to if it it's handled simply enough.
Unfortunately the only way you will ever know if the page is still active is to have a client-side technology (like JavaScript) tell you that it is.
You can add a "counter" page that isn't meant to be viewed directly, but instead is accessed via JavaScript after a 30 second page timer has expired. The act of JavaScript accessing that page will trigger the counter logic.