I'm trying to create a mobile application for my school's Gradebook system, as their current website design is quite unattractive. I don't want to scrape the information directly off the HTML in the website because it seems slow and inefficient, so I'm trying to figure out how to retrieve the raw data that the website retrieves before displaying it as HTML content.
The website will call something like https://.../RetrieveReport, and an HTML page will immediately be returned to the client side. I'm wondering if there is any way to peer into the handler on the asp.net side to understand where it is getting the raw data from. Please let me know if this is possible. Thanks in advance :)
Related
I’ve had success doing web scraping with Microsoft Access, using MSXML2.XMLHTTP objects and Regex. I’ve been exploring the web scraping possibilities of Power Automate, and see that it doesn’t have regex, but can execute regex scripts from Excel. The problem: accessing the relevant data in the first place.
Take a look at this link: https://letterboxd.com/tiff_net/list/2022-toronto-international-film-festival/ When you try to extract data from one of the entries, nothing useful is available.
And yet all the information I want is contained behind it:
You can display the source behind a web page by using Edge as your browser and adding “view-source:” to the beginning of the web address you want to go to. But then what? How do you get the HTML source into a variable where you can work on it? With MSXML2.XMLHTTP, you just access the responseText property. Can something like this be done with Power Automate or are you limited to scraping sites with extraction-compatible objects?
I'm researching the possibility of styled reporting on Wordpress, fed by Smartsheet data. My idea is to on click or even on frequency/date, get grid content from SmartSheet API and publish to a Wordpress post. I want the columns/cells/rows to output as html objects (ideally integration code allows me to create classes based on column and/or cell content logic) so I can style with CSS.
As someone with strong technical skills but no deep coding ability, am I being crazy to attempt self-teaching and building this? I feel like logically it seems pretty possible. If it is I would learn and execute myself.
Thoughts on how feasible, potential roadblocks?
Getting data from Smartsheet via its API into a Wordpress page is achievable. The biggest hurdle I can see you running into is since the Smartsheet API doesn't have support for CORS you won't be able to make requests to the Smartsheet API directly from the page on the front end. You'll need to build out some back end code that lives on your server that interacts with the Smartsheet API and your Wordpress site calls that code to get the data it needs.
I am currently making a processing program, where a part of it will be to acess some information from at website. The website will be an HTML file, where some information is stored, which i need to acess and parse. I know how to open a html file, but my problem is that it is supposed to acess a list, which is generated after a login on the website. How do i do that?
This is the website, right after loading the HTML file:
http://i.imgur.com/kGIkyle.png
After a login, the website will begin to spit out data every two seconds.
I wanna acess the data in the ordered list, and i wanna acess it every two seconds in my processing program. How do i do that?
This is the website, after a login, after a moment.
http://i.imgur.com/O743fNJ.png
When you use a web browser to submit a login, you're really interacting with the server. Usually the web browser submits a POST request containing the login information (like a username and password), and the server responds with the next webpage to load.
The details of this are going to depend on the website you're interacting with. Some websites might use AJAX to submit the data and then trigger some JavaScript to run.
The point is, you're going to have to understand exactly how the underlying web server and webpage works. Then you're going to have to use the rules of those interactions to issue the appropriate requests from your Processing code.
It might be as simple as submitting the login credentials in the url itself and then just scraping the information from the webpage.
More likely, you're going to have to interact with some kind of web API and do the requests yourself. Google "Java post request" for more info.
Of course, all of this assumes that the website is open to people using it. If this website isn't yours, it could also be locked down and unavailable to you.
I'm trying to build a single page application that can be viewed offline, and that can be packaged (preferably as an HTML page), and emailed out. It needs to be able to load from the file I email out, but not hit a server at all on load time. It can load JS libraries from their servers, but the page itself will not be hosted anywhere.
This seems like a simple problem, but I'm having trouble exporting my current spa site into a static format that can be emailed.
Currently the site is built in ASP.NET MVC 4 as a WebAppSPA. The application is a bunch of graphs built from data in a JSON file.
Thanks in advance for the help!
Looking to prefill a webpage (a form in a wordpress site) from a lotus notes application. My investigations to date show that using the query strings behind the urlname should work - however I am missing something here.
www.url.com/register?fname="bob"&sname="smith" etc...
I thought this could be done, but am not sure where the starting point would be?
Ideal solution - webservice from the owner of url.com to allow us to populate and get a response once done. They are not keen to do this as it does not save them any time, just our business.
I am looking to understand how to prepopulate the webpage so that the user just has to submit the form at url.com.
You could go mega-sneaky and use the Selenium WebDriver to launch a browser session on your client. It has full control over the browser and can sniff out the fields you need to populate. I wrote some simple code for starters - but the samples on the original web site might be more comprehensive.