I'm new to stackoverflow (Hello World!). I have some basic understanding of JS, C++, HTML, and CSS and I have been looking in this and other forums but I am having problems figuring out this one, mostly because I don't know what this would be called (TLDR at the bottom):
Essentially, I would like build a chrome extension that extracts data from a website (in this case, copart - a website where people sell cars) and create a link from it that opens another window to one of three car evaluators (edmunds, kbb, nada). I fix cars as a hobby but it's a pain to have to input vehicle info over and over so I wanted to automatize the process as much as possible. Hopefully this will help others as well.
E.g. a generic link to edmunds is: https://www.edmunds.com/ford/escape/2018/appraisal-value/?vin=XXXXXXXXXXXXXX. I would like to know how to extract the make, model, year and VIN, in this case, from copart (Example copart page). On Kbb, e.g., all I see that can automatized is inputting the vin into the window and clicking "Go". Is there a way to have the plugin automatically select "VIN" and copy the VIN into the field while clicking the "Go" button?
Kbb
I know, a lot of questions. I'm also not quite sure what this would be called? A crawler? A scraper? A craper? :)
Either way, here the basic (TLDR) question:
How to create a chrome plugin that extracts data from one website, opens a URL using that data, and which then performs an action like switching a label, populating a textbox, and clicking a button on that URL?
I have only posed this question here so if there's a better place to put it, please let me know.
Mark
Extracting data from one website and searching more for scraped data in other website
1. For this project you can use combination of selenium and scrapy
2. Since both are dynamic page powered by javascript do need to check on security constraints
3. Can make use of spider under scrapy each spider with support of selenium
4. there is need of pressing Go button that can be achieved using selenium
Related
I'm creating a VCL Application with Delpi 10.3 and want to support some web functionality by having the user enter the ISBN of a book into a TEdit component and from there passing/sending this value to a search field on this website: https://isbnsearch.org after which the website looks up the ISBN and displays the Author of the book. I want to somehow access the information (i.e Author) presented by the search result and again use it in my application.
This is my GUI, for a better idea of what I want to accomplish:
What code can I use for this? Any other feasible suggestions or approaches are acceptable.
When performing a search on that website, it simply loads a page with a specific URL query string...
https://isbnsearch.org/search?s=suess
The above example is when I search for "suess", so you can easily concatenate a search URL.
You can use any HTTP component, such as TIdHTTP, to load this search page, then use an HTML parser to scrape the page and read what you need. Much, much easier than trying to read through the TWebBrowser.
In the end, you won't actually display the HTML (I mean you can if you want to), but the idea is to read the data and display it in your own format.
On that specific page, start by locating the ul element with id searchresults. Then, each li element contains individual results. Unfortunately, this website uses pagination, and only shows 10 results per page. To do this, call this page again with another parameter &p=2 for the 2nd page, &p=3 for the 3rd page, and so on.
On the other hand, that is the worst way to acquire such information. What you should be doing is using a proper API which gives you machine-friendly data. The service you are referencing doesn't appear to have an option, but here's an example of one which does:
https://openlibrary.org/dev/docs/api/books - this also appears to provide you MUCH more information than the one you're using.
I am writing a program for managing an inventory. It serves up html based on records from a postresql database, or writes to the database using html forms.
Different functions (adding records, searching, etc.) are accessible using <a></a> tags or form submits, which in turn call functions using http.HandleFunc(), functions then generate queries, parse results and render these to html templates.
The search function renders query results to an html table. To keep the search results page ideally usable and uncluttered I intent to provide only the most relevant information there. However, since there are many more details stored in the database, I need a way to access that information too. In order to do that I wanted to have each table row clickable, displaying the details of the selected record in a status area at the bottom or side of the page for instance.
I could try to follow the pattern that works for running the other functions, that is use <a></a> tags and http.HandleFunc() to render new content but this isn't exactly what I want for a couple of reasons.
First: There should be no need to navigate away from the search result page to view the additional details; there are not so many details that a single record's full data should not be able to be rendered on the same page as the search results.
Second: I want the whole row clickable, not merely the text within a table cell, which is what the <a></a> tags get me.
Using the id returned from the database in an attribute, as in <div id="search-result-row-id-{{.ID}}"></div> I am able to work with individual records but I have yet to find a way to then capture a click in Go.
Before I run off and write this in javascript, does anyone know of a way to do this strictly in Go? I am not particularly adverse to using the tried-and-true js methods but I am curious to see if it could be done without it.
does anyone know of a way to do this strictly in Go?
As others have indicated in the comments, no, Go cannot capture the event in the browser.
For that you will need to use some JavaScript to send to the server (where Go runs) the web request for more information.
You could also push all the required information to the browser when you first serve the page and hide/show it based on CSS/JavaScript event but again, that's just regular web development and nothing to do with Go.
I've Googled around for about an hour, without luck.
I'm trying to build an advanced contact formula i a Wordpress environment. The contact formula should do this:
At first, there should only be a single dropdown-menu available, like this:
Option1
Option2
Option3
If 'Option1' is selected, then some information should be shown, and then another dropdown-menu is shown, as such:
Option4
Option5
Option6
If 'Option2' is selected, then it should again show some information, but then show three other options, like this:
Option7
Option8
Option9
And so on. Many survey-formulars are built in this way - but I need to make it, to make a support-site for some different products.
I tried Googling for Wordpress-plugins, but without any luck (I must admit, I'm unsure of, what I should search for). I've used Wufoo-formula's before, but it doesn't appear to me, that Wufoo has that option.
How is this made the easiest and the best? Hand-coding it using HTML, javascript and CSS? Or are there a cool online-tool that I'm unaware of?
I had the same problem and made some research, which ended up in some results.
1. Buy the required functions
If you have a great number of forms to create and the money to invest, you could use something like Ninja Forms and buy the Conditional Logic plugin. With this you can create multiple forms and connect their elements with the needed criteria to show or hide them or to change their values.
https://ninjaforms.com/extensions/conditional-logic/
2. Code the form
Actually we have only one form we use. This is the reason, because I decided to save my money and did it myself.I expanded our wordpress theme by another page template and created the form that fits our needs in it. After that I added the PHP code to send the form via email on the top and the JavaScript code for the conditional logic on the bottom of my php file. In Wordpress I created a new page and applied the newly created page template.
I'm not sure if this is best practice, but for me it was an easy way to achieve my goal and save some time.
Best regards
I work at a gym at the front desk. The website we use takes awhile to navigate using the GUI. This console in the Google Chrome browser seems pretty powerful. Can someone please direct me to some sort of tutorial or even answer this question yourself?
How would I use the Google Chrome console (inspect element > console) to perform searches using the website's search ability?
Thanks for your help!
You will need some basic understanding of javascript and DOM to do this.
If your site has a simple form for searching, you could use something like this.
document.getElementById('IdOfSearchField').value = 'test';
document.getElementById('IdOfSearchForm').submit();
What is does is that it fiends the searchfield and sets its value to "test" and then it submits the search form.
On this site (sorry for it being in Swedish, it was the one I was currently working on) you could use the following to search for 'test'.
document.getElementById('query').value = 'test'; document.getElementById('SpeedSearchForm').submit();
If your site has jQuery loaded you could simplify this a bit.
$('#query').val('test'); $('#SpeedSearchForm').submit();
Another way of doing it would be to navigate directly to the searchresult-page with the proper querystring (if that is supported by your website. In my sample case, it would look like this (because the search page is located at /search and just need the querystring query to work).
window.location = '/search?query=test';
But as ajp15243 noted in the comments on the question, it all depends on how your site is built. It's also a bit messy to type all that for every search.
I have a website in which I have many categories, many sub-categories within each one and many products within each of those. Since the URLs are very user-unfriendly (they contain a GUID!!!), I would like to use a method which I think is called URL Masking. For example instead of going to catalogue.aspx?ItemID=12343435323434243534, they would go to notpads.htm. This would display the same as going to catalogue.aspx?ItemID=12343435323434243534 would display, somehow.
I know I could do this by creating a file for each category / sub-category (individual products cannot be accessed individually as it is a wholesale site - customers cannot purchase directly from the site). This would be a lot of work as the server would have to update each relevant file whenever a category / sub-category / product visibility changes, or a description changes, a name changes... you get the idea...
I have tried using server-side includes but that doesn't like it when a .aspx file is specified in an html file.
I have also tried using an iframe set to 100% width / height and absolutely positioned left 0 and top 0. This works quite well, but I know there are reasons you should not use this method such as some search engines not coping with it well. I also notice that the title of the "parent" page (notepads.htm) is not the title set in the iframe (logically this is correct - but another issue I need to solve if I go ahead and use this method).
Can anyone suggest another way I could do this, or tell me whether I am going along the right lines by using iframes? Thanks.
Regards,
Richard
PS If this is the wrong name for what I am trying to do then please let me know what it actually is so I can rename / retag it.
Look into URL Rewrites. You can create a regular expression and map it to your true url. For example
http://mysite.com?product=banana
could map to
http://mysite.com?guid=lakjdsflkajkfj3lj3l4923892&asfd=9234983920894893
I believe you mean URL Rewriting.
IIS 7+ has a rewrite module built in that you can use for this kind of thing.
URL Rewriters solve the problem you are describing - When someone requests page A, display page B - in a general way.
But yours is not a general requirement. You seem to have a finite uuid-to-shortname mapping requirement. This is the kind of thing you could or should set up in your app, yourself, rather than inserting a new piece of machinery into your system.
Within a default .aspx page, You'd simply do a lookup on the shortname from the url in a persistent table stored somewhere, and then call Server.Transfer() to the uuid-named page associated to that shortname.
It should be easy to prototype this.