I'm doing a project for my job and am trying to get attorneys contact information off of the GA bar website. I've searched for programs to do it, but am not finding anything for exactly what I need.
I need to get all of the attorneys information into excel and I figured there was some batch text program to do the job. There are over 50,000 contacts listed and I really dont think I need to click on every link to view the information to be able to access it.
I have opened up the code on the page with "inspect element" on chrome and saw that each attorney has a unique ID on the site. I feel like their info can easily be copied from the site, but I have very limited programming knowledge.
Any suggestions?
The first step is to check the terms of service / copyright of the GA bar website and make sure you are allowed to do that.
Before you start coding, you may consider requesting the contact info in database format. The information may be available for free, or for a fee that is less than the cost of you programming a solution.
If you must program, a very capable library for scraping the contents of a website is HTML Agility Pack. I suggest you become familiar with it, and post more detailed questions here as you get into specifics.
HTML Agility Pack does require programming. If you're looking for a non-programming solution, Stack Overflow is not the right place to ask.
Related
Context
I am volunteering to develop a website for an astronomer.
The website is made using Wix.com with some additional HTML/CSS/Javascript edits when needed.
Problem
The astronomer wants to embed his research publications into the website. These publications are visible here: Link
Unfortunately, I do not know how to perform this embedding.
What I have tried
I have tried to find Wix Apps or other third-party apps that can convert the database into HTML embeddings. Unfortunately, no such objects exist.
I have tried scouring the database's website to find any possibilities of embedding into an external site. Unfortunately, this does not exist.
Please let me know if there is any other information that you need, I would love to get this up and running soon :)
I have taught myself HTML/CSS and some JavaScript as a hobby, and have reached the point where I am comfortable building a clean simple website. The company I work for (we do nothing related to coding) has a website that is quite outdated so naturally I saw this as an opportunity for my first live site. I approached my Managers at work to take a look at my first mock up and they loved it and want me to revamp our current site.
The one concern they have with me being a rookie is the issue of web security. Essentially they want to make sure that the website I build leaves no vulnerability for someone to hack through our server, start editing our website, etc.
There are no interactive components to the current website. I plan to build this website with only HTML and CSS, and perhaps add some JavaScript later down the road once I progress in my learning a bit more. There are no account logins or areas to enter personal information anywhere on the site.
My question comes down to this -- what can I do to ensure that the website I build is not leaving our company vulnerable? I have done a lot of searching around Stack Overflow and other websites but I am not confident I am finding the correct information.
Details that might make a difference:
Our company website is hosted through godaddy.com
Our website is currently on Wordpress, but I will probably not use Wordpress for the new website
I greatly appreciate all of your help!
since there should be no direct interacting with the server ex PHP and AJAX only HTML CSS and js there should be no security issue, as js and HTML can't edit/delete/read server files, only server-side programming can. eventually you may want to invest in PHP almost no website is complete without PHP or AJAX. take me for example, I used to use just HTML, CSS, and js. then I wanted to do more.. log form answers to a file, show different pages based on the query string. these things are virtually impossible in a HTML, CSS, and js only environment. I would also recommend atleast getting free protection from cloud flare.. they give free shared hosting wildcard SSL, and free DDOS protection, granted for a business you might want to invest a little more than free but free would be a good starting point
also you could pay someone to test the vulnerability of your company, take for example OurMine. a legal hacking group.. you pay them to test your security and they do just that. (they claim they don't log anything that they get) otherwise there might be vulnerabilities you may not be aware of
When you are concerning about just AJAX call, I would help you out for following suggestion regarding "Function access rule from AJAX".
By adding "_" as a prefix for Function name, we can prevent function to be called from The Web publicly. This is the best practice when we need some specific function to be accessed via AJAX only.
Kindly, refer my answer given in other question.
[Website Security: How to learn?
I'm a relative beginner using Google Apps Script and JavaScript, but I've been playing around with bot for days now and I've created a few simple programs and I'd really like to try and get started on my dream project, even if it takes me forever. I'd like some advice on what I should use in terms of making the UI and what database I should use to hold the information (and if this app is even possible).
The App
I'd like to create an online novel management app that utilizes Google Drive as it's source for files. The UI would have a tree that showcases all the google drive files in the novel. When a scene is clicked, the scene opens up for editting.
Questions
Is this app a possibility?
If so, in terms of a UI, what do you think I should use? The google
provided UIbuilder? The HTML service - for example, can I have a
frame on the right that the google doc that needs to be editted
can open up in on the right?
Lastly, what database should I use? The database would have to store
chapter names and positions, as well as scene names, positions,
and the google doc ID that the scene corresponds to. I've got a
handle on ScriptDB and Spreadsheets... And if either of these two
aren't the best option, would some other database work better? And
why?
This app will, hopefully, be able to give an overview of a novel in tree form, allow you to open a particular scene and edit it, create new scenes, and also change the order in which the scenes are displayed. And then when the person finishes their novel, the app will compile all the scenes into one novel (also in google Drive).
Any insight or suggestions would be greatly appreciated!
Having a look at the questions you recently posted I think I have a pretty good idea of what you are trying to do and it looks like an exciting project... I can only encourage you to start it as soon as you can even if you're not comfortable with all the tools you will need to use, the best learning method is probably to work on something important to you.
Now your 3 questions : 1 - This is perfectly doable in the GAS environment and shouldn't be too hard to go through.
2 - the GUI builder is an easy way to start with UI but it lacks a number of features and tools that you will be needing (tree for example ) and is not so easy to expand if you ever need to. Depending on your knowledge in html, the choice is mainly between UiApp and html service... I would choose UiApp because I'm not good at all in html (but that's not relevant here ;-) but both are capable of building what you want, are easily expandable and not too hard to debug. The advantage could go to html service if you are going to look for 'nice looking features' because it opens the door to 3rd party tools... but again, this is a matter of personal choice.
3 - A recent post from Mogsdad showed that spreadsheet are faster than scriptDb for data storage and manipulation. I find it also easier since I can have a global view on data in the spreadsheet when debugging. Of course Spreadsheet must be considered as a container and data manipulated at array level to benefit from maximum performance. I use that in a lot of database application with full satisfaction.
Sorry for these "general considerations" that don't comply to sto standards ;-)
Yes, it seems that all of the things you are requesting are not too ridiculous. I recommend sticking to Google services because they are all easily integrated. To start off, you may want to use the UI builder/UI services. There may be a point in this project where you may want some functionality that the UiApp doesn't provide. At that point, you might want to switch over to HtmlService.
My answer is the same for the databases question. You might want to use a spreadsheet for your database so that you will be able to easily edit it by hand if you need to. You may not have the performance that another database would give you, but it will be fairly easy to test and mess around with your spreadsheet "database."
You could start out with getting the basics down. There's a serious amount of data out there. I would suggest you research on an "as-needed" basis. Design some work-/dataflow patterns for your app, for which you could try to use the Fluid UI extension for Chrome. Have a look at this from Mozilla on the designing of apps.
When you've gone through this you might want to have a look at Phonegap and the basics of web development and how you could combine the two.
There's also several ways of using/storing data. You could try WebSQL though it they no longer develop it. You could look at IndexedDB. You could try to use cookies.
Seriously, have a look around. You might also like the books of Wrox. They're very informative and have great work with reading demo's. Though the books are huge ;)
Does anyone know of any good tutorials that would show me how to create a sitemap similar to the image below. I can't figure out how to add the different sections underneath like the Your Account, FAQs, etc.
Any help would be extremely helpful. Thank you.
google search for instagram
What you want is what Google calls Sitelinks.
The process is automated and it's not possible right now to create them but you can manage them with the Google's web master tools. The algorithm used by Google to generate them is not public.
You can try this: http://www.xml-sitemaps.com/, or just google 'xml sitemap generator'
I remember that what happened (in my 'previous' life, when I had to take care of all the gory details of our company site) I just followed google's recommended seo suggestions. It was painstaking and slow, but over time when we started turning up at the top of sear results, that exactly how google presented us. It pulled relevant information on it's own and created that nice display. Looking at my old codebase I don't even see a sitemap file there. But I do remember using one of those online generators and then hand turning it a bit.
I am learning the html course from the available tutorials on the internet. And with that knowledge I have developed some html files and I believe there is more to go. These files consist of our old school friends and their present condition and what they're doing. I have created a bunch of html files. Like I have created a website for now named as www.mypage.com
Arjun
So in the href I'm just giving the path but those files are in my desktop pc itself. How do I put them on internet and share with my friends who are living somewhere out of this town. I want to reunite all of my school friends using these files. But where do I upload them and make it like a webiste to my friends? Is there any free way to do that? Or any possible way to reach it.
And I'm saying sorry if the question is not for the tag I mentioned. Please let me know and I remove the tag. I don't know what is the link to wikiposts to share my views. If you know then let me see the link. Thank you.
Find a web hosting service (Google knows lots)
Sign up
Follow their instructions
Arjun
And use relative uris
I use http://webhosting.uk.com ... for about £32 a year you get asp.net hosting and access to sql server.
then you simply ftp up to your website something like this (using windows explorer) ....
ftp://mysite.com <-- not a real link
... that would then open up the remote server as if it was a local folder so you could drag and drop your files straight over.
there's an online chat link on the top right of the homepage, the support staff will walk you through getting setup.
don't forget though ... the cost of the domain name that's the bit that turns your server ip address in to stuff like google.co.uk ... I highly recommend them.
I shall just expand on Quentin's answer, because it is clear that you are new.
What you need is a web hosting service. This is a service which hosts your html pages, meaning stores them in their own computers, and also displays them to the world as webpages. Web hosting services are usually paid, but there are some excellent free services if your content is not too big. A simple and free service that I would recommend is Google Sites. You could also try Google App Engine, where you have more freedom and control over your content, but for the same reason it is a little more advanced. But since you are learning html, I believe it is a wise idea to learn more about these services and related concepts.
When you upload your files, the html links need to be changed. They can no longer point to files in your hard drive. When you upload these files, there will be a directory structure in it. All you need to do is place hyperlinks with relative addresses.
And about what your website will be called, www.whatever.com, that is quite another business. For that, you need to register your own domain name, for which you must pay. If you don't want to, then your website will be labelled something under the hosting service domain name. This forum is not adequate to go into a more elaborate explanation of all this, but I think I have mentioned all the key terms, so do some research!