Related
I am trying to create a simple C++ brute forcing program to crack small passwords-maybe like 3-4 characters long, (as a science fair project) just to show people how it works, i.e. my topic is mainly about cyber security and data breach vulnerability and stuff.
So I got the brute forcing part covered, but what I want to change is that instead of asking users to enter their password within my program itself, I was wondering if it was possible to make a, say, an HTML page with a login screen, and the entered password is then saved to a txt file which I can then open from my program for the brute forcing part.
What I am trying to say is that most people trying this out wont be so convinced entering passwords into an output box. They might wonder if I had just printed out the password they entered. I saw other people's programs but all they did was cin>>password and then started the brute forcing part. The user does not know if his password was cracked or the programmer just did cout<< password.
So it would help me a lot by letting me know of methods in which I can achieve this(not just with HTML, anything is fine, as long as its not an output box).
Sorry for the long question! Just wanted to make sure I covered all the info.
Please help me!!!
P.S: I'm just a student (1 and a half years into C++ :)), so please try to keep it as simple as you can. Thanks in advance!!
I would simply print out as much internal information in the terminal/outputbox as possible. This way, people get a better understanding of the inner workings of your program. For example, you could print out every password that your program tried. On top of that, show your sourcecode to the people who are also interested in that.
Say I have a collection of websites for accountants, like this:
http://www.johnvanderlyn.com
http://www.rubinassociatespa.com
http://www.taxestaxestaxes.com
http://janus-curran.com
http://ricksarassociates.com
http://www.condoaudits.com
http://www.krco-cpa.com
http://ci.boca-raton.fl.us
What I want to do is crawl each and get the names & emails of the partners. How should I approach this problem, at a high-level?
Assume I know how to actually crawl each site (and all subpages) & parse the HTML elements -- I am using Oga.
What I am struggling with is how to make sense of data that is presented in a wide variety of ways. For instance, the email address for the firm (and or partner) can be found in one of these ways:
On the About Us page, under the name of the partner.
On the About Us page, as a generic catch-all email.
On the Team page, under the name of the partner.
On the Contact Us page, as a generic catch-all email.
On a Partner's page, under the name of the partner.
Or it could be any other way.
One way I was thinking about approaching the email, is just to search for all mailto a tags and filter from there.
The obvious downside for this is that there is no guarantee that the email will be for the partner and not some other employee.
Another issue that is more obvious is detecting the partner(s) names just from the markup. I was initially thinking I could just pull all the header tags and text in them, but I have stumbled across a few sites that have the partner names in span tags.
I know SO is usually for specific programming questions, but I am not sure how to approach this and where to ask this. Is there another StackExchange site that this question is more appropriate for?
Any advice on specific direction you can give me would be great.
I looked at the http://ricksarassociates.com/ website and I cant find any partners at all so in my opinion you better stand to gain from this if not you better look for some other invention.
I have done similar datascraping from time to time, and in norway we have laws - or should I say "laws" - that you are not allowed to email people however you are allowed to email the company - so in a way the same problem from another angle.
I wish I knew maths and algorythms by heart because I am sure there is a fascinating sollution hidden in AI and machine learning, but in my mind the only sollution I can see is building a rule set that over time probably gets quite complex. Maby you could apply some bayesian filtering - it works very well for email.
But - to be a little more productive here. One thing i know is inmportant, you could start by creating the crawler environment and building the dataset. Have the database for URLS so you can add more at any time, and start the crawling on what you have already so that you do your testing querying your own data with a 100% copy. This will save you enormous time instead of live scraping while tweaking.
I did my own search engine some years ago, scraping all NO domains however I needed only the index file that time. Took over a week alone just to scrape it down and I think it was 8GB of data just for that single file, and I had to use several proxyservers aswell to make it work due to problems with to much DNS traffik. Lots of problems that needed being taken care of. I guess I am only saying - if you are crawling a large scale you might aswell start getting the data down if you want to work efficient with the parsing later.
Good luck, and do post if you get a sollution. I do not think it is posible without an algorythm or AI though - people design websites the way they like and they pull templates out of their arse so there are no rules to follow. You will end up with bad data.
Do you have funding for this? If so its simpler. Then you could just crawl each site, and make a profile for each site. You could employ someone cheap to manual go through the parsed data and remove all the errors. This is probably how most people does it, unless someone already have done it and the database is for sale / available from webservice so it can be scraped.
The links you provide are mainly US site, so I guess you are focusing on English names. In that case, instead of parsing from html tags, I would just search the whole webpage for name. (There are free database of first name and last name) This may also work if you are donig this for some other Europe company, but it would be a problem for company from some countries. Take Chinese as an example, while there is a fix set of last name, one may use basically any combination of Chinese character as first name, so this solution won't work for Chinese site.
It is easy to find email from a webpage as there is a fixed format of (username)#(domain name) with no space in between. Again I won't treat it as html tags but just as normal string so that the email can be found no matter it is in mailto tag or in plain text. Then, to determine what email is it:
Only one email in page?
Yes -> catch-all email.
No -> Is name found in that page as well?
No -> catch-all email (can have more than one catch-all email, maybe for different purpose like info + employment)
Yes -> Email should be attached to the name found right before it. It is normal that the name should appear before the email.
Then, it should be safe to assume the name appear first belongs to more important member, e.g. Chairman or partner.
I have done similar scraping for these types of pages, and it varies wildly from site to site. If you are trying to make one crawler to sort of auto find the information, it will be difficult. However, the high level looks something like this.
For each site you check, look for element patterns. Divs will often have labels, ID's, and classes which will easily let you grab information. Perhaps you find that many divs will have a particular class name. Check for this first.
It is often better to grab too much data from a particular page, and boil it down on your side afterwards. You could, perhaps, look for information which comes up on a screen by utilizing type (is link) or regex (is email) to look for formatted text. Names and occupation will be harder to find by this method, but might be related positionally on many pages to other well formatted items.
Names will often be affixed with honorifics (Mrs., Mr., Dr., JD, MD, etc.) You could come up with a bank of those, and check against them for any page you end up on.
Finally, if you really wanted to make this process general purpose, you could do some heuristics to improve your methods based off of expected information; names, for example, are most often within a particular list. If it was worth your time, you could check certain text for whether it matches a list of more common names.
What you mentioned in your initial question seems that you would have a lot of benefit with a general purpose Regular Expressions crawler, and you could make improvements on it as you know more about the sites which you interact with.
There are excellent posts on this topic with a lot of useful links throughout these webpages:
https://www.quora.com/What-is-a-good-web-scraper-for-pulling-emails-names-etc-even-if-the-contact-info-is-another-page-deep-a-browser-add-on-is-a-plus
http://www.hongkiat.com/blog/web-scraping-tools/
http://www.garethjames.net/a-guide-to-web-scraping-tools/
http://www.butleranalytics.com/15-web-scraping-tools/
Some of the examined applications are working in macOS.
First of all, hello! I am new to... everything basically.
Is there a way to use any web coding (maybe HTML5?), to analyze the tones of a song, specifically a piano song, and give out some kind of code every time a specific tone, or a note is hit, which can then be further used for other stuff?
If not, do you know of any other methods of getting something like this done? I want a webpage to be able to analyze a song by the different notes, and then have something happen when the different notes are getting played.
I myself only picked up coding recently but now know html and php. I just thought I'd say that doing what you want is definitely not possible for someone that is new to everything. HTML is a markup language, it just organises everything on the page.
To achieve what you want you would need a lot of coding knowledge. Tone recognition is the kind of thing you see on extremely expensive production software such as cubase. I wish it were simple to do such a task as it would come in very handy for the site I have built. However it is not. If you are really set on this idea google something like tone recognition software although I doubt there will be anything suitable that can be easily implemented into a website.
My advice would be either spend a lot of time learning and mastering many front and backed coding platforms or spend a lot of money hiring someone to do it for you.
If you want to accomplish the same task but in a simpler but still intricate way I would scrap the notion of tone recognition. If you had an onscreen keyboard where each key was a form submit button you could then attach some PHP code to each key with an if(isset([''])) function. When a certain key is pressed something is outputted by the code or in PHP words echoed. This would require extensive knowledge of HTML, CSS, PHP and jquery. Probably some others thrown in their too.
You probably won't be able to execute that yourself but it is a much easier and viable approach if you want to get someone to do it for you.
Since you are new to coding world I would like to recommend you to do these free online courses from Udacity which is an awesome initiative from one of the top teachers of coding world to democratize education.
Udacity is a digital university on a mission to democratize education
Course material is excellent and you can follow subjects on your own pace and many other benefits for free and they will provide you with signed certificated too when you clear final exam of a subject.
I am not connected to Udacity in any way. :) I am myself following some courses even after I have a CS degree.
Now on to your answer your questions :
HTML is a markup language which displays structured data to webpage. It takes data input via forms and pass it on to the backend system and displays the data it receives from the backend system.
There are two major divisions of any website or webapplication :
1. Frontend
2. Backend
Frontend is what you see on the webpage and how you interact with the webpage.
Backend is actually responsible for providing data for frontend to display. Both of them are nothing without each other.
So what you are trying to achieve can be done on backend only but the input will be taken through HTML. Only HTML is not enough to do this. You need a powerful programming language to do this stuff on the backend.
I would like to tell you this project is very advanced and it takes years of programming experience (and CS degree too but its not mandatory if you are determined enough) to create such project. If its a business idea you better higher someone or some company in software and web development field to build this for you.
If you have any question then simple comment and ask me.
Welcome to the world of softwares and SO too. :)
How do I go about explaining that I am a "Web Application Developer" and not a "Web Site Designer" to prospective clients - without talking myself out of the project?!
Often I am approached to "design a web site" for someone where it turns out to be more of a "brochureware" presentation site and less of a real web application.
While I am a highly skilled developer, I am not a graphic artist. That said, I would still like to be able to close deals with prospects without disqualifying myself. Simply stating, "I do backend work, not frontend" will quickly end the conversation, and along with it my opportunity for work.
Sure, I can just subcontract the project to a real designer and mark up his rate, but I would rather be up front with the client that I am not going to be the guy doing the actual work and they would be paying $120/hr for $60/hr work.
...and then they will ask price - is visual design often quoted hourly like app dev is, or will I get sucked into the oblivion that is fixed fees?
First thought: You sound like you're a good developer, which is a great foundation in this situation because people feel that we have a broad understanding of technology -- and they're right! You're correct in your insecurity though - a good developer is not exactly what they're after.
They want a solution, and if you can become the guy who knows how to navigate all the complexities involved in delivering that solution... Well, then you're the guy. And you're much more valuable.
And you don't have to stop coding, either. Which has always been my phobia.
So that's more or less how I present myself to my clients. I'm a developer, but I'm also a competent project manager. I know how to take your project from here to done even though it may involve designers, IT guys, additional developers, datacenters, etc.
Working with other professionals: You probably aren't in a position to hire folks. That's fine, but find good partners, and you can market your self as a shop or consulting group that is good at more than one thing. Believe me, people will appreciate that! For instance, we have a fantastic partner in Romania that we use for a lot of design, and we add value because we know how to get results from them and handle all the communication and so on. My customers don't want to talk to 3 to 6 people, they want a single point of contact who is accountable for all of it.
One thought here: try to find people who do this full time. Moonlighters and people doing side jobs have constantly let me down. I sympathize; I know what it's like to be spread too thin.
Fixed prices: You will probably have to deal with fixed prices, and you will need to hold your partners to fixed prices (or have some other mechanismf or controlling costs) if that's the case. Once the relationship becomes more comfortable, I usually try to give clients a range of what I think a job will take -- 5 to 8 hours, etc. And I let them know that fixed prices are a little bad for them because I have to assume some risk, which means they get charged more.
Most companies still want actual fixed prices for larger projects though and many large companies are shockingly comfortable with being overcharged in this situation. Fixed prices seems to be unfortunate fact of life in the contract development space.
First of all, I think you're exaggerating a bit - you don't really want to have a $30/hour designer working on a $150/hour website. If you do, and your client doesn't notice a difference, they deserve what they get.
And if we're talking about, say, $60 - $75/hour range vs $150/hour (which, IMO, is more realistic), you have to count in the time you spend working with said designer, so it's not like remaining $75-$90 goes directly into your pocked with you doing nothing.
If I am a client, I want a FINISHED PRODUCT and I'm paying YOU to deliver. Consider yourself a general contractor. I DON'T WANT to deal with designer / tester / technical writer / what have you myself - that's what I'm paying you for. My time may be a lot more valuable to me than $90 / hour difference. I may (will) get a second / third / fourth bid if I think you're too expensive.
That said, if you need to placate your conscience you can always bill couple hours less.
I tend to say something along the lines of "My job is to develop the stuff you don't see, but that make your site work"... And that is actually quite true.
... And you might add something that says that building a website is an activity that requires several very different abilities, like :
creating something that looks nice, and is usable ; that's a job on its own ; do your client have designed to logo of their company ?
integrating this nice looking thing into a dynamic application ; that one is your/my job ^^ Your clients send/receive e-mails every day, but do they know how it works ?
administration of the hardware / server : your clients use a computer every day to go in the Internet, use Word/Excel and the like, but do they know how to install those ? How to secure their network ? How to deal with security alerts the right way ?
You can use analogies with other processes ; designing / building / using / repairing a car, for instance ; some people understand better when we just don't speak about computers at all...
The way I always try and explained it, since I'm a Developer not a Designer is.
Developer -> Makes it work.
Designer -> Makes it pretty.
Of course there are vest generalizations, but for the most part they're true.
-> Developers creates the product
-> Designers create the package
Do you bill enough hours that you could consider hiring a program manager/product owner type person at least part time to do most of your customer interaction? Because if you do, you can abstract the "design/code" questions away from the customer; let the program manager really focus on figuring out what the customer wants built.
It sounds like your average customer isn't that technically savvy. If that's the case, figure out a way to speak their language (which most experienced program managers can do). They really don't care about the difference between a designer, a developer, and a tooth fairy that shows up and delivers the site that they want.
It's also worth considering whether you really want to take on projects that are mostly web design. If not, you should work on improving your marketing materials to focus on what you do best (interactive line of business web applications, or whatever).
You can steer the conversation toward what business problem the customer wants to solve, and propose the statement of work that best matches their objective. I think this will do the most good, and then it's never a question of misleading them. It's about packaging up a solution that matches what they want.
"I do backend work, not frontend" will quickly end the conversation, and along with it my opportunity for work.
This is not necessarily a bad thing. It's generally a good idea to be honest with prospective clients, and if they do not understand exactly what your job entails up front, then they may end up expecting too much from you when the job is done, or for future projects.
If you do really want to push your services on them, you could always try to point out exactly what kinds of benefits you will be offering to them. Make a point in explaining how much value can be added to the end-product by using your expertise.
A car may look great on the store floor, but if its powered by a lawn mower engine held down with duct tape and chewing gum, the only thing it's going to be doing is looking pretty and collecting dust in your garage.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 8 years ago.
Improve this question
A (non-technical) friend of mine has asked me how to make a website. I get this question all the time. After a few questions I found out that she has an idea that could turn into a commercial site. I described three options to her:
a) Get a book/enroll in a class/follow some online tutorials and learn how to do it. She's pretty smart and her personality seems like a good match for this sort of thing so I'm sure she could learn but she doesn't have a lot of time spare. Maybe if she started with one of those WYSIWYG editors at first? I stressed that this would take a longer than a couple of weekends of playing around.
b) Hire someone to build it. Ranges from ultra cheap to ultra expensive, crappy to good and everything in between. I didn't mention sites like Rentacoder because she hasn't worked on a project like this before and doesn't know what to ask for. At this stage she'd likely ask for a Youtube-MySpace-Google for a few hundred bucks because she doesn't yet understand just how much is involved.
c) Find someone technical and partner up with them. I explained that this can either work really well or be a disaster because she'd have to give up some of her ownership of the idea.
How do you respond in these situations?
Depending on the nature of the required website (ie whether she needs shopping carts etc) I often recommend first creating a blog, although it's often not the best format, it can be used to quite good effect at times. I've seen a number of small retailers for example, using a blog to advertise their wares.
It depends on the person and their motivation. Your friend sounds like she should take option b) or c). She's probably not so much interested in the technical aspects of making a web site as she is in seeing her idea come to life, or maybe running her own company based on the idea.
Well in general there are several steps:
Determine the subject of the website.
Determine the target audience.
Decide on a general layout and look and feel.
Decide which techniques to use.
Design the overall structure of the site.
Collect content and images.
Implement the site.
Most of the times, these steps are carried out by more than a single person. Because they all require their own specialization.
I think that the only realistic options are B and C. Non-techies will almost never come to grips with real web site development. With all due respect to those who have advocated technologies such as ASP.NET or PHP - it won't work. In fact, you are likely to be fielding questions for weeks on end as to why things don't work. You'll have to bear these questions until the person, having failed, gives up.
If they have the resources, then I would strongly recommend the "hire an expert" route.
In either realistic case, they must get a legal agreement in place. If she does hire an expert, be sure that the agreement expressly stipulates that this is a "work for hire". If your friend doesn't demand a work-for-hire clause then she will have no legal means to stop the developer from using the exact same code she just paid for in creating a competing site (at least in the U.S.). Just to emphasize: they would have no legal means to prevent the developer from starting their own competing site without a work-for-hire clause - the courts won't even hear the suit.
Now, if it is really just a brochure site or something similar, and they still want your advice, then it depends on whether they use a Mac or something else.
If they are a Mac user, I tell them to try booting up iWeb and using it to feed a .me account. Just plug in some pictures and some text, upload to your .me account and you are done.
If they are a Windows user, I direct them to Register.com or a similar online Web Site template-based site builder.
For your sake, don't volunteer to help unless you are really sure you have the time to essentially build the whole thing for them! You can ruin friendships this way: friends and family, having no clue what really goes into the construction of a site, almost invariably assume that it is "trivial" for you. If you delay or fail to get things done quickly enough, they'll almost always assume that you are blowing them off and they'll resent you for it (can you tell I've been there?).
It depends on how confident she is that this thing will succeed or, alternatively, how unwilling she is to share ownership. I'd definitely not recommend (a) for something that has commercial value. Beyond the time-to-market factor, initial impressions are something that is very hard to overcome. If she has the cash and is possessive of ownership, then (b) could work well. It's a bigger financial risk, though.
I'd probably go with (c), but then again I'm on the other side of the equation. There are lots of ways of structuring a partnership that would help her maintain ownership, though probably not complete ownership since no one wants to work for nothing. Some combination of b/c is probably best - some less pay in exchange for a small stake in the business. I'd definitely see a lawyer before doing (c) and maybe even before doing (b) just to make sure any agreement she has with the developer precludes them taking the code and running.
There are really two questions here: should she learn how to write a website and, if so, how.
If she has a commercial idea and she only wants to learn how to write websites to market it, I would suggest that she does not bother. Outsource instead.
If you have someone who to trust then only 3rd option is realistic. first one takes years to give out good result and with second one you never know what you get.
Buy her a book for her birthday on ASP.NET or PHP.
A bit of topic but here it goes...
We have ONE web designer at our company(we are about 10 programmers), we usually make some jokes like: "You do not work, you just sit there making drawings all day long" (just for the record, she is very competent and does her job pretty well).
So one time she stares at my monitor for about 2 minutes while i am working on a web application, and at some point she says: "Sooo that's what you do? Formulas and stuff?".
From that day on we all say "I do formulas and Stuff" when asked for what we do at work, as for most people, saying that and saying you develop web applications is the exact same thing.
Developing a web site is not as trivial as places like GeoCities or Google Pages would make it appear.
Here is a list of things you need to know about or consider when publishing a site.
If she owns a Mac or doesn't mind to acquire one, I would suggest her to have a look at iWeb. Otherwise, starting a blog on a platform such as blogger would be a nice first step for a non-technical person.
I always give the same answer, HTML. I say what it does and then tell them to get a book. If they get what it is about they will follow their own course of action.
Unless you're a web designer, 'how do I make a website' is a similar question to 'how do I make a television'. You don't, you buy one. So this question is probably 'How do I procure a website?'.
This breaks down into:
Working out what you want:
What user features do you want
What do you want the UI to look like?
What admin features do you need?
Do you already have a graphic design/logo/color scheme, etc.
Finding someone to make it for you
How do you find them? Referrals? Local web dev companys? Friends?
Do you care which technology they use?
Are you going to project manage it?
How do you know they're doing a good job?
Are you clear on issues like scaling, support, maintenance?
Ideally, find a single company who is comfortable helping you work through all the issues. Less ideally but also good, find a consultant or company that will help you with the top level stuff and assist with production of technical specs that you can then take to a development company to produce.
(Edited for formatting)