Related
How do you deal with a client who has different time estimates for the software product than yours?
I am going to describe a scenario that is not mine, but that captures broadly the same problem. I am working as a subcontractor to a large company that has a programming department. The software project we are working on is in an area that the department believe they have a handle on, but because their expertise and mine are very different we tend to get different results.
Example: At the start of the project I suggested one way of development which they rubbished as being unrealistically difficult and suggested integrating a different framework (one they are familiar with) with the programming language we are using (Python) to get more or less the same result.
Their estimate for this integration: less than a week (they haven't done the integration before).
My estimate for the integration: above two weeks.
Using my suggested way to get the result needed (including using matplotlib among other libraries used elsewhere within the project): 45 minutes. This is not an estimate, the bit was actually finished in 45 minutes.
Example: for the software to be integrated with their internal system, they needed to provide a web service for me to use. They provided a broken one, though it does work with their internal tool (doesn't work with .Net or Java mainstream packages among other options). They maintain that it is my fault that the integration has taken longer than the time estimated.
The problem is not that they don't know, the problem is that they have enough knowledge about programming to be dangerous (in my opinion). Is there some guidelines for how to deal with this type of situation? A way for expectation management? Or may be I shouldn't get involved in such projects from the start and in this case what are the telltale signs?
If a client isn't happy with a time estimate, don't do the work. If they think they can do it better or faster, tell them to go ahead.
The one thing I never allow is for my estimates to be modified. That's something that caught me out early on in my career but we learn our lessons.
If clients were so good at doing the work, they wouldn't be hiring me. I'd simply point out that they hired me for my expertise so why are they disregarding that expertise. Of course, if they were to allow the scope of the project to change (i.e., less work), that would be another matter, and one up for discussion.
If you didn't lock in exactly what they were meant to provide as part of the deal, then it's a "he says, she says" situation and, unfortunately, the customer controls the purse strings. However, often, the greatest power you can have is the ability to just walk away.
No-one says you have to do the job.
Of course, all that advice above is worth every cent you paid for it :-)
I don't know your specific circumstances.
Or may be I shouldn't get involved in such projects from the start and in this case what are the telltale signs?
My answer for sure. If you can avoid those projects, do it.
Some signs : people thinking they know how to do things when you can guess they can't. The "oh no let's not use this perfectly suitable tool because I don't know it" is a major indicator that the person is technically challenged.
first of all, it is no fun to be in such an environment.
So, if you like to have fun at your job, and you do not need to take this job for extenuating financial reasons, then simply do not take the job that is not fun.
Since that is hardly realistic in many cases, you will end up with the job and need to manage the situation as best you can. One way is to make sure there is a paper trail documenting your objections and concerns with the plan. Try not to be overtly negative, but try to be constructive and present valid alternatives. Here you will need to feel out the political landscape, determine if the 'boss' will be appreciative or threatened by your commentary, and act accordingly.
Many times there are other issues that management is dealing with that you are not aware of. Be cautious of this fact, and maybe ask the management team if this is the case, again without being condescending or negative.
Finally, if you have alternatives that take less time than the meetings it would take to discuss them, just try it in a sandbox, and show it off. This would go a long way to 'proving' your points. Caution here is that you could be accused of not being a team player, or of wasting resources, or not following direction. Make sure this is mitigated by doing these types of things on your own time, or after careful consideration of how long you are spending on these things as well as how vested your boss seems to be on the alternatives.
hth
I ran into the same problem with integration. Example: for the
software to be integrated with their internal system, they needed to
provide a web service for me to use...They maintain that it is my
fault that the integration has taken longer than the time estimated.
Wow very similar to what I was experiencing with a client. The best thing I can suggest is to keep good documentation. In the end that is what saved me. When it came to finger pointing I had all of the emails and facts in order and was prepared to defend my self.
One thing I would suggest is to separate out a target/goal and an estimation. I would not change my estimate unless it involved actually removing features or something is revealed that would make it easier. Tell them you will try to hit the target in anyway you can and you care about the business goal. However, your estimate will not change. If its getting no where and they are just dense then smile and nod and take it if its the only gig around.
Was just writing about this in my blog
How to estimate the WRONG way
I am a programmer. I have about 5 years experience of programming in different kind of languages. I was concerning about my code speed, about optimizing the memory that uses my code, and about good coding style and so on. But have never thought how secure my code is. So I have disassembled my code to see what can do a hacker. Would it be easy to crack my code?
And I saw that it is! It is very easy, because I was storing
serial number as a string
encryption-decryption codes as well
So if someone has the minimal knowledge of assembler he/she can just simple dissembler and after 10-20 minutes of debugging my code is cracked!!! Even it could be done by opening the exe with notepad I guess! :-)
So what I am asking are the following:
Where I should store that kind of secure information’s?
What are the common strategies of delivering a secure code?
First thing you must realize is that you'll never prevent a determined reverser from cracking any protection schemes because anything that the code can do, the reverser will eventually find out how to replicate it. The only way you can achieve any sort of reliable protection is to have the shipped program be nothing more than a dumb client and have the brunt of the software on some server the reverser has no access to.
With that out of the way, you can certainly make it harder for a would be reverser to break your protections. Obfuscation is the sort of first step in achieving this. I have no experience using obfuscators but I'm sure you can find some suggestions for some on SO. Also if you're using a lower level language like C/C++, simply compiling the code with full optimization and stripping all debugging symbols gets you a decent amount of obfuscation.
I read this article a few years ago, but I still think it's techniques hold up today. It's one of the developers of a video game called Spyro talking about the set of techniques they used to prevent piracy. They claim it wasn't until 3 months after the release that a cracked version became available, which is fairly impressive.
If you are concerned about piracy, then there are many avenues you can take. Making the code security tighter (obfuscation, license codes, binding the software to a particular PC, hardware/dongle protection, etc) is one, but it's worth bearing in mind that every piece of software can be cracked if someone sufficiently talented can be bothered.
Another approach is to consider the pricing model for your software. If you charge $1000 a copy, then there is a big incentive for someone to have a go at cracking it. If you only charge $5 then why should anyone bother to crack it?
So what is needed is a balance. Even the most basic protection will stop ordinary people making casual copies. Beyond that, simple techniques (obfuscation and license codes) and a sensible pricing strategy will hold most would-be crackers at bay by making it not worth the bother of cracking. After that, you start getting into ever more sophisticated techniques (dongles/CDs needing to be present to run the software, only being able to run the software after logging on to an online licensing system) that take a lot of effort/cost to implement and significantly increase the risk of annoying genuine customers (remember how annoyed everyone got when they bought half life but it wouldn't let them play the game?) - unless you have a popular mainstream product (i.e. a huge revenue stream to protect), there probably isn't much point going to that much effort.
Make it web app.
It will generally not be well-protected unless there's an external service doing the checking that you are in control of - and that service can still be spoofed by those who really wants to "crack" it. Instead, trust the customer and provide only minimal copyright protection. I'm sure there was an article or podcast about this by Joel Spolsky somewhere... here's another related SO question.
I have no idea if it will help but Windows provides (since 2000) a mechanism to retrieve and store encrypted information and you can also salt this storage on a per-application basis if needed: Data Protection API (DPAPI)
This is on a machine or a user level but storing serials and perhaps some keys using it might be better than having them hidden in the application?
What sort of secure are you talking about?
Secure from the perspective that you are guarding your users data well? If so, study some real cryptography and utilize Existing libraries to encrypt your data. The win32 API is pretty good for this.
But if you're talking about stopping a cracker from stealing your application? There are many methods, but just give up. They slow crackers down, they don't stop them.
Look at How to hide a string in binary code? question
First you have to define what your code should be secure against, being secure as such is meaningless.
You seem to be worried about reverse engineering and users generating license codes without paying, though you don't say so. To make this harder you can obfuscate your code and key information in various ways. There area also techniques to make the use of debuggers harder, to prevent the reverse engineer from stepping through the code and seeing the information in clear.
But this only makes reverse engineering somewhat harder, not impossible
Another common security threat is execution of unwanted code, for example via buffer overflows.
A simple technique for doing this is to xor over all your code and xor back when you need it... but this needs an innate knowledge of assembly... I'm not sure, but you could try this:
void (*encryptionFunctn)(void);
void hideEncryptnFunctn(void)
{
volatile char * i;
while(*i!=0xC0) // 0xC0 is the opcode for ret
{
*i++^=0x45; // or any other code
}
}
To prevent against hackers viewing your code, you should use an obfuscator. An obfuscator will use various techniques which make it extremely difficult to make sense of the obfuscated code. Some techniques used are string encryption, symbol renaming, control flow obfuscation, etc. Check out Crypto Obfuscator which additionally also has external method call hiding, Anti-Reflector, Anti-Debugging, etc
The goal is to erect as many obstacles as possible in the path of a would-be hacker.
EDIT:
Wow, the initial response to this question was quite negative. I think I might have triggered some pretty strong emotions by using the word "best"; it seems like a few people latched onto that word and decided to dismiss my question right away.
Obviously, there are many, many situations in which no single approach is "best", or at least, what ends up being the best solution to one problem will often not be the best solution for other, even similar, problems. I get that. But now let me try to elaborate on the reasoning behind what I'm actually asking.
I tend to find it easiest to explain myself using analogies, so here goes. In my current job I work almost exclusively in .NET. .NET has a lot of functionality built into the framework. A prime example is the System.Collections.Generic namespace, which has a bunch of collection classes that (almost) no .NET developer in his/her right mind would bother re-developing from scratch, because very good implementations are already there. If I am working on a problem that requires a doubly linked list, I'm not going to decide, "Okay, time to write a doubly linked list class"; I'm just going to use the LinkedList<T> that's already there, or, at most, extend it or wrap it with my own class that adds some extra functionality.
Am I saying the "best" version of a doubly linked list is LinkedList<T> from .NET? Of course not. That would be absurd. But I highly doubt .NET's implementation of LinkedList<T> is drastically different from most other established libraries' implementations of collections that are intended to serve the same purpose (that of a doubly linked list). On the other hand, I am relatively confident that if I were to write my own implementation from scratch, there'd be a considerable number of issues with it, in terms of robustness, performance, flexibility, etc. for one simple reason: not that I'm stupid, or lazy, or don't care about good code--simply that I'm one person, and I'm not an expert on linked lists, and I haven't thought of everything that needs to be taken into consideration when designing one.
But I happen to be a developer who does take an interest in how things are implemented internally. And so it would be nice if I could check out a page where some variant of a well thought-out design for a linked list--or for any fairly established concept for which robust, efficient implementations have been written--were available to view. (By the way, yes I am aware that the source code for .NET's LinkedList<T> is available. I'm just using that as an example; really I am talking about all problems with solutions for which good, working implementations exist.)
Now, I talked about this being something that is open; let me elaborate on that. I am not talking about sites like SourceForge.net, or CodePlex, or Google Code. These are all sites for hosting projects, i.e., applications or libraries tailored for some specific industry or field or otherwise categorizable purpose. What I'm talking about is something like this:
http://en.wikibooks.org/wiki/Category:Algorithms_and_data_structures
Maybe I should have just provided that link in the first place, as it probably illustrates what I'm getting at better than anything I've written so far. But I think the main point that differentiates what I'm asking about from any other site I've seen is that I was specifically wondering if there could be some way to work on a new problem--so, something for which there aren't necessarily any well-known, established implementations, again as in my linked list example--collaboratively, in a wiki-esque fashion, but not tied to any specific open-source project.
So, as a conclusion of sorts, I was kind of envisioning a situation like the following: I find myself faced with a new problem. Maybe it isn't common enough to be something that is addressed in a framework like .NET. But it's common enough that some developers here and there are independently working on it. If a website exists like what I'm imagining, maybe at some point one of those developers working on the problem could post an idea on that website, and over time others might discover it and suggest improvements/modifications, and given enough time and participation, a pretty darn good implementation might result from all this collaboration. And from there, eventually, something like this implementation might be considered fairly "standard", just like a linked list implementation, or a quicksort implementation, or, I don't know, some well-known pseudo-random number generator.
Does this make any more sense to anyone now? I feel quite confident that what I'm talking about is not absurd, but hey, if that's what people think, then maybe it is.
Open source projects are very popular. Some of these are libraries suited for specific purposes, the best of which include some very well-written code.
However, if you're interested in contributing to an open source project, finding a project that is well-suited to your skills can be quite a task. At the same time, if you're interesting in using an open source project in your own work, finding a project that is well-suited to your needs can also be difficult, especially when, for example, open-source library X has a lot of functionality you could use, as does library Y, and these two libraries' capabilities overlap so that integrating both into your code could be messy.
We've all seen questions, here on Stack Overflow and elsewhere on the web, posted by one developer: "How would I implement this idea?" and answered by others, often accompanied by a plethora of example code. Sometimes these answers link to an open source project/library that provides functionality similar to what the poster is asking about.
My question is: are there any well-known websites or other sources that are open in nature and provide "best-known implementations" for common (or even not-so-common) programming problems, but not associated with any particular open source project?
As a generic example, suppose I have a need for some algorithm that does X. I post a question on SO or some other site requesting ideas, asking for suggestions on how best to implement it. One person points me to project P1, which contains some code that performs something very similar to this algorithm. Another person points me to project P2. Someone else writes some sample code and says, "maybe you could do it like this."
It seems to me, if there are all these different versions of this idea floating around out in the world, it would make sense for there to be a site, somewhat in the vein of Wikipedia, where a quasi-"official" implementation ("official" is not the right word; I'm just having trouble thinking of a better one right now) could be published and modified as improvements are developed/discovered.
I feel like I have stumbled across a few different sites like this in the past, but I'm interested to know if anyone else has found any resources like what I'm describing.
The very idea is absurd. It means that there's one, single opinion on "best-known implementations" with no changes based on other people having better ideas.
It implies a that best practices are static and can be accumulated into a single repository.
If they could be collected, then Google would have them and would simply charge for access.
Interestingly, they don't have all the best practices. Interestingly, they have to expend mountains of computing power looking for more information. Then people (like you) have to read and think and judge and decide.
The read-think-judge-decide is really hard to eliminate. Unless, of course, you want someone to think for you. In which case, there are many companies who have a single solution that requires less thinking. Call Microsoft or Oracle or IBM. They have solutions that are all in one place, unified best practices, no reading, no thinking, no judging, no deciding required.
Open -- by definition -- means it's impossible to have a single authoritative source.
Here is something, maybe not the best implementations. But a book called Design Patterns contains what is considered by many programmers some of the best patterns to follow!
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
I wonder what would be a good definition of term "over-engineering" as applied to software development. The expression seems to be used a lot during software design discussions often in conjunction with "excessive future-proofing" and it would be nice to nail down a more precise definition.
Contrary to most answers, I do not believe that "presently unneeded functionality" is over-engineering; or it is the least problematic form.
Like you said, the worst kind of over-engineering is usually committed in the name of future-proofing and extensibility - and achieves the exact opposite:
Empty layers of abstraction that are at best unnecessary and at worst restrict you to a narrow, inefficient use of the underlying API.
Code littered with designated "extension points" such as protected methods or components acquired via abstract factories - which all turn out to be not quite what you actually need when you do have to extend the functionality.
Making everything configurable to "avoid hard-coding", with the effect that there's more (complex, failure-prone) application logic in configuration files than in source code.
Over-genericizing: instead of implementing the (technically uninteresting) functional spec, the developer builds a (technically interesting) "business rule engine" that "executes" the specs themselves as supplied by business users. The net result is an interpreter for a proprietary (scripting or domain-specific) language that is usually horribly designed, has no tool support and is so hard to use that no business user could ever work with it.
The truth is that the design that is most easily adapted to new and changing requirements (and is thus the most future-proof and extensible) is the design that is as simple as possible.
Contrary to popular belief, over-engineering is really a phenomena that appears when engineers get "hubris" and think they understand the user.
I made a simple diagram to illustrate this:
In the cases where we've considered things over engineered, it's always been describing software that has been designed to be so generic that it loses sight of the main task that it was initially designed to perform, and has therefore become not only hard to use, but fundimentally unintelligent.
To me, over-engineering is including anything that you don't need and that you don't know you're going to need. If you catch yourself saying that a feature might be nice if the requirements change in a certain way, then you might be over-engineering. Basically, over-engineering is violating YAGNI.
The agile answer to this question is: every piece of code that does not contribute to the requested functionality.
There is this discussion at Joel on Software that starts with,
creating extensive class hierarchies for an imagined future problem that does not yet exist, is a kind of over-engineering, and is therefore, bad.
And, gets into a discussion with examples.
If you spend so much time thinking about the possible ramifications of a problem that you end up interfering with the solving of the problem itself, you may be over-engineering.
There's a fine balance between "best engineering practices" and "real world applicability". At some point you have to decide that even though a particular solution may not be as "pure" from an engineering standpoint as it could be, it will do the job.
For example:
If you are designing a user management system for one-time use at a high school reunion, you probably don't need to add support for incredibly long names, or funky character sets. Setting a reasonable maximum length and doing some basic sanitizing should be sufficient. On the other hand, if you're creating a system that will be deployed for hundreds of similar events, you might want to spend some more time on the problem.
It's all about the appropriate level of effort for the task at hand.
I'm afraid that a precise definition is probably not possible as it's highly dependent on the context. For example, it's much easier to over-engineer a web site that displays glittering ponies than it is a nuclear power plant control system. Redundancies, excessive error checking, highly instrumented logging facilities are all over-engineering for a glittering ponies app, but not for a nuclear power plant control system. I think the best you can do is have a feeling about when you are applying too much overhead to your features for the purpose of the application.
Note that I would distinguish between gold-plating and over-engineering. In my mind, gold-plating is creating features that weren't asked for and will never be used. Over-engineering is more about how much "safety" you build into the application either by coding checks around the code or using excessive design for a simple task.
This relates to my controversial programming opinion of "The simplest approach is always the best approach".
Quoting from here: "...Implement things when you actually need them, never when you just foresee that you need them."
To me it is anything that would add any more fat to the code. Meat would be any code that will do the job according to the spec and fat would be any code that would bloat the code in a way that it just adds more complexity. The programmer might have been expecting a future expansion of the functionality; but still it is fat.!
My rough definition would be 'Providing functionality that isnt needed to meet the requirements spec'
I think they are the same as Gold plating and being hit by the Golden hammer :)
Basically, you can sit down and spent too much time trying to make a perfect design, without ever writing some code to check out how it works. Any agile method will tell you not to do all your design up-front, and to just create chunks of design, implement it, reiterate over it, re-design, go again, etc...
Over-engeneering means architecting and designing the applcation with more components than it really should have according to the requirements list.
There is a big difference between over-engeneering and creating an extensible applcaiton, that can be upgraded as reqirements change. If I can think of an example i'll edit the post.
Over-engineering is simply creating a product with greater functionality, quality, generality, extensibility, documentation, or any other aspect than is required.
Of course, you may have requirements outside a specific project -- for example, if you forsee doing future similar applications, then you might have additional requirements for extendability, dependent on cost, that you add on to the project specific requirements.
When your design actually makes things more complex instead of simplifying things, you’re overengineering.
More on this at:
http://www.codesimplicity.com/post/what-is-overengineering/
Disclaimer #1: I am a big-picture BA. I know no code. I read this site all the time. This is my first post.
Funny I was just told by my boss that I over-engineered a new software produce we're planning for mentoring (target market HR people). So I came here to look up the term.
They want to get something in place to sell now, re-purposing existing tools. I can't help but sit back and think, fewer signups, lower retention, if it doesn't allow some of the flexibility we talked about. And mainly, have a highly visual UI that a monkey could use.
He said we could plan future phases to improve the product, especially the UI. We have current customers waiting on "future improvements" that we still aren't doing. They need it though, truly need it.
I am in the process of resigning so I didn't push back.
But my definition would be.............making sure it only does as little as possible, for as cheap as possible, and still be passable for the thing you say it is. Beyond that is over engineering.
Disclaimer #2: This site helped me land my next job implementing a more configurable software.
I think the best answers to your question can be found in this other qestion
The beauty of Agile programming is that it's hard to over engineer if you do it right.
I'm working on a product which is meant to be simple to use and simple to set up, the competition largely requiring a long set up period and in some cases going as far as a bespoke solution for each customer. One part of our application is now expanding based on customer requests and it is looking like we'll need to make it very flexible so each customer can have a lot of control over how it behaves for them. The problem being that I don't want to make the system too configurable, as I believe this then makes it more complex to learn and to work with. I'm also concerned it opens the door to someone messing things up for themselves, kind of like handing them a gun, although I'm not actually pointing it at their foot for them.
Has anyone else faced a similar dilemma of putting power in users hands? How did you solve it? and what was the result?
I don't normally like to subscribe to the idea that all users are stupid, but there is a rule which can still be applied:
If you give them the opportunity, they WILL break it
Now it is up to you whether or not to give them the ability to do potentially dumb things. Or better yet, develop it so that when they do do the stupid voodoo that they do, it can be reverted or recovered from error state gracefully.
I highly recommend you read Joel's Controlling Your Environment Makes You Happy, which can be described as a treatise on user interface design but is really about usability with a healthy dab of psychology thrown in.
The section I'm referring to is Choices:
Every time you provide an option,
you're asking the user to make a
decision.
This is something I strongly agree with. Many developers, product managers and so on take the easy route and instead of figuring out what users actually need, they just give them a choice. You see this in enterprise bloatware like Clearcase or PVCS where there are so many options--90% of which you'd never change--indicating the designers have tried to make it all things to all men rather than doing one or two things exceptionally well.
Instead it just does lots of things badly.
Keep it simple, follow conventions, don't overwhelm the user with pointless and unnecessary choices and make the software behave like a normal user would expect. That alone would set you apart from an awful lot of other products.
Personally I like the TurboTax model (http://turbotax.intuit.com/). When creating a tax return, I get a simple, tell-me-like-I'm-five wizard that takes me step-by-step through the process, but I can step outside the process at any time and use more advanced features, returning to the process later.
Make it easy and simple and uncluttered for your user to do what they're going to do 80% of the time, but give them the power to deliberately step outside of the norm.
Interesting timing for your question. In the U.S. this is Income Tax week. Filling out the ol' 1040 and associated subforms should give us some sympathy for what users endure.
Lessons I take away are:
Only ask questions that relate to the user domain; avoid questions relating to the software system; and if you can derive the answer or suggest a most likely answer, do so.
Put related questions together (as long as they are normally entered by the same person using data most likely available at the same place and time, which is the definition of related for these purposes).
Make it support incremental input. It should be easy to enter the data they have, and defer completing it when the rest is available.
Show status validity and completeness. Make it clear and obvious how far they are to having validatable data.
Make it interruptable. Make sure it's possible to interrupt the process, leave the application, come back, and resume where they left off.
Yup, it's harder to program. Embrace it.
There are at least two ways to build a good software product:
Focus on a narrow set of functionality, and implement that functionality very well.
Design your system to be customizable (ideally, through scripting.) If you do the base system right, it will be easy to provide the basic, no options, just-do-what-I-want functionality on top of the customization layer.
Unfortunately, there are many more ways to create a bad software product.
Your questions implies that you can either provide a flexible solution OR make it foolproof.
I wouldn't put it like that. To me this is rather a matter of user expectations and the question in the first place would be:
How can I meet all important user expectations (even if they conflict with each other) without corrupting the application?
For instance a web application which has a menu, breadcrumb navigation, a site map and a search offers together with the inline links five different ways to find what you're looking for and how to go there.
That way most users can find fast and easily the functionality they are expecting and therefore the need for an extensive documentation might actually decrease.
So the answer might be to offer several different carefully chosen ways to solve one specific task, while each of them can be streamlined independent to avoid user mistakes.
The answer with this lies in who your end-users are. I used to write software that got used by professional sports coaches. While these guys were definitely good at what they did, they were hardly proficient at computer use, so our configurability was kept to a minimum (at least as far as what could be done in the GUI).
On the other hand, if you're dealing with power users, adding options is usually not a bad thing as long as they aren't intrusive.
It's all about who's going to be getting them.
Read Jeff Atwood's Training Your Users. It's a great article with some very useful links.
I like the approach of Firefox towards this. The basic options are accessible in the option menu, all the rest is under about:config. Thus you have an easy interface and an incredible flexibility if you need it.
I've had great success, and been happiest as an user, when using sensible defaults. In other words, make the most common use case easy (or even better, free), but give users the ability to step outside of that use case when the situation calls for it.