Related
I am really finding it tough to figure out the insights about how does a social networking site (Facebook being a reference) manage their comments and notifications for its users.
How would they actually store the comments data? also how would a notification be stored and sent to all the users that. An example scenario would be that a friend comments on my status and everyone that has liked my status including me gets a notification for that. Also each user has their own read/unread functionality implemented, So I guess there is a notification reference that is stored for each user. But then there would be a lot of redundancy of notification information. If we use a separate table/collection to store these with reference of actual notificatin, then that would create realtime scalability issues. So how would you decide which way to trade-off. My brain crashes when I think about all this. Too much stuff to figure with not a ot of help available over the web.
Now how would each notification be sent to the all the users who are supposed to receive that.. and how would the data structure look like.
I read a lot of implementations those suggest to use MySql. My understanding was that the kind of data (size) that is, it would be better to use a NoSql for scalability purpose.
So how does MySql work well for such use cases, and why is a NoSql like Mongo not suggested anywhere for such implementation, when these are meant to be heavily scalable.
Well, I know a lot of questions in one. But I am not looking for a complete answer here, insights on particular things would also be a great help for me to build my own application.
The question is extremely broad, but I'll try to answer it to the best of my ability.
How would they actually store the comments data? also how would a notification be stored and sent to all the users that.
I generally don't like answering questions like this because it appears as if you did very little research before coming to SO. It also seems like you're confused with application and database roles. I'll at least start you off with some material/ideas and let you decide on your own.
There is no "silver bullet" for a backend design, especially when it comes to databases. SQL databases are generally very good at most database functionality, and rightfully so; it's a technology that is very mature and has stood the test of time for a reason. Most NOSQL solutions are specialized for particular purposes. For instance: if you were logging a lot of information, you might want to look at Cassandra. If you were dealing with a lot of relational data, you would want to use something like Neo4j (or PostgreSQL/MySQL for RMDBS). If you were dealing with a lot of real-time data, you might want to look at Redis.
It's dumb to ask NOSQL vs SQL for a few reasons:
NOSQL is a bad term in general. And it doesn't mean "No SQL". It means "Not Only SQL". Unfortunately the term has encapsulated even the most polar opposite of databases.
Only you know your application's full functionality. Even if I knew the basics of what you wanted to achieve, I still couldn't give you a definitive answer. Nor can anyone else. It's highly subjective, and again, only YOU know EXACTLY what your application should do.
The biggest reason: It's 2014. Why one database? Ten years ago "DatabaseX vs DatabaseY" would have been a practical question. Now, you can configure many application frameworks to reliably use multiple databases in a matter of minutes. Moral of the story: Use each database for its specialized purpose. More on polyglot persistence here.
As far as Facebook goes: a five minute Google search reveals what backend technologies they've used in the past, and it's not that difficult to research some of their current backend solutions. You're not Facebook. You don't need to prepare for a billion users right now. Start with simple, proven technologies. This will let you naturally scale your application. When those technologies start to become a bottleneck, then be worried about scalability.
I hope this helped you with starting your coding journey, but please use Stack Overflow as a last resort if you're having trouble with code. Not an immediate go-to.
How do you deal with a client who has different time estimates for the software product than yours?
I am going to describe a scenario that is not mine, but that captures broadly the same problem. I am working as a subcontractor to a large company that has a programming department. The software project we are working on is in an area that the department believe they have a handle on, but because their expertise and mine are very different we tend to get different results.
Example: At the start of the project I suggested one way of development which they rubbished as being unrealistically difficult and suggested integrating a different framework (one they are familiar with) with the programming language we are using (Python) to get more or less the same result.
Their estimate for this integration: less than a week (they haven't done the integration before).
My estimate for the integration: above two weeks.
Using my suggested way to get the result needed (including using matplotlib among other libraries used elsewhere within the project): 45 minutes. This is not an estimate, the bit was actually finished in 45 minutes.
Example: for the software to be integrated with their internal system, they needed to provide a web service for me to use. They provided a broken one, though it does work with their internal tool (doesn't work with .Net or Java mainstream packages among other options). They maintain that it is my fault that the integration has taken longer than the time estimated.
The problem is not that they don't know, the problem is that they have enough knowledge about programming to be dangerous (in my opinion). Is there some guidelines for how to deal with this type of situation? A way for expectation management? Or may be I shouldn't get involved in such projects from the start and in this case what are the telltale signs?
If a client isn't happy with a time estimate, don't do the work. If they think they can do it better or faster, tell them to go ahead.
The one thing I never allow is for my estimates to be modified. That's something that caught me out early on in my career but we learn our lessons.
If clients were so good at doing the work, they wouldn't be hiring me. I'd simply point out that they hired me for my expertise so why are they disregarding that expertise. Of course, if they were to allow the scope of the project to change (i.e., less work), that would be another matter, and one up for discussion.
If you didn't lock in exactly what they were meant to provide as part of the deal, then it's a "he says, she says" situation and, unfortunately, the customer controls the purse strings. However, often, the greatest power you can have is the ability to just walk away.
No-one says you have to do the job.
Of course, all that advice above is worth every cent you paid for it :-)
I don't know your specific circumstances.
Or may be I shouldn't get involved in such projects from the start and in this case what are the telltale signs?
My answer for sure. If you can avoid those projects, do it.
Some signs : people thinking they know how to do things when you can guess they can't. The "oh no let's not use this perfectly suitable tool because I don't know it" is a major indicator that the person is technically challenged.
first of all, it is no fun to be in such an environment.
So, if you like to have fun at your job, and you do not need to take this job for extenuating financial reasons, then simply do not take the job that is not fun.
Since that is hardly realistic in many cases, you will end up with the job and need to manage the situation as best you can. One way is to make sure there is a paper trail documenting your objections and concerns with the plan. Try not to be overtly negative, but try to be constructive and present valid alternatives. Here you will need to feel out the political landscape, determine if the 'boss' will be appreciative or threatened by your commentary, and act accordingly.
Many times there are other issues that management is dealing with that you are not aware of. Be cautious of this fact, and maybe ask the management team if this is the case, again without being condescending or negative.
Finally, if you have alternatives that take less time than the meetings it would take to discuss them, just try it in a sandbox, and show it off. This would go a long way to 'proving' your points. Caution here is that you could be accused of not being a team player, or of wasting resources, or not following direction. Make sure this is mitigated by doing these types of things on your own time, or after careful consideration of how long you are spending on these things as well as how vested your boss seems to be on the alternatives.
hth
I ran into the same problem with integration. Example: for the
software to be integrated with their internal system, they needed to
provide a web service for me to use...They maintain that it is my
fault that the integration has taken longer than the time estimated.
Wow very similar to what I was experiencing with a client. The best thing I can suggest is to keep good documentation. In the end that is what saved me. When it came to finger pointing I had all of the emails and facts in order and was prepared to defend my self.
One thing I would suggest is to separate out a target/goal and an estimation. I would not change my estimate unless it involved actually removing features or something is revealed that would make it easier. Tell them you will try to hit the target in anyway you can and you care about the business goal. However, your estimate will not change. If its getting no where and they are just dense then smile and nod and take it if its the only gig around.
Was just writing about this in my blog
How to estimate the WRONG way
A little background: I have the opportunity to present the idea of a public API to the management of a large car sharing company in my country. Currently, the only options to book a car are a very slow web interface and a hard to reach call center. So I'm excited of the possiblity of writing my own search interface, integrating this functionality into other products and applications etc.
The problem: Due to the special nature of this company, I'll first have to get my proposal trough a comission, which is entirely made up of non-technical and rather conservative people. How do I explain the concept of an API to such an audience?
Don't explain technical details like an API. State the business problem and your solution to the business problem - and how it would impact their bottom line.
For years, sales people have based pitches on two things: Features and Benefit. Each feature should have an associated benefit (to somebody, and preferably everybody). In this case, you're apparently planning to break what's basically a monolithic application into (at least) two pieces: a front end and a back end. The obvious benefits are that 1) each works independently, so development of each is easier. 2) different people can develop the different pieces, 3) it's easier to increase capacity by simply buying more hardware.
Though you haven't said it explicitly, I'd guess one intent is to publicly document the API. This allows outside developers to take over (at least some) development of the front-end code (often for free, no less) while you retain control over the parts that are crucial to your business process. You can more easily [allow others to] add new front-end code to address new market segments while retaining security/certainty that the underlying business process won't be disturbed in the process.
HardCode's answer is correct in that you should really should concentrate on the business issues and benefits.
However, if you really feel you need to explain something you could use the medical receptionist analogue.
A medical practice has it's own patient database and appointment scheduling system used by it's admin and medical staff. This might be pretty complex internally.
However when you want to book an appointment as a patient you talk to the receptionist with a simple set of commands - 'I want an appointment', 'I want to see doctor X', 'I feel sick' and they interface to their systems based on your medical history, the symptoms presented and resource availability to give you an appointment - '4:30pm tomorrow' - in simple language.
So, roughly speaking using the receptionist is analogous to an exterior program using an API. It allows you to interact with a complex system to get the information you need without having to deal with the internal complexities.
They'll be able to understand the benefit of having a mobile phone app that can interact with the booking system, and an API is a necessary component of that. The second benefit of the API being public is that you won't necessarily have to write that app, someone else will be able to (whether or not they actually do is another question, of course).
You should explain which use cases will be improved by your project proposal. An what benefits they can expect, like customer satisfaction.
I'm working in a small development group. We are building and improving our product.
Half a year ago we couldn't think about higher characteristics, such as usability, because we had so many problems with our product. Many bugs, high technical debt, low performance and other problems kept us from being able to focus on usability.
With time we've improved our process substantially. What we've done:
Real Agile iterations
Continuous integration
Testing(unit-tests, functional Smoke tests, performance)
Code quality is 'good'
Painless deployment process
So we are now producing stable, reliable releases. The following quote (paraphrased) describes our current situation:
first - make it work; after that, make it reliable; after that, make it usable
We are geeks, so we can't 'make' a great UI by ourselves.
So what should we do? What direction can you recommend?
Maybe we should hire Usability experts part-time or full-time?
How can we explain the importance of Usability to our stakeholders?
How do we convince them that this is useful?
What do your Business people say will make you the most money? Do that. Maybe usability is the next thing to do, maybe more features, maybe a different product. It's not something a "geek" will necessarily be able to guess.
I'm in the same boat as you are - I basically live on the command line, and I'm completely out of touch with the modern UI (both web and desktop application).
The solution for me was using a real UI developer for all my GUIs, and I just live in the back-end as it were.
There are quite a few benefits of this arrangement:
You don't have to debug your own crappy UIs anymore :) that's their job, and they're better at it than you, so no worries.
Your code will naturally gravitate to a MVC or at least tiered API approach, which is easier to code against for all parties involved.
Good UI people know what questions to ask end users, and know when those users don't know what they're talking about. I certainly don't have that skill.
You can do what you do best, and they do what they do best, making a stronger team overall.
The cons are obvious - you need to not only find the money for a talented UI dev, but you need to find a talented UI dev!
Now, I can't speak for you and your company's position in your market etc etc (I also don't do buisnessspeak :) ) but if you can afford another hire, it will give back more to the team than the cost of the position. It did for me!
You ask, "How can we explain the importance of Usability to our stakeholders?" but I'm not sure that you yourselves get it!
Interaction design (iD) and usability aren't things that you can tack on to an existing products when the "important" things are done. They should be there from the very first start, preferably done in small iterations with small tests and studies. I'm talking about cheap and dirty iD/usability, stuff like lo-fi prototyping, user testing with just four people, having enough stats to be able to detect user errors and such.
If you don't to iD/usability from the start, you risk ending up with the same crappy product as your competitors and/or providing users with band aids when they need surgery.
What do your users want ? They're probably the people best placed to identify requirements.
You are the ones who know and understand the product, so don't assume that just because someone else has 'usability expert' in their title that hiring them will somehow make your product usable.
Also, don't undercut your own instincts for usability. As a programmer, you use software all the time, what products do find the most usable? Think about what you like about them and compare them to your product.
Think about what your product does, and imagine that you are the person having to use the product and imagine how you would want it to work. Think of what a user wants to accomplish using your product, and imagine the steps they would have to go through to do it. Does it seem easy to understand what to do? Can it be done in fewer steps?
Most importantly, talk to your customers. Find out what they found confusing or difficult to accomplish. See if they have come up with their own workarounds for using your product in ways you didn't initially picture.
If you put as much thought, planning and effort into usability as you did into improving the reliability and deployment, you will end up with a much better product.
When analyzing the next step it really all comes down to business requirements & goals.
What is upper management like? Are they tech-savy? Are they open to new ideas? Do they think that the current product needs adjustment, improvement, etc? Is the product still in high demand? Is the marketplace changing such that the product/service will soon be obsolete? etc. etc. etc.
IF there are real business reasons for spending the $/time/resources then you can begin to explore product improvements. At that point consider the opinions of previous posters regarding user opinion.
I know so many geeks including myself who know usability, so one way would be learning it. Another way bringing someone in who can do UI design and usability.
To convince them that usability is important:
It's useless if you can't use it!
I don't know what sort of product you build but you always got clients, and clients always love usable applications. This will increase sales, happy client count and decrease tech support.
What does it do for your users? What do they think about the usability? Maybe it's not an ssue for them.
Make it more valueable to your users. Deliver more business value. Help your customers getter a better return on their investment. Do this by making it do more of what they need it to do, to do it better (more accurately, more quickly, more reliably more useably), or to do it at lower cost (less infrastructure needed to run it, reduced maintenance costs because you improved reliability), more flexibly (deals with their business changes)...
Lots of dimensions which do connect with the technical ones you refer to (usability reliabilty stability etc). But paying customers normally care about their business needs/features, not your technical ones that deliver them.
Go talk to your users (or potential users)
My one-liner about the importance of usability:
What use is a reliable system that is not usable?
If you have an existing product with existing users, then what makes you think that your current UI is not usable?
Do you suspect that there are some minor changes you can make that will greatly improve usability or is something more revolutionary required? If the latter, then what about the needs of your existing users, will they be willing to re-learn a whole new UI?
Edit
A user interface can be considered "poor" for a variety of reasons...
It is just plain ugly / old fashioned / does not "look like a Windows application"
It uses metaphors or workflows which do not relate to things that the user understands or wants to do
The first of these is relatively easy to fix, especially if you hire in a great designer. The fix would be the equivalent of redecorating your lounge and buying a new sofa and TV. Same room, different experience. Your existing users would still be able to use this application.
Fixing the second of these gets a lot more complex and involved, and might really impact your codebase. It's hard to comment further without knowing more about your application.
I think the answer is in the order of things, you say its:
"first - make it work; after that, make it reliable; after that, make it usable"
But the most important thing here is "make it work". Acceptance criteria for a functionality to "work" is that it is in fact - usable. If not, it will not be executed. Then it's just a block of dead code. And dead code should not be in the system in the first place.
Make a UI.
Then throw that away, and make another after you tried to use the first one. Then simplify as much as you can. Any time you can programmatically determine what the user wants from inputs, instead of multiple explicit choices, do so. Too many buttons induces paralysis.
Personally, I've found that when good developers deal with clients, they often get sucked into the after-sales support process and this process has been difficult to reverse, so was just interested to hear the various strategies that developers employ in maintaining a healthy, useful relationship that keeps clients using the right person at the right time.
So do you and, if so, how do you deal with clients?
Just a tip: Write down every single thing a client says to you.
Most of the projects I work on are done on time-and-materials contracts, which means: we give the customer an initial estimate of how long the project will take but bill for actual hours worked, whether over or under the estimate (I don't know why a client would agree to this, but they do). Once the project is "complete" and in production, we set up a service extension to the time-and-materials contract, creating a block of billable hours to cover after-sales support. When a client is aware that they're being billed for all contact with us, they tend to keep that contact to a minimum.
One other point: I've found that it's best to communicate with clients via email where possible. It's a much more efficient way to transfer information (assuming everyone involved can write), and it leaves a permanent record of what the client told you to do.
I'd go the opposite of what have been said.
The client is your number one information source
Avoid intermediaries (human and technical)
Keep tracks (not to use it against the customers, even if it can happen, but because he pays to get what he wants)
Communicate - on your initiative - in a short regular basis but for small amount of times.
Any doubt can be cleared asking the good questions. The guy don't want that ? Get rid of it (even if you like it better). The guy want that ? Why not, add time and money on the contract.
You must train your communication skills
Most of what has been said here before is essentially related to the fact that programmers usually have poor communications skills. So they fall into the typical traps :
customers give them bad info
they waste time
they get stressed
At the end, nobody is happy.
But with trained communication skills you will learn to direct when, how long and about what your chats will be, and so :
Make any deal quick and nice
Give confidence to the client
Understands what the client wants (not what he says he wants)
Ensure is satisfied with the answer (even if it's nonsens for you)
Everybody will be happier : the customer will feel good and let you work in peace while you will have the information to keep working. Eventually, the resulting software will be better.
Think talking to customer is boring ? They think it too. And paperwork is boring as well, but you must do it, so do it well instead of looking for excuses.
This is a pain we feel as well. Once you help out a customer it is too easy for the customer to directly contact the developer later on and request support. And since we usually aim to please, and probably feel sort of responsible when the application we built for them has a problem, we too often give the customer a quick helping hand.
I think that the developers should be separated from the customers, but this requires that the company has a support/concultancy department which can fix the problem instead. They in turn should be free to contact the developer, unless it's a huge company with a mainstream application where there is a less risk that the problem can be traced back to a problem with the sourcecode.
But let me tell you, I understand how difficult this is. I've been working in our consultancy shop for many years, starting from support and now I'm mostly managing the other consultants and developing. There are a lot of customers (like hundreds) who feel they have a personal relationship with me, and assume that they can call me directly even after years and years.
My tip is to make sure you have a good network of concultants and supportworkers who can help the customer for you, and have them contact you instead if they can't figure it out.
I just finished my education and am working at my first job, but here is what we do:
I communicate through a third party from the same company with "higher rank". The third party is someone knowledgeable of the requirements the software should have, but not in software engineering. When I ask about specifications, or send them proposals he distills the essence of their answers send them to me.
I think this way of working with stuff limits the amount of bullying a customer can get away with when it comes to changing specs, expanding specs etc.
For me it's especially useful since I'm only 21 years old, and people might have trouble believing I can get things done.
best practices:
Remember the client is the one who signs the checks.
Users work for the client.
Refer any user requests to the client for approval.
Always deal with the client because they understand that everything you do will cost them money.
If the client wants after the sale support and is willing to pay for it then give it to him cheerfully.
Oh and what MusiGenesis said!
The best way is to never ever ever give your direct line to a customer. Have them go through Tech support (if it exists) first. We employ this method and it works well. The software developers are the last resort - for things that support simply can't do/don't know how to fix -- such as a DBA not knowing that the servers are instanced. But it will cut down on the "it's not connecting to the internets" type of phone calls.
You could also force all support requests to go through email/secretary. At that point, you can discern which ones are critical, and which ones can be solved with a simple 'tutorial' on how to fix the problem.
And as stated above - record EVERYTHING in an exchange with a customer. Doing so prevents the 'well he said she said' deal that customers can fall into.
Then again -- if you're getting a ton of customer support issues, you should be looking at the cause of it - whether it's a training issue, or whether the software is legitimately buggy.
In our company, every developer is also a salesman. If I step over the door of a Customer then I'm in a good position to make more business.
They know me and I have credabillity because I've allready delivered to them.
I have knowledge about their business
I use my knowledge to ask questionas about other parts in their business
I plant hooks to them when I talk to them, in their best interrests of course.
I make clear that we are not a "hit and run"-company, but there to really support their business.
Maybe this is not how all company does, but I think you should use the people you have that allready has a foot inside the customers company to really work with them and make more business and tie the customer tighter to you.
I personally think developers should never interact with clients. This is why you have the Q/A team. They get requirements, hand them to developers, discuss any issues, schedule development progress meetings. If developers have questions, the go to the Q/A personnel responsible for the requirements and documentation. Developers are engineers, not salesmen or negotiators. They should be given environment to develop stable, working code without getting distracted by customer phone calls. This is how many companies deal with customers regardless of company size. In the end, your chances of completing a project on time are higher than when you customer calls up and decides to change requirements or requests a feature. Which would probably mean you have to go back a couple of iterations and change something that may break everything completed past that point.
Lots and lots of communication. Communication can be as simple as checking in with your customers by stopping by at their desks (if you are co-located) or keeping in touch over the phone. The more personal the communication is (in-person beats phone call, phone call beats email, etc.), the stronger your relationship will be.
Another good conflict resolution practice I've used is keeping as much information as possible in a single, shared place. I've used a bug/feature database (JIRA), a wiki, and even a network share drive for this purpose, but the point is that neither party has exclusive lock/write access. Updates can be made together with your customers, and there is a clear, public record of the change history of your system.