Reputation system: weighted points vs unweighted points? - html

I am developing one small reputation system, and I faced one problem.
So, in my example I want to create a website for pictures with 4 different types of user; let's call them: amateur, good, very good, professional.
Each user can upload a picture and this picture can get rated by other users. When the user reaches certain number of points, he passes to the next level
(e.g. if he was amateur, he becames good).
Now my question is: How should I develop this reputation system?
Should I include weighted points or unweighted?
E.g. if professional user gives 5 stars to a amateur user pic, should this bring more points then when a good user gives 5 stars to an amateur user pic?
Also for negative points.
Which path should I choose? How to choose the right solution?
Is there some name for this kind of "problems" so I can read about it?
Could you address me on what should I pay attention with both solutions and what are the pro and cons?
Thank you
PREVIOUSLY READ:
I read about SO pointing system, precisely this:
https://meta.stackexchange.com/questions/57278/if-a-user-has-good-rep-on-a-particular-tag-shouldnt-his-votes-on-that-tag-weig
This answer made me think about. Which way should I choose?

This is a tremendously difficult question to answer, without knowing the specifics of a use case. Is there a reason to believe that those who have reached higher levels are more authoritative in delivering ratings to others? It may be the case. It is notable, however, that you are almost by definition creating a system which will lead to conformity - that may or may not be what you are looking to do.
There are some interesting implementation questions, if you decide to go ahead with such a system. Does the relative ranking of two individuals matter, or just the one who is rating? (i.e. if you are "good" and I am "very good", should my vote on your image be the same as if you are "amateur" and I am "very good"? or should it hold less weight in the first case since we are presumably closer individuals?)
If I rate someone when I am very good, and then later I cross the threshold into professional, does my rating get scaled by the "very good" or does it later get updated when I update?
If I am an individual who currently has no ratings by anyone, and a Professional individual rates me as Professional, should that count for more or less than if I am someone who has been called an Amateur by 100s of Amateurs, and then a single Professional rates me as a Professional? In other words, how does the variance of ratings get reconciled with the variance of weightings?
So, in some type of summary: If there is reason to believe that those who rank more highly will be better judges of how others rank, and you do not mind uniformity of the highly ranked individuals, then perhaps a weighted ranking system may work.
If you are looking to implement a weighted ranking system, then you need to decide exactly what matters. Is it the relative weights? Is it the explicit weights?
My recommendation would be to think about specific scenarios. Enumerating how this should work specifically may help you generalize a solution.
As some readings of interest:
https://www.microsoft.com/en-us/research/project/trueskill-ranking-system/
https://en.chessbase.com/post/the-sonas-rating-formula-better-than-elo-

The reputation of a voter should almost certainly influence the weight of her vote and how much it affects the votee's reputation, as long as you believe that a person's reputation directly correlates with her ability to judge the reputation of others.
Take a look at the Elo rating system and its peers (with R implementation here). Player ratings -- analogous to your reputation calculations -- depend on their opponents' ratings, not just a simple win/loss incrementing.

Related

Modeling an RTS or how Blizzard was able to put together Starcraft 1 & 2?

Not sure if this question is related to software development, but I hope someone can at least point me in the right direction(no, not that direction..)
I am very curious as to how Blizzard achieve such a balance of strategic/tactic forces in their games? If you look at Starcraft 1 or now 2, each race have unique features that sort of counterpart other unique features of other races and all together create a pretty beautiful(to my mind at least) balance.
Is there some sort of area of mathematics that could help model these things? How do they do it basically?
I don't have full answer but here is what I know. Initially, when game is technically ready the balance is not ideal. When they started first public beta there were holes in balance that they patched very fast. They let players (testers) play as is and captured statistics of % of wins per race and tuned the parameters according to it. When the beta was at the end ratio was almost ideal: 33%/33%/33%.
I've no idea how Blizzard specifically did it - it might have just been through a lot of user testing.
However, the entire field of CS and statistics dedicated to these kinds of problems is simulation. Generally the idea is to construct a model of your system and then generate inputs according to some statistical distribution to try to understand the behaviour of that model. In the case of finding game balance, you would want to show that most sequences of game events led to some kind of equilibrium. This would probably involve some kind of stochastic modeling.
Linear algebra, maybe a little calculus, and a lot of testing.
There's no real mystery to this sort of thing, but people often just don't think about it in the rigorous terms you need to get a system that is fairly well-balanced.
Basically the designers know how quickly you can gather resources (both the best case and the average case), and they know how long it takes to build a unit, and they know roughly how powerful a unit is (eg. by reference to approximations such as damage per second). With this, you can ensure a unit's cost in resources makes sense. And from that, it's possible to compare resource gathering with unit cost to model the strength of a force growing over time. Similarly you can measure a force's capacity for damage over time, and compare the two. If there's a big disparity then they can tweak one or more of the variables to reduce it. Obviously there are many permutations of units but the interesting thing is that you only really need to understand the most powerful permutations - if a player picks a poor combination then it's ok if they lose. You don't want every approach to be equally good, as that would imply a boring game where your decisions are meaningless.
This is, of course, a simplification, but it helps get everything in roughly the right place and ensure that there are no units that are useless. Then testing can hammer down the rough edges and find most of the exploits that remain.
If you've tried SCII you've noticed that Blizzard records the relevant data for each game played on B.Net. They did the same in WC3, and presumably in SC1. With enough games stored, it is possible to get accurate results from statistical analysis. For example, the Protoss should win a third of all match-ups with similarly skilled opponents. If this is not the case, Blizzard can analyze the games where the Protoss won vs the games where they lost, and see what units made the difference. Then they can nerf those units (with a bit of in-house testing), and introduce the change at the next patch. Balancing is a difficult problem - a change that fixes problems in top-level games may break the balance in mid-level games.
Proper testing in a closed beta is impossible - you just can't accumulate enough data. This is why Blizzard do open betas. Even so the real test is the release of the game.

Coding Competition, language agnostic guidelines?

I might be doing a coding competition soon, I was wondering if anyone made one and what where the guidelines/ process.
I'd like to make the competition appealing to all devs, and I m trying to come up with ideas as to how.
the scenario is: There is an event running and we(of the coding competition) will have a room that we can use (either to code or for questions, etc), however, ideally the task for the competition should be assignet and they should eb able to go and do other things, if they are so inclined.
what i wonder is what kind of challenges to give, and most importantly, what is the criteria to "win" teaching and learning good coding standards takes a looong time, and I d like to think that if you ve been coding for longer you ll do things right and quick... but in a competition, you would be cutting corners...
I would really appreciate your input on this
A competition that's appealing to all devs? That sounds... difficult. But if you want to make the competition about problem solving and algorithms, then I am a big fan of the Sphere Online Judge. Basically this is a repository of programming puzzles but you can also become a problem setter and create problems or contests on the site.
It supports a huge number of programming languages, from the "popular" ones to more obscure ones. Programs will typically read from standard in, and write to standard out. The standard judge program will simply diff a program's output with the expected output, but more elaborate judges are possible. You also set a time limit for the execution of submissions, which usually requires programmers to be more clever than brute force.
Winner is whoever solved the most problems. Ties are broken by time of correct submissions, with some time penalty for wrong submissions.
Guidelines
Limit the languages which can be submitted. If you don't, you may get proprietary languages which require a special purchased compiler or some other inconvenience.
Correctness
This is easy. Provide an easy-to-read unit test in all languages you will accept. This will allow simple, automatic testing of submissions, and will guide the interface of the solution.
Challenges
Create a theme. Make it focused, but not too specific as to require certain paradigms or language features. Then develop challenges based around that theme.
Assign points to each challenge. Give more points to more difficult problems. Be sure to review each challenge carefully and have a team attempt them before giving points so you can make a more accurate decision.
As #miorel mentioned in his answer, time limits and memory limits are wonderful. Set a time limit per test per challenge, or at least monitor them and have these metrics contribute to the points given for solving the problem.
You should look at the ACM competition. Each year they have collegiate programming competitions. These are language agnostic. The archive is located here.
http://www.ntnu.edu.tw/acm/
To start improve yourself, you need a project you can work on, a problem you want to solve, something you want to achieve. Without any context and a destination you want to end, you won't be able to learn all the necessary methods and all the connections of a language.
There is a competition, taking place 2 times a year, it is called ludum dare.
It also doesn't matter which language you are writing, you just have to create a game within 48h ( compo, just one person and all assets created by yourself ) and 72h ( jam, a team working together, can purchase assets ). After the competition when every uploaded his game, the voting starts. This will take like 20 days where everybody can upvote your game or you can upvote other peoples' games. There are taking part approximately 3000 people.
Every time the competition starts, there is a voting on 5 days in sequence. Everyday you vote on a set of themes which can be possibly the theme you will have to create a game for. My last competition had the theme "unconventional weapon". After the voting ended, the competition starts and you have to think about a game with (in my case ) an unconventional weapon and start coding a game you like.
This is not about being the best, you should start looking at other peoples projects after the competition ended. You can learn a lot of other people, ways they solve their problem and I am sure you would improve your self every time taking place in such a contest.
It's gonna be hard to designed a coding competition suitable for a wide array of languages, since languages typically serve different purposes. I'd suggest that what you're looking for doesn't exist.

Nielson's usablity scale

Just wondering if anyone out there knows of a standard survey (preferably based off Jacob Nielson's work on usability) that web admin's can administer to test groups for usability?
I could just make up my own but I feel there as got to be some solid research out there on the sort of judgments on tasks I should be asking.
For example
Q:: Ask user to find profile page
Do I ...
A.) Present them with standard likert scale after each question
B.) Present them the likert after all the questions
..
Then what should that likert be, I know Nielson's usability judgments scale is based on Learnability, Efficiency of Use, Memorability, Error Rate, Satisfaction but I can only imagine a likert I would design that would effectively measure satisfaction...how am I suppose to ask a user to rank the Memorability of a site after one use on a 1-5 scale? Surely someone has devised a good way to pose the question?
A few recommendations:
Don't determine your standard exclusively by listening to the users and waiting for their feedback. Nielsen says that rule #1 in usability is "Don't listen to users"; it's more important to watch them work.
Here is an FAQ regarding development of Likert questionnaires. I would err on the side of simplicity and brevity if you are going to ask users a list of questions after every task. There are advantages and disadvantages to both of the options you are considering. If you make a user wait until they have finished all of their tasks before they fill out a survey, they may not remember their initial difficulties with the interface as they adjust to its learning curve. On the other hand, if you ask them questions after each task, they may start rushing through the questionnaire as they get toward the end of the list of tasks. An extra option, depending on how many tasks you have, may be to have the user fill out a survey after every several tasks.
The University of Maryland HCI Laboratory maintains a Questionnaire for User Interaction Satisfaction, which is available for download and now on version 7.0. You may be able to use their survey, or at least tailor it for your use.
The short and easy System Usability Scale (SUS) has been found by Tullis and Stetson (2004) to psychometrically outperform other subjective scales including the renowned QUIS. Most SUS items seem related to learnability or memorability, along with a couple for efficiency. However, I wouldn’t try to break it into subscales; all items are highly intercorrelated suggesting this scale measures a single underlying construct.
I would doubt you can get a scale to measure each of Nielsen’s dimensions separately. A user can tell you if a product is “hard” to use, but it’s much more difficult for them to break it down further. They know it took a lot of work to do something, but was it because they couldn’t figure out an easier way (learnability)? Or maybe they had learned a better way on a previous task, but forgot it (memorability)? Or is that just the way it has to be (efficiency)? Users are not going to have sufficient information to make the distinction.
If you are specifically interested in each of Nielsen’s dimensions separately, then assess them separately and directly. You can measure learnability crudely through recording the number of errors or time between clicks, and precisely by how many trials it takes for users to learn the normative interaction sequence. For efficiency, after you train users to do the normative interaction sequence, record how long it takes them to do it. You can also get a pretty good answer analytically using something like GOMS-KLM. For memorability, bring the same users in a week or so later and compare their performance to that of the efficiency-measuring trial.
Like nearly all subjective scales, the SUS is primarily useful for comparing the overall subjective experience of different products. It’s hard to know what to make out of a single score without something to compare it to. These scales won’t tell what specific problems a product has or why it has them (e.g., to help you determine improvements). For that, qualitative observation and debriefing your test participants is best.

Where is the dividing line between complete lack of planning and analysis paralysis?

In my very short time working in the programming field, I've seen two extremes:
Projects where little to no planning was done and thus become maintenence nightmares.
Projects that are perpetually in the planning stages and don't move from there.
It seems like the latter oftentimes happen as a reaction to the former. Where is the happy medium? And more importantly, if a project is moving in one of these directions, what is the best way to move it towards said happy medium?
In my own personal experience, I have found that 'decisions' are my bottle neck.
If this is the case, then :
List all your design options
Pick an option(s) (pick a few if you can't decide on one)
List the risks of the best option(s)
For each risk, brainstorm a solution, then design a conclusive proof of concept and write it.
If your proof of concepts proves it will NOT work, then toss that option, and pick another one.
A 'Proof Of Concept' is a minimal app to prove something. (mine are usually 1-6hrs)
If you have a situation where 2 or more options are equal, give yourself a time limit (like 5 minutes, not 2 months) and make a decision ... any decision, and don't look back.
And trust yourself to be able to deal with any problems you will hit which you did not take into account at design time.
Initial planning should be roughly O(log n) where n is expected total development time.
If you have to push in a week, sketch something on a napkin.
If you have a month, the first day is for initial design.
If you have a year, spend a week.
This does assume that you revisit planning iteratively, and don't just go all commando-style on your codebase, without adult supervision :-).
Analysis paralysis can have many symptoms. One that I have noticed is that same questions are asked each meeting and no resolution is reached. If you can point this out to people that might be able to help them admit that the planning process is stagnating.
If you can, at the start of the project, state that you want to cover a certain percentage of the requirements in the planning stage, say like 80-90%. This way there is a clear goal and you aren't trying for perfection. You can revisit the planning and analysis later just don't let it hold things up.
I think it depends on 2 factors:
The length of the project
Is it a 1 week project?
Is it a 1 year project?
Or somewhere in between?
The risk of the changes being introduced in the project
Are they architectural in nature, potentially affecting a lot of the original code?
Or are you simply adding a new feature?
Obviously, it's a combination of the above 2 factors. It simply doesn't make sense to spend 1 month designing a feature which will take 2 days to implement and is of little risk to the architecture. I'm picturing a matrix here of length/risk/design time tradeoffs.
There was some interesting advice in Code Complete 2, which I am currently in the process of reading. I can't remember the exact wording, so I am paraphrasing here, but it said something along the lines of:
The 2 biggest mistakes you can make in a design are:
1. Attemping to design EVERYTHING (you will fail)
2. Designing NOTHING before implementation
Finding the happy medium between these 2 is the key to successful design and planning.
Great questions - with no absolute answer - this is what makes experience meaningful. Experience including:
how much detail can you actually get from any user/sponsor and from this specific group
how much detail does your current team need (technical/business specific skill levels)
how open are your sponsors/users in helping during the development (how agile of a team do you have - does it include the sponsor/user?)
how good are you are identifying gaps
A big factor is the system being developed - the more 'life' critical the more details you will need (heart monitor compared to a web page).
The more complex, cost/time constrained, life-critical the more up front work - the less the more you can detail out as you do the work (prototype to production)

Is my reputation system secure?

BOUNTY: To get bounty either show me how to play the system, or explain why you think it's impossible to play it.
I'm developing a reputation system for a site that allows you to start your own blog of sorts, and to comment and have favorites etc.
I aim for a very short ruleset that is easy to understand for the users and can't be "played". Note: when I write "answers", I mean answers to comments, which is
the equivalent of Stack Overflow's comments to answers.
To create an account you need one Mojo. Without it you can only answer other people's comments.
When you have an account, you get one Mojo a day.
Each day you can only post as many posts and comments as you have Mojo. You can answer as many comments as you like. You don't lose Mojo when you post.
You can give one Mojo to someone who has less, but only once to each person.
Alternatively, you can burn one Mojo to take one Mojo away from someone who has less, but only once.
You can't have your Mojo back.
The site is supposed to be invite-only, but non-members can answer member's comments, and maybe they'll be able to start their own blogs too, I'm not sure yet. The idea is for a system based mainly on seniority, where power can be transferred between users only in a very limited manner so basically there's no way to inflate your reputation (Mojo only flows down.)
Giving and taking Mojo is supposed to be an act on the level of marking someone as a friend or as an enemy.
I don't want Mojo itself to be a motivator for people to be part of the community. I want them to use it to influence how other people behave on site. By burning someone with your Mojo you effectively limit how much they can post on the site. By giving someone Mojo you allow them to express more.
I'm also planning to add ways to get extra Mojo, like "Post of the Day" or "Favorite Blog of the Week" contests with prizes of around 3-10 Mojo. But one of my main goals is to avoid inflation.
Here are some consequences of your design. These may or may not be what you intended, but I point them out so that you're aware of them.
A very Mojo-ful user who has been mostly quiet is not going to care at all about stepping on some toes, because they have a huge bank of Mojo from which to draw. This seems to go against your goal of limiting negative behavior.
Likewise, users who contribute absolutely nothing to your site still get Mojo just by virtue of having an account. But an otherwise valuable contributor who makes one off-color post that's disliked by the community will be silenced until he has enough Mojo to post again.
If someone has something very valuable to contribute, he has to make sure to have "reserve Mojo" at all times -- that is, he must ensure that he isn't at his posting limit. If he doesn't, he might lose the opportunity to say something useful that would earn him more Mojo.
The rate at which people can accrue Mojo is limited by the size of the community. If there are few people who are handing out Mojo, pretty soon they won't be able to hand out Mojo anymore, since there won't be anyone left who hasn't already received Mojo from them.
The oldest users will effectively become an invincible cabal whose ideas may define and shape your site. Since you can only reduce someone else's Mojo if they have fewer Mojo than you, these users can make statements your community vehemently disagrees with and have their Mojo ratings remain intact.
In general, by restricting the supply of Mojo, you have created an economic system in which people will probably be more hesitant to contribute to a discussion. They will need to more carefully weigh what they say, since a post that is disliked by the community may prevent them from speaking further if they get too much negative Mojo.
Playing the system
Either show me how to play the system, or explain why you think it's impossible to play it.
Suppose we define "playing the system" as "artificially changing the value or quantity of one's Mojo in ways you didn't intend". I would say your system is safe from artificial inflation of Mojo, but at the cost of stifling discussion that would have otherwise taken place. You must be able to make the following guarantees:
Flow condition. Mojo only "flows down"; it is not possible for a lower-ranked user to transfer Mojo to a higher-ranked user.
Creation condition. It is difficult for users to take actions which can generate their own Mojo. Also, it is either impossible to make new accounts, or they carry such a heavy penalty that no one will want to do it.
For example, if you cannot satisfy the creation condition, then malicious early users will simply make an army of new accounts. They keep a number of these new accounts in reserve, quietly accruing Mojo. They then use them as a "bury brigade" to drain Mojo of factions or ideas they disagree with. Although they will lose Mojo when they do so, the collective Mojo of the bury brigade will be constant as long as they don't go hitman on more than a single hated foe per day.
This is clearly not what you intended.
Artificial deflation
However, your system is not safe from artificial deflation of Mojo. To see why, imagine that users stream into your site to accept their invitations. Imagine a user, Mallory, with k Mojo points. Because Mojo can only flow down, there is no way for a user with k or fewer Mojo points can express their disagreement with Mallory. Only users with k+1 or more Mojo can do so.
Mallory's reign of terror will continue unchecked unless there are enough users with k+1 or higher Mojo. In fact, if Mallory is an early enough user, there may not be any users who have the power to reduce her Mojo. Indeed, because you've artificially made Mojo scarce, they may not want to burn a Mojo to express their opinion, given how precious each Mojo point is -- since that also weakens them and makes their opinions more vulnerable to attack.
In short, if there are enough (or maybe even just a few) Mallorys, you will be reduced to playing traffic cop and cleaning up after Mallorys instead of improving your site. The system can no longer be self-policing. Each Mojo point has now become worth much more than before, because people will see from the example of Mallory and her ilk that it is better not to burn a Mojo to open oneself to attack. Thus, Mojo deflation.
This is also clearly not what you intended.
The flaw in your system is that it focuses on length of membership, not quality of contribution. An inactive user who contributes nothing gains status simply by virtue of having joined a long time ago.
Why do you want to limit the amount of content users can post? I'd suggest a simple system that lets other members flag a post as "bad" (just like SO). Once a certain number of members (say 5) all mark the post as "bad," it is automatically removed. Members with too many "bad" posts are booted off.
I have a long answer, but the summary is: your system does not get rid of the bad users, but only delayed them from doing harm.
...can't be "played"
Giving and taking Mojo is supposed to be an act on the level of marking someone as a friend or as an enemy.
I want them to use it to influence how other people behave on site.
The only way to see if your system is secure is to be evil and game it. Stackoverflow is full of nice people, but for this question to be answered, its best to think evil thoughts.
/me wears goatee
To create an account you need one Mojo. Without it you can only answer other people's comments.
Hi, I'm MrValdez. I'm a nice guy. Here's some friendly answers to your comments.
When you have an account, you get one Mojo a day.
You can give one Mojo to someone who has less, but only once to each person.
Hey, this is a cool site. I'll invite my friends over.
/me creates dummy accounts and give them Mojo.
Hey, here's a blog post. What do you guys think?
/me logins into dummy accounts and post like crazy.
You can only post as many posts and comments as you have Mojo.
You can answer as many comments as you like.
You can't have your Mojo back.
Via a post or private message: Awww man, I'm out of Mojo. Can you give me some so I can answer [dummy account #1]'s post? His post is awesome and I got a great response which I think he'll enjoy.
"Post of the Day" or "Favorite Blog of the Week" contests with prizes of around 3-10 Mojo.
I'll make a post and have my dummies vote me.
10 days later, I would have mojo = (number of dummy accounts) * 10 + (Number of users fooled into giving me Mojo).
/me gives the Mojo to dummy account who goes into a flaming spree. /me washes hands of the deed.
What happens next depends on your current community. They could start flaming back or they could ignore me and my dummies until we go away.
The worst case is that they will fight back and all of your planing and system would go to waste.
The best case is that they would be mature enough not to entertain the bad guy. BUT, if your current community is mature enough, why would you still need such a complicated system to encourage people to be nice?
I recommend that you should spend time building the community. In my opinion, if you want a good community, you should spend more time with the community and helping set the tone for the rest to follow.
Case-in-point: StackOverflow started with a community of programmers who are genuinely interested in helping other people. We got a great sense of community because we all have the same interest. We know the top users won't abuse their moderator's power because we explicitly gave them our trust over a long period of time (in contrast with your system where time and trickery are used to rank a person's status). And finally, the developers of the site actually posts questions and answers as well as "talk" to us via a weekly podcast.
As a (pretend) bad guy, I don't have the heart or the resources to destroy such a tight community.
updates:
To avoid having people create multiple accounts to boost their main account you can only give Mojo to people with less Mojo (and each user can only give another user Mojo once).
Same problem. I'll just have the main account give Mojo to the dummy accounts until I get almost even Mojo across the different accounts. In addition, that rule can be easily circumvent by making more dummy accounts.
The scoring on "Post of the day" type of things is something I have yet to work on.
If you want us to comment on the scoring, update your post and we'll update our answers. But my opinion still haven't changed, you're just applying band-aid to the wrong problem.
Other posters have mentioned that your system encourages idle accounts. From the point of view of a bad guy, I wouldn't want to have an idle account. I would want to get as many account recognized by the community as a legitimate account. This would cause more damage when I finally decide to attack.
There are a lot of good answers here, but I think what you are looking for is a way to "game" the system such that no user, no matter how powerful, can limit how you behave. Thus this is my attempt to game the system in this way.
Necessary precondition: Someone invites you to the site/gives you one mojo, since it takes one mojo to create an account.
Then,
I have my one mojo account.
Each day, I use my one mojo to create a new account (I "invite" them and give them one mojo).
Then each day, I use the mojo of my created accounts to create another account for each account I already have (obviously burning each accounts Mojo down to zero each day).
IE Day 1
1 account creates another account - so I have 2
Day 2 - Each account creates another account - so I have 4
Day 3 - 8
Day 4 - 16 accounts
Day 5 - 32 accounts
Day 6 - 64 accounts
Day 7 - 128 accounts
Day 8 - 256 accounts
Day 9 - 512 accounts
Day 10 - 1024 accounts
Day 11- 2048 accounts
I repeat until I have enough accounts that even the highest ranked user won't have enough mojo to silence me - instead of competing by increasing my individual mojo, I compete by increasing my "group's" mojo - If I post something and you don't like I just repost under another one of my thousands of accounts. Instead of being a bigger dinosaur I would be a swarm of mosquitoes.
It looks pretty good, but remember that some users will eventually find ways to game any system.
Hopefully you can get a vague idea about the effectiveness of your reputation system after the public beta.
So that is about security.
A problem (may be it is how you designed it to be) I see with the system is that unlike SO or any other similar community site, you need to burn your points to contribute to the system rather than gaining points for those actions.
This decision may harm the growth of content in the site. Users will think like "I have a small number of points. So I will keep it and let my reputation grow". This means that your current system does not promote growth of content in the system.
When you have an account, you get one
Mojo a day.
i don't know but getting paid for doing nothing does not make much sense
overall, i would say your system will not work out. a reputation system should be primarily based on the (good or bad) things people do and not on the number of days people have an account.
Mojo is not a currency, it's
determines your status among other
users.
that's exactly what money does ;)
I can see where you are coming from, but as long as you hand out free Mojo(s), it will be possible to game the system.
But aside from that, your system has a side effect that you probably haven't thought of:
You launch the service on Dec 31st.
MrFirstUser registers as your first user, followed by MrSecondPlace and a few others
In the following days, a bunch of others register users
On January 3rd, everyone from the first day has 3 Mojo, and everyone else has fewer. MrFirstUser writes a comment, and MrSecondPlace gives him a Mojo for it (they have the same amount of Mojo, so I assume he is allowed to do that).
Now, MrFirstUser has 4 Mojo, and everyone else has 3 or fewer.
Now, MrFirstUser has exactly (numberOfDaysRunning + 1) Mojo, and he can never get any more, since all other users have too little Mojo to give him more. The only way he could get another Mojo, would be if User 3 gave 1 Mojo to User 4, who then gave a Mojo to MrFirstUser. Beyond that, it becomes even more involved: the number of other users who would have to participate in this human pyramid of Mojo contributions increasing exponentially.
In other words, the first thing I would definitely do to game the system, would be to create as many first-day users as humanly (robotically) possible, or at least make sure my own account was created on the first day, since that would make me an automatic Jon Skeet -- since noone would ever go significantly higher than 1 or 2 above me (only prizes could help them).
To fix your system, I would do three things:
No more automatic Mojo; Mojo is earned through upvotes, gifts and special prizes only
Allow giving Mojo to all other users (if there must be a limit, cap the gifts so no more than 10% of one's total Mojo can be given to one user)
Keep the system invite-only, and penalize users whose invitees misbehave or get too many downvotes/'report user's
As far as I understand, no one can have as much mojo as someone who joined on the first day and never "burned" any. He can even do nothing and still be at the top of the mojo list. I wouldn't call this reputation, just something like seniority, and it doesn't have much correlation with achievements.
far too complicated. KISS
Is there any good reason why your reputation system exist? Joel Spolsky's Building Communities with Software mentioned that if you want your users to participate, you shouldn't be making it hard. By setting up a complicated reputation system, aren't you limiting your community too much?
In software, as in architecture, design decisions are just as important to the type of community that develops or fails to develop. When you make something easy, people do it more often. When you make something hard, people do it less often. In this way you can gently encourage people to behave in certain ways which determine the character and quality of the community. Will it feel friendly? Is there thick conversation, a European salon full of intellectuals with interesting ideas? Or is the place deserted, with a few dirty advertising leaflets lying around on the floor that nobody has bothered to pick up?
Your system is basically a popularity contest with feedback, so expressing popular sentiments will be rewarded and expressing unpopular sentiments will be penalized. This will discourage intelligent conversation and encourage the echo chamber effect that has occurred on Reddit around issues such as Ron Paul.
Also, as Paul Graham noted in a recent essay, a joke or a slogan is easier to understand and upvote than a research paper, so your system encourages simple posts.
say the maximum posts anyone wants to do is 15 in a day - that's a lot, but you only have to be a member for 2 weeks, and then there's no point in accruing more mojo because you've maxed out what you can do with it.
So at this point you don't care too much if you get more, and you'd need quite a lot of coordination to knock someone down.
Pies,
I've been thinking about reputation algorithms (that's how I found this thread).
You have to model your reputation algorithm after real world:
1) Older people are not necessarily better or more interesting, or should have more reputation points
2) Very young people can't have much of a reputation as they are new and did not contribute anything to the community that they can get reputation points for
3) Opinion / vote of a truly respectable / reputable person weighs more than that of some Joe Schmo.
4) If a bunch of Joe Schmos gang against a truly reputable individual, their combined mob opinion still should not prevail this individual's opinion
5) Possessing a lot of money (or mojos) does not translate into having a better reputation score (otherwise rich people would ALWAYS ultimately prevail over smart people).
6) Associating with more reputable members of a community passes some of their reputation juice to their friends and family (and it doesn't matter how much these friends and family contribute -- they are cool just because they know or related to someone cool).
Thus, my conclusions:
1) You can't increase reputation score based on age
2) You can "penalize" very young accounts (same as Google Sandbox), which will at least solve a problem of creating a number of accounts and immediately voting something up or down. Some time should pass until user account "matures". Maybe even as little as 24 hours.
3) Same actions (that can cause your or other members reputation score to increase or decrease) can't be same for all members. Value of up or down vote of a more reputable member should be more than of a member with low reputation, and opposite for low reputation member.
4) Even an army of members with average reputation score way below targeted member's reputation score, they should not be able to decrease this member's score. Though it would sort of make sense if their average reputation score was closer, in which case the mass effect should trigger something that we can call a revolution. (Though in history revolutions have not always result in a good thing for the communities where they happened).
5) Reputation can't be the "currency" of your site. Something else can, but not reputation.
6) If you have a friendship concept, than being friends with someone with higher reputation should automatically pass some of that reputation to you. And that should be happening if you become friends with someone you know (at least on your web site), not befriending anybody and everybody.
No reputation system is secure, without real life verification. If anonymous people can be invited then it can always be gamed, with sufficient effort and false identities.
Your aim should be to increase the cost of such gaming above the potential gain, which will depend on what your system gets used for.
The main cost you can use to inhibit power gains is forcing users to contribute positively reviewed contributions before you increase their standing.
How you do this, and how you make it difficult for them to leverage existing identities to falsely rate as positive the contributions of new identities they are trying to give power to, is the nuts and bolts of the system; and should make use of an existing user base to spot and penalise spurious false positives.
You can make use of game theory by rewarding people for being in the majority. So if A accuses B of wrong-rating, it is highlighted and lots of people get to vote on it. Those who cast their mojo on the side of the vote that ends up winning, get it back and increase in reputation for reliability (it is worth having more than one type of reputation), while those in the minority, lose a triangle number of mojo (one the first time, 2 the next, 3 the next, 6 the next, 10 the next, and so on)
Your system is impossible to game, because it is impossible to join it in the first place.
You say a user needs one mojo to join. How do they get that one mojo if they are not already a member?