I want to design a traffic exchange script that counts incoming traffic and tries to return a n:m (1in:2out for example) ratio of traffic. In the database, I want to have the sites info (SITE), then track traffic by site and ip (DAILY-HITS), then I want to have the trade counted per hour or day - not sure yet (SITE-TOTAL-HITS). I would love any suggestions I can get for designing a well designed database for handling traffic trades.
Right now I'm worried about tracking the incoming hits, later worry about returning. So basically my real question is, how can I design a database that can be efficiently used for returning the perfect amount of traffic for say the last 24 hours. The problem I'm having is, when it comes to programming, I want to have the best designed database for getting information on the last 24 hours and making sure I stay within the n:m ration. Heres what I've designed so far for the database:
SITE (just basic info):
id,
url,
title,
description.
DAILY-HITS:
id,
site_id,
ip,
date (include time - will be broken down per hour),
amount (count how many hits from this ip for this hour).
SITE-TOTAL-HITS (probably be updated every hour via script - useful later for counting last 24 hours):
id,
site_id,
year,
month,
day,
hour.
Any tips, suggestions or point me in the right direction would be greatly appreciated. Thanks in advance.
Here is my take on it for what it's worth:
Site:
Same as what you have
Traffic:
id,
site_id,
ip,
request_DateTimeStamp <- this is a date and time for the url request
no count here, just logging here is why:
assume you do have a count column
- url is requested
- lock up a record in the database with the same date and hour and ip
- Found, update the account
- Not Found, Create a new record and set the count to 1
this is a long process for a table that will presumably be updated a lot, within an hour span multiple request from the same ip will try to update the same record and they will have to wait on each other.
take out that count column and your process is simply to log the incoming traffic by always creating new records.
as far as data analysis is concerned, you can do it in couple of ways:
you can try building cubes: http://datacharmer.blogspot.com/2010/01/multi-dimensional-cubes-in-mysql.html
or you could create information aggregation tables like site_total_hits and just update them using nightly jobs or is often as you wanna run them depending on how accurate they have to be.
these are my 2 cents :D
You can use HitLeap to do that.
What is HitLeap? HitLeap is a Traffic Exchange, also known as an autohits service We help you increase your website hits, rankings (alexa, google) and more. Our affiliate program gives cash and traffic commissions of up to 50%. How does it work? After signing up, you will submit all the websites you want to send traffic to. Then you will earn free traffic by viewing other people's websites. Alternatively, you could buy a traffic package from us.
Go tho this link to sign up
Related
in vue.js app the main focus is working with prospects. prospects have many things like contacts, listings, and half a dozen other objects/tables.
they also have interactions, which could have 30 or more per prospect, while most things like emails or phones would have 1-3 results. I load 50 prospects at a time in to the front end
I'm trying to decide if loading it all into the front end to work 50 prospects at a time is a good idea, or if i should have a json column with interactions as part of the prospects table that i would update each time an interaction is saved, with minimal info like date, type, subject...
it seems like an extra step (and duplicate data, how important is that?) to update the json column with each interaction, but also seems like it would save looking up and loading data all the time
I'm not a programmer, but have been teaching myself how to do something i need done for my business with tutorials and youtube, any opinions from those who deal with this professionally would be appreciated
also, if anyone wants to tell me how to ask this question in a better formatted way, I'm open ears
thanks
Imagine if you have 1000 data, but you are sending only 50 of them, and your user do a filter by price. Will you display only the filtered data from 50 or 1000 of them?
That depends on whether you want to expose all 1000 data to the front end. It's a choice between that, and calling your server api everytime.
If you are calling the server, consider using a cache like Redis to store your results .
Pseudo code.
Request Received
Check Redis Cache - Redis.get('key')
If key exist - return cache.
Else -
check mysql for latest results.
Redis.set('key', latest results);
CreateRequest Received
- Write to mysql
- Redis.delete('key') // next request to view will create new cache with fresh data.
Your key can be anything like, e.g your url ('/my/url')
https://laravel.com/docs/8.x/redis
I am planning on creating a database to track user's time in/time out M-F. Every week should begin on monday and run through sunday.
I have a table filled with my entire user population, so I know which users I need to create entries for and where they belong to. I have proposed so far, a table consisting of the following fields to track the entries (along with example data to fill the fields):
Field Name in table (Example of possible data)
---------------------
Employee (John Smith) 'String
Unit (Quality Assurance) 'String
WeekOf (9/9/13) 'date
InMonday (6:30) 'string, validate either a time in/out or N/A if holiday/vacation
OutMonday (3:15) '^^
HoursWorkedMonday (8.00) 'total hours worked
VacationMonday (0.00) 'if N/A for time, should have hours here
OvertimeMonday (0.00) 'any additional work hours should go here
For this instance, I would have to create In/Out for each day of the week (and perhaps track the date that each day is for). Is this extraneous or is there a seemingly better organization to tracking weekly time measurements? Should I use one table with a unit indicator or multiple tables for each unit?
Usually it's one table with a Date field, an In field and an Out field. That's pretty much standard timesheet data. Take a look at how this guy has it set up.
Make sure you're using an Employee ID in the timesheet, and then you would have a corresponding Employee table with all relevant info (ID, Name, Address, whatever else you store on him/her).
While this project is technically feasible, I have to question the value of making it yourself in Access.
The main issue is with security:
As a desktop program, this can be very easy to hack without precautions. Keep in mind that with Access, the user interface and the designer interface are by default the same thing.
If this is going to be a simple, straightforward db, a motivated user just needs to open the navigation panel and they can add/edit/delete all the timesheets.
If you hide the navigation panel, the user can just do a quick google search and learn to hit F11 (or find it by accident, either way)
You can try regularly (daily? hourly?) transferring the data from the publicly accessible back-end to an archive db that is not accessible to the general users. This can work, but still gives them a window to edit records. And if you don't do the transfer right, they can still add old records.
As a webform on a SharePoint, this can be fairly secure. I'd recommend this if you have Sharepoint.
You should also consider your development time. This is a very common business task across many industries, from restaurants to factories to schools. As such, there's a huge number of cheap web-based options already out there that you can start using today. I'll even assume some of these include summary reviews breaking out numbers by departments as well.
I've never researched these myself, but a quick google search found this interesting page: http://en.wikipedia.org/wiki/Comparison_of_time_tracking_software
Okay..
I am making a web base application,that will be connected to a sms gateway.
It is basically an attendance app for colleges.
The Attendance will be updated using a sms by the teacher.
Now,the main part-
What I want to do is,the teacher should not be able to correct the attendance after 10minutes of sending the 1st message.i.e,the database should accept a correction or new message for the same class and the same teacher only for 10 minutes after the 1st attendance is recieved in the database.
So only recieving from a particular number should be blocked and also only if it is for the same class...
I hope the question is clear :o
Thankyou
This is not the sort of thing that you should be enforcing at the DB level, it belongs in your application code. If you can't connect time, number & class together in your DB, it's time to change your schema.
As Sean McSomething mentioned, this is not done at database level, this is business logic that should be checked just before interacting with a database. The best practice actually is to simply add a column time_created and before updating simply check if NOW() and time_created interval is less than 10 minutes. It's a pretty trivial job, but don't bother trying to do this in database with some stored procedures or other stuff, as it will make your application almost un-debuggable and very sloppy.
Check if there is a row with active number and active class, if there are no - insert, if there are any - check if this row's time_created is greater than 10 minutes ago, if it's not - update, ignore otherwise.
I'm developing an application that
stores an e-mail address (to a user) in a table.
stores the number of days the user would like to stay in the table.
takes the user off the table when the number of days is up.
I don't really know how to approach this, so here are my questions:
Each second, do I have the application check through every table entry for the time that's currently stored in, let's say, the time_left column?
Wouldn't (1) be inefficient if I'm expecting a significant number (10,000+) users?
If not (2), what's the best algorithm to implement for such a task?
What's the name of what I'm trying to do here? I'd like to do some more research on it before and while I'm writing the script, so I need a good search query to start with.
I plan on writing this script in Perl, although I'm open to suggestions with regards to language choice, frameworks, etc... I'm actually new to web development (both on the back-end and front-end), so I'd appreciate it if you could advise me precisely.
Thank you!
*after posting, Topener asked a valid question:
Why would you store users if they won't get requested?
Assume the user is just sitting in the database.
Let's say I'm using the user's e-mail address every 5 minutes from the time the user was added to the database (so if the user's entry was born at 2:00PM-October 18, the user would be accessed at 2:05, 2:10, etc...).
If the user decides that they want out of the database in 10 days, that means their entry is being accessed normally (every 5 minutes from 2:00PM-October 18) until 2:00PM-October 28.
So to clarify, based on this situation:
The system would have to constantly compare the current time with the user's expiration date, wouldn't it?
you should not store the time_left variable, bt you should store vaildTo. This way, whenever the user is requested from the database, you can check if it is valid.
If not, then do whatever you want with it.
This approach wont let you make any cronjobs, or will cost you extramload.
Hey Mr_spock I like the above answer from Topener. Instead of storing a number of days the user would like to be valid, store the day the user would like to be be removed.
Adding a field like validToDate, which would be a DATETIME field type, you can do a query like
delete from tablename where validToDate <= NOW()
where
the italicized text is a SQL query
tablename is the name of the table in question
NOW() is a valid sql function that returns the current DATETIME
validToDate is a field of type DATETIME
This has what ever efficiency SQL server promises, I think it is fairly good.
You could write a separate program/script which makes the delete query on a set interval. If you are on a Linux machine you can create a cron job to do it. Doing it every second may become very resource intensive for slower machines and larger tables, but I don't believe that will become an issue for a simple delete query.
I apologize if this has been asked before, but I'm pretty new to this and unable to find an answer that addresses the situation I'm faced with.
I'm trying to put together a database to run behind our company website. The database will store information on customer invoices and payments. I'm trying to figure out if I should create a field for the invoice balance, or if I should just have it calculate when the customer account is accessed? I don't want to create redundant data, and don't want to have the chance that somehow the field wouldn't be updated, and would thus be incorrect...but I also don't want to create to large of a burden on the server - especially if we pull up an overview of customer accounts - which would need to then calculate the balance of every account. Right now we are starting from scratch, so I want to set it up right!
We are anticipating having a couple hundred customer accounts by the end of the year, but will most likely be up to a couple thousand by the end of next year. (Average number of invoices per customer would be roughly 2-3 per year.)
There are probably other things to consider as well. For example, what if your invoice consist of ID's of products in another table... and the prices of those other products change? When you go to generate the invoice, you'll have the wrong total in there for what the guy actually paid 6 months ago. So if its a situation like that, you'll probably want to store the total on the invoice. And I wouldn't worry too much about doing a little math if you go the other route, it's not likely to be a huge bottleneck.
Yes, remember that items/goods could and will change their prices over time. You need to have the invoice balance as of the day of the purchase. Calculating the balance on the fly could lead to wrong balances later on.
Invoice balance is essential data to store, however I think you meant account balance since you referred to that later.
Storing the account balance would be denormalizing it, and that's not how accounting databases are typically designed. Always calculate account balance from invoices minus payments. Denormalizing is almost always a bad idea, and if you need to optimize in the future, there are other places to cache data that are more efficient than the database.
In your use case, a query like that on a few thousand rows would be negligible anyway, so don't optimize before you have to.