I made a node app that reads every 100ms a table in a MySQL database, which get updated from a other app (lua), and compares the result with the last result. If the table has changed the app will perform an action. The reading process needs ~0.05ms and it works all perfect. But now am I asking myself wehter this is an acceptable solution or very unprofessional.
If you are limited with only the database, I would use two timestamp columns like updated_at and deleted_at (with indexes maybe - should ask a DB professional) and store the last read time on the client, and then only query the changes made since. This way it'll be more comfortable when using a table with millions of rows.
And except that, It comes to mind that you can use something like websockets to update the client when an update/insert/delete will occur/had occured.
There's another question about this matter:
Which is more efficient to send WebSocket updates with a MySQL database change
Late Edit
Your "If the table has changed the app will perform an action." sentence is probably the key here. You can just check for something like this:
select count(*) from table
where updated_at > last_update_time or deleted_at > last_update_time
then you can decide for to do that action or not.
Related
In my MySQL database I have a user table. I need to perform search as you type with typo over the user name field. There are few very old question on this topic. I tested the builtin full text search of mysql but it didn't work as expected (it does not handle typo) [I knew but I tried anyway].
What's my best option? I thought there should be an easy solution nowadays. I'm thinking about replicating the user table on elasticsearch and do the instant search from there, but I'd really like to avoid the syncronization nightmare that this will cause.
Thanks!!
You could use SOUNDEX for mysql. We have tried that but I can say that it does not work that well and it also makes the search a bit slow.
We Had a similar issue and switched to ES.
What we did is as follows:
Created a trigger for the table that will be synced to ES. The
trigger will write to a new table. The columns of such a table would
be:
IdToUpdate Operation DateTime IsSynced
The Operation would be create, update, delete. IsSynced will tell
whether the update is pushed to ES.
Then add a corn job that would query this table for all rows that will have issynced set to say '0', Add those ID's and operation to a Queue like RabbitMQ. And set the ISSynced to 1 for those ID's
The reason to use RabbitMQ is that it will make sure that the update is forwarded to ES. In case of failure we can always re-queue the the object.
Write a consumer to get the objects from the queue and update ES.
Apart from this you will also have to create a utility that will create an ES index from the database for first time use.
And you can also look at Fuzzy Search of ES that will handle typo's
Also Completion suggester which also supports fuzzy search.
I would like to create a View that only returns new data added to the database since the last time the View was run. I know this sounds like a simple question, and there a millions of question just like it on here, but there are some restrictions in my case.
I can't use Stored Procedures, Functions or Triggers.
The user accessing the View only has read access to the database.
I posted a similar question here and was going to add an UPDATE statement to update a table with the current DateTime, and then check this table at the beginning of the View and only query after this DateTime. But the user accessing the View only has read access now.
I was also thinking there might be a type of timestamp column that updates to the current time when read with a SELECT statement, but apparently not.
Just for reference there is only ONE user accessing this view.
Any help much appreciated.
If the user only has read access to the database, then you have to store the relevant information at the application layer and not in the database. It simply is not possible to store information in the database if you don't have write access -- or at least information not part of some other process.
The recommendation would be for the application to keep track of the record returned -- keeping track of the maximum CreatedAt value or id. Then access the database using something like:
select v.*
from view v
where id > $id;
The subsequent code would update $id in the application.
I am currently working on a j2ee web application. The application features a way for users to reset their passwords if they forget them.
I have a database table with 3 columns: username, key, and timestamp.
When the user requests a password change, I add an entry in that table with their username and a random key (making sure that their are no duplicate keys in the table, also that a user can only appear once in the table). I also add the current time. I then send them an e-mail with a link to the application that contains their key, something like:
mysite.com/app/reset?key=abcxyz123
The servlet that handles this request looks at the key in the url to find the matching entry in the reset table to determine which user the key belongs to. If the key doesn't match an entry, I show an error page, if it does, I show the password reset screen. Once the user changes their password, I manually delete the entry from that reset table.
I am trying to implement the equivalent of a time to live for the password reset links, so that I don't have entries loitering in the table unnecessarily, and I thought of 2 options, the first of which I have implemented:
1) Create an EJB Timer that fires every minute that will delete entries in the reset table where the timestamp is older than 30 minutes. This is a manual process in that I am using hibernate as my jpa implementation, so I retrieve all the entries from the table, examine their timestamps, and delete the old ones.
2) Create a database job that deletes rows over a certain age?
My question is, does anyone see any drawbacks to the first approach, and second, is the 2nd option even possible with mysql? I figure that if I can use the 2nd approach, I can get rid of the timer, and let the database handle the time to live aspect of the password reset links, and that may be more efficient.
I haven't been doing j2ee development for that long, but based on the knowledge that I have, these seemed like 2 logical approaches. I welcome any input.
3) Create script that will connect to db, execute delete, disconnect. Then you can schedule this script via operating system e.g. crontab.
Regarding option 1 - Drawback of that solution is that it uses application server resources for stuff that can be done on database only and is not dependent/uses any application logic.
Benefit is that whole app is self contained and you don't need any additional installation/setup task on database as with 2 and 3.
I have a database in MySQL. The values in column named Curr_BaL is updated by different operations performing on it. The application, which is written in Java, accesses that database. When it runs, by default it should retrieve the last updated value. However, I also want to be able to get the value at a specific DATE entered by the user.
I have tried to do my best, but have not successful yet, and my whole application depends on that data.
Your problem is not entirely clear. What I can understand is that you need a way to have your users aware of this "last updated" value.
You have several designs approach for this. I think that the simpler would be to fetch this value when you're authenticating your user, and set it to its session information, so it will be available at any time.
You can also have some kind of service caching this value (since I guess is the same for all users).
A very important thing you didn't mentioned is who updates this value, is an external application? is a process on the same application?.
What I can understand, users date more priority then automaticaly date. Simple way for it's using triggers. Below may be useful:
CREATE OR ALTER trigger on_table_ins for TABLE
active before insert position 0
AS
BEGIN
IF (NEW.DATEFIELD IS NULL) THEN NEW.DATEFIELDD='now';
END
It correct for firebird, so see manual for triggers and insert current date(time) for your RDBMS.
I like to update my table every second. I know this is possible in AJAX but is this possible in MySQL query? I found MySQL event, can I get solution from this?
" i wanna check condition **if date_time > now() ** then update status as 1 . is this possible"
it does not seems like you need special status to be setup...
this condition can be checked when data is pulled (if need to be marked execute UPDATE and SELECT when pull),
also it can be done as cron job every minute (not sure can be done every second), however if it very related with user being on page - ajax could be the way to do it and downgrade performance at this same time
It is possible to write an sql query that will set an update status equal to 1 when the date value for that record is out of date, however you will still need a scheduled task to run this sql query from time to time.
If you are able to run code on your server, then you should write up a script that periodically runs your query against the database.
so cron is a scheduler in linux to run anything periodically.
So let's say your script that contains "**if date_time > now() ** then update status as 1" is called updateIfOld.php
then you should make crontab runs "php updateIfOld.php" every second.
Here are the manuals to use crontab: http://www.manpagez.com/man/5/crontab/Ta3HK4KosQPd8aT8BQ&usg=AFQjCNErp1Hz19N7xJwVY1wisQNxmtgpQQ&sig2=D4NQu19AJnBZil9J54V8ww
If you could actually tell a bit more about this situation and why u need this to be done, it will help us to give a better solutions.
With assumptions on what you are trying to achieve , we can give the best.
Anyway sending too many ajax and updating query every second is not an good option.
Here is an idea,
if you can store the expiry time for each row and you can set them set status to 1 where your condition matches.
Anyway i think there must be some reason to change the status to 1 (may be for making them not to display/consider).. If we know the exact reason, i think i can give a better solution..