Sorry, I am a newbie programmer with insufficient database control knowledge.I am not sure whether now i need to redesign the whole system flow or add on some feature in it.
Currently I have a few client that running on different computer.
Current Architecture:
-When the network is on,
Each clients will update into a same server, then server will sync data back to all local database located in each client.
-when the network is down,
Each clients will access to the local database, and save their update list, until they are reconnect back to the server, the update list will pass to server and will doing update to the server again.
The architecture is doing fine when the customer will only use one of the client (once), so the update apply to same customer only apply once and therefore no data conflict is occurred.
Now the problem come, one of the new feature of the system, like appointment function, if one of the appointment type is fail, customer is allow to make appointment again but not the type he had failed before.
But if system is down, customer is allowed to make the appointment with the same type again and he might success on the second try and the record will delete the previous failed information/ or if this information is updated before the client that holding the failed record connecting back to the server, the old record will overwrite the new record.
Currently i have some idea but i hope some one here will give me a guide, either other design or some other precaution should i take on this flow.
Some of the idea:
If the system is down, some function that might cause conflict should be not allowed.
-(Yet this is somehow decrease the availability of the system)
The update list will only work until all client is reconnected to the server, It compare which data is fresh and conflict, and will choose the fresh data to update and flag up the conflict.
-(Somehow i think it is the worst idea.by binding the client all together and restraint the update list to update in time, we can't confirm the data in server is the latest and might cause more problem to those online client. )
When the connection is flag down, record down all the update in another table. Just use some program/excel? to compare and check on the conflict?
-(probably the idea that cost the most)
Is there any third party software that can help on my situation?Or I should just change the system flow on updating and retrieving the data?
Related
I'm trying to make a live react control panel, so when you push a button on the web control panel the data (true or false) goes to the SQL database (phpmyadmin) and the when the data changes te SQL database should trigger a script on the raspberry pi that will turn the light on.
I know how to write data to the SQL database and how to control a lamp with a raspberry pi but I dont know how to trigger or execute something when data in the SQL database gets updated.
It needs to live, like react in max 20 ms or something. Can anyone help me with this?
The SQL Database runs on Ubuntu and is phpmyadmin based.
Greets,
Jules
Schematic:
DataUpdateGraphical
It's not a good idea to use a trigger in MySQL to activate any external process. The reason is that the trigger fires when the INSERT/UPDATE/DELETE executes, not when the transaction commits. So if the external process receives the event, it may immediately go query the database to get other details about that data change, and find it cannot see the uncommitted data.
Instead, I recommend whatever app is writing to the database should be responsible for creating the notification. Only then can the app wait until after the transaction is confirmed to be committed.
So your PHP code that handles the button press would insert/update some data the database, and check that the SQL completed without errors (always check the result of executing an SQL statement) and the transaction committed.
Then the same PHP code subsequently calls your script, or posts an even to a message queue that the script is waiting for, or something like that.
Just don't use the MySQL as a poor man's message queue! It's not the right tool for that.
The same advice applies to any other action you want to do external to the database. Like sending an email, writing a file, making an http API call, etc.
Don't do it in an SQL trigger, because external actions don't obey transaction isolation. The trigger or one of the cascading data updates could get rolled back, but the effect of an external action cannot be rolled back.
MySQL doesn't have a way to deliver an event to external software from within a trigger. That's what you need to have your database push events to your app.
(Actually, it's possible to install a user-defined function that sends an industry-standard stomp messsage to a message queue system like rabbitmq . But you will have to control the entire server, AND your database administrator, to get that installed.)
The alternative: run a query every so often to retrieve changed information, and push it to your app. That's a nasty alternative: polling is a pain in the xxx neck.
Can you get your server app to detect changes as it UPDATEs the database? It'll take some programming and testing, but it's a good solution to your problem.
You could use redis instead of / in addition to MySql. redis sends events to web servers whenever values change, which is close to perfect for what you want to do. https://redis.io/topics/notifications
I have an Azure SQL db where I am executing a change with a c# call (using await db.SaveChangesAsync();)
This works fine and I can see the update in the table, and in the APIs that I call which pull the data. However, roughly 30-40 minutes later, I run the API again and the value is back to the initial value. I check the database and see that it is indeed back to the initial value.
I can't figure out why this is, and I'm not sure how to go about tracking it down. I tried to use the Track Changes SQL command but it doesn't give me any insight into WHY the change is happening, or in what process, just that it is happening.
BTW, This is a test Azure instance that nobody has access to but me, and there are no other processes. I'm assuming this is some kind of delayed transaction rollback, but it would be nice to know how to verify that.
I figured out the issue.
I'm using an Azure Free Tier service, which is done on a shared virtual machine. When the app went inactive, it was being shut down, and restarted on demand when I issued a new request.
In addition, I had a Seed method in my Entity Framework Migration set up to set the particular record I was changing to 0, and when it restarted, it re-ran the migration, because it was configured to do so in my web config.
Simply disabling the EF Migrations and republishing does the trick (or when I upgrade to a better tier for real production, it will also go away). I verified that records outside of those expressly mentioned in the Migration Seed method were not affected by this change, so it was clearly that, and after disabling the migrations, I am not seeing it any more.
I'm building a platform that is supposed to serve many corporate users at the same time. The system contains a lot of records that from time to time require updating. It is important that every change is logged and appropriately stored. I use a Laravel 6 implementation for the back-end and Angular 6 for the front-end. The front-end sends requests to the back-end via HTTP requests. The data is stored in a MySQL database.
Users load a specific dataset in either read-only or in read-write mode. In read-only mode there is no need to lock the resource as the user is aware that he can only read the data. In read-write mode the dataset should be locked from the moment onwards that the user requested the data such that the dataset cannot be requested by a different user while someone is working on the data. The user working on the data then has the data open in a webform for editing. As soon as the user either saves, cancels or closes the browser the data should be unlocked.
Now locking the data in the database is not the problem, I keep a table of which datasets are locked for editing and whenever a user tries to access this data an error is thrown. Also releasing the data when the user either saves of cancels is not a problem, I just release the lock in the table.
However, since there is no interaction with the back-end when the browser is closed while editing, the dataset remains locked indefinitely (I could fix this with timestamps and cronjobs and so on but those are not an option for the company infrastructure). I'm wondering how to fix this?
One idea I had myself, but no idea on how to implement this, is to keep the HTTP connection open between the client and the server and have the server (laravel) execute some code to release the locks when the connection is closed. Any tips, hints or pointers for me to continue from here?
Use a timestamp field without cron job to indicate how long a record is locked. If the current time is later than the timestamp, consider the record unlocked.
While the web form is open, you can use an ajax request every couple of minutes to update this timestamp for the next couple of minutes, thus keeping the record locked.
If the user closes the browser window, then the timestamp field does not get updated and the record will be unlocked in couple of minutes.
I'm new to MySQL, but however I need MySQL to work as it will be at the center of my new SANS (Server Address Name System) system. The reason for this system is to provide a replacement system for gameservers, since the default Gamespy service that some games use is being switched off at the end of next month.
The function of MySQL in SANS is to store the IPs and ports of active gameservers (which are patched to send info to MySQL), and then make the clients (again, patched to retrieve the information from MySQL) add the servers to their in-game server lists.
Of cause, the issue here is that gameservers can easily go offline for any one of 1,000+ reasons, and we don't really want the client's game showing gameservers that are offline, mainly because:
If we need to block any fake gameservers, these fake gameservers will still be in the server list (and also the MySQL database)
It will clog up the server list very quickly
Temporary servers such as home, development and test servers will still be in the list
If a servers' IP and/or port changes for any reason (for example the server IP is dynamic), there will be duplicate servers in the list, and clients may not know which one to pick.
I've thought of a couple of solutions, including making the client ping each gameserver in turn to check to see if it is online, but this is not ideal for a couple of reasons:
The server computers' administrator may have WAN ping switched off, meaning that although our gameserver may be online, it won't show in the list
The pings of clients may be seen as suspicious behaviour to the various server administrators that administrate the networks that the server computers sit on, meaning that the client could be blocked because of this.
I've thought of a simple solution: get MySQL (or phpMyAdmin) to remove each table row 10 seconds after it has been added.
Is this sort of behaviour even possible?
I'm on Windows Server 2008 R2, with latest MySQL server and Xampp.
I think you could use a MySQL trigger to accomplish this (I'm not sure about the 10 second delay), but I believe there's a better solution:
You could add a column called Status to whichever table stores the gameserver information.
Then you could use flags to differentiate types of gameservers: fake, test, active, inactive, etc.
Next you would filter what the user sees to only show active gameservers.
If the server doesn't report back every 10 seconds, the flag is simply marked as inactive.
And finally you could schedule a job to run once a day to clean up records older than 24 hours.
If this doesn't work for your particular problem, let me know and I'll look into coding the trigger.
I have a table in my database which contains all of the users for my application. Unfortunately, when I launched my application, I didn't think to include a column which tracked the time at which a particular user signed up, and now I wish I had (bad idea, yes indeed).
Is there, by any shred of luck, a way that MySQL tracks when a particular record is inserted (such as in record metadata???), and would allow me to grab it and insert in into a new dedicated column for this purpose?
I am running on a shared cPanel host, so I doubt I have access to the MySQL logs.
Thank you for your time.
Only if you have binary logging enabled will you be able to trace exact times for the transaction.
http://dev.mysql.com/doc/refman/5.5/en/binary-log.html
Its not just for replication, but also a form of transactional recording in case of emergency.