I have a MySQL database with a REST API for my main application hosted on Azure. I am setting up a hardware sensor with an additional database that will capture data multiple times a second. When a value changes by a specific threshold of the current value or after a specific time interval I want to make an API call to update the main database.
ie) Threshold is 10%. Last value was 10 this value is 12; this will set a trigger to call API and add to main database.
Can a trigger be added to the second database to make a HTTP request? Is there benefit to using another RDBMS in this case instead of MySQL? Does PubNub/Firebase make sense in this situation?
Related
I am very thank full if anyone help me to resolve this issue.
Detail: I have a mysql table where 110000+ record stored with Unique application(column) number.
Now there is Detail API which will take only 1 application number at a time in URI parameter and return details of that application number it will take Approx 1 min to respond for now.
Now i need to update that records(multiple column) using Cron Job Scheduling to always updated record in database.
and it will progressively update.
Flow: Get application number from database -> Call detail API -> update there record on database.
Issue: there is large number of record in database so we can not call API for all application number in ones.
I am using Laravel 7 and Guzzle HTTP client 7.2 HTTP Client for API calling.
Any suggestion are welcome !
Update: (Tested) I am thinking to do something like this and agree with #bhucho comment to call cron in every 15 minute
We create one more column in table for last_updated_id default 1
and we will write a query to fetch application number something like and get 100 or 500 records in one slab using Laravel take methods from here Database Queries
$latestIdInColumn = myTableClassRef::max(last_updated_id);
$applications = myTableClassRef::select('application_number)->where('id', '>', $latestIdInColumn)
->take(100)
->get()->toArray();
Here we call detail API and update for each application number, when there is last application update done we will store that id in last_updated_id.
When again cron call we have that last_updated_id so we will apply same filter in next query where id '>' $latestIdInColumn. now we will get next 100/500 records where id is greater then $latestIdInColumn.
if ( $applications )
{
For all other available id
} else {
when record not found.
Here we will reset the last_updated_id = 1
}
Now function will fetch from id 1.
It is not tested yet i just planing to do this, i am happy to have
feedback on this.
Note: Now API not taking 1Min to respond I resolved this issue.
Update 2: It is working good.
Background: We rely heavily on views in our app and one of the ways in which we are looking to optimize them is by parameterization. We've got that aspect all figured out, but in order to make it work, I must be able to set a user parameter for the customer (tenant) id. I want to do this at a base level so that it is set for every single connection.
For example the script I want to run is simply:
Set #Param_CustomerId:=1234
where 1234 is the customer id based on the authentication.
Here's the question: Where can I set up code that will get run every time a new MySQL Connection is established?
I think you can run that code in the registerConnectionServices method of a class which would extend the default DatabaseServiceProvider (and use that extended provider instead of the default).
I've got two questions regarding Orion subscriptions.
If we register entity with provider application url in Orion and create a subscription for it (e.g. sending updates every 15 minutes), what will happen if there is no data in Orion's local DB? Will it query data provider to fetch data from specified url and then return this a subscription update, or will it return nothing?
This is somewhat related to the first one. Is there an option to specify "max duration" of an attribute value in Orion's local DB (e.g. if an attribute is not updated in 1 hour, delete it's value)?
We have the following example in mind: A subscription is set up for an entity to send update every 15 minutes to our server. Updates from sensors to Orion should be done every 5 minutes. Now, if something is not working with the sensor and it stops sending updates, we will get the last value stored in Orion DB forever, unless there's the "max-duration" option for that attribute that deletes the value if it's not changed in specified time period.
When subscription update is triggered, in case that there's no value for that attribute in Orion's local DB, it should query the provider application for data (Q1).
Regarding 1, I guess you refer to ONTIMEINTERVAL subscriptions. At the present moment at Orion 0.23.0 (it may change in the future) the notifications sent due to that kind of subscriptions are populated from Orion entities database, without querying Context Providers.
Regarding 2, there isn't such option (duration applies to registrations and subscriptions, but not to entity attributes). However, it would be easy to implement at client side: you can have an attribute named X_last_update to store the last update time of the X attribute and check regularly for attributes which last update is two old to be deleted.
I have a column Quantity in a table Inventory of MySQL which gets updated from multiple sources. I needed to maintain a track in the table on another column called QuantityLog on the last updated time of the Quantity and the source which did it. Something like this should be the content of QuantityLog column (Text type) (only the latest update details is required):
<Log>
<UpdateTime>2015-02-23 12:00:01 PM</UpdateTime>
<Source> Feeder application</Source>
</Log>
I am aware of how to do it using trigger if only the update time is required. However, with the trigger approach is there any other mechanism to get the source and use this too?
Do note pls that I am trying to perform this via triggers only as any other mechanisms of using my application to do this will require me to change in all applications that make this change and I am not inclined to do that.
There is no way MySql can know the "feeder application", unless there is a variable or table filled with that value. If you have this, it is easy to create a trigger that updates this info into the Inventory table on each change of the Quantity field.
However, if your applications use unique mysql users to connect to the database, you can of course use the CURRENT_USER() built in function inside your TRIGGER. Alternatively, CONNECTION_ID() might be helpful when tracking who did what. For example, you can create a new table that logs the connection id of your application. In that table you could write the application name, the PID and other stuff. Of course this would mean to change our application a bit by adding the appropriate insert statement after a connection is established. The overhead should be small, since usually connections are held in pools and do not get re-created all the time.
I am working on a project and would like a notification each time a new record is added to a specific table in a MySQL DB. I would like a small popup to be displayed each time a new record is added, but without refreshing the page. I heard AJAX was the way to go, but am not very familiar with it.
Nope it won't. The database won't tell you when a record is inserted. You can use AJAX to send a request to the server. Then, that server can query for changes. It can send a response indicating whether there is a change. Then, the AJAX request's response handler can show a message accordingly.
But implementing this will cause quite a load on both the webserver and the database server. So if you do this, choose the timing wisely. Don't execute this procedure 10 times a second or you'll kill your server as soon as you hit a 100 visitors.
To solve your problem, break it up in two pieces:
1. Get the actual AJAX request to work. Let the server return dummy values and try to handle them correctly. Hint: Use JQuery.ajax (or even JQuery.get) to ease your life.
2. Get the server to query for changes. If you want to monitor a single table this can easily be done. Add a timestamp column to the table if you don't already have one. You can configure it so that it will be updated each time the table is updates. Then, query for the highest timestamp. Don't forget to add an index to that column!
You can experiment with other solutions too. Add a trigger that alters a date/time in a different table. That way, the polling only needs to query that single column instead of the 'max' query.
To handle the change correctly, I think its best to let Javascript hold on to the last timestamp. Send the timestamp back in the response. Javascript can compare the timestamp to the last timestamp and show a message if needed. This way, you won't need to keep the timestamps in the session.