Is is possible to update data or insert data from grafana to mysql. I have a requirement to insert/update information in mysql using a UI. Now I am already using grafana so wanted to know if there is any way we can use grafana to update or insert information
There is no input panel for taking user input and inserting that data to MySQL or any other. (Grafana v7.0)
For very minimalist input data you can use grafana variables as a hack.
Create example table for storing input in MySQL
CREATE TABLE `grafana_variable` (`variable` VARCHAR(64)) ENGINE = InnoDb;
In Grafana dashboard click settings icon:
then click:
add variable:
return to dashboard and you should see at the top:
create a new visualization panel or add another query in any already existing panel and add SQL query:
INSERT INTO `grafana_variable` VALUES ('${myvar}');
now every time you type into that field
and click away from it or use ie tab key the data will be inserted to the MySQL database.
mysql> select * from grafana_variable;
+---------------------+
| variable |
+---------------------+
| this is just a test |
+---------------------+
1 row in set (0.08 sec)
The downside of this hack is that the visualization tile will have in its upper left corner error icon:
with information (when you hover over it):
Found no column named time or time_sec
because the sql query of the visualization tile is inserting data instead of selecting it from database therefore there is no time or time_sec data.
Grafana also does not handle multiple MySQL statements so you cannot repair that by adding before or after the INSERT INTO ... statement second one with SELECT FROM. Maybe this could be somehow patched by using sub queries or something similar but I did not investigate that further.
Because the error as above can be due to multiple reasons not related to the hack it will be better to use the hack query in separate visualization tile - the one that you will remember (or even give descriptive panel tile) so you know that the error there is nothing unusual.
See grafana varialbes for more info about variables.
There is also useful list of already existing variables you can use same way as myvar I created. See section MACROS here For eg:
$__timeFrom()
and
$__timeTo()
have the start and the end of displayed time range.
Permissions and Security
The whole hack works because the MySQL user that grafana uses is allowed to execute INSERT statement, but if grafana is able to execute ANY statement then make sure that MySQL user that grafana uses is not allowed for example to execute statements like DROP ... or any other that is not related to INSERT data into the grafana_variable table as in example.
If you use MySQL as datasource for displaying data the grafana user should also be able to execute SELECT statements.
But nothing else than that.
Read about MySQL user premissions
Must hide Update as temporary table
I am working with postgres and Grafana. Probably it is quite similar.
With postgres you have to hide the update like this:
WITH UPDAT_TEMP AS (UPDATE MyTable SET MY_ROW='MY_TEXT' WHERE ID=99)
SELECT 1;
If you want to update various rows, you have to put various temporary tables.
The
Update my_table (rows) values(...)
does not work for me under Grafana.
Therefore:
WITH UPDAT_TEMP AS (UPDATE MyTable SET MY_ROW='MY_TEXT' WHERE ID=99),
WITH UPDAT_TEMP2 AS (UPDATE MyTable SET MY_ROW2='MY_TEXT2' WHERE ID=99)
SELECT 1;
Related
Sorry for asking this question, but I am a beginner in SQL, my colleague at work build a view, which I need as datasource for a report, however since this view is based on several other views it takes like 45 minutes to execute the query. This is way to long. Therefore I created a table from that view, initial execution time is the same, but once in place it executes in seconds.
In Microsoft SQL Server 2014 I used the following query:
select *
into [dbo].[MAT_v_demnew_daily_am_all_data]
from [dbo].[v_demnew_daily_am]
This works fine, but since the view is updated daily I also need to refresh the table everyday. When I now execute the above mentioned query I get the message that the table already exists.
That's why I tried to use 'insert' in this case:
insert into [dbo].[MAT_v_demnew_daily_am_all_data]
select *
from [dbo].[v_demnew_daily_am]
Here I have the problem that it not only inserts the additional data but also the already existing data, so in the end I have duplicates.
As a workaround I now manually delete the [dbo].MAT_v_demnew_daily_am_all_data] table and then execute the select * into query.
Now I am looking for an easier solution, is it possible to having the table deleted by query and in the same query create a new one by select * into or is it possible to only insert new data from the view to the table so that I don't get duplicates.
Moreover, is it possible to have such SQL statement being executed automatically on a daily basis, maybe by .bat file and windows task scheduler?
I know that the source of all problems is the View and that we should improve that, but looking for a short term solution first.
Thanks so much.
Mathias
Try this:
IF OBJECT_ID('dbo.MAT_v_demnew_daily_am_all_data', 'U') IS NOT NULL
DROP TABLE dbo.MAT_v_demnew_daily_am_all_data
SELECT INTO dbo.MAT_v_demnew_daily_am_all_data FROM dbo.v_demnew_daily_am
This query is reusable on a daily basis.
You can create one stored procedure including this query.
Then you only need to execute the stored procedure.
Updated
Before you create the stored procedure, please check if you have the permission.
Then try:
create procedure [procedure_name]
as
IF OBJECT_ID('dbo.MAT_v_demnew_daily_am_all_data', 'U') IS NOT NULL
DROP TABLE dbo.MAT_v_demnew_daily_am_all_data
SELECT INTO dbo.MAT_v_demnew_daily_am_all_data FROM dbo.v_demnew_daily_am;
After you create it:
EXEC [procedure_name];
I'd like to know if it is possible to create an SQL function that will automatically update my View and add to it new tables in my DB.
My DB consist of multiple tables (same data structure) and are named as follow "MM_DD", now I would like to create a VIEW that joins all this data ( pretty simple , see query below) but I wish to automate the process so every time a new table is added the view will update.
CREATE OR REPLACE VIEW `viewTable` AS
select *,
md5(CONCAT(`columnA`,`columnB`)) AS myPK_id
from `05_06`
union all
select *,
md5(CONCAT(`columnA`,`columnB`)) AS myPK_id
from `05_08`
ect...
What I am doing at the moment is using PHP every time a table is added. It loops through the tables and create / update the view.
select * from information_schema.tables WHERE table_name LIKE '%05%'
Now that I have an array of table names -> create my Query string -> replace view...
Is this possible to do this in SQL?
If so, what is the correct approach?
Write a stored procedure to rebuild your view. You'll need to use what MySQL internally calls "prepared statements" so you can use string manipulation.
You'll still use your SELECT ... FROM information_schema.tables query to drive this stored procedure.
Once you get this working, you can embed it in a mySQL event, and arrange to run it automatically. For example, you could run it at the same time late at night.
Or, you can invoke the stored procedure immediately after you create one of these tables.
I want to replicate certain table from one database into another database in the same server. This tables contain exactly the same fields.
I was considering to use MySQL Replication to replicate that table but some people said that it will increase IO so i find another way to create 3 Trigger (Insert, update and Delete) that will perform exactly the same thing like what i expect.
My Question is, which way is better? Is it using MySQL replication is better even though it's in the same server or using Trigger to replicate the data is better.
Thanks.
I don't know what is your goal, but I got mine getting use of the VIEW functionality.
I had two different applications with separate databases but in the same Mysql server. Application2 needed to get a few data from Application1. In general, this is a trivial situation that you can handle with USE DB1; or USE DB2; as your needing, but my programming framework does not work very well with multiple DBs.
So, lets see my solution...
Here is my select query to retrieve this data:
SELECT id, name FROM DB1.customers;
So, using DB2 as default schema, I've created a VIEW:
USE DB2;
CREATE VIEW app1_customers AS SELECT id, name FROM DB1.customers;
Now I can retrieve this data in DB2 as a regular table with a regular SELECT statement.
SELECT * FROM DB2.app1_customers;
Hope ts useful. BR
Assuming you have two databases on the same server i.e DB1 and DB2 and the table is called tbl1 and it is sitting in DB1 you can query the table like this:
USE DB1;
SELECT * FROM tbl1;
USE DB2;
SELECT * FROM DB1.tbl1;
This way you wont need to copy the data and worry about extra space and extra code. You can query a table in another database on the same server. Replication and triggers are not your answer here. You could also create a view to encapsulate the SQL statement.
Definitely triggers is the way to go. Having another server (slave) will need to spare several MB for installation, logs, cpu and memory usage.
I'd use triggers to keep both tables equal. If you want to create a table with the same columns definition and data use:
USE db2;
CREATE TABLE t1 AS SELECT * FROM db1.t1;
After that, go ahead and create the triggers for Update, Insert and Delete statemetns.
Also you could ALTER the new table to a different engine like MEMORY or add indexes to see if you can improve something.
I have a project where I want to be able to manage several instances of the same database on several people's localhosts. I want each developer to be able to reset their DB back to the canonical origin no matter what state they get their DB into. To this end I maintain a standard database file. It holds the schema using CREATE TABLE table_name IF NOT EXISTS {
However, I want to be able to add to the starting table structures as needed as this project moves along. To do this, I would love to be able to do something like the following ALTER TABLE table_name ADD COLUMN IF NOT EXISTS column_name but that does not seem to exist. I did notice a stored procedure floating around the internet that solves this, but I wanted to know if something simpler is able to achieve the goal I have in mind. Thank you for the time and help.
It won't be possible to do it with plain SQL. Stored procedure should work the best: read information_schema and check if the column is present. If not - execute the alter statement.
One option is to execute your ALTER statement without checking anything:
ALTER TABLE table_name ADD column_name VARCHAR(40);
(change VARCHAR(40) to whatever you need)
If the column didn't already exist, then the statement creates it.
If the column already existed, the statement does nothing and returns an error. Just ignore the error and continue.
Is it possible to use any sort of logic in MySQL without using any procedures? My web hosting does not let me create any procedures so I'm looking for a workaround.
The type of thing I want to do is only add an item to a table if it doesn't already exist. Or add a column to a table if it's not already there. There are some operations that can be done such as CREATE TABLE IF NOT EXISTS and so on, but some operations I require do not have such luxuries :(
I realised late on that my lovely procs won't work and so I tried writing IF/ELSE logic as top-level queries, but for MySQL, IF ELSE blocks only seem to work inside functions/procs and not at the global scope.
Any workarounds greatfully received - I've already asked the hosting to grant me privileges to create procedures but no reply as yet...
I suppose you don't have access to the INFORMATION_SCHEMA either. You can possibly find solutions but it would be better, in my oninion, to:
Change your hosting provider. Seriously. Pay more - if needed - for a MySQL instance that you can configure to your needs. You only have a crippled DBMS if you are not allowed to create procedures and functions.
Posible workarounds for the specific task: You want to add a column if it doesn't exist.
1) Just ALTER TABLE and add the column. If it already exists, you'll get an error. You can catch that error, in your application.
2) (If you have no access to the INFORMATION_SCHEMA) maintain a version of the schema, for your database.
The best solution that I can think of would be to use an additional language with SQL. For example, you can run a query for a specific record, and based on the response that you get, you can conditionally run an INSERT statement.
For inserting a table if it doesn't exist, try using the SHOW TABLES statement and testing whether or not a name exists in the result set.
MySQL supports INSERT IGNORE. and INSERT ... ON DUPLICATE KEY UPDATE.
The following will insert a new row, but only if there is no existing row with id=10. (This assumes that id is defined as a unique or primary key).
INSERT IGNORE INTO my_table (id, col1, col2) values (10, "abc", "def");
The following will insert a new row, but if there is an existing row with id=10 (again, assuming id is unique or primary), the existing row will be updated to hold the new values, instead of inserting a new row.
INSERT INTO my_table (id, col1, col2) values (10, "abc", "def")
ON DUPLICATE KEY UPDATE col1=VALUES(col1), col2=VALUES(col2)
Also, CREATE TABLE supports the IF NOT EXISTS modifier. So you can do something like:
CREATE TABLE IF NOT EXISTS my_table ...
There are many other similar options and modifiers available in MySQL. Check the docs for more.
Originally I created a big script to create or update the database schema, to make it easier to deploy database changes from my local machine to the server.
My script was doing a lot of "if table 'abc' exists and it doesn't have a FK constraint called 'blah'" then create an FK constraint called 'blah' on table 'abc'... and so on.
I now realise it's not actually necessary to check whether a table has a certain column or constraint etc, because I can just maintain a schema-versioning system, and query the DB schema-version when my app starts, or when I navigate to a certain page.
e.g. let's say I want to add a new column to a table. It works like this:
Add a new migration script to the app code, containing the SQL required to add the column to the existing table
Increment the app's schema-version by 1
On app startup, the app queries the DB for the DB's schema-version
If DB schema-version < app schema-version, execute the SQL migration scripts between the two schema-versions, and then update the DB schema-version to be the same as the app
e.g. if the DB's schema-version is 5 and the app version is 8, the app will apply migration scripts 5-6, 6-7 and 7-8 to the DB. These can just be run without having to check anything on the DB side.
The app is therefore solely responsible for updating the DB schema and there's no need for me to ever have to execute schema change scripts on the local or remote DB.
I think it's a better system than the one I was trying to implement for my question.