Imagine we have 100 IoT devices collecting data every 5 seconds. We have a central table with 100 entries -- one per IoT device. The entries have columns such as name, battery, longitude, latitude.
I now want a column that points me to a new records table. There is a records table for each IoT device. This table holds the time-value pairs for the measurements that this specific device collects.
My questions are:
1- Is this the right way of thinking?
2- Can I automatically generate these tables as I insert entries to the central table?
3- Is there a pointer notion that enables me to be redirected to a table?
I am new to SQL, and I use MariaDB and SQLAlchemy.
You should not create separate table for each device. You need to create single table for all measurements:
create table devices (
id int unsigned primary key auto_increment,
name varchar(64),
battery varchar(64),
longitude decimal(5, 2),
latitude decimal(5, 2)
);
create table measurements (
device_id int unsigned references devices(id) on delete cascade ,
measurement_type int,
value int,
index device_idx (device_id)
);
SQL online
In high-load case you can use table partitioning
Related
I have a main table called results. E.g.
CREATE TABLE results (
r_id INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
r_date DATE NOT NULL,
system_id INT NOT NULL,
FOREIGN KEY (system_id) REFERENCES systems(s_id) ON UPDATE CASCADE ON DELETE CASCADE
);
The systems table as:
CREATE TABLE systems (
s_id INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
system_name VARCHAR(50) NOT NULL UNIQUE
);
I'm writing a program in Python with MySQL connector. Is there a way to add data to the systems table and then auto assign the generated s_id to the results table?
I know I could INSERT into systems, then do another call to that table to see what the ID is for the s_name, to add to the results table but I thought there might be quirk in SQL that I'm not aware of to make life easier with less calls to the DB?
You could do what you describe in a trigger like this:
CREATE TRIGGER t AFTER INSERT ON systems
FOR EACH ROW
INSERT INTO results SET r_date = NOW(), system_id = NEW.s_id;
This is possible only because the columns of your results table are easy to fill in from the data the trigger has access to. The auto-increment fills itself in, and no additional columns need to be filled in. If you had more columns in the results table, this would be harder.
You should read more about triggers:
https://dev.mysql.com/doc/refman/8.0/en/create-trigger.html
https://dev.mysql.com/doc/refman/8.0/en/triggers.html
I have a view which describes some statistics of another table, e.g.:
create table people (
playerid INT not null auto_increment,
name VARCHAR(70),
primarykey (playerid)
);
create view personinfo_oranges (playerid, number_of_oranges) as
select playerid, count(orangeid) from oranges
...
;
Unfortunately, there's loads of data in the tables that person_info looks at (e.g. oranges) and the view is very slow to load. This is apparently a known issues with views. So I created another table that stores the data for some of the people (e.g. when they become allergic to oranges):
create table person_summary (
playerid INT not null,
number_of_oranges INT,
foreign key (playerid) references players(playerid)
);
Ideally I would like to create a view that searches person_summary, and, if not found, looks in personinfo_oranges. I could do this logic in my Model layer, but how can I do it in MySQL?
A my-sql database table is having millions of data records.This table consists of a primary key [say user id],serial number [can have duplicates] and some other columns which allows null values.
Eg: say the schema is
CREATE TABLE IF NOT EXISTS SAMPLE_TABLE (
USER_ID INTEGER NOT NULL,
SERIAL_NO NOT NULL,
DESCRIPTION VARCHAR(100),
PRIMARY KEY (USER_ID)
)ENGINE INNODB;
Now I want to search a data row,based on the serial number.
I tried first adding a unique index including both columns [user id and serial no.] as
CREATE UNIQUE INDEX INDEX_USERS ON U=SAMPLE_TABLE (USER_ID,SERIAL_NO);
and then search for the data query based on serial number as below;
SELECT * FROM SAMPLE_TABLE WHERE SERIAL_NO=?
But it didn't success and I'm getting OOM error in mysql server side when I execute above select query. Appreciate any help on this.
You should not have added user_id intobthecindex you created. You just need an index on serial_no for that query.
If you provides necessary codes,it would be better than given explainations..However first you should find the id references to seraial number,then search the column corresponding to id
I created a table in MySQL. I want to store a name, a surname and a vector of doubles, but the problem is: how do I create a vector column in MySQL? My vector contains 130 elements.
There are essentially two ways you can do that.
A simple one is to create a LONGBLOB or LONGTEXT field where you will store a serialized version of your vector.
But this is a quite ugly solution from a database modeling perspective because the DBMS is not capable of performing searches or to index the content of those vectors.
The correct way would be to use two tables in a 1-to-many relationship.
It means, you would have a table table_A with the following structure:
CREATE TABLE table_A ( -- records
id INT UNSIGNED NOT NULL AUTO_INCREMENT,
name TEXT,
surname TEXT,
PRIMARY KEY (id)
);
And a table table_B containing the values in the vector and an association with their respective records in table_A:
CREATE TABLE table_B ( -- vector values
parent INT UNSIGNED NOT NULL,
id INT UNSIGNED NOT NULL, -- for indexing, in case the order of the vector elements matter
value TEXT,
PRIMARY KEY (parent, id),
FOREIGN KEY (parent) REFERENCES table_A (id) ON DELETE CASCADE ON UPDATE CASCADE
);
Working exemple: http://sqlfiddle.com/#!2/79521/2
With this format you are capable of allowing the DBMS to perform searches and manage the values of the vectors.
I suggest you to take a look at the JSON data type. This way you can store your vector in a more efficient way than text or varchar, and you can access your data directly form MySQL without having to parse the whole thing.
Take a look at this link : https://dev.mysql.com/doc/refman/5.7/en/json.html
You'll just have to store your vector in a JSON form like this [24, 12, 54, 39, ...]
I'm working on designing a new database that will need to handle an enormous amount of data. It will be a data warehouse system, and will thus be organized around a central hub table:
create table hub(id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY,
date_time DATETIME NOT NULL, bit_of_data INT NOT NULL);
When this table grows very large, it seems that it will be necessary to partition it based on the 'date_time' column, with each partition being, say, one month of data. However, there will also be another table:
create table other_data(id BIGINT NOT NULL PRIMARY KEY,
more_data INT NOT NULL, FOREIGN KEY(id) REFERENCES hub(id));
This second table will contain records for about 90% of the ids that appear in the main 'hub' table. I'd like to partition the 'other_data' table as well as the 'hub' table, and have the partitions basically match up with each other. Is there any way to partition the 'hub' table on a date range, and then also partition the 'other_data' table on the same date range?
Thanks!
This can be done only by adding a (redudant) date column in the other_data table.