I need to store a set of numbers in a MySQL database. I need some help to determine the best table structure to use.
There are 20 numbers that will be stored in each row, along with an ID. The numbers can range from 1 - 80 and there are no repeats in this series of numbers.
Initially I created a table structure with 21 columns, an ID and 20 columns that store each individual number.
Id | Num1 | Num2 | Num3 | Num4 | Num5 | etc.. |
----------------------------------------------------------
0001 | 1 | 4 | 15 | 22 | 39 | 43 |
0002 | 3 | 5 | 22 | 43 | 55 | 58 |
0003 | 1 | 3 | 5 | 6 | 15 | 26 |
I've also thought of a table with 81 columns, an ID and 80 boolean columns that would represent each individual number.
Id | 1 | 2 | 3 | 4 | 5 | etc.. |
----------------------------------------------------------
0001 | True | False | False | True | True | False |
0002 | False | False | True | False | True | False |
0003 | True | False | True | False | True | True |
Can anyone give some advice to the pros and cons of each table structure, and which would be easier to use when searching this table.
For example, we would need to search for every row that contains 1,2,5,66, and 79.
Or every row that contains 16,33, and 4.
Any guidence would be appreciated.
What you're looking for is called database normalization; a way to organize data that prevents duplication and anomalies (like changing one record inadvertently changing another record).
Higher-normal forms depend on the meaning of your data, which you have not told us, but to start you should avoid ordered or indeterminate columns (like Num1, Num2, ...) and split your columns into rows:
ID Num
0001 1
0001 4
0001 15
...
0002 3
0002 5
...
In general, any time you find yourself adding a bunch of columns that depend on their position you are making a mistake. SQL has many functions for aggregating, combining, sorting, and reporting on rows. Use the features of SQL to produce the results you want; don't try to make your database schema look like the final printed report.
In answer to your comment, a query that returns only IDs that have Nums 1, 4, and 15, and no other ID:
select ID from YourTable
where Num in (1, 4, 15)
group by ID
having Count(ID) = 3
If Nums can be duplicated you will want something like having count(distinct ID). If you can have different counts of Nums to match you will have to create a temporary table of Nums to match and use having count(ID) = (select Count(Num) from TemporaryTable).
Note that SQL Server already has a master..spt_values table of integers to use in such situations; I do not know if MySql has such a thing, but they are easy to generate if you need one.
Related
Does anyone know how to find ranges that overlap, using MySQL? Essentially, as seen on table below (just for illustrating the problem as the actual table contains 1000+ ranges), I am trying to fetch all ranges that overlap inside of a table.
Thanks!
RANGES
| count | Begin | End | Comment |
| 1 | 1001 | 1095 | overlaps with ranges 2, 3 |
| 2 | 1005 | 1030 | overlaps with ranges 1, 3 |
| 3 | 1017 | 1020 | overlaps with ranges 1, 2 |
| 4 | 1110 | 1125 | no overlap |
One method is a self join and aggregation:
select r1.count, r1.begin, r1.end,
group_concat(r2.count order by r2.count) as overlaps
from ranges r1 left join
ranges r2
on r1.end >= r2.begin and
r1.begin <= r2.end and
r1.count <> r2.count
group by r1.count, r1.begin, r1.end;
On a table with 1000 rows, this will not be fast, but it should be doable. You may want to validate the logic on a smaller table.
This assumes that count is really a unique identifier for each row.
Note that count and end are poor choices for column names because they are SQL keywords.
Here is a db<>fiddle.
I have a special scenario to fetch "unique" row.
Let's say the database is like below
| id | userid | value | others |
|----|--------|-------|---------|
| 1 | 111 | 10 | string1 |
| 2 | 112 | 30 | string2 |
| 3 | 112 | 30 | string3 |
| 4 | 113 | 50 | string4 |
what I want to achieve is to fetch the unique rows based on the "userid" so I'am able to sum all values.
the expect output row can be either id: 1 2 4 or 1 3 4 (both is acceptable for this special case because same id guarantees same value, or in general, get just one row from those row with same userid. ), so the sum will be 90.
Note: DB is extended from Eloquent\model
My old approach is to get DB::unique('userid'); then for each userid DB::where('userid', $id)->value('value'), add the result to sum; I just believe there might be a better approach.
There is Illuminate\Support\Facades\DB in Laravel, it can return the Query Builder. Not recommend to use a model that is named DB.
So just change another name.
By the way, for Eloquent\Model, you can use groupBy and sum too:
Model::groupBy('user_id')->sum('value');
I want to sort the user record according to city (chosen from the drop-down list). like if I pass city_id 22 in my query then i want all the row first which are having city_ids 22 then the rest of the rows.
I know WHERE find_in_set('22',city_ids) will give me the correct result but it will not return the all rows so I want to achieve it using some ORDER BY .
I have tried ORDER BY FIND_IN_SET('22',city_ids) but its not working. How do I fix this, any best way?
User Table:
Id Name city_ids
1 AAAAA 10,22,30
2 BBBBB 11,28
3 CCCCC 15,22,44
4 DDDDD 19,99,
5 EEEEE 55,27,22
Want Sorted Output like below:
Id Name city_ids
1 AAAAA 10,22,30
3 CCCCC 15,22,44
5 EEEEE 55,27,22
2 BBBBB 11,28
4 DDDDD 19,99,
You can do:
ORDER BY (FIND_IN_SET('22', city_ids) > 0) DESC
This puts matches first.
Then you should fix your data model. It is broken, broken, broken. Storing lists of ids in a string is wrong for many reasons:
The data types are (presumably) wrong. The ids are numbers and should not be stored as strings.
Storing multiple values in a column is not the SQL way to store things.
Ids should have properly declared foreign key relationships, which you cannot declare.
SQL does not have very good functions for processing strings.
The resulting queries cannot take advantage of indexes or partitioning, impeding performance.
SQL has this really great data structure for storing lists of things. It is called a table, not a string column.
The expression:
FIND_IN_SET('22', city_ids) > 0
will return 1 for all rows where '22' exists in column city_ids and 0 for the others.
So, after that you need add one more level for sorting by id ascending:
ORDER BY
FIND_IN_SET('22', city_ids) > 0 DESC,
id
See the demo.
Results:
| Id | Name | city_ids |
| --- | ----- | -------- |
| 1 | AAAAA | 10,22,30 |
| 3 | CCCCC | 15,22,44 |
| 5 | EEEEE | 55,27,22 |
| 2 | BBBBB | 11,28 |
| 4 | DDDDD | 19,99 |
In my table I have two columns "sku" and "fitment". The sku represents a part and the fitment represents all the vehicles this part will fit on. The problem is, in the fitment cells, there could be up to 20 vehicles in there, separated by ^^. For example
**sku -- fitment**
part1 -- Vehichle 1 information ^^ vehichle 2 information ^^ vehichle 3 etc
I am looking to split the cells in the fitment column, so it would look like this:
**sku -- fitment**
part1 -- Vehicle 1 information
part1 -- Vehicle 2 information
part1 -- Vehicle 3 information
Is this possible to do? And if so, would a mySQL db be able to handle hundreds of thousands of items "splitting" like this? I imagine it would turn my db of around 250k lines to about 20million lines. Any help is appreciated!
Also a little more background, this is going to be used for a drill down search function so I would be able to match up parts to vehicles (year, make, model, etc) so if you have a better solution, I am all ears.
Thanks
Possible duplicate of this: Split value from one field to two
Unfortunately, MySQL does not feature a split string function. As in the link above indicates there are User-defined Split function's.
A more verbose version to fetch the data can be the following:
SELECT SUBSTRING_INDEX(SUBSTRING_INDEX(fitment, '^^', 1), '^^', -1) as fitmentvehicle1,
SUBSTRING_INDEX(SUBSTRING_INDEX(fitment, '^^', 2), '^^', -1) as fitmentvehicle2
....
SUBSTRING_INDEX(SUBSTRING_INDEX(fitment, '^^', n), '^^', -1) as fitmentvehiclen
FROM table_name;
Since your requirement asks for a normalized format (i.e. not separated by ^^) to be retrieved, it is always better to store it in that way in the first place. And w.r.t the DB size bloat up, you might want to look into possibilities of archiving older data and deleting the same from the table.
Also, you should partition your table using an efficient partitioning strategy based on your requirement. It would be more easier to archive and truncate a partition of the table itself, instead of row by row.
E.g.
DROP TABLE IF EXISTS my_table;
CREATE TABLE my_table (user_id INT NOT NULL PRIMARY KEY,stuff VARCHAR(50) NOT NULL);
INSERT INTO my_table VALUES (101,'1,2,3'),(102,'3,4'),(103,'4,5,6');
SELECT *
FROM my_table;
+---------+-------+
| user_id | stuff |
+---------+-------+
| 101 | 1,2,3 |
| 102 | 3,4 |
| 103 | 4,5,6 |
+---------+-------+
SELECT * FROM ints;
+---+
| i |
+---+
| 0 |
| 1 |
| 2 |
| 3 |
| 4 |
| 5 |
| 6 |
| 7 |
| 8 |
| 9 |
+---+
SELECT DISTINCT user_id
, SUBSTRING_INDEX(SUBSTRING_INDEX(stuff,',',i2.i*10+i1.i+1),',',-1) x
FROM my_table
, ints i1
, ints i2
ORDER
BY user_id,x;
+---------+---+
| user_id | x |
+---------+---+
| 101 | 1 |
| 101 | 2 |
| 101 | 3 |
| 102 | 3 |
| 102 | 4 |
| 103 | 4 |
| 103 | 5 |
| 103 | 6 |
+---------+---+
I've checked out a few of the stackoverflow questions and there are similar questions, but didn't quite put my fingers on this one.
If you have a table like this:
uid cat_uid itm_uid
1 1 4
2 1 5
3 2 6
4 2 7
5 3 8
6 3 9
where the uid column in auto_incremented and the cat_uid references a
category of relevance to filter on and the itm_uid values are the one
we're seeking
I would like to get a result set that contains the following sample results:
array (
0 => array (1 => array(4,5)),
1 => array (2 => array(6,7)),
2 => array (3 => array(8,9))
)
An example issue is - select 2 records from each category (however many categories there may be) and make sure they are the last 2 entries by uid in those categories.
I'm not sure how to structure the question to allow an answer, and any hints on a method for the solution would be welcome!
EDIT:
This wasn't a very clear question, so let me extend the scenario to something more tangible.
I have a set of records being entered into categories and I would like to select, with as few queries as possible, the latest 2 records entered per category, so that when I list out the contents of those categories, I will have at least 2 records per category (assuming that there are 2 or more already in the database). A similar query was in place that selected the last 100 records and filtered them into categories, but for small numbers of categories with some being updated faster than others can lead to having the top 100 not consisting of members from every category, so to try to resolve that, I was looking for a way to select 2 records from each category (or N-records assuming it's the same per-category) and for those 2 records to be the last entered. A date field is available to sort on, but the itm_uid itself could be used to indicate inserted order.
SELECT cat_uid, itm_uid,
IF( #cat = cat_uid, #cat_row := #cat_row + 1, #cat_row := 0 ) AS cat_row,
#cat := cat_uid
FROM my_table
JOIN (SELECT #cat_row := 0, #cat := 0) AS init
HAVING cat_row < 2
ORDER BY cat_uid, uid DESC
You will have two extra columns in the results, just ignore them.
This is the logic:
We sort the table by cat_uid, uid descending, then we start from the top and give each row a "row number" (cat_row) we reset this row number to zero whenever cat_uid changes:
---------------------------------------
| uid | cat_uid | itm_uid | cat_row |
| 45 | 4 | 34 | 0 |
| 33 | 4 | 54 | 1 |
| 31 | 4 | 12 | 2 |
| 12 | 4 | 51 | 3 |
| 56 | 6 | 11 | 0 |
| 20 | 6 | 64 | 1 |
| 16 | 6 | 76 | 2 |
| ... | ... | ... | ... |
---------------------------------------
now if we keep only the rows that have cat_row < 2 we get the results we want:
---------------------------------------
| uid | cat_uid | itm_uid | cat_row |
| 45 | 4 | 34 | 0 |
| 33 | 4 | 54 | 1 |
| 56 | 6 | 11 | 0 |
| 20 | 6 | 64 | 1 |
| ... | ... | ... | ... |
---------------------------------------
This is called an adjacent tree model or a parent-child tree model. It's one of the simplier tree model where there is only 1 pointer or 1 leaf. You would solve your query with a recursion or using a Self Join. Sadly MySQL doesn't support recursive queries, maybe it's working with prepared statements. I want to suggest you an Self Join. With a Self Join you can get all the rows from the right side and the left side with a special condition.
select t1.cat_uid, t2.cat_uid, t1.itm_uid, t2.itm_uid From t1 Inner Join t2 On t1.cat_uid = t2.cat_uid