Let's say there is a table like the following:
id | number
----|----------
1 | 1
4 | 6
5 | 2
14 | 3
now I need all numbers that are not set between 1 and the highest number that is set. Here the highest number is 6, so I need all numbers that are not set between 1 and 6. Here it is: 4 and 5
But how can I achieve this with spring and hibernate jpa with a MySQL database?
The highest number is easy. Sort numbers DESC and then the first one. But then return all missing numbers that are not in the database? Is this possible?
One way: select each number that is smaller then the highest number and check if the returned object is null. So first check 5, then 4, then 3, ... But this is of course very slow on big databases.
So another idea was: get all numbers that are set and get the missing numbers on java side with the difference of two lists (one list with the numbers out of the database, the other list with the numbers from 1 to the highest number of the database). But on big databases it is also dumb to get everything. (Let's say, there are 1 million entries and only one number is missing.)
The third idea: something like select where number NULL would be perfect. But for this the database would have to be initialized with all possible numbers ever there. So that is also not possible.
Is there a possible way? Am I overseeing something?
Related
I'm not sure if SSRS is dumb, or I am (I'm leaning towards both).
I have a dataset that (as a result of joins etc) has some columns with the same values duplicated across every row (fairly standard database stuff):
rid cnt bid flg1 flg2
-------------------------------
4 2882 1 17 3
5 2784 1 17 3
6 1293 1 17 3
18 9288 2 4 9
20 762 2 4 9
Reporting based on cnt is straightforward enough. I can also make a tablix that shows the following:
bid flg1 flg2
------------------
1 17 3
2 4 9
(Where the tablix is grouped by Fields!bid.Value and the columns are just Fields!flg1.Value and Fields!flg2.Value respectively.)
What I can't figure out is how to display the sum of these values -- specifically I want to show that the sum of flg1 is 21 and the sum of flg2 is 12 -- not the sum of every row in the dataset (counting each value more than once).
(Note that I'm not looking for a sum of distinct values, as they may not be unique. I want a sum of one value from each bid group, because it's from a table join so they will always have the same value.)
If possible, I'd also like to be able to do a similar calculation at the top level of the report (not in any tablix); although I'd settle for hiding the detail row if that's the only way.
Obviously, Sum(Fields!flg1.Value) isn't the answer, as this either returns 51 (if on the first row inside the group) or 59 (if outside it).
I also tried Sum(Fields!flg1.Value, "bid") but this wasn't considered a valid scope.
I also tried Sum(First(Fields!flg1.Value, "bid")) but apparently you're not allowed to sum first values for some weird reason (and may have had the same scope problem anyway).
Using Sum(Max(Fields!flg1.Value, "bid")) does work, but feels wrong. Is there a better way to do this?
(Related: is there a good way to save the result of that calculation so that I can later also show a Sum of those totals without an even hairier expression?)
There are two basic ways to do this.
Do what you have already done (Sum(Max(Fields!flg1.Value, "bid")))
Sum the rendered values. To do this check the name of the cell containing the data you want (check it's properties) and then use something like =SUM(ReportItems!flg1.Value) where flg1 is the name of the textbox, which is not necessarily always the same name as the field.
Supposing I have 1000 numbers from 1 -> 1000, and a user can have any of the 1000 combination (eg: 4, 25, 353..).
How can I efficiently store that combination in a MySQL DB.
What I thought. I can use the power of 2, and store each number in a really large int, like:
1 -> 01
2 -> 10
4 -> 100
etc.
So if I happen to get the number 6 (110) I know the user has the combination of numbers 2, 4 (2 | 4 = 6) .
So we can have 2^1000 combinations, 125byte. But that is not efficient at all since bigint has 8bytes and I cant store
that in MySQL without using vachars etc. Nodejs cant handle that big number either (and I dont as well) with 2^53-1 being the max.
Why I am asking this question; can I do the above with base 10 instead of 2 and minimize the max bytes that the int can be. That was silly and I think making it to base10 or another base out of 2 changes nothing.
Edit: Additional thoughts;
So one possible solution is to make them in sets of 16digit numbers then convert them to strings concat them with a delimiter, and store that instead of numbers. (Potentially replace multiple 1's or 0's with a certain character to make it even smaller. Though I have a feeling that falls into the compression fields, but nothing better has come to my mind.)
Based on your question I am assuming you are optimizing for space
If most users have many numbers from the set then 125 bytes the way you described is the best you can do. You can store that in a BINARY(125) column though. In Node.js you could just a Buffer (you could use a plain string but should use a Buffer) to operate on the 125 byte bit-field.
If most users have only a few elements in the set then it will take less space to have a separate table with two columns such as:
user_id | has_element (SMALLINT)
---------------------
1 | 4
1 | 25
1 | 353
2 | 7
2 | 25
2 | 512
2 | 756
2 | 877
This will also make queries cleaner and more efficient for doing simple queries like SELECT user_id FROM user_elements WHERE has_element = 25;. You should probably add an index on has_element if you do queries like that to make them many times more efficient than storing a bitfield in a column.
I have some columns in mysql table with field vaues are seperated with commas. fields like IP address and running_port_ids, dns_range or subnet etc. running a cron to check every hour whether the ports are used or not on the appliance. if ports are used against each appliance running_port_ids(like 2,3,7) are inserted with comma seperated values.
How to process the data so that i can get a reports which ports are less used (i have a static list of port ids) in ascending order like below by grouping of address, running_port_ids and insert date for a date range of one month.
address port usage%
10.2.1.3 3 1
10.3.21.22 2 20
there are thousands of record now in the table with comma seperated running_port_ids. is there any methods available in MySql to do this?
Any help much appreciated.
If you can convert your data model to a n:m relation (or "link table"), i.e. normalize your data model, this is pretty easy using grouping (or "aggregate") functions. So I'd advise to revise your data model and introduce a table containing one row for each of the ports, in stead of storing this de-normalized in a text column.
A typical example would be: "student has many classes", and a property of this relation is "attendance":
Student
id name
1 John
2 Jane
Course
id name
1 Engineering
2 Databases
Class
id courseid date room
1 1 2015-08-05 10:00:00 301
2 1 2015-08-13 10:00:00 301
3 1 2015-09-03 10:00:00 301
StudentClass
studentid classid attendance
1 1 TRUE
1 2 FALSE
1 3 NULL
2 1 TRUE
2 2 TRUE
2 3 NULL
In this case, you can see the relation between student and class is normalized, i.e. every other value is stored vertically in stead of horizontally. This way, you can easily query things like "How many classes did John miss?" or "How many students did not miss any class". NULL in the example shows that we can not yet tell anything about the attendance (as the date is in the future), but we do know that they should attend.
This is the way you should keep track of properties of a relation between two things. I can't really make out what you're trying to build, but I'm pretty sure you need a similar model.
Hope this helps.
Let's say I have this strings in a MySQL table:
id | hash
1 | 462a276e262067573e553b5f6a2b4a323e35272d3c6b6227417c4f2654
2 | 5c2670355b6e503f39427a435a423d6d4c7c5156344c336c6c244a7234
3 | 35785c5f45373c495b70522452564b6f4531792b275e40642854772764
...
millions of records !
Now I have a set of substrings (6 character size), for example this:
["76e262", "435a42", "75e406", "95b705", "344c33"]
What I want is to know how many of these substrings are in each string, so the result could be:
id | matches
63 | 5
34 | 5
123 | 3
153 | 3
13 | 2
9 | 1
How can achieve this in a fast way ?
Real numbers and sizes are:
1) Table with 100.000/200.000 hashes
2) Main Hash size: 256 bytes
3) Substring of mini-hashes: 16 of 32 each one
NOTE: I'd like to avoid the "%LIKE%" since it's 16 likes for each row, and millions rows
You can accomplish this by using the Aho-Corasick algorithm: http://en.wikipedia.org/wiki/Aho%E2%80%93Corasick_string_matching_algorithm
MySQL doesn't have a function for that, so you'd need to write your own or consider using a language like java or c to massage the data.
How about a different approach?
You could also consider having a shifting mechanism for your data and the check on the shifting. For example, if your key is 462a276e262067573e553b5f6a2b4a323e35272d3c6b6227417c4f2654 and you know that your hash will have 58 chars, then you would have these variations:
62a276e262067573e553b5f6a2b4a323e35272d3c6b6227417c4f26544
2a276e262067573e553b5f6a2b4a323e35272d3c6b6227417c4f265446
a276e262067573e553b5f6a2b4a323e35272d3c6b6227417c4f2654462
276e262067573e553b5f6a2b4a323e35272d3c6b6227417c4f2654462a
...
Each one of these would be in a column, every one of them would be indexed.
So your query would be simply:
Select * from table where hash like "a27e262%" or s1 like "a27e262%" ...
Note that this would be MUCH faster than LIKE "%value%" as the column is indexed and the LIKE is only checking the begins with.
There are many disadvantages to this solutions: space required for the extra columns, insertion and update time would increase because of the time calculating the shifted columns, and time required to process the result of the select. But you wouldn't need to implement the algorithm in mysql.
You could also require that the minimum length of the string being searched is 6 chars, so you won't need to shift the whole string, only to keep the first 6 digits. If a match is found then you keep looking for the next 6 digits on the next match.
I would like help with sql query code to push the consequent data in a specific column down by a row.
For example in a random table like the following,
x column y column
6 6
9 4
89 30
34 15
the results should be "pushed" down a row, meaning
x column y column
6 null or 0 (preferably)
9 6
89 4
34 30
SQL tables have no inherent concept of ordering. Hence, the concept of "next row" does not make sense.
Your example has no column that specifies the order for the rows. There is no definition of next. So, what you want to do cannot be done.
I am not aware of a simple way to do this with the way you are showing the table being formatted. If your perhaps added two consecutively numbered integer fields that provide row number and row number + 1 values, you could join the table to itself and get that information.
After taking a backup of you table:
Make a PHP function that will:
- Load all values of Y into an array
- Set Y = 0 (MYSQL UPDATE)
- load the values back from PHP array to MYSQL