Lead function for missing records - mysql

Question

I am using below query
select id,
number_sequ,
startvalue
lead(startvalue,1,0) over (partition by id order by number_sequ) AS End_value
from mytable
to populate the following output
id number_sequ startvalue End_value
---- ----- ---------- -----------
AAA 1 30 20
AAA 2 20 10
AAA 4 10 15
AAA 5 15 0
BBB 1 12 23
BBB 3 23 34
BBB 4 34 0
But there are missing records in sequence
id number_sequ startvalue End_value
---- ----- ---------- -----------
AAA 3
BBB 2
I have tried different ways to find out missing numbers in Sequence and try to insert with 0 values. after that i can use lead function. unable to find out efficient way
INSERT INTO mytable (id, number_sequ, startvalue)
select id ,number_sequ ,'0'
from mytable
where (some condition to specify missing data)
Can any one help me to solve above issue.

Answer 1

You can get the missing values with the following approach: generate all possible values and then filter out the ones that exist.
select i.id, n.n, 0 as start_value
from (select id, min(number_seq) as min_ns, max(number_seq) as max_ns
from mytable
group by id
) i join
(select row_number() over (partition by number_seq) as n
from mytable
) n
on n.n <= i.max_ns left join -- just a bunch of numbers
mytable t
on t.id = i.id and
t.number_seq = n.n
where t.id is null;
You can pop the insert before the select to insert these values into your table.
Note that this generates the sequence numbers you need using the original data. So it assumes that your table has enough rows for the numbers you need.

Answer 2

If the missing values are always in between existing values, you can find the gaps using the Snowflake's JavaScript UDTFs
For example, here's a function that finds gaps in a sequence, and then we use it to generate "empty" rows:
create or replace table x(id int, seq int, startVal int) as select * from
values(1,1,11),(1,2,12),(1,4,14),(2,2,22),(2,5,25);
CREATE OR REPLACE FUNCTION find_gaps(SEQ float)
RETURNS TABLE (GAP float)
LANGUAGE JAVASCRIPT
AS '
{
initialize: function(argumentInfo, context) {
this.lastRow = null;
},
processRow: function (row, rowWriter, context) {
let curRow = row.SEQ;
if (this.lastRow == null || this.lastRow + 1 == curRow) {
this.lastRow = curRow;
} else {
while (this.lastRow + 1 < curRow) {
this.lastRow++;
rowWriter.writeRow({GAP: this.lastRow});
}
}
}
}'
;
select id, seq, startVal from x
union all
select id, gap, 0 from x,
table(find_gaps(seq::float)
over (partition by id order by seq));
----+-----+----------+
ID | SEQ | STARTVAL |
----+-----+----------+
1 | 1 | 11 |
1 | 2 | 12 |
1 | 4 | 14 |
2 | 2 | 22 |
2 | 5 | 25 |
2 | 3 | 0 |
2 | 4 | 0 |
1 | 3 | 0 |
----+-----+----------+
You can use variations of this function also e.g. if you know the range of your values per id, just feed it expected min/max as well. Also, you might need something special if your input contains NULL values (but then - what should be the result? :))

Answer 3

Apart from the suggested solutions, if you still want to stick to Lead function,
Lead function analyses data that has values, the result of it can have null values based on partition but the data it uses for analyses should have values. In my view, What you did is right to include the missing sequence in your result.
http://www.mysqltutorial.org/mysql-window-functions/mysql-lead-function/