Here is mysql data
id usr good quant delayed cart_ts
------------------------------------------------------
14 4 1 1 0 20100601235348
13 4 11 1 0 20100601235345
12 4 4 1 0 20100601235335
11 4 1 1 0 20100601235051
10 4 11 1 0 20100601235051
9 4 4 1 0 20100601235051
15 4 2 1 0 20100601235350
16 4 7 1 0 20100602000537
17 4 3 1 0 20100602000610
18 4 3 1 0 20100602000616
19 4 8 1 0 20100602000802
20 4 8 1 0 20100602000806
21 4 8 1 0 20100602000828
22 4 8 1 0 20100602000828
23 4 8 1 0 20100602000828
24 4 8 1 0 20100602000828
25 4 8 1 0 20100602000828
26 4 8 1 0 20100602000829
27 4 8 1 0 20100602000829
28 4 9 1 0 20100602001045
29 4 10 1 0 20100602001046
I need to group fields in witch usr & good has duplicated values with summing quant field
for getting smth like this:
id usr good quant delayed cart_ts
------------------------------------------------------
14 4 1 2 0 20100601235348
13 4 11 2 0 20100601235345
12 4 4 2 0 20100601235335
15 4 2 1 0 20100601235350
16 4 7 1 0 20100602000537
17 4 3 2 0 20100602000610
19 4 8 9 0 20100602000802
28 4 9 1 0 20100602001045
29 4 10 1 0 20100602001046
Which MySQL query I need to do to have this effect?
SELECT id,usr,good,SUM(quant),delayed,cart_ts FROM table GROUP BY usr,good
Related
I have table
AccID
AccName
Parent
Balance
1
A
0
0
2
B
0
0
3
A-1
1
0
4
B-1
2
0
5
A-1-1
3
100
6
A-1-2
3
100
7
B-1-1
4
0
8
B-1-2
7
300
9
B-1-3
7
100
i need help to produce :
AccID
AccName
Parent
SumBalance
1
A
0
200
2
B
0
400
3
A-1
1
200
4
B-1
2
400
5
A-1-1
3
100
6
A-1-2
3
100
7
B-1-1
4
400
8
B-1-2
7
300
9
B-1-3
7
100
using mysql without using With CTE ie i need normal sql that sum the balance
This is the webpage I am scraping: http://laxreports.sportlogiq.com/nll/GS2200.html
Below is the code for the spider I created:
import scrapy
class MatchesSpider(scrapy.Spider):
name = 'matches'
allowed_domains = ['laxreports.sportlogiq.com']
start_urls = ['http://laxreports.sportlogiq.com/nll/GS2200.html']
def parse(self, response):
tables = response.xpath('//table')
print(tables)
table = tables[0].xpath('//tbody')
I see 22 tables that have been selected for this XPath expression but my problem is that I don't fully understand how to select each individual table and extract its contents.
I am a beginner in scrapy and after searching online for a solution all I see is how to select the tables using the class or ID which in this case is not an option.
You can do that using only pandas
Code:
import pandas as pd
dfs = pd.read_html('https://laxreports.sportlogiq.com/nll/GS2200.html')
df = dfs[10]#.to_csv('d.csv', index = False)
print(df)
Output:
0 1 2 3 4 5 6 7 8 9 10 11 12
0 # Name G A +/- PIM S SOFF LB T CT FO TOF
1 2 W.Malcom 0 0 0 0 1 1 1 4 0 - 11:28
2 3 T.Edwards 0 0 -2 2 0 0 8 1 2 7-18 20:28
3 4 J.Sullivan 0 0 -3 2 0 0 3 0 0 - 15:29
4 11 T.Stuart 0 0 -3 0 0 0 4 1 1 - 21:09
5 14 W.Jeffrey 0 1 -1 0 0 0 9 2 1 - 19:17
6 16 R.Lee 2 1 2 0 9 4 6 6 1 - 23:13
7 17 C.Wardle 2 0 1 2 5 3 4 2 2 - 20:55
8 18 R.Hope (A) 0 0 -2 2 0 0 11 0 0 - 22:02
9 20 J.Ruest 3 2 3 0 8 1 3 2 0 - 24:16
10 23 J.Gilles 0 0 -1 0 0 0 4 0 3 - 14:44
11 27 S.Carnegie 0 0 -1 0 0 0 3 0 0 - 12:19
12 37 D.Coates (C) 0 0 0 0 1 0 1 0 0 1-1 2:31
13 51 E.McLaughlin 0 5 2 0 7 3 5 7 0 - 21:41
14 55 D.Kinnear 0 1 2 0 2 0 2 1 0 0-2 10:14
15 67 K.Killen 1 1 0 0 6 1 4 2 0 - 16:42
16 82 J.Cupido (A) 0 1 -1 0 3 0 4 1 0 - 20:52
17 86 J.Lintz 0 1 -1 0 0 0 4 0 1 - 19:26
18 30 T.Carlson 0 0 NaN 0 0 0 0 0 0 - NaN
19 45 D.Ward 0 0 NaN 0 0 0 0 1 0 - NaN
20 NaN Totals: 8 13 NaN 8 42 13 76 30 11 8-21 NaN
Looking on how to update a part of string.
String looks like this: '0 0 0 6 7 2 6 8 6 1 7 5 3 2 4 9 9 4'
What I want is to change first 9 numbers of string to 0.
Any suggestions on how to do in only with MySQL?
Quick and dirty if your string format/length is fixed:
-- Concat with hardcode prefix with tail of the string:
SELECT CONCAT("0 0 0 0 0 0 0 0 0 ", SUBSTRING("0 0 0 6 7 2 6 8 6 1 7 5 3 2 4 9 9 4", 19))
-- Update
UPDATE {TABLE}
SET {COLUMN} = CONCAT("0 0 0 0 0 0 0 0 0 ", SUBSTRING({COLUMN}, 19))
Try this (it is tested):
select
CONCAT(
LEFT('0 0 0 6 7 2 6 8 6 1 7 5 3 2 4 9 9 4', INSTR('0 0 0 6 7 2 6 8 6 1 7 5 3 2 4 9 9 4','9')-1)
,'0',
RIGHT( '0 0 0 6 7 2 6 8 6 1 7 5 3 2 4 9 9 4',CHAR_LENGTH('0 0 0 6 7 2 6 8 6 1 7 5 3 2 4 9 9 4')-1- CHAR_LENGTH(LEFT('0 0 0 6 7 2 6 8 6 1 7 5 3 2 4 9 9 4', INSTR('0 0 0 6 7 2 6 8 6 1 7 5 3 2 4 9 9 4','9')-1)))
)
Assume that s='0 0 0 6 7 2 6 8 6 1 7 5 3 2 4 9 9 4' (and can be every strings)
select
CONCAT(
LEFT(s, INSTR(s,'9')-1)
,'0',
RIGHT(s,CHAR_LENGTH(s)-1- CHAR_LENGTH(LEFT(s, INSTR(s,'9')-1)))
)
see Demo: http://rextester.com/OIMXCY58613
want to join two tables and produce a result like ..ie.,
Table : 1
-------------------------------
Text val1 val2 val3 val4
-------------------------------
Test 96 1 4 0
Test 96 3 4 0
Test 96 5 4 0
Test 96 7 4 0
Test 96 9 4 0
Test 96 11 4 0
Test 96 13 4 0
Test 96 15 4 0
Test 87 7 6 1
Test1 87 7 6 1
Test1 95 5 4 0
Test1 95 13 4 0
Test2 109 15 6 0
Test3 109 15 5 0
Test4 109 15 4 0
Test5 109 15 3 0
Test6 107 0 7 0
Test7 107 0 6 0
Test8 107 0 5 0
Test9 107 0 4 0
Table : 2
-------------------------------
ID val1 val2 val3 val4
-------------------------------
10 96 1 4 0
10 96 3 4 0
10 96 5 4 0
10 96 7 4 0
10 96 9 4 0
10 96 11 4 0
10 96 13 4 0
10 96 15 4 0
10 87 7 6 1
11 87 7 6 1
11 95 5 4 0
11 95 13 4 0
12 109 15 6 0
13 109 15 5 0
14 109 15 4 0
15 109 15 3 0
16 107 0 7 0
17 107 0 6 0
18 107 0 4 0
Output Table
-------------------------------
Text ID val1 val2 val3 val4
-------------------------------
Test 10 96 1 4 0
Test 10 96 3 4 0
Test 10 96 5 4 0
Test 10 96 7 4 0
Test 10 96 9 4 0
Test 10 96 11 4 0
Test 10 96 13 4 0
Test 10 96 15 4 0
Test 10 87 7 6 1
Test1 11 87 7 6 1
Test1 11 95 5 4 0
Test1 11 95 13 4 0
Test2 12 109 15 6 0
Test3 13 109 15 5 0
Test4 14 109 15 4 0
Test5 15 109 15 3 0
Test6 16 107 0 7 0
Test7 17 107 0 6 0
Test8 18 107 0 4 0
Kindly help me wih select query for the same.
select table1.TEXT
, table2.id
, table1.val1
, table1.val2
, table1.val3,table1.val4
from table1
join table2
on table1.val1 = table2.val
and table1.val2 = table2.val2
and table1.val3 = table2.val3
and table1.val4 = table2.val4
select table1.Text,table2.Id,table1.val1,table1.val2,table1.val3,table1.val4 from table1 inner join table2 on table1.val1 = table2.val1 and table1.val2 = table2.val2 and table1.val3 = table2.val3 and table1.val4 = table2.val4
Here is a sample data set
-----------------------------------------------------------------------------------------
id nameid name score diff include quantity grade
---------------------------------------------------------------------------------------
7 0002 MO 10 0 0 25 3
8 0002 MO 18 0 1 25 3
9 0002 MO 20 0 0 25 3
10 0002 MO 14 0 0 17 6
11 0002 MO 100 0 0 17 6
11 0002 MO 100 0 0 17 6
12 0003 MA 10 0 0 12 3
13 0003 MA 18 0 0 12 3
14 0003 MA 20 0 0 12 3
15 0003 MA 14 0 0 25 6
16 0003 MA 100 0 1 25 6
17 0003 MA 100 0 0 25 6
12 0004 MB 10 0 0 12 3
13 0004 MB 18 0 1 12 3
14 0004 MB 20 0 0 12 3
15 0004 MB 14 0 0 07 6
16 0004 MB 100 0 1 07 6
17 0004 MB 100 0 0 07 6
I have a query that returns the above table.
Note that in each group of six, there WILL be atleast one row that has value 1 in include column.
Look at ref: access query needed but not needed.
Also for each group of six, there are three rows that has grade = 3 and 3 rows that has grade = 6.
And corresspondingly, the grade 3 and grade 6 have the same quantity in that group.
What I want to do is filter out all the rows that have less then 15 quantity.
However, I still want to group them by 6.
I want to remove a "group" that has both quantity < 15 for both grade 3 and 6. From the above data set
I wwant the following result:
-----------------------------------------------------------------------------------------
id nameid name score diff include quantity grade
---------------------------------------------------------------------------------------
7 0002 MO 10 0 0 25 3
8 0002 MO 18 0 1 25 3
9 0002 MO 20 0 0 25 3
10 0002 MO 14 0 0 17 6
11 0002 MO 100 0 0 17 6
11 0002 MO 100 0 0 17 6
12 0003 MA 10 0 0 12 3
13 0003 MA 18 0 0 12 3
14 0003 MA 20 0 0 12 3
15 0003 MA 14 0 0 25 6
16 0003 MA 100 0 1 25 6
17 0003 MA 100 0 0 25 6
So basically if a group of six has include = 1 in any row, and either grade 3 or 6 quantity > 15 then I want the entire group.
"So basically if a group of six has include = 1 in any row, and either grade 3 or 6 quantity > 15 then I want the entire group."
My guess is this query will identify the candidate nameid groups:
SELECT DISTINCT nameid
FROM YourTable
WHERE
include = 1
AND quantity > 15
AND (grade = 3 OR grade = 6);
If I guessed correctly, you can save it as a separate query, or use it as a subquery, and INNER JOIN it to YourTable to limit the rows returned to only those where nameid meets your criteria. It might look close to this untested SELECT statement:
SELECT y.id, y.nameid, y.[name], y.score, y.diff, y.include, y.quantity, y.grade
FROM
YourTable AS y
INNER JOIN [
SELECT DISTINCT nameid
FROM YourTable
WHERE
include = 1
AND quantity > 15
AND (grade = 3 OR grade = 6)
]. AS q
ON y.nameid = q.nameid
ORDER BY y.nameid;
Edit: Add an index on nameid if you don't already have one.