want to join two tables and produce a result like ..ie.,
Table : 1
-------------------------------
Text val1 val2 val3 val4
-------------------------------
Test 96 1 4 0
Test 96 3 4 0
Test 96 5 4 0
Test 96 7 4 0
Test 96 9 4 0
Test 96 11 4 0
Test 96 13 4 0
Test 96 15 4 0
Test 87 7 6 1
Test1 87 7 6 1
Test1 95 5 4 0
Test1 95 13 4 0
Test2 109 15 6 0
Test3 109 15 5 0
Test4 109 15 4 0
Test5 109 15 3 0
Test6 107 0 7 0
Test7 107 0 6 0
Test8 107 0 5 0
Test9 107 0 4 0
Table : 2
-------------------------------
ID val1 val2 val3 val4
-------------------------------
10 96 1 4 0
10 96 3 4 0
10 96 5 4 0
10 96 7 4 0
10 96 9 4 0
10 96 11 4 0
10 96 13 4 0
10 96 15 4 0
10 87 7 6 1
11 87 7 6 1
11 95 5 4 0
11 95 13 4 0
12 109 15 6 0
13 109 15 5 0
14 109 15 4 0
15 109 15 3 0
16 107 0 7 0
17 107 0 6 0
18 107 0 4 0
Output Table
-------------------------------
Text ID val1 val2 val3 val4
-------------------------------
Test 10 96 1 4 0
Test 10 96 3 4 0
Test 10 96 5 4 0
Test 10 96 7 4 0
Test 10 96 9 4 0
Test 10 96 11 4 0
Test 10 96 13 4 0
Test 10 96 15 4 0
Test 10 87 7 6 1
Test1 11 87 7 6 1
Test1 11 95 5 4 0
Test1 11 95 13 4 0
Test2 12 109 15 6 0
Test3 13 109 15 5 0
Test4 14 109 15 4 0
Test5 15 109 15 3 0
Test6 16 107 0 7 0
Test7 17 107 0 6 0
Test8 18 107 0 4 0
Kindly help me wih select query for the same.
select table1.TEXT
, table2.id
, table1.val1
, table1.val2
, table1.val3,table1.val4
from table1
join table2
on table1.val1 = table2.val
and table1.val2 = table2.val2
and table1.val3 = table2.val3
and table1.val4 = table2.val4
select table1.Text,table2.Id,table1.val1,table1.val2,table1.val3,table1.val4 from table1 inner join table2 on table1.val1 = table2.val1 and table1.val2 = table2.val2 and table1.val3 = table2.val3 and table1.val4 = table2.val4
Related
This is the webpage I am scraping: http://laxreports.sportlogiq.com/nll/GS2200.html
Below is the code for the spider I created:
import scrapy
class MatchesSpider(scrapy.Spider):
name = 'matches'
allowed_domains = ['laxreports.sportlogiq.com']
start_urls = ['http://laxreports.sportlogiq.com/nll/GS2200.html']
def parse(self, response):
tables = response.xpath('//table')
print(tables)
table = tables[0].xpath('//tbody')
I see 22 tables that have been selected for this XPath expression but my problem is that I don't fully understand how to select each individual table and extract its contents.
I am a beginner in scrapy and after searching online for a solution all I see is how to select the tables using the class or ID which in this case is not an option.
You can do that using only pandas
Code:
import pandas as pd
dfs = pd.read_html('https://laxreports.sportlogiq.com/nll/GS2200.html')
df = dfs[10]#.to_csv('d.csv', index = False)
print(df)
Output:
0 1 2 3 4 5 6 7 8 9 10 11 12
0 # Name G A +/- PIM S SOFF LB T CT FO TOF
1 2 W.Malcom 0 0 0 0 1 1 1 4 0 - 11:28
2 3 T.Edwards 0 0 -2 2 0 0 8 1 2 7-18 20:28
3 4 J.Sullivan 0 0 -3 2 0 0 3 0 0 - 15:29
4 11 T.Stuart 0 0 -3 0 0 0 4 1 1 - 21:09
5 14 W.Jeffrey 0 1 -1 0 0 0 9 2 1 - 19:17
6 16 R.Lee 2 1 2 0 9 4 6 6 1 - 23:13
7 17 C.Wardle 2 0 1 2 5 3 4 2 2 - 20:55
8 18 R.Hope (A) 0 0 -2 2 0 0 11 0 0 - 22:02
9 20 J.Ruest 3 2 3 0 8 1 3 2 0 - 24:16
10 23 J.Gilles 0 0 -1 0 0 0 4 0 3 - 14:44
11 27 S.Carnegie 0 0 -1 0 0 0 3 0 0 - 12:19
12 37 D.Coates (C) 0 0 0 0 1 0 1 0 0 1-1 2:31
13 51 E.McLaughlin 0 5 2 0 7 3 5 7 0 - 21:41
14 55 D.Kinnear 0 1 2 0 2 0 2 1 0 0-2 10:14
15 67 K.Killen 1 1 0 0 6 1 4 2 0 - 16:42
16 82 J.Cupido (A) 0 1 -1 0 3 0 4 1 0 - 20:52
17 86 J.Lintz 0 1 -1 0 0 0 4 0 1 - 19:26
18 30 T.Carlson 0 0 NaN 0 0 0 0 0 0 - NaN
19 45 D.Ward 0 0 NaN 0 0 0 0 1 0 - NaN
20 NaN Totals: 8 13 NaN 8 42 13 76 30 11 8-21 NaN
I run mysqldump on a before and after picture of database before tests
mysqldump --no-create-info --no-create-db --tab=/data/tmp/db1 statsdb
and after tests
mysqldump --no-create-info --no-create-db --tab=/data/tmp/db2 statsdb
So then i do a diff
diff --brief /data/tmp/db1 /data/tmp/db2
Files /data/tmp/db1/details.txt and
/data/tmp/db2/details.txt differ
The trouble is that it results in the txt files, despite having same data created in the db1 and db2 directories do not display the lines in the same order.
Is there a way to force them to sort correctly? Here is the differing files for the record
[root#server ~]# cat /data/tmp/db2/enm_network_element_details.txt
2018-09-26 1 RNC 18.Q1-V.12.40 1 1 1 0
0
2018-09-26 1 MGW 17A-C.1.267 15 15 15 15
0
2018-09-26 1 ERBS 17.Q4-J.1.160 480 480 475 480
479
2018-09-26 1 SGSN-MME 16A-CP09 14 14 14
14 0
2018-09-26 1 MTAS MTAS-1.5 10 10 10 10
0
2018-09-26 1 SIU02 17A 56 56 56 54 0
2018-09-26 1 Router6672 R17B-GA 1 1 1 0
0
2018-09-26 1 RBS 18.Q1-U.4.750 350 350 350 350
0
2018-09-26 1 RNC 17B-V.9.1240 1 1 1 1
0
2018-09-26 1 Router6672 R18A-GA 104 104 104 104
0
2018-09-26 1 MINI-LINK-Indoor M16A-TN-5.4FP-L 18 18
18 5 5
2018-09-26 1 ERBS 18.Q1-J.1.220 1 0 0 0
0
2018-09-26 1 EPG 16B-R13C 6 6 6 6
0
2018-09-26 1 MINI-LINK-6352 M17B-6352-2.8 15 15 15
15 0
2018-09-26 1 MINI-LINK-Indoor M17.Q4-TN-6.1-L 41 41
41 4 4
2018-09-26 1 MINI-LINK-Indoor M17A-TN-6.0-LH- 9 9
9 9 9
2018-09-26 1 TCU02 17A 56 56 56 54 0
2018-09-26 1 DSC 17B-R1A 10 10 10 10 0
2018-09-26 1 RadioNode 17B-R6A19 550 550 550
550 0
2018-09-26 1 RadioNode 17Q4-R25A12 160 160 160
160 158
[root#server~]# cat /data/tmp/db1/enm_network_element_details.txt
2018-09-26 1 RadioNode 17Q4-R25A12 160 160 160
160 158
2018-09-26 1 RadioNode 17B-R6A19 550 550 550
550 0
2018-09-26 1 RNC 18.Q1-V.12.40 1 1 1 0
0
2018-09-26 1 MGW 17A-C.1.267 15 15 15 15
0
2018-09-26 1 ERBS 17.Q4-J.1.160 480 480 475 480
479
2018-09-26 1 MTAS MTAS-1.5 10 10 10 10
0
2018-09-26 1 SIU02 17A 56 56 56 54 0
2018-09-26 1 Router6672 R17B-GA 1 1 1 0
0
2018-09-26 1 SGSN-MME 16A-CP09 14 14 14
14 0
2018-09-26 1 RNC 17B-V.9.1240 1 1 1 1
0
2018-09-26 1 RBS 18.Q1-U.4.750 350 350 350 350
0
2018-09-26 1 Router6672 R18A-GA 104 104 104 104
0
2018-09-26 1 MINI-LINK-Indoor M16A-TN-5.4FP-L 18 18
18 5 5
2018-09-26 1 EPG 16B-R13C 6 6 6 6
0
2018-09-26 1 ERBS 18.Q1-J.1.220 1 0 0 0
0
2018-09-26 1 MINI-LINK-6352 M17B-6352-2.8 15 15 15
15 0
2018-09-26 1 MINI-LINK-Indoor M17.Q4-TN-6.1-L 41 41
41 4 4
2018-09-26 1 MINI-LINK-Indoor M17A-TN-6.0-LH- 9 9
9 9 9
2018-09-26 1 TCU02 17A 56 56 56 54 0
2018-09-26 1 DSC 17B-R1A 10 10 10 10 0
The option "--order-by-primary" might be what you need.
Source: https://dev.mysql.com/doc/refman/5.7/en/mysqldump.html#mysqldump-option-summary
We are developing an local shop reccomendation system and in one of our sql queries we had a problem
We want to fetch the companies which all users in same cluster rated , but if any one of the users in the same group doesnt rated the company we wouldnt want to fetch it
SELECT ml_user_clusters.primaryUser,ml_user_clusters.clusterId,ml_ratings.companyId,ml_ratings.rating,ml_user_labels.groupId FROM ml_user_clusters
LEFT JOIN ml_ratings ON ml_ratings.userId = ml_user_clusters.primaryUser
LEFT JOIN ml_company_user_labels ON ml_company_user_labels.companyId = ml_ratings.companyId
LEFT JOIN ml_user_labels ON ml_user_labels.groupId = ml_company_user_labels.labelId
WHERE ml_user_clusters.clusterId = 0
We've started to add a query like in the below but couldnt able to finish it with proper AND clause
And our data is like in the below: So in the result we would like to have only the companies which has groupId=6 because all users in the same cluster(clusterId=0) rated a company with groupId = 6
primaryUser clusterId companyId rating groupId
497 0 135 5 NULL
498 0 135 10 NULL
79 0 135 12 NULL
501 0 135 10 NULL
79 0 85 14 2
79 0 8 4 5
79 0 98 11 5
79 0 3 5 5
497 0 6 7 6
500 0 6 7 6
499 0 29 7 6
497 0 29 7 6
499 0 77 7 6
500 0 29 7 6
498 0 6 7 6
500 0 77 11 6
500 0 130 3 6
498 0 130 3 6
501 0 77 19 6
499 0 6 7 6
79 0 30 1 7
500 0 30 7 7
79 0 48 7 9
79 0 39 1 13
79 0 48 7 13
499 0 6 7 15
497 0 6 7 15
79 0 8 4 15
500 0 6 7 15
79 0 98 11 15
498 0 6 7 15
79 0 3 5 15
79 0 81 7 15
79 0 3 5 17
79 0 82 7 17
79 0 103 7 17
79 0 118 3 17
79 0 63 3 17
501 0 118 7 17
79 0 82 7 19
79 0 118 3 19
79 0 63 3 19
501 0 118 7 19
79 0 39 1 21
79 0 85 14 23
Expected output must be: (Because all unique users in Cluster=0 has rated a company which has GroupID=6 )
primaryUser clusterId companyId rating groupId
497 0 6 7 6
500 0 6 7 6
499 0 29 7 6
497 0 29 7 6
499 0 77 7 6
500 0 29 7 6
498 0 6 7 6
500 0 77 11 6
500 0 130 3 6
498 0 130 3 6
501 0 77 19 6
499 0 6 7 6
Do you have any idea how we can fix that problem?
Something like this should work,you should build a fiddle for better testing.
Explanation: you count distinct users grouped by group id and compare with the total number of distinct users.If the two match it means all users in that respective groupid have voted.
SELECT ml_user_labels.groupId
FROM ml_user_clusters
LEFT JOIN ml_ratings ON ml_ratings.userId = ml_user_clusters.primaryUser
LEFT JOIN ml_company_user_labels ON ml_company_user_labels.companyId = ml_ratings.companyId
LEFT JOIN ml_user_labels ON ml_user_labels.groupId = ml_company_user_labels.labelId
WHERE ml_user_clusters.clusterId = 0
GROUP BY ml_user_labels.groupId
HAVING COUNT(DISTINCT ml_user_clusters.primaryUser) =
(SELECT COUNT(DISTINCT ml_user_clusters.primaryUser)
FROM ml_user_clusters
LEFT JOIN ml_ratings ON ml_ratings.userId = ml_user_clusters.primaryUser
LEFT JOIN ml_company_user_labels ON ml_company_user_labels.companyId = ml_ratings.companyId
LEFT JOIN ml_user_labels ON ml_user_labels.groupId = ml_company_user_labels.labelId
WHERE ml_user_clusters.clusterId = 0)x
I have a table like this:
id c_id time value
1 4 1 12
2 4 2 5
3 4 3 6
4 4 4 48
5 4 5 1
6 4 6 121
7 5 1 121
8 5 2 321
9 5 3 2
10 5 4 1
11 5 5 54
12 5 6 4546
13 5 7 78
14 5 8 784
15 5 9 1
Now I want a table like this with a SELECT command:
time1 value1 time2 value2
1 12 1 121
2 5 2 321
3 6 3 2
4 48 4 1
5 1 5 54
6 121 6 4546
0 0 7 78
0 0 8 784
0 0 9 1
time1 and value1 is from the data with c_id=4,
time2 and value2 is from the data with c_id=5
Is it possible to create a SELECT command to do that?
I hope you can help
Yiu can use an inner join
select a.time as time1, a.value as value1, b.time as time2, b.value as value2
from my_table as a
inner join my_table as b on a.time = b.time
and a.c_id= 4
and b.c_id= 5;
Here is mysql data
id usr good quant delayed cart_ts
------------------------------------------------------
14 4 1 1 0 20100601235348
13 4 11 1 0 20100601235345
12 4 4 1 0 20100601235335
11 4 1 1 0 20100601235051
10 4 11 1 0 20100601235051
9 4 4 1 0 20100601235051
15 4 2 1 0 20100601235350
16 4 7 1 0 20100602000537
17 4 3 1 0 20100602000610
18 4 3 1 0 20100602000616
19 4 8 1 0 20100602000802
20 4 8 1 0 20100602000806
21 4 8 1 0 20100602000828
22 4 8 1 0 20100602000828
23 4 8 1 0 20100602000828
24 4 8 1 0 20100602000828
25 4 8 1 0 20100602000828
26 4 8 1 0 20100602000829
27 4 8 1 0 20100602000829
28 4 9 1 0 20100602001045
29 4 10 1 0 20100602001046
I need to group fields in witch usr & good has duplicated values with summing quant field
for getting smth like this:
id usr good quant delayed cart_ts
------------------------------------------------------
14 4 1 2 0 20100601235348
13 4 11 2 0 20100601235345
12 4 4 2 0 20100601235335
15 4 2 1 0 20100601235350
16 4 7 1 0 20100602000537
17 4 3 2 0 20100602000610
19 4 8 9 0 20100602000802
28 4 9 1 0 20100602001045
29 4 10 1 0 20100602001046
Which MySQL query I need to do to have this effect?
SELECT id,usr,good,SUM(quant),delayed,cart_ts FROM table GROUP BY usr,good