Somebody can guide me (maybe Simple and fast query if there is or some fast code) to convert my CSV data file (with commas separation):
1,A,C,Z,F,G
2,G,Q,R,C,
3,Z,G,Q,
4,C,F,
5,O,P,
6,O,X,Y,J,
7,A,P,X,
I have this table with ~1,000,000 records
like these 7 records that you see (In real Database A,B,C,... are words in string), Records 1 and 2 are common in G and C value and 2,3 and 1,3 and ...
I want to sync records if they have at least two common value like Records 1 & 2,3,4 (but record 5,6,7 haven't at least 2 shared values with others) and generate a list like this:
1 A C Z F G Q R
2 G Q R C A Z F
3 Z G Q A C F R
4 C F A Z G Q R
5 O P
6 O X Y J
7 A P X
at the end we must have 4 same records if we sort data and one others without sync:
1 A C F G Q R Z
2 A C F G Q R Z
3 A C F G Q R Z
4 A C F G Q R Z
5 O P
6 J O X Y
7 A P X
Maybe I do not use good term for my meaning, please see:
1 A C Z F G
2 G Q R C
record 1 has C and G common with Record 2 now 1 has not R and Q thus we must have 1 A C Z F G + Q and R and Record 2 has not A,Z and F thus we must have: 2 G Q R C + A,Z and F thus at the end we have:
1 A C Z F G Q R
2 G Q R C A Z F
I need all records Respectively in the queue from top to bottom.
wrote a delphi code but it is so slow.
Someone suggest me this groovy code:
def f=[:]
new File('Data.csv').readLines().each{
def items=it.split(',')
def name
items.eachWithIndex { String entry, int i ->
if(i==0){
name=entry
}
else if(entry){
if(!f[entry])
f[entry]=[]
f[entry]<<name
}
}
}
f.findAll {it.value.size()>1}
It is very fast (because of using a map file I think), but It only finds the common values.
If you would go for a SQL solution, then that csv data could be
put in a normalized table with the data unfolded per ID & WORD.
Once you have that, it becomes a matter of self-joining that table.
And concatinate the words back together in alphabetic order.
SqlFiddle test here
Not sure how fast this method would be on a table with 1000k records though.
But it's an interesting puzzle.
Sample data:
DROP TABLE IF EXISTS test_words;
CREATE TABLE IF NOT EXISTS test_words (
id int unsigned NOT NULL PRIMARY KEY,
words varchar(60) NOT NULL
);
INSERT INTO test_words (id, words) VALUES
(1,'A C Z F G'),
(2,'G Q R C'),
(3,'Z G Q'),
(4,'C F'),
(5,'P O'),
(6,'O X Y J'),
(7,'A P X');
Tally table with numbers:
DROP TABLE IF EXISTS tmp_numbers;
CREATE TEMPORARY TABLE IF NOT EXISTS tmp_numbers (
n int unsigned NOT NULL PRIMARY KEY
);
INSERT INTO tmp_numbers (n) VALUES (1),(2),(3),(4),(5),(6),(7),(8),(9),(10);
Unfolding the words:
DROP TABLE IF EXISTS test_words_unfolded;
CREATE TABLE test_words_unfolded (
word varchar(10) NOT NULL,
id int unsigned NOT NULL,
PRIMARY KEY (word, id)
);
INSERT INTO test_words_unfolded (word, id)
SELECT DISTINCT SUBSTRING_INDEX(SUBSTRING_INDEX(t.words,' ', nr.n),' ',-1) as word, t.id
FROM test_words AS t
JOIN tmp_numbers AS nr
ON CHAR_LENGTH(t.words) - CHAR_LENGTH(REPLACE(t.words,' ','')) >= nr.n - 1
AND SUBSTRING_INDEX(SUBSTRING_INDEX(t.words,' ', nr.n),' ',-1) != '';
Result table:
DROP TABLE IF EXISTS test_result;
CREATE TABLE IF NOT EXISTS test_result (
id int unsigned NOT NULL PRIMARY KEY,
words varchar(60) NOT NULL
);
INSERT INTO test_result (id, words)
SELECT q.id, GROUP_CONCAT(DISTINCT t3.word ORDER BY t3.word ASC SEPARATOR ' ') as words
FROM
(
SELECT t1.id, t2.id as id2
FROM test_words_unfolded t1
JOIN test_words_unfolded t2 ON t1.word = t2.word
GROUP BY t1.id, t2.id
HAVING COUNT(*) > 1 OR t1.id = t2.id
) q
LEFT JOIN test_words_unfolded t3 ON t3.id = q.id2
GROUP BY q.id
ORDER BY q.id;
SELECT *
FROM test_result
ORDER BY id;
Result:
id words
-- -----
1 A C F G Q R Z
2 A C F G Q R Z
3 A C F G Q R Z
4 A C F G Z
5 O P
6 J O X Y
7 A P X
Extra
To mark the words that have been added, the query to fill the result table becomes a bit more complicated.
SELECT
q2.id,
GROUP_CONCAT(DISTINCT CASE WHEN q2.ori = 1 THEN q2.word ELSE CONCAT('[',q2.word,']') END ORDER BY q2.word ASC SEPARATOR ' ') as words
FROM
(
SELECT
q1.id, t3.word,
MAX(CASE WHEN q1.id = t3.id THEN 1 ELSE 0 END) as ori
FROM
(
SELECT
t1.id, t2.id as id2
FROM test_words_unfolded t1
JOIN test_words_unfolded t2 ON t1.word = t2.word
GROUP BY t1.id, t2.id
HAVING COUNT(*) > 1 OR t1.id = t2.id
) q1
LEFT JOIN test_words_unfolded t3 ON t3.id = q1.id2
GROUP BY q1.id, t3.word
) q2
GROUP BY q2.id
ORDER BY q2.id;
Result:
id words
-- -----
1 A C F G [Q] [R] Z
2 [A] C [F] G Q R [Z]
3 [A] [C] [F] G Q [R] Z
4 [A] C F [G] [Z]
5 O P
6 J O X Y
7 A P X
Additional experiment here
I'm trying to convert the SQL below to Linq. I haven't figured out the syntax for the GROUP BY, the MIN() or the extra organization join conditions.
SELECT DISTINCT o.OrganizationHierarchyUnitLevelThreeNm, o.OrganizationHierarchyUnitLevelFourNm, a.LabAssetSerialNbr, MIN(a.SystemCreatedOnDtm) MinCreated
FROM vw_DimLabAsset a
INNER JOIN vw_FactWorker w ON a.LabAssetAssignedToWorkerKey = w.WorkerKey
INNER JOIN vw_DimOrganizationHierarchy o ON w.OrganizationHierarchyKey = o.OrganizationHierarchyKey
AND o.OrganizationHierarchyUnitLevelThreeNm IS NOT NULL
AND o.OrganizationHierarchyUnitLevelFourNm IS NOT NULL
GROUP BY o.OrganizationHierarchyUnitLevelThreeNm, o.OrganizationHierarchyUnitLevelFourNm, a.LabAssetSerialNbr
This is what I've managed to get so far:
var pphw = from a in Vw_DimLabAsset
where a.LabAssetHardwareStatus != "Retired" && (a.LabAssetHardwareSubStatus == null || a.LabAssetHardwareSubStatus != "Archive") && types.Contains(a.LabAssetTypeNm) // (a.LabAssetTypeNm == "u_cmdb_ci_prototype_system" || a.LabAssetTypeNm == "u_cmdb_ci_silicon")
join w in Vw_FactWorker on a.LabAssetAssignedToWorkerKey equals w.WorkerKey
join o in Vw_DimOrganizationHierarchy on w.OrganizationHierarchyKey equals o.OrganizationHierarchyKey
select new { o.OrganizationHierarchyUnitLevelThreeNm, o.OrganizationHierarchyUnitLevelFourNm, a.LabAssetSerialNbr };
Here is how I would translate the query:
var ans = (from a in vw_DimLabAsset
join w in vw_FactWorker on a.LabAssetAssignedToWorkerKey equals w.WorkerKey
join o in vw_DimOrganizationHierarchy on w.OrganizationHierarchyKey equals o.OrganizationHierarchyKey
where o.OrganizationHierarchyUnitLevelThreeNm != null && o.OrganizationHierarchyUnitLevelFourNm != null
group new { o, a } by new { o.OrganizationHierarchyUnitLevelThreeNm, o.OrganizationHierarchyUnitLevelFourNm, a.LabAssetSerialNbr } into oag
select new {
oag.Key.OrganizationHierarchyUnitLevelThreeNm,
oag.Key.OrganizationHierarchyUnitLevelFourNm,
oag.Key.LabAssetSerialNbr,
MinCreated = oag.Min(oa => oa.a.SystemCreatedOnDtm)
}).Distinct();
I have a query, which return (say)10 rows
SELECT EL.ID AS ID ,EL.x AS x ,EL.y AS y,EL.z AS z,
EL.k AS k, EL.a AS a,
EL.b AS b ,EL.c AS c,EL.d AS d ,EL.e AS e ,
EL.f AS f,EL.g AS g
FROM MYTABLE EL
WHERE EL.x = '2004'
AND EL.y = 'FYY'
AND EL.z = 'test'
AND EL.a = 'INTC'
AND EL.b = 321593
Along with this i want to join the row of same table (MYTABLE) where id =4 (1 row)
Which is the most preferred way using SQL server syntax?
SELECT EL.ID AS ID ,EL.x AS x ,EL.y AS y,EL.z AS z,
EL.k AS k, EL.a AS a,
EL.b AS b ,EL.c AS c,EL.d AS d ,EL.e AS e ,
EL.f AS f,EL.g AS g
FROM MYTABLE EL
WHERE (EL.x = '2004'
AND EL.y = 'FYY'
AND EL.z = 'test'
AND EL.a = 'INTC'
AND EL.b = 321593)
OR (EL.ID = 4)
System is not passing 'null' value if there is no value for
model.SpecialtyTypeDesc = spec.SpecialtyDescription;
I am unable to open the edit page if there is no value for Specialty for the employee. If the employee has specificity value, its working fine. I want to display the value as Null if the employee doestn't have any specialty in his personnel data.
public JsonResult GetSpecialityDepartment(int id){
EmployeeDepartments getDep = (from c in db.EmployeeDepartments where c.EmpId == id select c).FirstOrDefault();
Departments dept = (from c in db.Departments where c.Id == getDep.departmentId select c).FirstOrDefault();
EmployeeDetails details = (from c in db.EmployeeDetails where c.People_Id == id select c).FirstOrDefault();
SpecialtyType spec = (from c in db.SpecialityType where c.SpecialtyTypeId == details.SpecialtyTypeId select c).FirstOrDefault();
var data = new {
Department = dept.Name,
SpecialtyType = spec == null ? "" : spec.SpecialtyDescription
};
return Json(data);
}
public ActionResult Edit(int id) {
PersonnelLeaveAbsence leaveAbsence = (from c in db.PersonnelLeaveAbsence where c.PersonnelLeaveAbsenceId == id select c).FirstOrDefault();
EmployeeDepartments getDep = (from c in db.EmployeeDepartments where c.EmpId == leaveAbsence.PersonnelId select c).FirstOrDefault();
Departments dept = (from c in db.Departments where c.Id == getDep.departmentId select c).FirstOrDefault();
EmployeeDetails details = (from c in db.EmployeeDetails where c.People_Id == leaveAbsence.PersonnelId select c).FirstOrDefault();
SpecialtyType spec = (from c in db.SpecialtyType where c.SpecialtyTypeId == details.SpecialtyTypeId select c).FirstOrDefault();
PersonnelLeaveAbsenceModel model = new PersonnelLeaveAbsenceModel();
model.PersonnelLeaveAbsenceId = leaveAbsence.PersonnelLeaveAbsenceId;
model.PersonnelLeaveDate = leaveAbsence.LeaveDate;
model.LeaveAbsenceId = leaveAbsence.LeaveAbsenceTypeId;
model.PersonnelId = leaveAbsence.PersonnelId;
model.EmployeeRoleId = leaveAbsence.EmployeeRole;
model.StartTime = leaveAbsence.StartTime;
model.EndTime = leaveAbsence.EndTime;
model.DayType = Convert.ToBoolean(leaveAbsence.DayType);
model.DepartmentDesc = dept.Name;
model.SpecialtyTypeDesc = spec.SpecialtyDescription;
}
I have a query :
var list_transaction = from i in Dt.Transactions
join c in this.Dt.Customers on i.CustomerID equals c.ID
join e in this.Dt.Employees on i.EmployeeID equals e.ID
join p in this.Dt.Projects on i.ProjectID equals p.ID
where
i.CustomerID == idCus &&
i.TransactionStep == 3 &&
i.EmployeeID == e.ID &&
i.ProjectID == p.ID
select new {
VAT = (i.Taxable * i.Total * p.VATRate/100)
};
Problem : the VAT is the multiply of three value that have different datatype. Taxable is int, Total is money and VATRate is float.
So could anyone tell me, how can I cast that in this query?
Thanks so much.
Try this :
System.Convert.ToDouble(i.Total) + (i.Taxable * System.Convert.ToDouble(i.Total)
* p.VATRate / 100)