I have a table in my MySQL database, compatibility_core_rules, which essentially stores pairs of ids which represent compatibility between parts which have fields with those corresponding ids. Now, my aim is to get all possible compatibility pairs by following the transitivity of the pairs (e.g. so if the table has (1,2) and (2,4), then add the pair (1,4)). So, mathematically speaking, I'm trying to find the transitive closure of the compatibility_core_rules table.
E.g. if compatibility_core_rules contains (1,2), (2,4) and (4,9), then initially we can see that (1,2) and (2,4) gives a new pair (1,4). I then iterate over the updated pairs and find that (4,9) with the newly added (1,4) gives me (1,9). At this point, iterating again would add no more pairs.
So my approach is to create a view with the initial pairs from compatibility_core_rules, like so:
CREATE VIEW compatibility_core_rules_closure
AS
SELECT part_type_field_values_id_a,
part_type_field_values_id_b,
custom_builder_id
FROM compatibility_core_rules;
Then, in order to iteratively discover all pairs, I need to keep replacing that view with an updated version of itself that has additional pairs each time. However, I found MySQL doesn't like me referencing the view in its own definition, so I make a temporary view (with or replace, since this will be inside a loop):
CREATE OR REPLACE VIEW compatibility_core_rules_closure_temp
AS
SELECT part_type_field_values_id_a,
part_type_field_values_id_b,
custom_builder_id
FROM compatibility_core_rules_closure;
No problems here. I then reference this temporary view in the following CREATE OR REPLACE VIEW statement to update the compatibility_core_rules_closure view with one iteration's worth of additional pairs:
CREATE OR REPLACE VIEW compatibility_core_rules_closure
AS
SELECT
CASE WHEN ccr1.part_type_field_values_id_a = ccr2.part_type_field_values_id_a THEN ccr1.part_type_field_values_id_b
WHEN ccr1.part_type_field_values_id_a = ccr2.part_type_field_values_id_b THEN ccr1.part_type_field_values_id_b
END ccrA,
CASE WHEN ccr1.part_type_field_values_id_a = ccr2.part_type_field_values_id_a THEN ccr2.part_type_field_values_id_b
WHEN ccr1.part_type_field_values_id_a = ccr2.part_type_field_values_id_b THEN ccr2.part_type_field_values_id_a
END ccrB,
ccr1.custom_builder_id custom_builder_id
FROM compatibility_core_rules_closure_temp ccr1
INNER JOIN compatibility_core_rules_closure_temp ccr2
ON (
ccr1.part_type_field_values_id_a = ccr2.part_type_field_values_id_a OR
ccr1.part_type_field_values_id_a = ccr2.part_type_field_values_id_b
)
GROUP BY ccrA,
ccrB
HAVING -- ccrA and ccrB are in fact not the same
ccrA != ccrB
-- ccrA and ccrB do not belong to the same part type
AND (
SELECT ptf.part_type_id
FROM part_type_field_values ptfv
INNER JOIN part_type_fields ptf
ON ptfv.part_type_field_id = ptf.id
WHERE ptfv.id = ccrA
LIMIT 1
) !=
(
SELECT ptf.part_type_id
FROM part_type_field_values ptfv
INNER JOIN part_type_fields ptf
ON ptfv.part_type_field_id = ptf.id
WHERE ptfv.id = ccrB
LIMIT 1
)
Now this is where things go wrong. I get the following error:
#1146 - Table 'db509574872.compatibility_core_rules_closure' doesn't exist
I'm very confused by this error message. I literally just created the view/table only two statements ago. I'm sure the SELECT query itself is correct since if I try it by itself and it runs fine. If I change the first line to use compatibility_core_rules_closure2 instead of compatibility_core_rules_closure then it runs fine (however, that's not much use since I need to be re-updating the same view again and again). I've looked into the SQL SECURITY clauses but have not had any success. Also been researching online but not getting anywhere.
Does anyone have any ideas what is happening and how to solve it?
MySQL doesn't support sub-queries in views.
You'll have to separate them... ie. using another view containing the sub-query inside you main view.
Running the create statement for that view will render an error, not creating it, hence the doesn't exist error you're getting when querying it.
Related
I'm working on a MySQL Way of printing an "affiliate tree" and got the thing working with Common Table Expression. I'm using the following code right now:
WITH RECURSIVE recUsers AS
(
SELECT ID, username, sponsorID, 1 AS depth, username AS path
FROM users
WHERE id = 1
UNION ALL
SELECT c.ID, c.username, c.sponsorID, sc.depth + 1, CONCAT(sc.path, ' > ', c.username)
FROM recUsers AS sc
JOIN users AS c ON sc.ID = c.sponsorID
)
SELECT * FROM recUsers;
This selects the tree underneath the user with the id 1.
Now what I'd need to get is a way to pass that id as a parameter, so I don't need to define everything from the beginning every time I want to get the result.. So my idea is to put everything in a stored prodecure and pass the id in as a parameter.. However, so far I didn't get it working and always getting various errors that are very self speaking...
Basically what I've tried was
DELIMITER //
CREATE PROCEDURE getAffiliateTree(IN userid INT())
BEGIN
---my code here, the userid 1 replaced with userid
END//
DELIMITER;
However, this doesn't seem to work.. How can I get this done?
Two things I would suggest:
Use INT, not INT(). The optional length argument to integer types is deprecated in MySQL 8.0 (which I know you're using, because you're using CTE syntax). Even if you did use the length argument, using an empty argument is not legal syntax.
Make sure that the userid input parameter name is distinct from all of the columns in the tables you reference. That is, if the table has a column named userid (any capitalization), then change the name of your input parameter. Otherwise you may make ambiguous expressions like:
... WHERE userid = userid
Even though you intend one of these to be the column and the other to be the parameter, the SQL parser has no way of knowing that. It ends up treating both as the column name, so it's trivially true on all rows of the table.
Actually, a third thing I would suggest: when you ask questions, "it doesn't seem to work" isn't clear enough. Did it produce an error? If so, what was the full error message? Did it produce no error, but didn't give you the result you wanted? If so, show a mocked-up example of what you expected, and what the query produced that didn't match. It helps to be as clear as you can when you post questions, so readers don't have to guess what trouble you need help with.
before i use alias for table i get the error:
: Integrity constraint violation: 1052 Column 'id' in field list is ambiguous
Then i used aliases and i get this error:
unknown index a
I am trying to get a list of category name ( dependant to a translation) and the associated category id which is unique. Since i need to put them in a select, i see that i should use the lists.
$categorie= DB::table('cat as a')
->join('campo_cat as c','c.id_cat','=','a.id')
->join('campo as d','d.id','=','c.id_campo')
->join('cat_nome as nome','nome.id_cat','=','a.id')
->join('lingua','nome.id_lingua','=','lingua.id')
->where('lingua.lingua','=','it-IT')
->groupby('nome.nome')
->lists('nome.nome','a.id');
The best way to debug your query is to look at the raw query Laravel generates and trying to run this raw query in your favorite SQL tool (Navicat, MySQL cli tool...), so you can dump it to log using:
DB::listen(function($sql, $bindings, $time) {
Log::info($sql);
Log::info($bindings);
});
Doing that with yours I could see at least one problem:
->where('lingua.lingua','=','it-IT')
Must be changed to
->where('lingua.lingua','=',"'it-IT'")
As #jmail said, you didn't really describe the problem very well, just what you ended up doing to get around (part of) it. However, if I read your question right you're saying that originally you did it without all the aliases you got the 'ambiguous' error.
So let me explain that first: this would happen, because there are many parts of that query that use id rather than a qualified table`.`id.
if you think about it, without aliases you query looks a bit like this: SELECT * FROM `cat` JOIN `campo_cat` ON `id_cat` = `id` JOIN `campo` ON `id` = `id_campo`; and suddenly, MySQL doesn't know to which table all these id columns refer. So to get around that all you need to do is namespace your fields (i.e. use ... JOIN `campo` ON `campo`.`id` = `campo_cat`.`id_campo`...). In your case you've gone one step further and aliased your tables. This certianly makes the query a little simpler, though you don't need to actually do it.
So on to your next issue - this will be a Laravel error. And presumably happening because your key column from lists($valueColumn, $keyColumn) isn't found in the results. This is because you're referring to the cat.id column (okay in your aliased case a.id) in part of the code that's no longer in MySQL - the lists() method is actually run in PHP after Laravel gets the results from the database. As such, there's no such column called a.id. It's likely it'll be called id, but because you don't request it specifically, you may find that the ambiguous issue is back. My suggestion would be to select it specifically and alias the column. Try something like the below:
$categories = DB::table('cat as a')
->join('campo_cat as c','c.id_cat','=','a.id')
->join('campo as d','d.id','=','c.id_campo')
->join('cat_nome as nome','nome.id_cat','=','a.id')
->join('lingua','nome.id_lingua','=','lingua.id')
->where('lingua.lingua','=','it-IT')
->groupby('nome.nome')
->select('nome.nome as nome_nome','a.id as a_id') // here we alias `.id as a_id
->lists('nome_nome','a_id'); // here we refer to the actual columns
It may not work perfectly (I don't use ->select() so don't know whether you pass an array or multiple parameters, also you may need DB::raw() wrapping each one in order to do the aliasing) but hopefully you get my meaning and can get it working.
This one is a pretty good one IMO and I have not seen a close exampled on SO or Google so here you go. I need to do the following within a Perl application I am building. Unfortunately it can not be done directly in MySQL and will require DBI. In a nutshell I need to take Database1.tableA and locate every record with the column 'status' matching 'started'. This I can do as it is fairly easy (not very good with DBI yet, but have read the docs), but where I am having issues is what I have to do next.
my $started_query = "SELECT primary_ip FROM queue WHERE status='started'";
my $started = $dbh->prepare($started_query);
$started->execute();
while ( my #started = $started->fetchrow_array() ) {
# Where I am hoping to have the following occur so it can go by row
# for only rows with the status 'started'
}
So for each record in the #started array, really only contains one value per iteration of the while loop, I need to see if it exists in the Database2.tableA and IF it does exist in the other database (Database2.tableA) I need to delete it from Database1.tableA, but if it DOES NOT exist in the other database (Database2.tableA) I need to update the record in the current database (Database1.tableA).
Basically replicating the below semi-valid MySQL syntax.
DELETE FROM tableA WHERE primary_ip IN (SELECT primary_ip FROM db2.tablea) OR UPDATE tableA SET status = 'error'
I am limited to DBI to connect to the two databases and the logic is escaping me currently. I could do the queries to both databases and store in #arrays and then do a comparison, but that seems redundant as I think it should be possible within the while ( my #started = $started->fetchrow_array() ) as that will save on runtime and resources required. I am also not familiar enough with passing variables between DBI instances and as the #started array will always contain the column value I need to query for and delete I would like to take full advantage of having that defined and passed to the DBI objects.
I am going to be working on this thing all night and already ran through a couple pots of coffee so your helping me understand this logic is greatly appreciated.
You'll be better off with fetchrow_hashref, which returns a hashref of key/value pairs, where the keys are the column names, rather than coding based on columns showing up at ordinal positions in the array.
You need an additional database handle to do the lookups and updates because you've got a live statement handle on the first one. Something like this:
my $dbh2 = DBI->connect(...same credentials...);
...
while(my $row = $started->fetchrow_hashref)
{
if(my $found = $dbh2->selectrow_hashref("SELECT * FROM db2.t2 WHERE primary_ip = ?",undef,$row->{primary_ip}))
{
$dbh2->do("DELETE FROM db1.t1 WHERE primary_ip = ?",undef,$found->{primary_ip});
}
else
{
$dbh2->do("UPDATE db1.t1 SET status = 'error' WHERE primary_ip = ?",undef,$found->{primary_ip}");
}
}
Technically, I don't "need" to fetch the row from db2.t2 into my $found since you're only apparently testing for existence, there are other ways, but using it here is a bit of insurance against doing something other than you intended, since it will be undef if we somehow get some bad logic going and that should keep us from making some potential wrong changes.
But approaching a relational database with loop iterations is rarely the best tactic.
This "could" be done directly in MySQL with just a couple of queries.
First, the updates, where t1.status = 'started' and t2.primary_ip has no matching value for t1.primary_ip:
UPDATE db1.t1 a LEFT JOIN db2.t2 b ON b.primary_ip = a.primary_ip
SET a.status = 'error'
WHERE b.primary_ip IS NULL AND a.status = 'started';
If you are thinking "but b.primary_ip is never null" ... well, it is null in a left join where there are no matching rows.
Then deleting the rows from t1 can also be accomplished with a join. Multi-table joins delete only the rows from the table aliases listed between DELETE and FROM. Again, we're calling "t1" by the alias "a" and t2 by the alias "b".
DELETE a
FROM db1.t1 a JOIN db2.t2 b ON a.primary_ip = b.primary_ip
WHERE a.status = 'started';
This removes every row from t1 ("a") where status = 'started' AND where a matching row exists in t2.
I'm using LINQ to Entities to display paged results. But I'm having issues with the combination of Skip(), Take() and OrderBy() calls.
Everything works fine, except that OrderBy() is assigned too late. It's executed after result set has been cut down by Skip() and Take().
So each page of results has items in order. But ordering is done on a page handful of data instead of ordering of the whole set and then limiting those records with Skip() and Take().
How do I set precedence with these statements?
My example (simplified)
var query = ctx.EntitySet.Where(/* filter */).OrderByDescending(e => e.ChangedDate);
int total = query.Count();
var result = query.Skip(n).Take(x).ToList();
One possible (but a bad) solution
One possible solution would be to apply clustered index to order by column, but this column changes frequently, which would slow database performance on inserts and updates. And I really don't want to do that.
EDIT
I ran ToTraceString() on my query where we can actually see when order by is applied to the result set. Unfortunately at the end. :(
SELECT
-- columns
FROM (SELECT
-- columns
FROM (SELECT -- columns
FROM ( SELECT
-- columns
FROM table1 AS Extent1
WHERE EXISTS (SELECT
-- single constant column
FROM table2 AS Extent2
WHERE (Extent1.ID = Extent2.ID) AND (Extent2.userId = :p__linq__4)
)
) AS Project2
limit 0,10 ) AS Limit1
LEFT OUTER JOIN (SELECT
-- columns
FROM table2 AS Extent3 ) AS Project3 ON Limit1.ID = Project3.ID
UNION ALL
SELECT
-- columns
FROM (SELECT -- columns
FROM ( SELECT
-- columns
FROM table1 AS Extent4
WHERE EXISTS (SELECT
-- single constant column
FROM table2 AS Extent5
WHERE (Extent4.ID = Extent5.ID) AND (Extent5.userId = :p__linq__4)
)
) AS Project6
limit 0,10 ) AS Limit2
INNER JOIN table3 AS Extent6 ON Limit2.ID = Extent6.ID) AS UnionAll1
ORDER BY UnionAll1.ChangedDate DESC, UnionAll1.ID ASC, UnionAll1.C1 ASC
My workaround solution
I've managed to workaround this problem. Don't get me wrong here. I haven't solved precedence issue as of yet, but I've mitigated it.
What I did?
This is the code I've used until I get an answer from Devart. If they won't be able to overcome this issue I'll have to use this code in the end.
// get ordered list of IDs
List<int> ids = ctx.MyEntitySet
.Include(/* Related entity set that is needed in where clause */)
.Where(/* filter */)
.OrderByDescending(e => e.ChangedDate)
.Select(e => e.Id)
.ToList();
// get total count
int total = ids.Count;
if (total > 0)
{
// get a single page of results
List<MyEntity> result = ctx.MyEntitySet
.Include(/* related entity set (as described above) */)
.Include(/* additional entity set that's neede in end results */)
.Where(string.Format("it.Id in {{{0}}}", string.Join(",", ids.ConvertAll(id => id.ToString()).Skip(pageSize * currentPageIndex).Take(pageSize).ToArray())))
.OrderByDescending(e => e.ChangedOn)
.ToList();
}
First of all I'm getting ordered IDs of my entities. Getting only IDs is well performant even with larger set of data. MySql query is quite simple and performs really well. In the second part I partition these IDs and use them to get actual entity instances.
Thinking of it, this should perform even better than the way I was doing it at the beginning (as described in my question), because getting total count is much much quicker due to simplified query. The second part is practically very very similar, except that my entities are returned rather by their IDs instead of partitioned using Skip and Take...
Hopefully someone may find this solution helpful.
I haven't worked directly with Linq to Entities, but it should have a way to hook specific stored procedures into certain locations when needed. (Linq to SQL did.) If so, you could turn this query into a stored procedure, doing exacly what is required, and doing it efficiently.
Assuming from you comment the persisting the values in a List is not acceptable:
There's no way to completely minimize the iterations, as you intended (and as I would have tried too, living in hope). Cutting the iterations down by one would be nice. Is it possible to just get the Count once and cache/session it? Then you could:
int total = ctx.EntitySet.Count; // Hopefully you can not repeat doing this.
var result = ctx.EntitySet.Where(/* filter */).OrderBy(/* expression */).Skip(n).Take(x).ToList();
Hopefully you can cache the Count somehow, or avoid needing it every time. Even if you can't, this is the best you can do.
Could you please create a sample illusrating the problem and send it to us (support * devart * com, subject "EF: Skip, Take, OrderBy")?
Hope we will be able to help you.
You can also contact us using our forums or contact form.
Are you absolutely certain the ordering is off? What does the SQL look like?
Can you reorder your code as follows and post the output?
// Redefine your queries.
var query = ctx.EntitySet.Where(/* filter */).OrderBy(e => e.ChangedDate);
var skipped = query.Skip(n).Take(x);
// let's look at the SQL, shall we?
var querySQL = query.ToTraceString();
var skippedSQL = skipped.ToTraceString();
// actual execution of the queries...
int total = query.Count();
var result = skipped.ToList();
Edit:
I'm absolutely certain. You can check my "edit" to see trace result of my query with skipped trace result that is imperative in this case. Count is not really important.
Yeah, I see it. Wow, that's a stumper. Might even be an outright bug. I note you're not using SQL Server... what DB are you using? Looks like it might be MySQl.
One way:
var query = ctx.EntitySet.Where(/* filter */).OrderBy(/* expression */).ToList();
int total = query.Count;
var result = query.Skip(n).Take(x).ToList();
Convert it to a List before skipping. It's not too efficient, mind you...
Hiii,
I have a Database design as:
Table File (FileID, Name, Details)
Table Attributes (AttID, AttName, AttType)
Table AttValues (FileID, AttID, AttValue)
Till the runtime, it is not known how many Attributes are in a File and of what names.
And I want to display after insertion at Frontend in a manner like like:
FileID, FileName, Details, (Rows of Attribute Table as Column here).
i.e.,
Can anybody provide me a piece of code in Java or in MySQL to achieve this Pivoting Result.
Highly thanks full for your precious time.
Or is there any other better way to store data, So that I can get the desired result easily.
This requires two queries. First select the File:
SELECT * FROM File WHERE (...)
Then, fetch the Attributes:
SELECT *
FROM AttValues
JOIN Attributes ON (Attributes.AttId = AttValues.AttId)
WHERE FileId = $id
The latter query will provide you with one row per Attribute, which you can programmatically pivot for display on your frontend:
foreach(row in result) {
table.AddColumn(Header = row['AttName'], Value = row['AttValue']);
}
Adapt to your local programming environment as needed.
Of course, this only works for a single File or Files with the same attributes. If you want to display multiple files with different Attributes you can instead prefetch all AttNames:
SELECT Attributes.AttId, Attributes.AttName
FROM Attributes
JOIN AttValues ON (Attributes.AttId = AttValues.AttId)
WHERE FileId IN ( $list_of_ids )
Then load the values like this:
SELECT *
FROM AttValues
WHERE FileId IN ( $list_of_ids )
and use a local associative array to map from AttIds to column indexes.
As a final optimisation, you can combine the last two queries into an OUTER JOIN to avoid the third round trip. While this will probably increase the amount of data transferred, it also makes filling the table easier, if your class library supports named columns.
I answered a similar question recently: How to pivot a MySQL entity-attribute-value schema. The answer is MySQL-specific, but I guess that's OK as the question is tagged with mysql.