I have and old MySQL database where I need to insert new columns into tables (to support new parts of the front-end). But some of the old parts use SQL commands that depend on column count and order instead of their names. e.g.:
INSERT INTO `data` VALUES (null /*auto-id*/, "name", "description", ...)
When I add new columns into this table, I get the error:
1136 - Column count doesn't match value count at row 1
Right now I know about the INSERT which needs to be changed to:
INSERT INTO `data` (`name`, `desc`, ...) VALUES ("name", "description", ...)
The question is: are there any other commands that can use similar syntax that rely on an order or count of the columns instead of their names? I need to update all the old SQL commands before updating the DB and using trial & error method would be really long.
SELECTs are not a problem, because the front-end uses associative mapping and correctly uses their names everywhere so new columns will be just ignored. Also I'm sure there are no commands that modifying the DB structure (e.g. ALTER TABLE).
You ruled out data structure modifying queries, so this leaves us with insert, update, delete, and select.
Insert you are already aware of.
Update requires each updated field to be specified, so mostly that's ok. However, subqueries may be used in the where clause, and mysql allows multi-table updates, so my points around select do apply.
Delete applies to a whole record, so there is nothing that an extra field would influence. However, subqueries may be used in the where clause, so my points around select do apply.
You tried to rule out select, but you should not. It is not only the final resultset that can be influenced by a new field:
A subquery may use select * that and an extra field may cause error in the outer query. For example the newly introduced field mayhave the same name as another field in the outer query leading to ambiguous field name error.
If select * is used in union, then column counts may not match after adding a new field.
Natural joins may also be affected by an introduction of a new field.
Related
I have 1.7 million records in an access table sorted A to Z. the records are not unique and there are repeated records. I want to make them unique based on their frequency. if a record has been repeated 4 times I want the first one to get "-1" at the end of the record value, the second record get "-2" and so on. in this way similar records will become unique. all similar record are beside each other because of sorting. in excel I do this task by an If function (if this cell value<>the cell value above then "1" else above repeat number plus 1) but in access I don't know what to do (I'm a beginner).
finally I want to add a column to original table which is (original record value - repeat number).
I appreciate your help
Note about sort order:
Sort order in a relational database is not concrete like in a spreadsheet. There is no concept of rows being "next to each other", unless in context of an index. An index is largely a tool for the database to handle the data more efficiently (and to aid in defining uniqueness). The order itself is still largely dynamic because the order of a particular query can be specified differently from the index (or from storage order) and this does not change how the data is actually stored. Being "next to each other" is essentially a useless concept in SQL queries, unless you mean "next to each other numerically", for instance with an AutoNumber field or with the "repeat numbers" you want to add. Unlike in a spreadsheet, you cannot refer to the row "just above this row" or the "row offset by 2 from the 'current' row".
Solution
Regardless of whether or not you will use the AutoNumber column later, add a Long Integer AutoNumber column anyway. This column is named [ID] in the example code. Why? Because until you add something to allow the database to differentiate between the rows, there is technically no way using standard SQL to reliably reference individual duplicates since there is no way to distinguish individual rows. Even though you say that there are other differentiating columns, your own description rules out using them as a reliable key in referring to specific rows. (Even without such a differentiating column, Access can technically distinguish between rows. Iterating through a DAO.Recordset object in VBA would work, but perhaps not very elegant / efficient.)
Also add a new integer column for counting repeats, which below is named [DupeIndex]. A separate field is preferred (necessary?) because it allows continued reference to the original, unaltered duplicate values. If the reference number were directly updated, it would no longer match other fields and so would not be easily detected as a duplicate anymore. The following solution relies on grouping of ALL duplicate values, even those already "marked" with a [DupeIndex] number.
You should also realize that in comparing different data sets, that having separate fields allows more flexibility in matching the data. Having the values appended to the reference number complicates comparison, since you likely not only want to compare rows with the same duplication index, rather you will want to compare all possible combinations. For example, comparing records 123-1 in one set to 123-4 in another... how do you select such rows in an automated fashion? You don't want to have to manually code all combinations, but that's what you'll end up doing if you don't keep them separate like {123,1} and {123,4}.
Create and save this as a named query [Duplicates]. This query is referenced by later queries. It could instead be embedded as a sub query, but my preferences is to use saved queries for easier visualization and debugging in Access:
SELECT Data.RefNo, Count(Data.ID) AS Dupes, Max(Data.DupeIndex) AS IndexMax
FROM Data
GROUP BY Data.RefNo
HAVING Count(Data.ID) > 1
Execute the following to create a temporary table with new duplicate index values:
SELECT D1.ID, D1.RefNo,
IIf([Duplicates].[IndexMax] Is Null,0,[Duplicates].[IndexMax])
+ 1
+ (SELECT Count(D2.ID) FROM Data As D2
WHERE D2.[RefNo]=[D1].[RefNo]
And [D2].[DupeIndex] Is Null
And [D2].[ID]<[D1].[ID]) AS NewIndex
INTO TempIndices
FROM Data AS D1 INNER JOIN Duplicates ON D1.RefNo = Duplicates.RefNo
WHERE (D1.DupeIndex Is Null);
Execute the update query to set the new duplicate index values:
UPDATE Data
INNER JOIN TempIndices ON Data.ID = TempIndices.ID
SET Data.DupeIndex = [NewIndex]
Optionally remove the AutoNumber field and now assign the combined [RefNo] and new [DupeIndex] as primary key. The temporary table can also be deleted.
Comments about the queries:
Solution assume that [DupeIndex] is Null for unprocessed duplicates.
Solution correctly handles existing duplicate index numbers, only updating duplicate rows without an unique index.
Access has rather strict conditions for UPDATE queries, namely that updates are not based on circular references and/or that that joins will not produce multiple updates for the same row, etc. The temporary table is necessary in this case, since the query determining new index values refers multiple times in sub queries to the very column that is being updated. (If the update is attempted using joins on the subqueries, for example, Access complains that Operation must use an updatable query.)
I have a few tables in SQL that require content filtering, primarily for profanity. I want to allow my application(s) to insert data they want and have the server replace any profanity with asterisks such that I do not need to implement filtering on a variety of platforms.
I know triggers could be used for future, however, I am trying to determine the most efficient way to complete this task.
Here are some details:
There are 2 tables I need to ensure has content filtering as they are public facing: feedback and users. Here are the particular fields:
Table -> Fields
Feedback -> Subject, Message
Users -> Firstname, Lastname, Alias
I am relatively new to MySQL and know that having a table of values to replace may be the easiest-to-modify option.
My question is:
How would I join 2 tables and replace particular chars with asterisks using key words located in a third table?
I have these queries so far to locate the columns of interest, just not sure how to incorporate the replacement function and the ability to check both at the same time:
SELECT u.firstname, u.lastname, u.username FROM users u, feedback f, terms t;
SELECT f.subject, f.message FROM feedback f;
You are better off creating a new column (named alias or similar) and storing values with asterisks in there than writing a SELECT query and performing find-replace. Following are the advantages:
Handling this scenario in trigger means you will only perform this operation when a record gets inserted or updated, whereas in SELECT query, each read will need replacing.
You can't really use join here because (a) each value of feedback and user table needs to be compared with all the values of terms table and (b) this needs to be performed for all the columns that might contain these words. So, it's more of a use case for cursor than join.
Is it a standard behavior for HeidiSQL not to override fields that should be overriden in the result set?
I run SQL query, where two tables have the same fields, one table is main and other is from join. I have to select all fields from main table (e.g. use "*"), and some fields from joined table must override values of main table. And it actually works when we fetch results with PDO (cause as a result we got an array value with the same key overrides previous one). but in HeidiSQL I got this:
Ok, got it by myself.
According to documentation:
MySQL permits duplicate column names. That is, there can be more than one select_expr with the same name. This is an extension to standard SQL. Because MySQL also permits GROUP BY and HAVING to refer to select_expr values, this can result in an ambiguity:
SELECT 12 AS a, a FROM t GROUP BY a;
In that statement, both columns have the name a. To ensure that the correct column is used for grouping, use different names for each select_expr.
I haven't attached importance to this before.
My first mySQL project.
I am migrating a FileMaker DB to mySQL and having trouble with how to efficiently handle duplicate field (column) names from 3 left joined tables, combined with the fact that each table is supplying a large number of columns (50+). I understand the concept of aliasing columns. Is there a better way than to create several hundred alias lines to handle each column from each table? I've searched the site and not found a discussion of handling a large number of columns, which is common in FileMaker DBs...perhaps not in mySQL.
Current code is below, where I created the aliases for only ONE (WebPublish) of the ~50 fields for each of the 3 joined tables:
$query = "SELECT
Artwork.WebPublish as Artwork_WebPublish,
Artist.WebPublish as Artist_WebPublish,
Location.WebPublish as Location_WebPublish
FROM Review
LEFT JOIN Artwork ON Review._kf_ArtworkID = Artwork.__kp_ArtworkID
LEFT JOIN Artist ON Review._kf_ArtistID = Artist.__kp_ArtistID
LEFT JOIN Location ON Review._kf_LocationID = Location.__kp_LocationID
WHERE __kp_ReviewID = ?";
This query produces the desired response for one column from each joined table:
Array
(
[Artwork_WebPublish] => Yes
[Artist_WebPublish] => No
[Location_WebPublish] => Maybe
)
The question is whether I need to expand the aliases the long way to include 49 times more data.
Thanks for you help.
No, there's no SQL syntax for giving column aliases in a "batch" mode, for example applying the table name as a prefix to all columns (by the way, SQLite does support that feature by default).
One way to solve this is to refer to columns by ordinal position instead of by name, in whatever language you use to fetch the results.
Another solution is to define your tables with distinct column names so you avoid the name conflict. Some SQL identifiers, for example constraint names, are already required to be unique within the database they reside in, not only unique within a table. It may be a naming convention you want to use to apply the same rule to column names.
Let's say I have a table and I want to insert a row. The new row's key may already match an existing row's key in the table, in which case I want to update the existing row. Or, it may not exist in the table, in which case the new row should be inserted.
What is the most efficient way to perform such an operation? I was thinking of first doing a SELECT (perhaps with EXISTS) to see if a particular key is present, followed by an UPDATE if present and an INSERT if not. You would probably need to keep an UPDLOCK and a HOLDLOCK for this combination of statements in order to avoid race conditions, as well. This seems overly complicated and inefficient.
I was wondering if there was a more efficient way to do this in SQL Server 2008R2.
SQL Server 2008 and newer have a MERGE statement which does exactly that.
See the MSDN Books Online docs on MERGE for details.
Basically, you need four things:
a source (table or view or inline SELECT statement)
a target
a JOIN condition that links the two
statements for cases when there's a MATCH (rows exists in both source and target), NOT MATCHED (when row doesn't exist in the target yet) and so forth
So you basically define something like:
MERGE (targettable) AS t
USING (sourcetable) AS s
ON (JOIN condition between s and t)
WHEN MATCHED THEN
UPDATE SET t.Col1 = s.Col1, t.Col2 = s.Col2 (etc.)
WHEN NOT MATCHED THEN
INSERT(Col1, Col2, ..., ColN) VALUES(s.Col1, s.Col2, ......, s.ColN)
This is done as one statement and highly optimized by SQL Server.