I have to recreate a stored procedure which existed in DB1, and map the existing tables to new tables in the new database DB2.
In DB1, table there was column Fullname and in the new db there are two columns firstname, lastname. I could have concat but there is a user defined function also which truncates all special characters which I have to use.
How do I use the first name and last name columns as one full name column so that it fits this case statement?
I am getting the [P Name] from another table enrolled enr which is like the master table having all the names, getting names from either tables is conditioned as shown in case statement.
I tried searching all blogs and stackexchange but cannot get the desired reply.
masterfinancer = dbo.fn_RemoveSpecialChars(iif(enr.p_name = 'XXXXXXXXX,XXXXXXX(2)', 'XXXXX XXXX XXX GROUP', mv.master_vendor_name))
Sma_finace_key = iif(enr.p_name = 'XXXXXXXXX0L(2)', 1111, mv.ven_key)
[P Name] = case
when enr.p_name = 'INACTIVE ' or enr.p_name = 'UNASSIGNED'
then [P Name]
else dbo.fn_RemoveSpecialChars(enr.p_name)
end
You'll need to use the function on each separate field and join them together. Here I removed the IIF statements (which only work in SQL 2012) and created a FullName field that assumes you want "First, Last". I added COALESCE to avoid the FullName field being NULL when either the first or last names are NULL:
SELECT
masterfinancer = dbo.fn_RemoveSpecialChars(
CASE
WHEN enr.p_name = 'XXXXXXXXX,XXXXXXX(2)'
THEN 'XXXXX XXXX XXX GROUP'
ELSE mv.master_vendor_name
END)
, Sma_finace_key =
CASE
WHEN enr.p_name = 'XXXXXXXXX0L(2)'
THEN 1111
ELSE mv.ven_key
END
, [P Name] =
CASE
WHEN enr.p_name IN('INACTIVE ', 'UNASSIGNED')
THEN [P Name]
ELSE dbo.fn_RemoveSpecialChars(enr.p_name)
END
, FullName =
COALESCE(dbo.fn_RemoveSpecialChars(enr.last_name) + ', ', '') +
COALESCE(dbo.fn_RemoveSpecialChars(enr.first_name), '')
In my database there are 3 column which is Name, Age, Gender.
In the program, I only want to use 1 search button. When the button is clicked, the program determine which 3 of the textbox has input and search for the right data.
How do you work with the query? For example if Name and Gender has text, the query :
"Select * from table Where (Name = #name) AND (Gender = #gender)"
And when only name is entered, I only query for the name. Must I check textbox by textbox whether there is user input and then write multiple query for each of them? Or is there a better way to do this?
Edit (29/5/16) : I tried doing this another way like this
myCommand = New MySqlCommand("Select * from project_record Where
(FloatNo = #floatNo OR FloatNo = 'None') AND
(DevCompanyName = #devCompanyName OR DevCompanyName = 'None') AND
(DevType = #devType OR DevType = 'None') AND
(LotPt = #lotPt OR LotPt = 'None') AND
(Mukim = #mukim OR Mukim = 'None') AND
(Daerah = #daerah OR Daerah = 'None') AND
(Negeri = #negeri OR Negeri = 'None') AND
(TempReference = #tempRef OR TempReference = 'None')", sqlConn)
But as you can guess already it will not work efficiently as well because if I only enter input for DevType and leave other textboxes blank, the query will not pull up all the records for DevType only. It will just display as no records.
Select * from table
Where (Name = #name OR #name is Null)
AND (Gender = #gender OR #gender is Null)
...
it should be one query
Other answers have explained how to simplify the query. It is especially important to get rid of the ORs, since they inhibit any use of indexes.
Once you have the query build cleanly, you need to think about the dataset and decide which columns are usually used for filtering. Then make a few INDEXes for them. You won't be able to provide 'all' possible indexes, hence my admonition that you think about the dataset.
When building indexes, you can have single-column or multiple-column indexes. For your type of data, I would suggest starting with several 2-column indexes. Make sure each index starts with a different column.
For Where (Name = #name) AND (Gender = #gender), here are some notes:
INDEX(gender) is useless because of low 'cardinality';
INDEX(gender, name) might be useful, but the following would be better:
INDEX(name)
Things like name and DevCompanyName are virtually unique, so a 1-column index is probably good.
If you had gender and age, then INDEX(age, gender) might be useful.
MySQL will almost never use two indexes for a single SELECT.
By the way, the construction of the WHERE could be done in a Stored Procedure. You would need CONCAT, PREPARE, etc.
Original answer
(scroll down to see update)
Can you try the following:
build a list only including values of the textboxes that have an input
set a string of the join the items of that list together with the " AND " string
append that string to your standard SELECT statement
The code looks like this:
Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
Dim Predicate1 As String = Me.TextBox1.Text
Dim Predicate2 As String = Me.TextBox2.Text
Dim Predicate3 As String = Me.TextBox3.Text
Dim PredicateList As New List(Of String)
Dim WhereClause As String
Dim Query As String
If Predicate1 <> String.Empty Then
PredicateList.Add("Name=""" & Predicate1 & """")
End If
If Predicate2 <> String.Empty Then
PredicateList.Add("Age=""" & Predicate2 & """")
End If
If Predicate3 <> String.Empty Then
PredicateList.Add("Gender=""" & Predicate3 & """")
End If
WhereClause = String.Join(" AND ", PredicateList.ToArray)
Query = "SELECT * FROM TABLE WHERE " & WhereClause
MessageBox.Show(Query)
End Sub
Update
Further to the comments re SQL injection, here is an updated sample.
Dim Command As SqlClient.SqlCommand
Dim Predicate1 As String = Me.TextBox1.Text
Dim Predicate2 As String = Me.TextBox2.Text
Dim Predicate3 As String = Me.TextBox2.Text
Dim ParameterList As New List(Of SqlClient.SqlParameter)
Dim PredicateList As New List(Of String)
Dim BaseQuery As String = "SELECT * FROM TABLE WHERE "
If Predicate1 <> String.Empty Then
PredicateList.Add("name = #name")
ParameterList.Add(New SqlClient.SqlParameter("#name", Predicate1))
End If
If Predicate2 <> String.Empty Then
PredicateList.Add("age = #age")
ParameterList.Add(New SqlClient.SqlParameter("#age", Predicate2))
End If
If Predicate3 <> String.Empty Then
PredicateList.Add("gender = #gender")
ParameterList.Add(New SqlClient.SqlParameter("#gender", Predicate3))
End If
Command = New SqlClient.SqlCommand(BaseQuery & String.Join(" AND ", PredicateList.ToArray))
Command.Parameters.AddRange(ParameterList.ToArray)
COALESCE is your friend here. You can use it to make the where clause ignore comparisons where the parameter is NULL.
Select * from table Where (Name = COALESCE(#name,table.Name))
AND (Gender = COALESCE(#gender,table.Gender))
So, if the #name parameter is NULL, COALESCE(#name,table.Name) will return the value of the 'Name' column of the current row and (Name = COALESCE(#name,table.Name)) will always be true.
This assumes that if no value is entered in a textbox the corresponding parameter will be NULL. If instead it is a value such as 'None', you can use the NULLIF function to map 'None' to NULL
Select * from table Where
(Name = COALESCE( NULLIF( #name, 'None'), table.Name))
AND (Gender = COALESCE( NULLIF( #gender, 'None'), table.Gender))
How to implement a more efficient search?
The answer partly depends on what your definition of efficient is. I suspect you mean less code and fewer if blocks etc. But fundamentally, running a new SELECT * query to apply a filter is inefficient because your base data set can be all the rows and you just fiddle with the users View of it.
I have a DB with random data in columns for Fish, Color (string), Bird, Group (int) and Active which should be similar enough for Name, Age and Gender in the question - or that other long thing at the bottom.
DataTable
Fill a datatable and bind it to a DGV:
' form level object
Private dtSample As DataTable
...
' elsewhere
Dim sql = "SELECT Id, Name, Descr, `Group`, Fish, Bird, Color, Active FROM Sample"
Using dbcon As MySqlConnection = New MySqlConnection(MySQLConnStr)
' create SELECT command with the Query and conn
Dim cmd As New MySqlCommand(sql, dbcon)
...
daSample.Fill(dtSample)
daSample.FillSchema(dtSimple, SchemaType.Source)
End Using
dgv2.DataSource = dtSample
Going forward, we can filter the user's view of that table without issuing a new query.
Filter Controls
If some of the fields are limited to certain selections, for instance Gender, you can use a ComboBox instead of a TextBox. This is to help the user succeed and avoid typos (Make or Mael instead of Male; or here, correctly spelling Baracuda I mean Baraccuda, er Barracuda correctly.
For illustration purposes, Fish is something where the user can type in anything at all, but Bird is constrained to a set of choices. If there is a Bird table, cboBird can be bound or populated from it. But you may also be able to populate it from the master/base table:
Dim birds = dtSample.AsEnumerable.Where(Function(d) d.IsNull(5) = False).
Select(Function(d) d.Field(Of String)("Bird")).
Distinct.
ToArray()
cboBird.Items.AddRange(birds)
If "Finch" is a legal choice but there are none in the database, it wont show in the list. Depending on the app, this can be a Good Thing:
If the user filters on Finch and there a no resulting records, you won't need a MessageBox or StatusBar message explaining the empty result set.
If something is not in the list, you are signalling up front that there are none of those. It then becomes a matter of training why a known element isnt in the list.
On the other hand, you'd have to repopulate those filter controls each time before they are used in case new records were added recently. If the controls are on a Dialog or different TabPage, this is easy to do as needed.
It isnt always applicable, but it can help the user avoid typos.
It depends on the app whether either method is of value.
DBNull / 'none'
I am not sure why you are adding 'none' to each clause. If someone want to see all the 'John` or all the 'Cod' records, it doesn't seem like they would also be interested in 'none'. Personally, Null/DBNull seems a better way to handle this, but it is easy to add or not add either form.
It would seem more valuable to filter to just those with DBNull/None. The code above for the Bird List filters out DBNull and I would do so for none as well. Then, before the result is added to the ComboBox, add a `None' item first so it is at the top.
Again it depends on what the app does; Or = 'None', may make perfect sense in this case.
Filter
Using a TextBox for Fish and Group, a ComboBox for Bird and Color and a CheckBox for Active, the code can form the filter thusly:
Dim filterTerms As New List(Of String)
Dim filterFmt = "{0} = '{1}' "
' OR:
' Dim filterFmt = "{0} = '{1}' OR {0} Is Null"
' OR:
' Dim filterFmt = "{0} = '{1}' OR {0} = 'none'"
If String.IsNullOrEmpty(tbSearchFish.Text) = False Then
Dim txt = tbSearchFish.Text.Replace("'", "''")
filterTerms.Add(String.Format(filterFmt, "Fish", txt))
End If
If cboBird.SelectedIndex > -1 Then
filterTerms.Add(String.Format(filterFmt, "Bird", cboBird.SelectedItem.ToString))
End If
If String.IsNullOrEmpty(tbGroup.Text) = False Then
Dim n As Int32
If Int32.TryParse(tbGroup.Text, n) Then
filterTerms.Add(String.Format(filterFmt, "[Group]", n))
End If
End If
If cboColor.SelectedIndex > -1 Then
filterTerms.Add(String.Format(filterFmt, "Color", cboColor.SelectedItem.ToString))
End If
If chkActive.Checked Then
' NOTE: I do not have TreatTinyAsBoolean turned on
' for some reason
filterTerms.Add(String.Format(filterFmt, "Active", "1"))
End If
If filterTerms.Count > 0 Then
Dim filter = String.Join(" AND ", filterTerms)
dtSample.DefaultView.RowFilter = filter
Dim rows = dtSample.DefaultView.Count
End If
Use whichever filterFmt is appropriate for what the app needs to do
A filter term is only added to the list if the related control has a value (as per above, this could include a 'None').
For the TextBox, it escapes any embedded ticks such as might be found in names like O'Malley or D'Artgnan. It replaces one tick with two.
Since Group is a numeric, a valid Int32 input is tested
If there are elements in the filterTerms list, a filter string is created
The filter is applied to the DefaultView.Filter (you can use also use a DataView or a BindingSource) so that the code need not query the database to provide filter capabilities.
Rows will tell you how many rows are in the current View.
The only halfway tricky one is a Boolean like Gender or Active because those actually resolve to three choices: {Any/Either, A, B}. For that, I would use a ComboBox and ignore it for SelectedIndex 0 as well. I didn't bother with this because the Combo concept is amply covered. Result:
Is it More "Efficient"?
It still depends.
It doesn't re-query the database to get rows the app can already have.
No new DBConnection, DBCommand or other DBProvider objects are created, just a list.
No need to dynamically create a SQL statement with N parameters in a loop to avoid SQL injection/special words and chars.
It doesn't even query the database for the items for the filter terms. If there is a static list of them in the DB, they could be loaded once, the first time they use the filters.
It is easy to remove the filter, no need to query yet again without WHERE clauses.
A ComboBox where applicable helps the user find what they want and avoid typos.
Is the SQL "cleaner". more "efficient? The code doesn't really mess with new SQL, just some WHERE clauses.
Is there less code? I have no idea since we just see the result. It doesnt string me as a lot of code to do what it does.
In my database there are 3 column which is Name, Age, Gender. In the program, I only want to use 1 search button. When the button is clicked, the program determine which 3 of the textbox has input and search for the right data.
And when only name is entered, I only query for the name. Must I check textbox by textbox whether there is user input and then write multiple query for each of them? Or is there a better way to do this?
SELECT * FROM `table`
WHERE (`name` = #name AND `name` IS NOT NULL)
OR (`age` = #age AND (`age`>0 OR `age` IS NOT NULL))
OR (`gender` = #gender AND `gender` IS NOT NULL);
With the above query if all text boxes have value, the result will not be one record (as if you where using logical AND between fields). If you want only that record you will filter it server-side with php from the rest of the results.
You can check the results on your own in this Fiddle
EDIT
In order to solve the above inconvenience (not bringing easily single results when needed) i got a little help from this answer and re-wrote the above query as:
SELECT *, IF(`name`=#name, 10, 0) + IF(`age`=#age, 10, 0) + IF(`gender`=#gender, 10, 0) AS `weight`
FROM `table`
WHERE (`name` = #name AND `name` IS NOT NULL)
OR (`age` = #age AND (`age`>0 OR `age` IS NOT NULL))
OR (`gender` = #gender AND `gender` IS NOT NULL)
HAVING `weight`=30;
OR to still get all records with a weight on result
SELECT *, IF(`name`=#name, 10, 0) + IF(`age`=#age, 10, 0) + IF(`gender`=#gender, 10, 0) AS `weight`
FROM `table` WHERE (`name` = #name AND `name` IS NOT NULL)
OR (`age` = #age AND (`age`>0 OR `age` IS NOT NULL))
OR (`gender` = #gender AND `gender` IS NOT NULL)
ORDER BY `weight` DESC;
You were pretty close. Let's look at
(FloatNo = #floatNo OR FloatNo = 'None')
So you want the field either to be the given input or 'None'? But there are (supposedly) no records in your table with FloatNo 'None'. What you really want to do is find out whether the input is none (i.e. empty):
(FloatNo = #floatNo OR #floatNo = '')
And for the case the user types in a blank by mistake, you can ignore this, too:
(FloatNo = #floatNo OR TRIM(#floatNo) = '')
The whole thing:
myCommand = New MySqlCommand(
"Select * from project_record Where
(FloatNo = #floatNo OR TRIM(#floatNo) = '') AND
(DevCompanyName = #devCompanyName OR TRIM(#devCompanyName) = '') AND
(DevType = #devType OR TRIM(#devType) = '') AND
(LotPt = #lotPt OR TRIM(#lotPt) = '') AND
(Mukim = #mukim OR TRIM(#mukim) = '') AND
(Daerah = #daerah OR TRIM(#daerah) = '') AND
(Negeri = #negeri OR TRIM(#negeri) = '') AND
(TempReference = #tempRef OR TRIM(#tempRef) = '')", sqlConn)
What is wrong with your approach?
Just change
(FloatNo = #floatNo OR FloatNo = 'None')
to
(FloatNo = #floatNo OR FloatNo = '' or FloatNo IS NULL)
And do that for every criteria.
Your query will respect empty values and NULL values after that.
I have created a table with 10 columns in matlab database toolbox. How do i retrieve the values and show them in matlab GUI? when I use the code below, I get Undefined function 'fetch' for input arguments of type 'double' error. Can anybody tell me what is wrong with my code?
f = getappdata(0,'fvalue');
%Set preferences with setdbprefs.
setdbprefs('DataReturnFormat', 'cellarray');
setdbprefs('NullStringRead', 'null');
%Make connection to database.
conn = database('marine_invertebrates', '', '');
%Read data from database.
curs = fetch(conn, sprintf(['SELECT description.commonName'...
' , description.scientificName'...
' , description.kingdom'...
' , description.phylum'...
' , description.subphylum'...
' , description.class'...
' , description.order'...
' , description.family'...
' , description.genus'...
' , description.species'...
' FROM marine_cbir.description WHERE description.imageID = "%s"'], num2str(f)));
curs = fetch(curs);
close(curs);
%Assign data to output variable
results = curs.Data;
commonName = set(handles.edit11,'String');
display (commonName);
scientificName = set(handles.edit1,'String');
display (scientificName);
kingdom = set(handles.edit2,'String');
display (kingdom);
phylum = set(handles.edit3,'String');
display (phylum);
subphylum = set(handles.edit4,'String');
display (subphylum);
class = set(handles.edit5,'String');
display (class);
order = set(handles.edit6,'String');
display (order);
family = set(handles.edit7,'String');
display (family );
genus = set(handles.edit8,'String');
display (genus);
species = set(handles.edit9,'String');
display (species );
Im looking to update 500+ records in my mysql database so that the fields will be a value combination of an $incremental_value+db_user_first_name+#some_static_text. An example of the wished outcome:
1_firstname#staticstring.com, 2_george#staticstring.com, 3_johnny#staticstring.com etc.
I've been playing around with some approach as the following, but that naturally doesn't work (modified for hopefully better clarification).
UPDATE user
SET email = (($incremental_value+1)+(user.first_name))"#staticstring.com"
WHERE email = "empty#empty.com"
The correct syntax for string concatenation in MySQL is the concat() function:
UPDATE user cross join
(select #i = VALUETOSTART) var
SET email = concat(#i := #i + 1, '_', user.first_name, '#staticstring.com')
WHERE email = 'empty#empty.com';
Were are facing a big problem with string length control in SQL Server 2008.
A brief recap of our system:
import data in a persistent staging area from *.txt file (semicolon as separator), using bulk insert in SQL Server environment;
in PSA table all columns are varchar(MAX);
cleaning operations using insert statement based on a select with multiple where conditions.
The problem we deal with is on a single column type and length, in fact in data warehouse level it has to be numeric and its lengths must not exceed 13 digits.
The select is the following:
select cast(LTRIM(RTRIM(data_giacenza)) as numeric),
LTRIM(RTRIM(codice_socio)),
LTRIM(RTRIM(codice_gln)),
LTRIM(RTRIM(tipo_gln)),
LTRIM(RTRIM(codice_articolo_socio)),
LTRIM(RTRIM(codice_ean_prodotto)),
LTRIM(RTRIM(codice_ecat_prodotto)),
LTRIM(RTRIM(famiglia)),
LTRIM(RTRIM(marca)),
LTRIM(RTRIM(classificazione_liv_1)),
LTRIM(RTRIM(classificazione_liv_2)),
LTRIM(RTRIM(classificazione_liv_3)),
LTRIM(RTRIM(classificazione_liv_4)),
LTRIM(RTRIM(modello)),
LTRIM(RTRIM(descrizione_articolo)),
cast(LTRIM(RTRIM(giacenza)) as numeric),
cast(LTRIM(RTRIM(acquistato)) as numeric), 'X' FROM psa_stock a
where EXISTS
(
SELECT 0
FROM(
SELECT
data_giacenza
,codice_socio
,codice_gln
,codice_articolo_socio
FROM psa_stock
where
LEN(LTRIM(RTRIM(data_giacenza))) = 8 and LEN(LTRIM(RTRIM(codice_socio))) = 3
and LEN(LTRIM(RTRIM(codice_gln))) = 13 and LEN(LTRIM(RTRIM(tipo_gln))) = 3
and LEN(LTRIM(RTRIM(codice_articolo_socio))) <= 15
and (LEN(LTRIM(RTRIM(codice_ean_prodotto))) <= 13 or LEN(ISNULL(codice_ean_prodotto, '')) = 0)
and (LEN(LTRIM(RTRIM(codice_ecat_prodotto))) = 9 or LEN(ISNULL(codice_ecat_prodotto, '')) = 0)
and LEN(LTRIM(RTRIM(famiglia))) = 2
and (LEN(LTRIM(RTRIM(marca))) <= 20 or LEN(ISNULL(marca, '')) = 0)
and (LEN(LTRIM(RTRIM(modello))) <= 30 or LEN(ISNULL(modello, '')) = 0)
and (LEN(LTRIM(RTRIM(descrizione_articolo))) <= 50 or LEN(ISNULL(descrizione_articolo, '')) = 0)
and LEN(LTRIM(RTRIM(giacenza))) <= 5
and LEN(LTRIM(RTRIM(acquistato))) <= 5
and (LEN(LTRIM(RTRIM(classificazione_liv_1))) <= 15 or LEN(ISNULL(classificazione_liv_1, '')) = 0)
and (LEN(LTRIM(RTRIM(classificazione_liv_2))) <= 15 or LEN(ISNULL(classificazione_liv_2, '')) = 0)
and (LEN(LTRIM(RTRIM(classificazione_liv_3))) <= 15 or LEN(ISNULL(classificazione_liv_3, '')) = 0)
and (LEN(LTRIM(RTRIM(classificazione_liv_4))) <= 15 or LEN(ISNULL(classificazione_liv_4, '')) = 0)
and ISNUMERIC(ltrim(rtrim(REPLACE(data_giacenza, ' ', '')))) = 1
and ISNUMERIC(ltrim(rtrim(REPLACE(codice_gln, ' ', '')))) = 1
and ISNUMERIC(LTRIM(RTRIM(REPLACE(giacenza, ' ', '')))) = 1 and charindex(',', giacenza) = 0
and ISNUMERIC(LTRIM(RTRIM(REPLACE(acquistato, ' ', '')))) = 1
and ISNUMERIC(ltrim(rtrim(REPLACE(codice_ean_prodotto, ' ', '')))) = 1
and ISNUMERIC(ltrim(rtrim(REPLACE(codice_ecat_prodotto, ' ', '')))) = 1
and codice_socio in (select codice_socio from ana_socio)
and tipo_gln in (select tipo from ana_gln)
and codice_gln in (select codice_gln from dw_key_gln)
group by
data_giacenza
,codice_socio
,codice_gln
,codice_articolo_socio
having COUNT (*) = 1
) b
where
a.data_giacenza = b.data_giacenza and
a.codice_articolo_socio = b.codice_articolo_socio and
a.codice_socio = b.codice_socio and
a.codice_gln = b.codice_gln)
The critical field is codice_ean_prodotto.
In fact, it allows to consider also values as SEAGAT7636490026751,NE20000003039,NE20000002168 which are not numeric and, the first, overlap maximum dimensions.
As result, the insert statement gives back
String o binary data would be truncated
error and fails the insertion.
Thanks in advance! I look forward your help!!!
Enrico
Have you tried executing that query, and adding codice_ean_prodotto = 'NE20000003039' to the where clause? Be sure that these are the actual field that is giving you the problem. If the select returns a row with that where clause, then something's wrong with the logic.
I'm leaning towards your having COUNT (*) = 1 clause in the EXISTS subquery - is it possible to have more than one record for these specific keys? As long as your PK is made up of those 4 fields (data_giacenza, codice_articolo_socio, codice_socio, codice_gln), you shouldn't need the GROUP BY and HAVING clauses at all. If you're not joining on your primary key, it could be that it is the culprit.
It's hard to tell without seeing your data model, however.
I figured out what was wrong.
In the inner select, we were excluding from the selection all records not respecting format constraints and duplication (the meaning of count(*)=1), extracting only the PK of the destination table.
But when selecting with PK we retrieve also those record that were duplicates, but were excluded by the format constraint, leading the insert to error due to dimension issues.
Now I divided the steps:
Duplicates lookup and deletion
Selection with format constraints
It works!