I'm building an object to search orders in my database. There are a bunch of possible parameters that the user can set, and they can set as many as then want for each search. I've created setter methods to collect all the parameters needed for the search.
My question is this. What would be "best practice"
Storing the parameters, and building the WHERE clause when the doSearch method is called
Building the WHERE clause as parameters are set
I'd like to understand the reason behind any recommendation.
Note that the object is instatiated for each search, so I don't have to worry about a second search with different parameters.
You should separate the code for your order search from the code that builds the SQL. The SQL should be built in a derivative (or Strategy derivative) of the OrderSearch class. Once you have made this separation, it doesn't really matter when you build the SQL.
To make this a bit more plain. Given a class named OrderSearch which has a bunch of setter methods for the search criteria, you'd like to have a subclass named OrderSearchSQLBuilder. Notice that the subclass depends on the base class, and that the base class is independent of the subclass. This is very important. This independence allows you to disregard whether the SQL is built in the setter methods, or in the search method. See The Dependency Inversion Principle (DIP).
Once you have this kind of seperation, you can replace the derivative with other strategies. For example, if you wanted to test your application without connecting it to the SQL database, you could create a dummy in-ram database and create a derivative of OrderSearch that dealt with that dummy database. The rest of the application would be blissfully unaware, and your tests would then be independent of the horrors of database connections, pre-existing data, etc.
In your method, simply use the parameters in your dynamic SQL that does the search. That way the where clause is built just prior to the SQL getting run. You will simply pass the search parameters into your method as arguments.
Something like this...
<cffunction name="getByAttributesQuery" access="public" output="false" returntype="query">
<cfargument name="id" type="numeric" required="false" />
<cfargument name="userName" type="string" required="false" />
<cfargument name="firstName" type="string" required="false" />
<cfargument name="lastName" type="string" required="false" />
<cfargument name="createdAt" type="date" required="false" />
<cfargument name="updatedAt" type="date" required="false" />
<cfargument name="orderby" type="string" required="false" />
<cfset var qList = "" />
<cfquery name="qList" datasource="#variables.dsn#">
SELECT
id,
userName,
firstName,
lastName,
createdAt,
updatedAt
FROM users
WHERE 0=0
<cfif structKeyExists(arguments,"id") and len(arguments.id)>
AND id = <cfqueryparam value="#arguments.id#" CFSQLType="cf_sql_integer" />
</cfif>
<cfif structKeyExists(arguments,"userName") and len(arguments.userName)>
AND userName = <cfqueryparam value="#arguments.userName#" CFSQLType="cf_sql_varchar" />
</cfif>
<cfif structKeyExists(arguments,"firstName") and len(arguments.firstName)>
AND firstName = <cfqueryparam value="#arguments.firstName#" CFSQLType="cf_sql_varchar" />
</cfif>
<cfif structKeyExists(arguments,"lastName") and len(arguments.lastName)>
AND lastName = <cfqueryparam value="#arguments.lastName#" CFSQLType="cf_sql_varchar" />
</cfif>
<cfif structKeyExists(arguments,"createdAt") and len(arguments.createdAt)>
AND createdAt = <cfqueryparam value="#arguments.createdAt#" CFSQLType="cf_sql_timestamp" />
</cfif>
<cfif structKeyExists(arguments,"updatedAt") and len(arguments.updatedAt)>
AND updatedAt = <cfqueryparam value="#arguments.updatedAt#" CFSQLType="cf_sql_timestamp" />
</cfif>
<cfif structKeyExists(arguments, "orderby") and len(arguments.orderBy)>
ORDER BY #arguments.orderby#
</cfif>
</cfquery>
<cfreturn qList />
</cffunction>
Don't build the where clause before it is needed to execute the search. You may end up with a user interface that feeds parameters in iterations, and you don't know when you have everything. Also, you may never execute the search, so why worry about the where clause.
I don't think it makes much difference, but I think it seems better practice to build the WHERE clause when you doSearch. I don't think it should be the responsibility of a setter for a parameter to add to a WHERE clause string somewhere.
On an SQLServer database it is more efficient to include all parameters in the where clause rather than building it on the fly. Then have an database index which includes all the columns you will search on. This ensures the index is always used when executing the statement.
Dosn't quite sound like your abstraction is quite right.
Search may be better as a method of a "orders" object. pass in the parameters to the search function and build the query manually as russ suggested. Anything which is specific to orders rather than the search can then be set on the orders init method.
It is possible that you may want to build an orders search object but this should be done via an orders object, to keep your front end code simple.
Option 1 is your best bet. Option 2 sounds dangerous. What if a parameter is updated? How do you replace it in your WHERE clause?
I would do something like this:
<cffunction name="doSearch" access="public" output="false" returntype="query">
<cfset var qList = "" />
<cfquery name="qList" datasource="#variables.dsn#">
SELECT
...
FROM
...
WHERE 0=0
<cfif len(getID())>
AND id = <cfqueryparam value="#getID()#" CFSQLType="cf_sql_integer" />
</cfif>
<cfif len(getUserName())>
AND userName = <cfqueryparam value="#getUserName()#" CFSQLType="cf_sql_varchar" />
</cfif>
</cfquery>
<cfreturn qList />
</cffunction>
Related
Question: What data type do I use when invoking JSON data from a database using ColdFusion?
Background: My application needs to pull some JSON data from a database and parse the data into an HTML form.
Below is my code so far:
<cftry>
<cfinvoke component="UserData.cfc.data" method="editData" returnvariable="editReturn">
<cfinvokeargument name="formID" value="#URL.dataID#">
</cfinvoke>
<cfset ReBuild = DeserializeJSON(#editReturns#)>
<cfcatch type="Any">
<cfoutput>
<hr>
<h1>Other Error: #cfcatch.Type#</h1>
<ul>
<li><b>Message:</b> #cfcatch.Message#
<li><b>Detail:</b> #cfcatch.Detail#
</ul>
</cfoutput>
<cfset errorCaught = "General Exception">
</cfcatch>
</cftry>
UserData.cfc.data:
<cffunction name="editData" access="public" returntype="any">
<cfargument name="formID" required="yes">
<!--- Select --->
<cfquery name="UserData" datasource="RC">
SELECT Data
FROM USER_Forms
WHERE ID = <cfqueryparam value="#ARGUMENTS.formID#">
</cfquery>
<!-- The information pulled from the database should be a Serialized JSON data. -->
<cfreturn UserData>
</cffunction>
Error Message:
Other Error: Expression
Message: Complex object types cannot be converted to simple values.
Detail: The expression has requested a variable or an intermediate expression result as a simple value. However, the result cannot be converted to a simple value. Simple values are strings, numbers, boolean values, and date/time values. Queries, arrays, and COM objects are examples of complex values.
The most likely cause of the error is that you tried to use a complex value as a simple one. For example, you tried to use a query variable in a cfif tag.
When I added the data to the database I used the following process:
<cfset ForDBInsert = SerializeJSON(formCopy)>
<!-- Then I INSERTED ForDBInsert into the database columnn. -->
Try DeserializeJSON(editReturns.data) - notice I took out the # they are not needed when passing arguments this way. Looks like you are trying to deserialize the entire query object rather than the string itself.
Please consider the following code which I am using to get the data posted by Sendgrid.
<cftry>
<cfset incomingData = toString(getHttpRequestData().content) />
<cfset djs = DeserializeJSON(incomingData)/>
<cfset a = "0">
<cfset b = "">
<cfset c = "0">
<cfset d = "0">
<cfset e = "">
<cfset f = "">
<cfset g = "">
<cfset h = "">
<cfset i = "">
<cfset k = "#NOW()#">
<cfset l = "">
<cfset m = "">
<cfset n = "">
<cfoutput>
<cfloop from="1" to="#arraylen(djs)#" index="i">
<cfset a = "0">
<cfset b = "">
<cfset c = "0">
<cfset d = "0">
<cfset e = "">
<cfset f = "">
<cfset g = "">
<cfset h = "">
<cfset i = "">
<cfset k = "#NOW()#">
<cfset l = "">
<cfset m = "">
<cfset n = "">
<cfif StructKeyExists(djs[i],'p')>
<cfset a = djs[i].p />
</cfif>
<cfif StructKeyExists(djs[i],'q')>
<cfset b = djs[i].q />
</cfif>
<cfif StructKeyExists(djs[i],'r')>
<cfset c = djs[i].r />
</cfif>
<cfif StructKeyExists(djs[i],'s')>
<cfset d = djs[i].s />
</cfif>
<cfif StructKeyExists(djs[i],'t')>
<cfset e = djs[i].t />
</cfif>
<cfif StructKeyExists(djs[i],'u')>
<cfset f = djs[i].u />
</cfif>
<cfif StructKeyExists(djs[i],'v')>
<cfset g = djs[i].v />
</cfif>
<cfif StructKeyExists(djs[i],'w')>
{
<cfset i = djs[i].w />
<cfset k = dateAdd("s", i, createDateTime(1970, 1, 1, 0, 0, 0))/>
}
</cfif>
<cfif StructKeyExists(djs[i],'x')>
<cfset l = djs[i].x />
</cfif>
<cfif StructKeyExists(djs[i],'y')>
<cfset m = djs[i].y />
</cfif>
<cfif StructKeyExists(djs[i],'z')>
<cfset n = djs[i].z />
</cfif>
<cfstoredproc procedure="sp1" datasource="db1">
<cfprocparam cfsqltype="cf_sql_bigint" value="#a#">
<cfprocparam cfsqltype="cf_sql_varchar" value="#left(b,199)#">
<cfprocparam cfsqltype="cf_sql_integer" value="#c#">
<cfprocparam cfsqltype="cf_sql_integer" value="#d#">
<cfprocparam cfsqltype="cf_sql_varchar" value="#left(e,199)#">
<cfprocparam cfsqltype="cf_sql_varchar" value="#left(f,199)#">
<cfprocparam cfsqltype="cf_sql_varchar" value="#left(g,499)#">
<cfprocparam cfsqltype="cf_sql_varchar" value="#left(h,199)#">
<cfprocparam cfsqltype="cf_sql_timestamp" value="#k#">
<cfprocparam cfsqltype="cf_sql_varchar" value="#left(l,199)#">
<cfprocparam cfsqltype="cf_sql_varchar" value="#LEFT(m,499)#">
<cfprocparam cfsqltype="cf_sql_varchar" value="#left(n,99)#">
<cfprocparam cfsqltype="cf_sql_varchar" value="XX.XX.X.XX">
</cfstoredproc>
<cfstoredproc procedure="sp2" datasource="db2">
<cfprocparam cfsqltype="cf_sql_bigint" value="#a#">
<cfprocparam cfsqltype="cf_sql_varchar" value="#left(b,199)#">
<cfprocparam cfsqltype="cf_sql_integer" value="#c#">
<cfprocparam cfsqltype="cf_sql_integer" value="#d#">
<cfprocparam cfsqltype="cf_sql_varchar" value="#left(e,199)#">
<cfprocparam cfsqltype="cf_sql_varchar" value="#left(f,199)#">
<cfprocparam cfsqltype="cf_sql_varchar" value="#left(g,499)#">
<cfprocparam cfsqltype="cf_sql_varchar" value="#left(h,199)#">
<cfprocparam cfsqltype="cf_sql_timestamp" value="#k#">
<cfprocparam cfsqltype="cf_sql_varchar" value="#left(l,199)#">
<cfprocparam cfsqltype="cf_sql_varchar" value="#LEFT(m,499)#">
<cfprocparam cfsqltype="cf_sql_varchar" value="#left(n,99)#">
<cfprocparam cfsqltype="cf_sql_varchar" value="XX.XX.X.XX">
</cfstoredproc>
</cfloop>
</cfoutput>
</cftry>
As clear from the above, my code is dependent on the stored procedure on the MySQL Database. I am thinking of getting rid of stored procedure and find a different way where I can accomplish what I am looking for without any stored procedure.Is there a better way other than using Stored Procedure to store the incoming data into database?
Is there a better way other than using Stored Procedure to store the incoming data into database? No, not really. A better question is why you feel the need to stop using the stored procedures?
You really only have three options when interacting with a database; stored procedures, in-line queries (parameterized), or Object Relational Mapping (ORM). Sure you could replace your stored procedure calls with in-line queries or ORM but I don't think you will really gain anything.
Some of the benefits of using stored procedures rather than in-line queries are:
Reduced server/client network traffic
The commands in a procedure are executed as a single batch of code. This can significantly reduce network traffic between the server and client because only the call to execute the procedure is sent across the network. Without the code encapsulation provided by a procedure, every individual line of code would have to cross the network.
Stronger security
Multiple users and client programs can perform operations on underlying database objects through a procedure, even if the users and programs do not have direct permissions on those underlying objects. The procedure controls what processes and activities are performed and protects the underlying database objects. This eliminates the requirement to grant permissions at the individual object level and simplifies the security layers.
The EXECUTE AS clause can be specified in the CREATE PROCEDURE statement to enable impersonating another user, or enable users or applications to perform certain database activities without needing direct permissions on the underlying objects and commands. For example, some actions such as TRUNCATE TABLE, do not have grantable permissions. To execute TRUNCATE TABLE, the user must have ALTER permissions on the specified table. Granting a user ALTER permissions on a table may not be ideal because the user will effectively have permissions well beyond the ability to truncate a table. By incorporating the TRUNCATE TABLE statement in a module and specifying that module execute as a user who has permissions to modify the table, you can extend the permissions to truncate the table to the user that you grant EXECUTE permissions on the module.
When calling a procedure over the network, only the call to execute the procedure is visible. Therefore, malicious users cannot see table and database object names, embed Transact-SQL statements of their own, or search for critical data.
Using procedure parameters helps guard against SQL injection attacks. Since parameter input is treated as a literal value and not as executable code, it is more difficult for an attacker to insert a command into the Transact-SQL statement(s) inside the procedure and compromise security.
Procedures can be encrypted, helping to obfuscate the source code. For more information, see SQL Server Encryption.
Reuse of code
The code for any repetitious database operation is the perfect candidate for encapsulation in procedures. This eliminates needless rewrites of the same code, decreases code inconsistency, and allows the code to be accessed and executed by any user or application possessing the necessary permissions.
Easier maintenance
When client applications call procedures and keep database operations in the data tier, only the procedures must be updated for any changes in the underlying database. The application tier remains separate and does not have to know how about any changes to database layouts, relationships, or processes.
Improved performance
By default, a procedure compiles the first time it is executed and creates an execution plan that is reused for subsequent executions. Since the query processor does not have to create a new plan, it typically takes less time to process the procedure.
If there has been significant change to the tables or data referenced by the procedure, the precompiled plan may actually cause the procedure to perform slower. In this case, recompiling the procedure and forcing a new execution plan can improve performance.
Some of this is specific to SQL Server but most applies to any database
Reference - Stored Procedures (Database Engine)
It depends what that stored proc does. Does it only insert one row of data?
I suggest that you use the loop to put your data into an array or struct then make ONE database call (either via cfquery or cfstoredproc) to insert the data.
I've been building a new coldfusion application leveraging object oriented approach and stored procedures. While everything works like a charm, the app is very fast due to SPs and optimized mysql code, I would really appreciate if you would help me clear one thing up :) I have dbcode.cfc which as you probably already guessed stores all queries with stored procedures. That said, generally what's the best approach when it comes to storing global DNS parameters?
1.this can be used for one global DSN
<cfset this.datasource ="myDB">
2.this can also be used for one global DSN
<cffunction name="onApplicationStart" returnType="boolean" output="false">
<cfset application.dsn = "myDB">
<cfset application.username = "userName">
<cfset application.password = "password">
<cfreturn true>
</cffunction>
2.1 everything as above but onRequest
3.in my case I could also create a global variables within the dbcode.cfc
<cfset variables.dsn = "myDB">
<cfset variables.username = "userName">
<cfset variables.password = "password">
4.additionally one could use something like this for setting multiple datasources
<cffunction name="onApplicationStart" returnType="boolean" output="false">
<cfset application.myDSNs = StructNew()>
<cfset application.myDSNs.1 = "myDB1">
<cfset application.myDSNs.2 = "myDB2">
.
.
.
<!---something similar for usernames and passwords--->
</cffunction>
so what's the most efficient way to handle this sort of thing?
Let me see if I can answer each part of your DSN questions.
Setting the datasource as a part the application.cfc constructor is OK
Setting up the DSN in OnApplicationStart() works too, but you need to use:
<cfset application.datasource = "myDB">
2.1 Setting the datasource on every request is pointless, unless you datasource changes with every request. I suspect is does not
You should not touch queries.cfc. That is internal to ColdFusion and has the potential to break everything on your server.
You may want to read on up on the difference between arrays and structs. It looks like what you wanted to do was:
<cfset application.myDSNs = ["MyDB1", "MyDB2]>
They would then be accessable via:
#application.myDSNs[1]#
#application.myDSNs[2]#
I would not do this approach. Databases are typically not arrays of anything. Each server has its own purpose. You DSNs should be reflective of one of more of the following:
The Server
The Database
Purpose
I have a problem with saving a huge amount of records to database using CFWheels. Here is an example:
<cfloop from="1" to="10000" index="i">
<cfset var newUser = model("user").new()>
<cfset newUser.name = "Test"&i>
<cfset newUser.save()>
</cfloop>
This causes java.lang.OutOfMemoryError
Please help me how to solve this problem.
Looping over multiple database calls leading to OOM is a known ColdFusion bug. Fortunately, there is a workaround, use <cfthread/>. You should be able to alter your code as such:
<cfloop from="1" to="10000" index="i">
<cfset threadName = "thread" & createUuid()>
<cfthread name="#threadName#">
<cfset var newUser = model("user").new()>
<cfset newUser.name = "Test"&i>
<cfset newUser.save()>
</cfthread>
<cfthread action="join" name="#threadName#">
</cfloop>
In this situation, you're using the thread solely for its side effect, running in a different context so that it doesn't get retained on the heap. Thus the immediate join right after declaring the thread, so it's not actually running anything in parallel.
You can try to run the Garbage collector:
http://www.beetrootstreet.com/blog/index.cfm/2009/6/25/Clearing-ColdFusion-memory-using-garbage-collection-when-memory-gets-low
There are a couple fairly inefficient things going on here. First, it's generating 1,000 user objects, which isn't really a good idea to do in a single request in ColdFusion. Second, it's running 1,000 database queries, which isn't really a good idea to do in any programming language.
I would stop using model objects for a case like this and figure out how to condense the logic into a single database query. The ORM stuff in Wheels is generally very useful, but it has its limits in situations like this one.
For example, if you're using SQL Server 2008, you can do this inside your user model to keep everything under a single call to cfquery:
<cffunction name="batchCreate">
<cfquery datasource="#get('dataSourceName')#">
INSERT INTO
#this.tableName()# (#this.columnNameForProperty("name")#)
VALUES
<cfloop from="1" to="10000" index="i">
(<cfqueryparam cfsqltype="cf_sql_varchar" value="Test#i#">)
<cfif i lt 10000>,</cfif>
</cfloop>
</cfquery>
</cffunction>
Of course, the query will look different if you're using MySQL or another database engine.
I'm moving from an MS Access backend to mySQL. This used to work but now doesn't and I can't figure the problem.
<cfargument required="false" name="expiry" type="any" default="" />
<cfquery datasource='#arguments.dsn#'>
INSERT INTO users(expiry)
VALUES (<cfqueryparam value="#arguments.expiry#" cfsqltype="CF_SQL_TIMESTAMP"/>)
</cfquery>
The database field is set to datetime and default NULL
The argument is populated from a form field which is either empty, or a javascript validated date. It chokes on empty formfield.
Before you mess with the DSN settings, I would also try changing your <cfqueryparam> to the following:
<cfqueryparam value="#arguments.expiry#" cfsqltype="CF_SQL_TIMESTAMP" null="#len(arguments.expiry) eq 0#" />
This will pass a true null in the event that the argument value is an empty string.
CF's implementation of the JDBC driver for MySQL doesn't handle NULL dates very well.
You need to add a config flag to your DSN connection string settings (under advanced) in the CF admin
&zeroDateTimeBehavior=convertToNull
Should set you right.
Rob