I'm trying to improve performance by replacing a dynamic field (a transient getter with no underlying database representation) with a derived field so that I can use, e.g., Criteria to query my model. The original dynamic field was pretty simple:
Client resolveClient() {
if (prevCall && prevCall.client) {
return prevCall.client
} else {
return client
}
}
I don't know how to reproduce that with a single MySQL statement, so I figured I would go ahead and stick it into a stored function, defined as follows:
CREATE FUNCTION `request_client`(requestId long) RETURNS varchar(255) CHARSET utf8
begin
declare pci long;
declare clientId long;
declare clientName varchar(255);
select request.prev_call_id
from request
where request.id = requestId
into pci;
if pci is not null then
select call_history.client_id
from call_history
where call_history.call_id = pci
into clientId;
else
select request.client_id
from request
where request.id = requestId
into clientId;
end if;
select clients.client_name
from clients
where clients.client_id = clientId
into clientName;
return clientName;
end;
And then I call that function in a derived field:
String derivedFieldName
static mapping = {
derivedFieldName formula: '(select stored_function(id))'
}
The problem is that now when I run any query on the domain, even as simple as Request.list(), I get the following exception:
Class: java.io.StreamCorruptedException
Message: invalid stream header: 32303135
For extra fun, this is an abstract domain class. I don't know if that really makes any difference; it's still persisted to the database like any other domain, and I'm calling the query on the abstract class itself, not an implementation.
The most frustrating thing is that the derived field itself does work! I can successfully retrieve a client name using it; I just can't query the overall domain.
Finally, I am pretty confident that the derived property is the issue, as I have commented it out and can then successfully query the domain.
If anybody comes across this later, the problem actually was the abstract class. And not just that it's an abstract class -- it's an abstract domain class. Apparently, Grails doesn't support derived properties on those.
To move querying to the database I just had to start saving the resolved client into my Request domain :/
Related
Using the example from the Spring docs, I'm trying to return a value from a mySQL function. I keep getting the error Can't set IN parameter for return value of stored function call;.
I created a mySQL function that works fine (ran in MySQL Workbench). I've written a SimpleJdbcCall statement, set up the parameters as per Spring docs example but consistently get this error. If I turn the function into a procedure, the code works, I just have to retrieve the return value from the result set.
I used https://docs.spring.io/spring/docs/3.0.0.M3/reference/html/ch13s05.html, section 13.5.8 as reference.
CREATE FUNCTION `ScenarioRegistration`(
environment VARCHAR(45),
username VARCHAR(15),
scenario_name VARCHAR(45)) RETURNS int(11)
A couple of SELECT statements followed by an INSERT then
RETURN scenario_id; // The inserted id
Java code:
SimpleJdbcCall simpleJdbcCall = new SimpleJdbcCall(getJdbcTemplate())
.withFunctionName("ScenarioRegistration")
.withoutProcedureColumnMetaDataAccess();
simpleJdbcCall.addDeclaredParameter(new SqlParameter("environment"
,Types.VARCHAR));
simpleJdbcCall.addDeclaredParameter(new SqlParameter("username"
,Types.VARCHAR));
simpleJdbcCall.addDeclaredParameter(new SqlParameter("scenario_name"
,Types.VARCHAR));
SqlParameterSource parameters = new MapSqlParameterSource()
.addValue("environment", environment)
.addValue("username", username)
.addValue("scenario_name", scenario);
simpleJdbcCall.setReturnValueRequired(true);
Integer scenario_id = simpleJdbcCall.executeFunction(
Integer.class, parameters);
All I want the routine to do is give me back the id of the newly inserted scenario.
What I get is:
SQL [{? = call scenarioregistration(?, ?)}]; Can't set IN parameter for return value of stored function call.
I find it interesting that it's taken my THREE input values and changed them to an output and TWO input values.
Anyone enlighten me as to the problem and how to fix it?
Thanks,
Steven.
I would refer to the latest docs here for your answer. It appears Spring is trying to infer the output because you didn't explicity specify one.
Per the docs above there are two valid approaches on calling the desired function with the SimpleJdbcCall:
Inferred Parameters
Because you've specified withoutProcedureColumnMetaDataAccess, Spring isn't going to look and see what the ins/outs are to your function. If you want it easy, just don't specify that and you should be able to do:
SqlParameterSource parameters = new MapSqlParameterSource()
.addValue("environment", environment)
.addValue("username", username)
.addValue("scenario_name", scenario);
Integer scenarioId = new SimpleJdbcCall(getJdbcTemplate())
.withFunctionName("ScenarioRegistration")
.executeFunction(Integer.class, parameters);
Explicit Parameters
If you want to keep withoutProcedureColumnMetaDataAccess turned off for whatever reason, you can do:
Integer scenarioId = new SimpleJdbcCall(getJdbcTemplate)
.withFunctionName("ScenarioRegistration")
.withoutProcedureColumnMetaDataAccess()
.useInParameterNames("environment", "username", "scenario_name")
.declareParameters(
new SqlOutParameter("scenario_id", Types.NUMERIC),
new SqlParameter("environment", Types.VARCHAR),
new SqlParameter("username", Types.VARCHAR),
new SqlParameter("scenario_name", Types.VARCHAR)
).executeFunction(Integer.class, parameters);
Note: It appears that order is critical in this example. The output parameter should be declared first, and the subsequent named IN parameters come last. That is, the order of the parameters ? are ordinal in [{? = call scenarioregistration(?, ?, ?)}])
Alternative NamedParameterJdbcTemplate Solution
Another way to invoke your function is via an actual JDBC call. This could hypothetically save you the grief of using the fine tuning of the SimpleJdbcCall.
Integer scenarioId = namedParameterJdbcTemplate.queryForObject(
"SELECT ScenarioRegistration(:environment, :username, :scenario_name)",
parameters,
Integer.class);
I tried to implement the akka-http rest example provided at
https://github.com/ArchDev/akka-http-rest
but I'm stuck with the
slick.SlickException: This DBMS allows only a single column to be returned from an INSERT, and that column must be an AutoInc column.
at slick.jdbc.JdbcStatementBuilderComponent$JdbcCompiledInsert.buildReturnColumns(JdbcStatementBuilderComponent.scala:67)
Here is the Scala Code:
Signup API:
path("signUp") {
pathEndOrSingleSlash {
post {
entity(as[UsernamePasswordEmail]) { userEntity =>
complete(Created -> signUp(userEntity.username, userEntity.email, userEntity.password))
}
}
}
}
AuthService.scala
def signUp(login: String, email: String, password: String): Future[AuthToken] =
authDataStorage
.saveAuthData(AuthData(UUID.randomUUID().toString, login, email, password.sha256.hex))
.map(authData => encodeToken(authData.id))
AuthDataStorage.scala
...
override def saveAuthData(authData: AuthData): Future[AuthData] =
db.run((auth returning auth).insertOrUpdate(authData)).map(_ => authData)
...
Since I'm new to Scala and Slick, can anyway provide the information why this exception is occurring even though I've defined O.AutoInc in Model. I'm using MySQL RDBMS
The problem is with returning auth. Instead of returning auth i.e complete object, Just return the auto-increment Id id. Slick does not support returning the complete object, though it compiles correctly. It does not generate a valid sql query.
Once you can get access to the auto-increment id then you can build the AuthData using the argument of the function.
Code:
(auth returning auth.map(_.id)).insertOrUpdate(authData)).map(id => authData.copy(id = id))
The exception is the result of a MySQL behavior. As the Slick documentation states:
Many database systems only allow a single column to be returned which must be the table’s auto-incrementing primary key. If you ask for other columns a SlickException is thrown at runtime (unless the database actually supports it).
Change the saveAuthData method to return the id column on an upsert:
override def saveAuthData(authData: AuthData): Future[AuthData] =
db.run((auth returning auth.map(_.id)).insertOrUpdate(authData))
.map(idFromDb => authData.copy(id = idFromDb.getOrElse(authData.id)))
In the above code, idFromDb is a Some[Int] for an insert and a None for an update.
I have defined in my db something like this
CREATE FUNCTION fun_totalInvestorsFor(issuer varchar(30)) RETURNS INT
NOT DETERMINISTIC
BEGIN
RETURN (SELECT COUNT(DISTINCT LOYAL3_SHARED_HOLDER_ID)
FROM stocks_x_hldr
WHERE STOCK_TICKER_SIMBOL = issuer AND
QUANT_PURCHASES > QUANT_SALES);
END;
Now I have received an answer from Stefan Zeiger (Slick lead) redirecting me here: User defined functions in Slick
I have tried (having the following object in scope):
lazy val db = Database.forURL("jdbc:mysql://localhost:3306/mydb",
driver = "com.mysql.jdbc.Driver", user = "dev", password = "root")
val totalInvestorsFor = SimpleFunction.unary[String, Int]("fun_totalInvestorsFor")
totalInvestorsFor("APPLE") should be (23)
Result: Rep(slick.lifted.SimpleFunction$$anon$2#13fd2ccd fun_totalInvestorsFor, false) was not equal to 23
I have also tried while having an application.conf in src/main/resources like this:
tsql = {
driver = "slick.driver.MySQLDriver$"
db {
connectionPool = disabled
driver = "com.mysql.jdbc.Driver"
url = "jdbc:mysql://localhost/mydb"
}
}
Then in my code with #StaticDatabaseConfig("file:src/main/resources/application.conf#tsql")
tsql"select fun_totalInvestorsFor('APPLE')" should be (23)
Result: Error:(24, 9) Cannot load #StaticDatabaseConfig("file:src/main/resources/application.conf#tsql"): No configuration setting found for key 'tsql'
tsql"select fun_totalInvestorsFor('APPLE')" should be (23)
^
I am also planning to call stored procedures that return one tuple of three values, via sql"call myProc(v1).as[(Int, Int, Int)]
Any ideas?
EDIT: When making
sql""""SELECT COUNT(DISTINCT LOYAL3_SHARED_HOLDER_ID)
FROM stocks_x_hldr
WHERE STOCK_TICKER_SIMBOL = issuer AND
QUANT_PURCHASES > QUANT_SALES""".as[(Int)]
results in SqlStreamingAction[Vector[Int], Int, Effect] instead of the suggested DBIO[Int] (from what I infer) suggested by the documentation
I've been running into exactly the same problem for the past week. After some extensive research (see my post here, I'll be adding a complete description of what I've done as a solution), I decided it can't be done in Slick... not strictly speaking.
But, I'm resistant to adding pure JDBC or Anorm into our solution stack, so I did find an "acceptable" fix, IMHO.
The solution is to get the session object from Slick, and then use common JDBC to manage the stored function / stored procedure calls. At that point you can use any third party library that makes it easier... although in my case I wrote my own function to set up the call and return a result set.
val db = Database.forDataSource(DB.getDataSource)
var response: Option[GPInviteResponse] = None
db.withSession {
implicit session => {
// Set up your call here... (See my other post for a more detailed
// answer with an example:
// procedure is eg., "{?=call myfunction(?,?,?,?)}"
val cs = session.conn.prepareCall(procedure.toString)
// Set up your in and out parameters here
// eg. cs.setLong(index, value)
val result = cs.execute()
val rc = result.head.asInstanceOf[Int]
response = rc match {
// Package up the response to the caller
}
}
}
db.close()
I know that's pretty terse, but as I said, see the other thread for a more complete posting. I'm putting it together right now and will post the answer shortly.
I'm using EF5.0 in an ASP.NET MVC app. My Entity Model is named 'DataModel'. Included in the model is a table-valued function that exists in my MSSQL database, named MatchingEntries. It returns a table of integer ids.
I've looked at the DataModel.Context.cs file, that gets generated via the .tt (T4) template file. It has the following code in it:
[EdmFunction("DataEntities", "MatchingEntries")]
public virtual IQueryable<Nullable<int>> MatchingEntries(string term)
{
var termParameter = term != null ?
new ObjectParameter("Term", term) :
new ObjectParameter("Term", typeof(string));
return ((IObjectContextAdapter)this).ObjectContext.CreateQuery<Nullable<int>>("[DataEntities].[MatchingEntries](#Term)", termParameter);
}
The error I am getting results from using this method twice within the one query, such as:
IQueryable<int> one = db.MatchingEntries("\"one*\"");
IQueryable<int> two = db.MatchingEntries("\"two*\"");
List<int> both = one.Intersect(two).ToList();
The error is:
A parameter named 'Term' already exists in the parameter collection. Parameter names must be unique in the parameter collection.
Parameter name: parameter
Is this a known limitation of the classes generated from an EDMX for table-valued functions? With LINQ2SQL I am able to execute this a a single query to the database (that does a JOIN between the 2 outputs from MatchingEntries) and it replaces the parameter name #Term with #p0 and #p1 for the two different instances of the call. I'd like to make Entity Framework do the same.
So, my question is, how can I get EF to work in the same manner and avoid the 'Duplicate parameter' error?
My fallback is to evaluate each call to db.MatchingEntries separately, by putting ToList() after them. My other idea has been to replace the ObjectParameter name in the T4 generated Context.cs class with something randomly generated each time. These feel like hacks that I should be able to avoid.
This answer is Linq to Entities specific. This doesn't have to be done in Linq to SQL (Linqpad).
Thanks to this question I got a pointer to a viable solution:
extend the autogenerated DBContext class (partial class)
add a method with two parameters in the partial class
at calling, pass an index as second parameter
Detailed Answer:
DataEntitys.my.cs:
[EdmFunction("DataEntities", "MatchingEntries")]
public virtual IQueryable<Nullable<int>> MatchingEntries(string term, int index)
{
string param_name = String.Format("k_{0}", index);
var termParameter = term != null ?
new ObjectParameter(param_name, term) :
new ObjectParameter(param_name, typeof(string));
return ((IObjectContextAdapter)this).
ObjectContext.CreateQuery<Nullable<int>>(
String.Format("[DataEntities].[MatchingEntries](#{0})", param_name),
termParameter);
}
Call the function:
foreach (string teil in such)
{
index++;
if (teil.Trim() != "")
res = res.Join(db.MatchingEntries("\"" + teil + "*\"", index), l => l.ID, s => s.KEY, (l, s) => l);
}
Hallo,
I have web service that has multiple methods that can be called. Each time one of these methods is called I am logging the call to a statistics database so we know how many times each method is called each month and the average process time.
Each time I log statistic data I first check the database to see if that method for the current month already exists, if not the row is created and added. If it already exists I update the needed columns to the database.
My problem is that sometimes when I update a row I get the "Row not found or changed" exception and yes I know it is because the row has been modified since I read it.
To solve this I have tried using the following without success:
Use using around my datacontext.
Use using around a TransactionScope.
Use a mutex, this doesn’t work because the web service is (not sure I am calling it the right think) replicated out on different PC for performance but still using the same database.
Resolve concurrency conflict in the exception, this doesn’t work because I need to get the new database value and add a value to it.
Below I have added the code used to log the statistics data. Any help would be appreciated very much.
public class StatisticsGateway : IStatisticsGateway
{
#region member variables
private StatisticsDataContext db;
#endregion
#region Singleton
[ThreadStatic]
private static IStatisticsGateway instance;
[ThreadStatic]
private static DateTime lastEntryTime = DateTime.MinValue;
public static IStatisticsGateway Instance
{
get
{
if (!lastEntryTime.Equals(OperationState.EntryTime) || instance == null)
{
instance = new StatisticsGateway();
lastEntryTime = OperationState.EntryTime;
}
return instance;
}
}
#endregion
#region constructor / initialize
private StatisticsGateway()
{
var configurationAppSettings = new System.Configuration.AppSettingsReader();
var connectionString = ((string)(configurationAppSettings.GetValue("sqlConnection1.ConnectionString", typeof(string))));
db = new StatisticsDataContext(connectionString);
}
#endregion
#region IStatisticsGateway members
public void AddStatisticRecord(StatisticRecord record)
{
using (db)
{
var existing = db.Statistics.SingleOrDefault(p => p.MethodName == record.MethodName &&
p.CountryID == record.CountryID &&
p.TokenType == record.TokenType &&
p.Year == record.Year &&
p.Month == record.Month);
if (existing == null)
{
//Add new row
this.AddNewRecord(record);
return;
}
//Update
existing.Count += record.Count;
existing.TotalTimeValue += record.TotalTimeValue;
db.SubmitChanges();
}
}
I would suggest letting SQL Server deal with the concurrency.
Here's how:
Create a stored procedure that accepts your log values (method name, month/date, and execution statistics) as arguments.
In the stored procedure, before anything else, get an application lock as described here, and here. Now you can be sure only one instance of the stored procedure will be running at once. (Disclaimer! I have not tried sp_getapplock myself. Just saying. But it seems fairly straightforward, given all the examples out there on the interwebs.)
Next, in the stored procedure, query the log table for a current-month's entry for the method to determine whether to insert or update, and then do the insert or update.
As you may know, in VS you can drag stored procedures from the Server Explorer into the DBML designer for easy access with LINQ to SQL.
If you're trying to avoid stored procedures then this solution obviously won't be for you, but it's how I'd solve it easily and quickly. Hope it helps!
If you don't want to use the stored procedure approach, a crude way of dealing with it would simply be retrying on that specific exception. E.g:
int maxRetryCount = 5;
for (int i = 0; i < maxRetryCount; i++)
{
try
{
QueryAndUpdateDB();
break;
}
catch(RowUpdateException ex)
{
if (i == maxRetryCount) throw;
}
}
I have not used the sp_getapplock, instead I have used HOLDLOCK and ROWLOCK as seen below:
CREATE PROCEDURE [dbo].[UpdateStatistics]
#MethodName as varchar(50) = null,
#CountryID as varchar(2) = null,
#TokenType as varchar(5) = null,
#Year as int,
#Month as int,
#Count bigint,
#TotalTimeValue bigint
AS
BEGIN
SET NOCOUNT ON;
BEGIN TRAN
UPDATE dbo.[Statistics]
WITH (HOLDLOCK, ROWLOCK)
SET Count = Count + #Count
WHERE MethodName=#MethodName and CountryID=#CountryID and TokenType=#TokenType and Year=#Year and Month=#Month
IF ##ROWCOUNT=0
INSERT INTO dbo.[Statistics] (MethodName, CountryID, TokenType, TotalTimeValue, Year, Month, Count) values (#MethodName, #CountryID, #TokenType, #TotalTimeValue, #Year, #Month, #Count)
COMMIT TRAN
END
GO
I have tested it by calling my web service methods by multiple threads simultaneous and each call is logged without any problems.