JDBI select on varbinary and uuid - mysql

A legacy mysql db table has an id column that is non-human readable raw varbinary (don't ask me why :P)
CREATE TABLE IF NOT EXISTS `tbl_portfolio` (
`id` varbinary(16) NOT NULL,
`name` varchar(128) NOT NULL,
...
PRIMARY KEY (`id`)
);
and I need to select on it based on a java.util.UUID
jdbiReader
.withHandle<PortfolioData, JdbiException> { handle ->
handle
.createQuery(
"""
SELECT *
FROM tbl_portfolio
WHERE id = :id
"""
)
.bind("id", uuid) //mapping this uuid into the varbinary
//id db column is the problem
.mapTo(PortfolioData::class.java) //the mapper out does work
.firstOrNull()
}
just in case anyone wants to see it, here's the mapper out (but again, the mapper out is not the problem - binding the uuid to the varbinary id db column is)
class PortfolioDataMapper : RowMapper<PortfolioData> {
override fun map(
rs: ResultSet,
ctx: StatementContext
): PortfolioData = PortfolioData(
fromBytes(rs.getBytes("id")),
rs.getString("name"),
rs.getString("portfolio_idempotent_key")
)
private fun fromBytes(bytes: ByteArray): UUID {
val byteBuff = ByteBuffer.wrap(bytes)
val first = byteBuff.long
val second = byteBuff.long
return UUID(first, second)
}
}
I've tried all kinds of things to get the binding to work but no success - any advice much appreciated!

Finally got it to work, partly thanks to https://jdbi.org/#_argumentfactory which actually deals with UUID specifically but I somehow missed despite looking at JDBI docs for hours, oh well
The query can remain like this
jdbiReader
.withHandle<PortfolioData, JdbiException> { handle ->
handle
.createQuery(
"""
SELECT *
FROM tbl_portfolio
WHERE id = :id
"""
)
.bind("id", uuid)
.mapTo(PortfolioData::class.java)
.firstOrNull()
}
But jdbi needs a UUIDArgumentFactory registered
jdbi.registerArgument(UUIDArgumentFactory(VARBINARY))
where
class UUIDArgumentFactory(sqlType: Int) : AbstractArgumentFactory<UUID>(sqlType) {
override fun build(
value: UUID,
config: ConfigRegistry?
): Argument {
return UUIDArgument(value)
}
}
where
class UUIDArgument(private val value: UUID) : Argument {
companion object {
private const val UUID_SIZE = 16
}
#Throws(SQLException::class)
override fun apply(
position: Int,
statement: PreparedStatement,
ctx: StatementContext
) {
val bb = ByteBuffer.wrap(ByteArray(UUID_SIZE))
bb.putLong(value.mostSignificantBits)
bb.putLong(value.leastSignificantBits)
statement.setBytes(position, bb.array())
}
}
NOTE that registering an ArgumentFactory on the entire jdbi instance like this will make ALL UUID type arguments sent to .bind map to bytes which MAY not be what you want in case you elsewhere in your code base have other UUID arguments that are stored on the mysql end with something other than VARBINARY - eg, you may have another table with a column where your JVM UUID are actually stores as VARCHAR or whatever, in which case you'd have to, rather than registering the UUID ArgumentFactory on the entire jdbi instance, only use it ad hoc on individual queries where appropriate.

Related

Reading a row with a NULL column causes an exception in slick

I have a table with a column type date. This column accepts null values,
therefore, I declared it as an Option (see field perDate below). When I
run the select query through the application code I get the following exception
slick.SlickException: Read NULL value (null) for ResultSet column
problem.This
is the Slick table definition:
import java.sql.Date
import java.time.LocalDate
class FormulaDB(tag: Tag) extends Table[Formula](tag, "formulas") {
def sk = column[Int]("sk", O.PrimaryKey, O.AutoInc)
def formula = column[Option[String]]("formula")
def notes = column[Option[String]]("notes")
def periodicity = column[Int]("periodicity")
def perDate = column[Option[LocalDate]]("per_date")(localDateColumnType)
def * =
(sk, name, descrip, formula, notes, periodicity, perDate) <>
((Formula.apply _).tupled, Formula.unapply)
implicit val localDateColumnType = MappedColumnType.base[Option[LocalDate], Date](
{
case Some(localDate) => Date.valueOf(localDate)
case None => null
}, { sqlDate =>
if (sqlDate != null) Some(sqlDate.toLocalDate) else None
}
)
}
Your mapped column function just needs to provide the LocalDate to Date conversion. Slick will automatically handle Option[LocalDate] if it knows how to handle LocalDate.
That means changing your localDateColumnType to be:
implicit val localDateColumnType = MappedColumnType.base[LocalDate, Date](
Date.valueOf(_), _.toLocalDate
)
Chapter 5 of Essential Slick covers some of this, as does the section on User Defined Features in the Manual.
I'm not 100% sure why you're seeing the run-time error: my guess is that the column is being treated as an Option[Option[LocalDate]] or similar, and there's a level of null in there that's being missed.
BTW, your def * can probably be:
def * = (sk, name, descrip, formula, notes, periodicity, perDate).mapTo[Formula]
...which is a little nicer to read. The mapTo was added in Slick 3 at some point.

Fetch the response from sql, store it in a object and use conditions?

I have two sql statements to be executed with a validity check. My need is that I execute the 1st query and store the response in one object and check the object is empty or not and execute the second query if it is not empty.
So, I have tried something like
In rolerepository.scala=>
override val allQuery = s"""
select UserRoles.* from
(select CASE rbac.roleTypeID
ELSE rbac.name JOIN dirNetworkInfo ni
ON UserRoles.PersonID = ni.PersonID
where ni.Loginname = {loginName}
and UserRoles.roleName in ( 'Business User ','Administrator')"""
(This is just some sample of the query - it is not fully written here.)
Then I map it to an object with model class written outside
override def map2Object(implicit map: Map[String, Any]):
HierarchyEntryBillingRoleCheck = {
HierarchyEntryBillingRoleCheck(str("roleName"), oint("PersonID")) }
Then I have written the getall method to execute the query
override def getAll(implicit loginName: String):
Future[Seq[HierarchyEntryBillingRoleCheck]] = {
doQueryIgnoreRowErrors(allQuery, "loginName" -> loginName) }
Then I have written the method to check whether the response from the 1st sql is empty or not. This is were I'm stuck and not able to proceed further.
def method1()= {
val getallresponse = HierarchyEntryBillingRoleCheck
getallresponse.toString
if (getallresponse != " ")
billingMonthCheckRepository.getrepo()
}
I am getting an error (type mismatch) in last closing brace and I don't know what other logic can be used here.
Can any one of you please explain and give me some solution for this?
And i also tried to use for loop in controller but not getting how to do that.
i tried ->
def getAll(implicit queryParams: QueryParams,
billingMonthmodel:Seq[HierarchyEntryBillingRoleCheck]):
Action[AnyContent] = securityService.authenticate() { implicit request
=> withErrorRecovery { req =>
toJson {
repository.getAll(request.user.loginName)
for {
rolenamecheck <- billingMonthmodel
}yield rolenamecheck
}}}}
You don't say which db access method you are using. (I'm assuming anorm). One way of approaching this is:
Create a case class matching your table
Create a parser matching your case class
use Option (or Either) to return a row for a specific set of parameters
For example, perhaps you have:
case class UserRole (id:Int, loginName:String, roleName:String)
And then
object UserRole {
val sqlFields = "ur.id, ur.loginName, ur.roleName"
val userRoleParser = {
get[Int]("id") ~
get[String]("loginName") ~
get[String]("roleName") map {
case id ~ loginName ~ roleName => {
UserRole(id, loginName, roleName)
}
}
}
...
The parser maps the row to your case class. The next step is creating either single row methods like findById or findByLoginName and multi-row methods, perhaps allForRoleName or other generic filter methods. In your case there might (assuming a single role per loginName) be something like:
def findByLoginName(loginName:String):Option[UserRole) = DB.withConnection { implicit c =>
SQL(s"select $sqlFields from userRoles ur ...")
.on('loginName -> loginName)
.as(userRoleParser.singleOpt)
}
The .as(parser... is key. Typically, you'll need at least:
as(parser.singleOpt) which returns an Option of your case class
as(parser *) which returns a List of your case class (you'll need this if multiple roles could exist for a login
as(scalar[Long].singleOpt) which returns an Option[Long] and which is handy for returning counts or exists values
Then, to eventually return to your question a little more directly, you can call your find method, and if it returns something, continue with the second method call, perhaps like this:
val userRole = findByLoginName(loginName)
if (userRole.isDefined)
billingMonthCheckRepository.getrepo()
or, a little more idiomatically
findByLoginName(loginName).map { userRole =>
billingMonthCheckRepository.getrepo()
...
I've shown the find method returning an Option, but in reality we find it more useful to return an Either[String,(your case class)], and then the string contains the reason for failure. Either is cool.
On my version of play (2.3.x), the imports for the above are:
import play.api.db._
import play.api.Play.current
import anorm._
import anorm.SqlParser._
You're going to be doing this sort of thing a lot so worth finding a set of patterns that works for you.
WOW I don't know what's happening with the formatting here, I am really attempting to use the code formatter on the toolbar but I don't know why it won't format it, even when pressed multiple times. I invite the community to edit my code formatting because I can't figure it out. Apologies to OP.
Because I find Play's documentation to be very tough to trudge through if you're unfamiliar with it, I won't just leave a link to it only.
You have to inject an instance of your database into your controller. This will then give it to you as a global variable:
#Singleton
class LoginRegController #Inject()(**myDB: Database**, cc: ControllerComponents ) {
// do stuff
}
But, it's bad practice to actually use this connection within the controller, because the JDBC is a blocking operation, so you need to create a Model which takes the db as a parameter to a method. Don't set the constructor of the object to take the DB and store it as a field. For some reason this creates connection leaks and the connections won't release when they are done with your query. Not sure why, but that's how it is.
Create a Model object that you will use to execute your query. Instead of passing the DB through the object's constructor, pass it through the method you will create:
object DBChecker {
def attemptLogin(db:Database, password:String): String = {
}}
In your method, use the method .withConnection { conn => to access your JDBC connection. So, something like this:
object DBChecker {
def attemptLogin(db:Database, password:String):String = {
var username: String = ""
db.withConnection{ conn =>
val query:String = s"SELECT uploaded_by, date_added FROM tableName where PASSWORD = $password ;"
val stmt = conn.createStatement()
val qryResult:ResultSet = stmt.executeQuery(query)
// then iterate over your ResultSet to get the results from the query
if (qryResult.next()) {
userName = qryResult.getString("uploaded_by")
}
}
}
return username
}
// but note, please look into the use of PreparedStatement objects, doing it this way leaves you vulnerable to SQL injection.
In your Controller, as long as you import the object, you can then call that object's methods from your controller you made in Step 1.
import com.path.to.object.DBChecker
#Singleton
class LoginRegController #Inject()(myDB: Database, cc: ControllerComponents ) { def attemptLogin(pass:String) = Action {
implicit request: Request[AnyContent] => {
val result:String = DbChecker.attemptLogin(pass)
// do your work with the results here
}

How to insert a data into RDB (MySQL) using Spark?

I'm trying to insert a data into MySQL table via Spark SQL.
Here is my table:
CREATE TABLE images (
id INT NOT NULL AUTO_INCREMENT,
name VARCHAR(100) NOT NULL,
data LONGBLOB NOT NULL
);
and my Spark code:
case class Image(name: String, data: Array[Byte])
def saveImage(image: Image): Unit = {
sqlContext.sql(s"""INSERT INTO images (name, data) VALUES ('${image.name}', ${image.data});""".stripMargin)
}
But I get an error:
java.lang.RuntimeException: [1.13] failure: ``table'' expected but identifier images found
INSERT INTO images (name, data)
^
What is wrong with my code?
Finally, I've found a solution. I can use a trick to save data into MySQL using Spark SQL. The trick is to create a new DataFrame and then to persist it. Here is an example:
def saveImage(image: Image): Unit = {
val df = sqlContext.createDataFrame {
sc.parallelize(
Image(
name = image.name,
data = image.data
) :: Nil
)
}
JdbcUtils.saveTable(df, url, "images", props)
}
And the model would be like this:
case class Image(
id : Option[Int] = None,
name : String,
data : Array[Byte]
)

Could not find implicit value for parameter c: anorm.Column[Float]

I got this similar question but it doesn't help me. (Anorm parse float values). And I can honestly say I didn't understand the solution of that question.
I am getting this complie time error:
could not find implicit value for parameter c: anorm.Column[Float]
at
def getInformation(id: Long): List[(Float, Float, Float)] = {
DB.withConnection { implicit con =>
val query = SQL("select principal,interest,value from myTable where userId={id} and status=true").on("id"->id)
val result = query().map { row =>
Tuple3(row[Float]("principal"), row[Float]("inetrest"), row[Float]("value"))
// ^
}.toList
return result
}
}
Maybe a short review of implicits help you. Let's construct a very basic example:
// some class which will be used as implicit (can be anything)
case class SomeImplicitInformation(maybe: Int, with: Int, data: Int)
// lets assume we have a function that requires an implicit
def functionRequiringImplicit(regularParameters: Int)(implicit imp: SomeImplicitInformation) {
// ...
}
// now if you try to call the function without having an implicit in scope
// you would have to pass it explicitly as second parameter list:
functionRequiringImplicit(0)(SomeImplicitInformation(0,0,0))
// instead you can declare an implicit somewhere in your scope:
implicit val imp = SomeImplicitInformation(0,0,0)
// and now you can call:
functionRequiringImplicit(0)
The error you get simply says that anorm.Column[Float] in not in the scope as implicit. You can solve it by adding it implicitly to your scope or pass it explicitly.
More detailed instructions for you: Since the Column companion object only provides an implicit for rowToDouble you simply have to use the code that is linked in your question. To get it to work put it before your result computation. Later you might want to place it in a val in some enclosing scope.
try this...
def getInformation(id: Long): List[(Float, Float, Float)] = {
DB.withConnection { implicit con =>
val query = SQL("select principal,interest,value from myTable where userId={id} and status=true").on("id"->id)
val result = query().map { row =>
Tuple3(row[Float]("principal").asInstanceOf[Float], row[Float]("inetrest").asInstanceOf[Float], row[Float]("value").asInstanceOf[Float])
}.toList
return result
}
}
implicit def rowToFloat: Column[Float] = Column.nonNull { (value, meta) =>
val MetaDataItem(qualified, nullable, clazz) = meta
value match {
case d: Float => Right(d)
case _ => Left(TypeDoesNotMatch("Cannot convert " + value + ":" + value.asInstanceOf[AnyRef].getClass + " to Float for column " + qualified))
}
}
Some functions can accept what we call implicit parameters. Such parameters can, under certain conditions, be derived from the context. If these parameters can't be found, then you have to specify them by hand. If you expect a parameter to be used as an implicit one, it must have been declared implicit, for instance this way :
implicit val myVal = ...
It can be done in the current block or in an enclosing one (in the class body, for instance, or even sometimes in the imports)
The error you get seems to be related to this feature. You're using a function that needs a parameter of type anorm.Column[Float]. The argument is defined to be implicit so that an implicit value can be used and your code may be more concise. Unfortunately, you don't seem to have such an implicit value in your code, so it fails.
Latest Anorm (included in Play 2.3) provides more numeric conversion (see details at http://applicius-en.tumblr.com/post/87829484643/anorm-whats-new-play-2-3 & in Play migration notes).
If you have missing converter, you can add an issue on Play github project.
Best

Passing functions and operating on their results within Scala's Actors

I'm implementing an actor-based app in scala and I'm trying to be able to pass functions between the actors for them to be processed only when some message is received by the actor.
import actors.Actor
import java.util.Random
import scala.Numeric._
import Implicits._
class Constant(val n:Number) extends Actor{
def act(){
loop{
receive{
case "value" => reply( {n} )
}
}
}
}
class Arithmetic[T: Numeric](A: ()=>T, B: ()=>T) extends Actor{
def act(){
receive{
case "sum" => reply ( A() + B() )
/* case "mul" => reply ( A * B )
*/
}
}
}
object Main extends App{
val c5 = new Constant(5)
c5.start
val a = new Arithmetic({c5 !! "value"}, {c5!!"value"} )
a.start
println(a!?"sum")
println(a!?"mul")
}
In the example code above I would expect the output to be both 5+5 and 5*5. The issue is that reply is not a typed function and as such I'm unable to have the operator (+,*) to operate over the result from A and B.
Can you provide any help on how to better design/implement such system?
Edit: Code updated to better reflect the problem. Error in:
error: could not find implicit value for evidence parameter of type Numeric[Any]
val a = new Arithmetic({c5 !! "value"}, {c5!!"value"} )
I need to be able to pass the function to be evaluated in the actor whenever I call it. This example uses static values but I'll bu using dynamic values in the future, so, passing the value won't solve the problem. Also, I would like to receive different var types (Int/Long/Double) and still be able to use the same code.
The error: Error in: error: could not find implicit value for evidence parameter of type Numeric[Any]. The definition of !!:
def !! (msg: Any): Future[Any]
So the T that Arithmetic is getting is Any. There truly isn't a Numeric[Any].
I'm pretty sure that is not your problem. First, A and B are functions, and functions don't have + or *. If you called A() and B(), then you might stand a chance... except for the fact that they are java.lang.Number, which also does not have + or * (or any other method you'd expect it to have).
Basically, there's no "Number" type that is a superclass or interface of all numbers for the simple reason that Java doesn't have it. There's a lot of questions touching this subject on Stack Overflow, including some of my own very first questions about Scala -- investigate scala.math.Numeric, which is the best approximation for the moment.
Method vs Function and lack of parenthesis
Methods and functions are different things -- see tons of related questions here, and the rule regarding dropping parenthesis is different as well. I'll let REPL speak for me:
scala> def f: () => Int = () => 5
f: () => Int
scala> def g(): Int = 5
g: ()Int
scala> f
res2: () => Int = <function0>
scala> f()
res3: Int = 5
scala> g
res4: Int = 5
scala> g()
res5: Int = 5