I am using Anorm for querying a MySQL database from Playframework 2.1. I created a prepared statement like this.
import play.api.db.DB
import anorm._
val stat = DB.withConnection(implicit c => SQL("SELECT name, email FROM user WHERE id=?").filledStatement)
Now how do I use it? Am I event doing this right? I am totally ignorant of the anorm API and I already went through the source code without gaining much insight.
Code examples are more that welcome.
A good example on the Anorm usage is given in the respective tutorial. It also contains some examples that pass dynamic parameters to the queries. You should start by writing your query and replace declare placeholders like {somePlaceholder} in the query string. You can later assign values using the .on() method like this:
SQL(
"""
select * from Country c
join CountryLanguage l on l.CountryCode = c.Code
where c.code = {countryCode};
"""
).on("countryCode" -> "FRA")
Or in your case:
import play.api.db.DB
import anorm._
val stat = DB.withConnection(implicit c =>
SQL("SELECT name, email FROM user WHERE id={id}").on("id" -> 42)
)
Related
Using the code shown below I can obtain the vendor type that corresponds to the SQLAlchemy generic type. In this case it is "VARCHAR(10)". How can I get the vendor type without creating a table?
engine = create_engine(DB_URL)
metadata_obj = MetaData()
table = Table('Table', metadata_obj,
Column('Column', types.String(10))
)
metadata_obj.create_all(bind=engine)
metadata_obj = MetaData()
metadata_obj.reflect(bind=engine)
print(metadata_obj.tables['Table'].columns[0].type)
You can't obtain the type directly, but you could use a mock_engine to generate the DDL as a string which can be parsed. A mock_engine must be coupled with a callable that will process the SQL expression object that it generates.
This snippet is based on the example code from the SQLAlchemy docs.
import sqlalchemy as sa
tbl = sa.Table('drop_me', sa.MetaData(), sa.Column('col', sa.String(10)))
def dump(sql, *multiparams, **params):
print(sql.compile(dialect=engine.dialect))
mock_engine = sa.create_mock_engine('postgresql://', executor=dump)
tbl.create(mock_engine)
Outputs
CREATE TABLE "Table" (
"Column" VARCHAR(10)
)
sqlalchemy.schema.CreateTable, could also be used, but binding it to an engine is deprecated, to be removed in SQLAlchemy 2.0.
from sqlalchemy.schema import CreateTable
print(CreateTable(tbl, bind=some_engine)
I created a glue job using the visual tab like below. First I connected to a mysql table as data source which is already in my data catalog. Then in the transform node, I wrote a custom sql query to select only one column from the source table. Validated with the data preview feature and the transformation node works fine. Now I want to write the data to the existing database table that has only one column with 'string' data type. Glue job succeeded but I dont see the data in the table.
Below is the automatic script generated from Glue Job Visual.
import sys
from awsglue.transforms import *
from awsglue.utils import getResolvedOptions
from pyspark.context import SparkContext
from awsglue.context import GlueContext
from awsglue.job import Job
from awsglue import DynamicFrame
def sparkSqlQuery(glueContext, query, mapping, transformation_ctx) -> DynamicFrame:
for alias, frame in mapping.items():
frame.toDF().createOrReplaceTempView(alias)
result = spark.sql(query)
return DynamicFrame.fromDF(result, glueContext, transformation_ctx)
args = getResolvedOptions(sys.argv, ["JOB_NAME"])
sc = SparkContext()
glueContext = GlueContext(sc)
spark = glueContext.spark_session
job = Job(glueContext)
job.init(args["JOB_NAME"], args)
# Script generated for node MySQL
MySQL_node1650299412376 = glueContext.create_dynamic_frame.from_catalog(
database="glue_rds_test",
table_name="test_customer",
transformation_ctx="MySQL_node1650299412376",
)
# Script generated for node SQL
SqlQuery0 = """
select CUST_CODE from customer
"""
SQL_node1650302847690 = sparkSqlQuery(
glueContext,
query=SqlQuery0,
mapping={"customer": MySQL_node1650299412376},
transformation_ctx="SQL_node1650302847690",
)
# Script generated for node MySQL
MySQL_node1650304163076 = glueContext.write_dynamic_frame.from_catalog(
frame=SQL_node1650302847690,
database="glue_rds_test",
table_name="test_customer2",
transformation_ctx="MySQL_node1650304163076",
)
job.commit()
For me the problem was the double-quotes of the selected fields in the SQL query. Dropping the use of double quotes solved it. There is no mention of it in the Spark SQL Syntax documentation
For example, I "wrongly" used this query syntax:
select "CUST_CODE" from customer
instead of this "correct" one :
select CUST_CODE from customer
Your shared sample code does not seem to have this syntax issue, but I thought putting the answer here might be of a help to others.
I have two models like this:
class McqQuestion(models.Model):
mcq_question_id = models.IntegerField()
test_id = models.ForeignKey('exam.Test')
mcq_right_answer = models.IntegerField()
class UserMcqAnswer(models.Model):
user = models.ForeignKey('exam.UserInfo')
test_id = models.ForeignKey('exam.Test')
mcq_question_id=models.ForeignKey('exam.McqQuestion')
user_answer = models.IntegerField()
I need to match the user_answer and mcq_right_answer. Able to do that by executing the below raw query.
rightAns=UserMcqAnswer.objects.raw('SELECT B.id, COUNT(A.mcq_question_id) AS RightAns\
FROM exam_mcqquestion AS A\
LEFT JOIN exam_usermcqanswer AS B\
ON A.mcq_question_id=B.mcq_question_id_id\
WHERE B.test_id_id=%s AND B.user_id=%s AND\
A.mcq_right_answer=B.user_answer',[test_id,user_id])
1) But the problem is that couldn't able to pass the result as JSONResponse because it says TypeError: Object of type 'RawQuerySet' is not JSON serializable
2) Is there any alternative to this raw query by using the objects and filtered querysets?
Django's serialize function's second argument can be any iterator that yields Django model instances.
So, in principle, you can use that raw SQL query that you worked on, using something like this:
query = """SELECT B.id, COUNT(A.mcq_question_id) AS RightAns\
FROM exam_mcqquestion AS A\
LEFT JOIN exam_usermcqanswer AS B\
ON A.mcq_question_id=B.mcq_question_id_id\
WHERE B.test_id_id=%s AND B.user_id=%s AND\
A.mcq_right_answer=B.user_answer"""%(test_id, user_id)
and then getting the json data you'll return, as:
from django.core import serializers
data = serializers.serialize('json', UserMcqAnswer.objects.raw(query), fields=('some_field_you_want', 'another_field', 'and_some_other_field'))
Good luck finding the best way to solve your issue
Edit: small fix, added an import
Using raw query is not recommended in Django.
When the model query APIs don’t go far enough, you can fall back to writing raw SQL.
In your case model query API can solve your problem. You can use the following view:
views.py
def get_answers(request):
test = Test.objects.get(name="Test 1")
answers = UserMcqAnswer.objects.filter(test_id=test, user=request.user).annotate(
is_correct=Case(
When(user_answer=F('mcq_question_id__mcq_right_answer'),
then=Value(True)),
default=Value(False),
output_field=BooleanField())
).values()
return JsonResponse(list(answers), safe=False)
Also you can consider Django Rest Framework for serialization of QuerySet.
I'm trying to introduce slick into my code to replace some existing jdbc code.
First of all I'd like to use a scala worksheet to run a really simple query, I want to pass in an integer id, and get back a string uuid. This is the simplest method in the whole codebase.
As I understand I need to make a connection to the database, setup an action, and then run the action. I have the following code:
val db = Database.forURL("jdbc:mysql://mysql-dev.${URL}/${DB}?autoReconnect=true&characterEncoding=UTF-8",
driver = "com.mysql.jdbc.Driver", user = "${user}",password= "${pass}")
val getUUID = sql"""SELECT ${UUIDFIELD} from users u WHERE u.${IDFIELD} = ${Id}""".as[String]
val uuid:String = db.run(getUUID)
println(uuid)
I'm pretty sure I don't have the driver setup correctly in the Database.forURL call, but also the worksheet is complaining that the result of db.run is not a string. How do I get to the string UUID value?
The db.run method returns a Future[_] type. You should use Await for getting result from it.
val db = Database.forURL("jdbc:mysql://mysql-dev.${URL}/${DB}?autoReconnect=true&characterEncoding=UTF-8",
driver = "com.mysql.jdbc.Driver", user = "${user}",password= "${pass}")
val getUUID = sql"""SELECT ${UUIDFIELD} from users u WHERE u.${IDFIELD} = ${Id}""".as[String]
val uuidFuture:Future[String] = db.run(getUUID)
import scala.concurrent._
import scala.concurrent.duration._
val uuid:String = Await.result(uuidFuture, Duration.Inf)
println(uuid)
I'd like to execute two scenarios that should be executed one after another and the data "produced" by the first scenario should be used as base for the second scenario.
So a case could be for example clearing of a credit card. The first scenarios is there to authorize/reserve of a certain amount on the card:
val auths = scenario("auths").during(durationInMinutes minutes) {
feed(credentials)
.feed(firstNames)
.feed(lastNames)
.feed(cards)
.feed(amounts)
.exec(http("send auth requests")
.post(...)
.check(...))}
The second one is there to capture/take the amount from the credit card:
val caps = scenario("caps").during(durationInMinutes minutes) {
feed(credentials)
.feed(RESPONSE_IDS_FROM_PREVIOUS_SCENARIO)
.exec(http("send auth requests")
.post(...)
.check(...))}
I initially thought about using the saveAs(...) option on check but I figured out that the saved field is only valid for the given session.
So basically I want to preserve the IDs I got from the auths scenario and use them in the caps scenario.
I cannot execute both steps in one scenario though (saveAs would work for that) because I have different requirement for both scenarios.
Quoting the documentation: "Presently our Simulation is one big monolithic scenario. So first let us split it into composable business processes, akin to the PageObject pattern with Selenium. This way, you’ll be able to easily reuse some parts and build complex behaviors without sacrificing maintenance." at gatling.io/Advanced Tutorial
Thus your there is no build-in mechanism for communication between scenarios (AFAIK). Recommendation is to structure your code that way that you can combine your calls to URIs subsequently. In your case (apart from implementation details) you should have something like this:
val auths = feed(credentials)
.feed(firstNames)
.feed(lastNames)
.feed(cards)
.feed(amounts)
.exec(http("send auth requests")
.post(...)
.check(...) // extract and store RESPONSE_ID to session
)
val caps = exec(http("send auth requests")
.post(...) // use of RESPONSE_ID from session
.check(...))
Then your scenario can look something like this:
val scn = scenario("auth with caps").exec(auths, caps) // rest omitted
Maybe even better way to structure your code is to use objects. See mentioned tutorial link.
More illustrative example (which compiles, but I didn't run it while domain is foo.com):
import io.gatling.core.Predef._
import io.gatling.http.Predef._
class ExampleSimulation extends Simulation {
import scala.util.Random
import scala.concurrent.duration._
val httpConf = http.baseURL(s"http://foo.com")
val emails = Iterator.continually(Map("email" -> (Random.alphanumeric.take(20).mkString + "#foo.com")))
val names = Iterator.continually(Map("name" -> Random.alphanumeric.take(20).mkString))
val getIdByEmail = feed(emails)
.exec(
http("Get By Email")
.get("/email/$email")
.check(
jsonPath("userId").saveAs("anId")
)
)
val getIdByName = feed(names)
.exec(
http("Get By Name")
.get("/name/$name")
.check(
jsonPath("userId").is(session =>
session("anId").as[String]
)
)
)
val scn = scenario("Get and check user id").exec(getIdByEmail, getIdByName).inject(constantUsersPerSec(5) during (5.minutes))
setUp(scn).protocols(httpConf)
}
Hope it is what you're looking for.