I am reading the tutorial here
But this tutorial uses sqlalchemy as following:
from datetime import datetime, timedelta
import unittest
from app import app, db
from app.models import User, Post
class UserModelCase(unittest.TestCase):
def setUp(self):
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite://'
db.create_all()
def tearDown(self):
db.session.remove()
db.drop_all()
in fact, I already wrote a web app that uses from flask_mysqldb import MySQL
the __init__.py in the tutorial looks like this:
from flask import Flask
from config import Config
from flask_sqlalchemy import SQLAlchemy
from flask_migrate import Migrate
app = Flask(__name__)
app.config.from_object(Config)
db = SQLAlchemy(app)
migrate = Migrate(app, db)
from app import routes, models
I have mysql=MySQL(app) in my own __init__.py
can I replace db by mysql in the first part of the code to test my app?
and what about app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite://'?
I haven't use the sqlalchemy,just use flask_mysqldb
Sol:
use testing.mysqld module
here
Reference:
Mock a MySQL database in Python
Related
I am trying to connect to Mysql. I have uploaded the corresponding java jar we are using , which is mysql-connector-java-5.1.49.jar , I uploaded to s3 bucket. I am using the following code to access to Mysql and I am failing with the Error
An error occurred while calling o93.getDynamicFrame.
com.mysql.cj.jdbc.Driver
from awsglue.transforms import *
from awsglue.utils import getResolvedOptions
from pyspark.context import SparkContext, SparkConf
from awsglue.context import GlueContext
from awsglue.job import Job
import time
from pyspark.sql.types import StructType, StructField, IntegerType, StringType
import boto3
import json
sc = SparkContext()
glueContext = GlueContext(sc)
spark = glueContext.spark_session
spark.conf.set("jars", "s3://xxxxx/jdbc-drivers/mysql-connector-java-5.1.49.jar")
client = boto3.client("secretsmanager" , region_name = "eu-west-1")
get_secret_value_response = client.get_secret_value(SecretId = "etl-1")
secret = get_secret_value_response["SecretString"]
secret=json.loads(secret)
username = secret.get("mysql_username")
password = secret.get("mysql_password")
url = secret.get("mysql_url")
table = secret.get("mysql_table")
connection_mysql_options_source_session = {
"url": url,
"dbtable": table,
"user": username,
"password": password,
"customJdbcDriverS3Path": "s3://xxxxx/jdbc-drivers/mysql-connector-java-5.1.49.jar",
"customJdbcDriverClassName": "com.mysql.cj.jdbc.Driver"}
# Read from JDBC databases with custom driver
df_session = glueContext.create_dynamic_frame.from_options(connection_type="mysql", connection_options=connection_mysql_options_source_session)
df_session.printSchema()
In job details section, I have referred to jar libs:
And I didn't define any connection in the connection section of job properties. I can't figure out why I am getting this error.
The strange thing is that I can connect with Crawler, data catalogue and also direct connection to the same server, but via Notebook & script I can't.
I want to pass a SQL query as parameter in a POST request. Ideally this SQL query will be further passed into MySQL connection, and then it can fetch data back.
Here is what I did:
These are basic modules and settings:
from flask import Flask
from flask_cors import CORS, cross_origin
app = Flask(__name__)
CORS(app)
from app import app
from flaskext.mysql import MySQL
mysql = MySQL()
app.config['MYSQL_DATABASE_USER'] = 'root'
app.config['MYSQL_DATABASE_PASSWORD'] = 'root'
app.config['MYSQL_DATABASE_DB'] = 'hibernate1'
app.config['MYSQL_DATABASE_HOST'] = 'localhost'
mysql.init_app(app)
This is the main.py, where I want to pass a SQL query:
from urllib import response
import pymysql
from app import app
from config import mysql
from flask import jsonify
from flask import flash, request
#app.route('/SQL/<query>', methods=['POST'])
def return_query(query):
conn = mysql.connect()
cursor = conn.cursor(pymysql.cursors.DictCursor)
cursor.execute(query)
Rows = cursor.fetchall()
respone = jsonify(Rows)
return respone
if __name__ == "__main__":
app.run()
This is test.py:
import requests
dictToSend = {"query": "select * from student"}
res = requests.post('http://localhost:5000/SQL', json = dictToSend)
print ('response from server:', res.text)
dictFromServer = res.json()
print(dictFromServer)
However, I got the following errors:
response from server: <!doctype html>
<html lang=en>
<title>404 Not Found</title>
<h1>Not Found</h1>
<p>The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.</p>
It says the requested URL was not found. But the following GET request works fine:
#app.route('/', methods=['GET'])
def return_hello():
return {"data": "hello"}
Could anyone tell me what I missed in the POST request? Thanks.
I have a selenium python automation test, it works fine, now I want to generate Html and JSON reports and have screenshots in the report using pytest. I am new to automation and python so I am not much aware of how its done.
following is my code
test_screenshot.py
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import pytest_html
from selenium.common.exceptions import InvalidSessionIdException
def test_Openurl(setup):
driver = setup["driver"]
url = setup["url"]
try:
driver.get(url)
except Exception as e:
print(e.message)
assert driver.current_url == URL
driver.save_screenshot("ss.png")
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
driver.save_screenshot("ss1.png")
driver.close()
conftest.py
import pytest
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.chrome.service import Service
def pytest_addoption(parser):
parser.addoption("--url", action="store", default="https://google.com/")
#pytest.fixture()
def setup(pytestconfig):
s = Service("C:/Users/Yash/Downloads/chromedriver_win32/chromedriver.exe")
driver = webdriver.Chrome(service=s)
driver.maximize_window()
yield {"driver":driver, "url": pytestconfig.getoption("url")}
I ran this using
pytest test_screenshot.py --url "https://www.netflix.com/in/"
Test case is passed. How do I generate HTML and JSON report?
I tried this
pytest -v -s --json-report --json-report-indent=4 --json-report-file=report/report.json --html=report/report.html test_screenshot.py
but got this error
ERROR: usage: pytest [options] [file_or_dir] [file_or_dir] [...]
pytest: error: unrecognized arguments: --json-report --json-report-indent=4 --json-report-file=report/report.json
inifile: None
You need to install these two libraries : https://pypi.org/project/pytest-json-report/ & https://pypi.org/project/pytest-html/
I am working on a Spark Application and I want to create a rest API in Django, below is my code
from django.shortcuts import render
from django.http import Http404
from rest_framework.views import APIView
from rest_framework.decorators import api_view
from rest_framework.response import Response
from rest_framework import status
from django.http import JsonResponse
from django.core import serializers
from django.conf import settings
import json
from pyspark import SparkContext, SparkConf, SQLContext
sc = SparkContext()
sql = SQLContext(sc)
df = Sql.read.format("jdbc").options(
url = "jdbc:mysql://127.0.0.1:3306/demo",
driver = "com.mysql.cj.jdbc.Driver",
dbtable = "tablename",
user = "xyz",
password = "abc"
).load()
totalrecords = df.count()
# Create your views here.
#api_view(["GET"])
def Demo(self):
try:
a = str(totalrecords)
return JsonResponse(a,safe=False)
except ValueError as e:
return Response(e.args[0],status.HTTP_400_BAD_REQUEST)
I want to know how will I run this code, as I have directly tried "python manage.py runserver" which is not working, so how to run this spark and django with django api and spark-submit with all required spark jar file?
To run this code you have to use spark submit only,
spark-submit --jars mysql.jar manage.py runserver 0.0.0.0:8000
or
spark-submit manage.py runserver
I am new to flask framework. I want to connect with a MySQL database
and my code in the __init__.py is
app = Flask(__name__)
app.config.from_object(Config)
db = SQLAlchemy(app)
migrate =Migrate(app,db)
but I am getting this error
Authentication plugin '{0}' is not supported".format(plugin_name))
sqlalchemy.exc.NotSupportedError:
(mysql.connector.errors.NotSupportedError) Authentication plugin
'caching_sha2_password' is not supported
(Background on this error at: http://sqlalche.me/e/tw8g)
Can anyone please help me?
Please install the following requirement using pip:
pip install flask-mysql
I perform my MySQL connection with Flask using similar code (tested now):
from flask import Flask
from flaskext.mysql import MySQL
app = Flask(__name__)
app.config['MYSQL_DATABASE_USER'] = 'youruser'
app.config['MYSQL_DATABASE_PASSWORD'] = 'yourpassword'
app.config['MYSQL_DATABASE_DB'] = 'yourdb'
app.config['MYSQL_DATABASE_HOST'] = 'yourhost'
mysql = MySQL(app)
mysql.init_app(app)
#app.route("/")
def hello_db():
conn = mysql.connect()
cursor =conn.cursor()
cursor.execute('''SELECT * from yourtable''')
data = cursor.fetchall()
return str(data)
if __name__ == "__main__":
app.run()
Please change the variables with your data (user/password etc) and try the connection.