I try to deploy an AWS lambda application, I implemented with the Chalice Python Framework. My app.py connects to a MySQL server and therefore has to
import mysql.connector
But on every invocation of one of my lambda functions I get an error in the log
'Unable to import module 'app': No module named mysql.connector'
I tried to add the mysql.connector to the requirements.txt file in the chalice project:
mysql_connector==2.1.6
And if I do so, 2 additional folders containing several files appear in the AWS lambda environment:
/mysql_connector-2.1.6.data
/mysql_connector-2.1.6.dist-info
But the error remains the same. How to deploy python mysql.connector with Chalice?
This finally worked for me:
lib_path=os.path.abspath(os.path.join(__file__, '..', 'mysql_connector-2.1.6.data', 'purelib'))
sys.path.append(lib_path)
import mysql.connector
Putting the "mysql_connector==2.1.6" into the "requirements.txt" file did install the mysql connector in lambda environment. I added the path of the package (../mysql_connector-2.1.6.data/purelib) to system path.
Related
I am trying to run the following code to train paddleOCR.
import paddle
import paddle.distributed as dist
But I'm getting this error:
ModuleNotFoundError: No module named 'paddle.distributed'
Even after I have installed paddle-client.
docker pull paddlepaddle/paddle:2.3.0-gpu-cuda11.2-cudnn8
I use this images which can work well.
You can try the paddlepaddle with 2.3.1 version，and quick install can refer to: https://www.paddlepaddle.org.cn/en
I'd like to connect to prestodb with SQLalchemy interface. I'm running prestodb==0.7.0 and SQLalchemy== 1.4.20 and SQLalchemy doesn't seem to have prestodb baked in:
NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:presto
Not much luck with registering the prestodb either:
from sqlalchemy.dialects import registry
import prestodb
from prestodb.dbapi import Connection
registry.register('presto', 'prestodb.dbapi', 'Connection')
from sqlalchemy.engine import create_engine
port = 8889
user = os.environ["USER"]
engine = create_engine(f'presto://{user}#presto:{port}/hive',
connect_args={'protocol': 'https', 'requests_kwargs': {'verify': False}})
db = engine.raw_connection()
# AttributeError: type object 'Connection' has no attribute 'get_dialect_cls'
Any ideas?
If you have a look at the Dialects docs you will see that Presto is a external dialect and needs to be installed separately. The Presto dialect is supported through PiHyve and can be installed using pip install 'pyhive[presto]'.
I am trying to use ImageDataBunch from fastai, and it worked fine, but recently when I ran my code, it showed this error ModuleNotFoundError: No module named 'fastai.vision'
Then, I upgraded my fastai version pip install fastai --upgrade. This error got cleared but landed in NameError: name 'ImageDataBunch' is not defined
Here's my code:
import warnings
import numpy as np
from fastai.vision import *
warnings.filterwarnings("ignore", category=UserWarning, module="torch.nn.functional")
np.random.seed(42)
data = ImageDataBunch.from_folder(path, train='.', valid_pct=0.2,
ds_tfms=get_transforms(), size=224, num_workers=4, no_check=True).normalize(imagenet_stats)
How can I fix this?
I actually ran into this same issue when I started using Colab, but haven't been able to reproduce it. Here was the thread describing what I and another developer did to troubleshoot: https://forums.fast.ai/t/no-module-named-fastai-data-in-google-colab/78164/4
I would recommend trying to factory reset your runtime ( "Runtime" -> "Factory Reset Runtime")
Then you can check which version of fastai you have (you have to restart the runtime to use the new version if you've already imported it)
import fastai
fastai.__version__
I'm able to run fastai.vision import * on fastai version 1.0.61 and 2.0.13
In Google Colab:
Upgrade fastai on colab:
! [ -e /content ] && pip install -Uqq fastai
Import necessary libraries:
from fastai.vision.all import *
from fastai.text.all import *
from fastai.collab import *
from fastai.tabular.all import *
Get the images and annotations:
path = untar_data(URLs.PETS)
path_anno = path/'annotations'
path_img = path/'images'
print( path_img.ls() ) # print all images
fnames = get_image_files(path_img) # -->> 7390 images
print(fnames[:5]) # print first 5 images
The solution that worked for me is to copy to (connect) my google drive & then run the cells. Source
You might have installed the older version of fastai. You need to upgrade to fastaiv2. You can upgrade fastai by using pip as shown below.
!pip install fastai --upgrade
Also check your fastai version using
import fastai
print(fastai.__version__)
I am using pyspark to do some work on a csv file, hence I need to import package from spark-csv_2.10-1.4.0.jar downloaded from https://repo1.maven.org/maven2/com/databricks/spark-csv_2.11/1.4.0/spark-csv_2.11-1.4.0.jar
I downloaded the jar to my local due to proxy issue.
Can anyone tell me what is the right usage of referring to a local jar:
Here is the code I use:
pyspark --jars /home/rx52019/data/spark-csv_2.10-1.4.0.jar
it will take me to the pyspark shell as expected, however, when I run:
df = sqlContext.read.format('com.databricks.spark.csv').options(header='true',inferschema='true').load('hdfs://dev-icg/user/spark/routes.dat')
the route.dat is uploaded to hdfs already at hdfs://dev-icg/user/spark/routes.dat
It gives me error:
: java.lang.NoClassDefFoundError: org/apache/commons/csv/CSVFormat
If I run:
df = sqlContext.read.format('com.databricks.spark.csv').options(header='true',inferschema='true').load('routes.dat')
I get this error:
py4j.protocol.Py4JJavaError: An error occurred while calling o72.load.
: java.lang.NoClassDefFoundError: Could not initialize class
com.databricks.spark.csv.package$
Can anyone help to sort it out for me? Thank you very much. Any clue is appreciated.
The correct way to do this would be to add the options (say if you are starting a spark shell)
spark-shell --packages com.databricks:spark-csv_2.11:1.4.0 --driver-class-path /path/to/csvfilejar.jar
I have not used the databricks csvjar directly, but I used a netezza connector to spark where they mention using this option
https://github.com/SparkTC/spark-netezza
I am using a Gson library for parsing Json data. I am trying to run a program from terminal as follows:
scala -classpath "*.jar" JsonParsing.scala
To which I am getting the following error:
JsonParsing.scala:2: error: object google is not a member of package com import com.google.gson.Gson
I am unsure as why this error is coming. When I have gson jar in accurate folder.
gson-2.2.2.jar
I am using import statements as follows:
import com.google.gson.Gson
import com.google.gson.JsonObject
import com.google.gson.JsonParser
Help on this error would be appreciated. Thanks.
Your dependancy not include google package.
You can use :
// https://mvnrepository.com/artifact/com.google.code.gson/gson
libraryDependencies += "com.google.code.gson" % "gson" % "2.8.0"
or download appropriate jar http://www.java2s.com/Code/Jar/g/gson.htm
Compile:
$ scalac -classpath <path_to_your_jar_files> -d classes " path/to/classes/you/want/to/compile/*
Execute:
$ scala -classpath classes:<path_to_your_jar_files> com.your.package.ClassYouWantToRun
This is not a good way of doing it because it's not scalable. You should be using a tool like SBT to build and run projects.