Sorry for this redundant post but no one else resolves my problem.
My goal
Import a CSV file on Neo4J.
My Problem
Neo.ClientError.Statement.ExternalResourceFailed
Couldn't load the external resource at: file:/AFG-ADM1.csv ()
My Environment
I work on my computer and install Neo4J with a docker-compose file as follows:
version: "3.1"
services:
neo4j:
image: neo4j:5.3.0
restart: unless-stopped
container_name: neo4j
# The ports that will be accessible from outside the container - HTTP (7474) and Bolt (7687).
ports:
- "7474:7474"
- "7687:7687"
# Uncomment the volumes to be mounted to make them accessible from outside the container.
volumes:
- ~/Documents/neo4j/neo4j.conf:/conf/neo4j.conf
- ~/Documents/neo4j/data:/var/lib/neo4j/data
- ~/Documents/neo4j/logs:/var/lib/neo4j/logs
- ~/Documents/neo4j/conf:/var/lib/neo4j/conf
- ~/Documents/neo4j/import:/var/lib/neo4j/import
# - ./metrics/server1:/var/lib/neo4j/metrics
# - ./licenses/server1:/var/lib/neo4j/licenses
# - ./ssl/server1:/var/lib/neo4j/ssl
environment:
- NEO4J_ACCEPT_LICENSE_AGREEMENT=yes
- NEO4J_AUTH=neo4j/password
- EXTENDED_CONF=yes
- NEO4J_EDITION=docker_compose
- NEO4J_initial_server_mode__constraint=PRIMARY
healthcheck:
test: ["CMD-SHELL", "wget --no-verbose --tries=1 --spider localhost:7474 || exit 1"]
user: 1000:1000
In my "~/Documents/neo4j/import" directory, I have the file "AFG-ADM1.csv".
To be sure, I went to look in my container:
My neo4j config file, neo4j.conf
# Setting that specifies how much memory Neo4j is allowed to use for the page cache.
server.memory.pagecache.size=100M
# Setting that specifies the initial JVM heap size.
server.memory.heap.initial_size=100M
# Enable server-side routing
dbms.routing.enabled=true
# Use server-side routing for neo4j:// protocol connections.
dbms.routing.default_router=SERVER
server.config.strict_validation.enabled=false
dbms.max_databases=$(my_setting.bat)
#dbms.security.allow_csv_import_from_file_urls=true
I try with and without opt dbms.security.allow_csv_import_from_file_urls to true
My cmd
LOAD CSV WITH HEADERS FROM "file:///AFG-ADM1.csv" AS csvLine
CREATE (p:Pipeline {name: csvLine.name, level: csvLine.Level, shapeGroup: csvLine.shapeGroup})
And get this error:
Using the rancher GUI, I'm trying to set up Nextcloud with MySQL database workloads on my AKS cluster. In the environment variables, I already have defined the admin user and password so why do I get this error on the create admin page?
Error while trying to create admin user: Failed to connect to the
database: An exception occurred in driver: SQLSTATE[HY000] [2054] The
server requested authentication method unknown to the client
I entered the Username and password correctly multiple times.
Below are my configurations for the database and nextcloud so far.
database workload:
Name: nextdb
Docker image: mysql
port: not set
I have the following variables:
MYSQL_ROOT_PASSWORD=rootpassX
MYSQL_DATABASE=nextDB
MYSQL_USER=nextcloud
MYSQL_PASSWORD=passX
volumes configuration:
Volume Type: Bind-Mount
Volume Name: nextdb
Path on the Node : /nextdb
The Path on the Node must be: a directory or create
Mount Point: /var/lib/mysql
nextcloud workload:
Name: nextcloud
Docker Image: nextcloud
Port Mapping:
Port Name : nextcloud80
Publish the container port: 80
Protocol: TCP
As a: Layer-4 load balancer
On listening port: 80
Environment variables:
MYSQL_DATABASE=nextDB
MYSQL_USER=nextcloud
MYSQL_PASSWORD=passX
MYSQL_HOST=nextdb
NEXTCLOUD_ADMIN_USER=admin
NEXTCLOUD_ADMIN_PASSWORD=adminPass
NEXTCLOUD_DATA_DIR=/var/www/html/nextcloud
Volumes:
Volume 1:
name: nextcloud
Volume Type: Bind-Mount
Path on the Node: /nextcloud
The Path on the Node must be: a directory or create.
Mount Point: /var/www/html
Volume 2
name: nextdb
Volume Type: Bind-Mount
Path on the Node: /nextdatabase
The Path on the Node must be: a directory or create.
Mount Point: /var/lib/mysql
What are the problems with my configurations?
Starting with version 8.02, MySQL updated the default authentication method for client connections. To revert to the older authentication method you need to explicitly specify the default authentication method.
If you are able to update your DB service in Rancher to pass the container argument --default-authentication-plugin=mysql_native_password that should revert MySQL to the older auth method.
Alternatively, depending on the MySQL image you are using, you can create a new Docker image from that base which replaces /etc/mysql/my.cnf inside the container. You should inspect /etc/mysql/my.cnf before you replace it, if there are any !includedir directives in the config file, you can place your supplemental configuration into an included folder using whatever filename you choose.
The supplemental configuration should look like this:
[mysqld]
default_authentication_plugin=mysql_native_password
I am having real problems trying to get a docker-compose script to initiate a mysql database and a Django project, but get the Django project to wait until the mysql database is ready.
I have two files, a Dockerfile and a docker-compose.yml, which I have copied below.
When I run the docker-compose.yml, and check the logs of the web container, it says that it cannot connect to the database mydb. However the second time that I run it (without clearing the containers and images) it connects properly and the Django app works.
I have spent a whole day trying a number of things such as scripts, health checks etc, but I cannot get it to work.
Dockerfile
FROM python:3.6
ENV PYTHONUNBUFFERED 1
RUN mkdir /code
WORKDIR /code
COPY ./ /code/
RUN pip install -r requirements.txt
RUN python manage.py collectstatic --noinput
docker-compose.yml
version: '3'
services:
mydb:
environment:
- MYSQL_ROOT_PASSWORD=password
- MYSQL_USER=django
- MYSQL_PASSWORD=secret
- MYSQL_DATABASE=dbMarksWebsite
image: mysql:5.7
ports:
# Map default mysql port 3306 to 3308 on outside so that I can connect
# to mysql using workbench localhost with port 3308
- "3308:3306"
web:
environment:
- DJANGO_DEBUG=1
- DOCKER_PASSWORD=secret
- DOCKER_USER=django
- DOCKER_DB=dbMarksWebsite
- DOCKER_HOST=mydb
- DOCKER_PORT=3306
build: .
command: >
sh -c "sleep 10 &&
python manage.py migrate &&
python manage.py loaddata myprojects_testdata.json &&
python manage.py runserver 0.0.0.0:8080"
ports:
- "8080:8080"
depends_on:
- mydb
First run (with no existing images or containers):
...
File "/usr/local/lib/python3.6/site-packages/MySQLdb/__init__.py", line 84, in Connect
return Connection(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/MySQLdb/connections.py", line 179, in __init__
super(Connection, self).__init__(*args, **kwargs2)
django.db.utils.OperationalError: (2002, "Can't connect to MySQL server on 'mydb' (115)")
Second run:
System check identified no issues (0 silenced).
March 27, 2020 - 16:44:57
Django version 2.2.11, using settings 'ebdjango.settings'
Starting development server at http://0.0.0.0:8080/
Quit the server with CONTROL-C.
I solved it using the following function in my entrypoint.sh:
function wait_for_db()
{
while ! ./manage.py sqlflush > /dev/null 2>&1 ;do
echo "Waiting for the db to be ready."
sleep 1
done
}
For anybody who is interested, I found a solution to this:
1 - I wrote a python script to connect to the database every second,
but with a timeout. I set this timeout to be quite high at 60
seconds, but this seems to work on my computer.
2 - I added the command to wait into my compose file.
It should mean that I can bring up a set of test containers for my website, where I can specify the exact version of Python and MySQL used.
The relevant files are listed below:
Dockerfile:
FROM python:3.6
ENV PYTHONUNBUFFERED 1
RUN mkdir /code
WORKDIR /code
COPY ./ /code/
RUN pip install -r requirements.txt
RUN python manage.py collectstatic --noinput
docker-compose.yml
version: '3'
services:
mydb:
container_name: mydb
environment:
- MYSQL_ROOT_PASSWORD=password
- MYSQL_USER=django
- MYSQL_PASSWORD=secret
- MYSQL_DATABASE=dbMarksWebsite
image: mysql:5.7
ports:
# Map default mysql port 3306 to 3308 on outside so that I can connect
# to mysql using workbench localhost with port 3308
- "3308:3306"
web:
container_name: web
environment:
- DJANGO_DEBUG=1
- DOCKER_PASSWORD=secret
- DOCKER_USER=django
- DOCKER_DB=dbMarksWebsite
- DOCKER_HOST=mydb
- DOCKER_PORT=3306
build: .
command: >
sh -c "python ./bin/wait-for.py mydb 3306 django secret dbMarksWebsite 60 &&
python manage.py migrate &&
python manage.py loaddata myprojects_testdata.json &&
python manage.py runserver 0.0.0.0:8080"
ports:
- "8080:8080"
depends_on:
- mydb
wait-for.py
'''
I don't like adding this in here, but I cannot get the typical wait-for scripts
to work with MySQL database in docker, so I hve written a python script that
either times out after ? seconds or successfully connects to the database
The input arguments for the script need to be:
HOST, PORT, USERNAME, PASSWORD, DATABASE, TIMEOUT
'''
import sys, os
import time
import pymysql
def readCommandLineArgument():
'''
Validate the number of command line input arguments and return the
input filename
'''
# Get arguments
if len(sys.argv)!=7:
raise ValueError("You must pass in 6 arguments, HOST, PORT, USERNAME, PASSWORD, DATABASE, TIMEOUT")
# return the arguments as a tuple
return (sys.argv[1], sys.argv[2], sys.argv[3], sys.argv[4], sys.argv[5], sys.argv[6])
def connectToDB(HOST, PORT, USERNAME, PASSWORD, DATABASE):
'''
for now, just try to connect to the database.
'''
con = pymysql.connect(host=HOST, port=PORT, user=USERNAME, password=PASSWORD, database=DATABASE)
with con:
cur = con.cursor()
cur.execute("SELECT VERSION()")
def runDelay():
'''
I don't like passing passwords in, but this is only used for a test docker
delay script
'''
# Get the database connection characteristics.
(HOST, PORT, USERNAME, PASSWORD, DATABASE, TIMEOUT) = readCommandLineArgument()
# Ensure timeout is an integer greater than zero, otherwise use 15 secs a default
try:
TIMEOUT = int(TIMEOUT)
if TIMEOUT <= 0:
raise("Timeout needs to be > 0")
except:
TIMEOUT = 60
# Ensure port is an integer greater than zero, otherwise use 3306 as default
try:
PORT = int(PORT)
if PORT <= 0:
raise("Port needs to be > 0")
except:
PORT = 3306
# Try to connect to the database TIMEOUT times
for i in range(0, TIMEOUT):
try:
# Try to connect to db
connectToDB(HOST, PORT, USERNAME, PASSWORD, DATABASE)
# If an error hasn't been raised, then exit
return True
except Exception as Ex:
strErr=Ex.args[0]
print(Ex.args)
# Sleep for 1 second
time.sleep(1)
# If I get here, assume a timeout has occurred
raise("Timeout")
if __name__ == "__main__":
runDelay()
For testing/development purposes, you could use a version of the MySQL image that has health checks (I believe there's a healthcheck/mysql image), or configure your own (see example here: Docker-compose check if mysql connection is ready).
For production use, you don't want to upgrade the database schema on startup, nor do you want to assume the database is up. Upgrading schema automatically encourages you to not think about what happens when you deploy a bug and need to rollback, and parallel schema upgrades won't work. Longer version: https://pythonspeed.com/articles/schema-migrations-server-startup/
Another option is to use a script to control the startup order, and wrap the web service's command.
In the docker-compose's documentation "wait-for-it" is one of the recommended tools, but other exists.
ALERT: WAIT BEFORE MARKING THIS AS A DUPLICATE, THERE ARE A LOT OF SIMILIAR QUESTIONS HERE, BUT NOTHING THAT WORKED FOR ME.
I'm starting to learn the php framework Symfony and I have some problems with Entities and migration. When i try to run the command bin/console make:migration on my git Bush, it gives me these errors:
In AbstractMySQLDriver.php line 106:
An exception occurred in driver: could not find driver
In PDOConnection.php line 31:
could not find driver
In PDOConnection.php line 27:
could not find driver
I am working with Symfony 4. My PC is a Windows 10 and the code is running in an Apache server made with XAMPP. The DB is local (always made with XAMPP) and it is MySQL (server MariaDB 10.4.6).The PHP version is 7.3.9.
This is my .env Symfony file:
###> symfony/framework-bundle ###
APP_ENV=dev
APP_SECRET=b9c673023606f26343aee9c3d24a8aba
#TRUSTED_PROXIES=127.0.0.1,127.0.0.2
#TRUSTED_HOSTS='^localhost|example\.com$'
###< symfony/framework-bundle ###
###> doctrine/doctrine-bundle ###
# Format described at https://www.doctrine-project.org/projects/doctrine-dbal/en/latest/reference/configuration.html#connecting-using-a-url
# For an SQLite database, use: "sqlite:///%kernel.project_dir%/var/data.db"
# Configure your db driver and server_version in config/packages/doctrine.yaml
DATABASE_URL=mysql://root#localhost:3306/symfony
###< doctrine/doctrine-bundle ###
###> symfony/swiftmailer-bundle ###
# For Gmail as a transport, use: "gmail://username:password#localhost"
# For a generic SMTP server, use: "smtp://localhost:25?encryption=&auth_mode="
# Delivery is disabled by default via "null://localhost"
MAILER_URL=null://localhost
###< symfony/swiftmailer-bundle ###
this is my doctrine.yaml:
doctrine:
dbal:
# configure these for your database server
driver: 'pdo_mysql'
server_version: '5.7'
charset: utf8mb4
default_table_options:
charset: utf8mb4
collate: utf8mb4_unicode_ci
url: '%env(resolve:DATABASE_URL)%'
orm:
auto_generate_proxy_classes: true
naming_strategy: doctrine.orm.naming_strategy.underscore
auto_mapping: true
mappings:
App:
is_bundle: false
type: annotation
dir: '%kernel.project_dir%/src/Entity'
prefix: 'App\Entity'
alias: App
In this file I tried to change the driver with php_pdo_mysql (in c:\xampp\php\ext I have a file with this name), the charset, the db version (in mariadb-10.4.6) but nothing changed.
In my php.ini (the once at c:\xampp\php ) I already removed the semicolon at extension=pdo_mysql but nothing changed.
I've also tried to uninstall and re-install doctrine, to create a new project, to search online for some sort of "mysql drivers" (maybe I searched the wrong things) and also to uninstall and re-install the entire xampp.
Nothing worked.
How can I solve this issue?
Seems that you don't use php.ini inside xampp but C:\Program Files (x86)\php\php.ini
Your php.ini try to search settings in C:\Program Files (x86)\php\php.ini
do you have another istance of php installed on your pc?
maybe in PATH environment php.exe path is out of folder of xampp
you can try to remove semicolon from C:\Program Files (x86)\php\php.ini to pdo_mysql and should fix your problem
but better solution is to fix PATH environment in windows to search php.exe in c:\xampp\php
I'm using docker for mac to run laradock and everything is alright, but the phpmyadmin can't not login by message
#2002 - No such file or directory — The server is not responding (or the local server's socket is not correctly configured).
Here is my .env file setting, and docker-compose.yml not change anythings
### PHP MY ADMIN ###
PMA_DB_ENGINE=mysql
PMA_USER=root
PMA_PASSWORD=1234
PMA_ROOT_PASSWORD=1234
PMA_PORT=8080
### MYSQL ##########
MYSQL_VERSION=8.0
MYSQL_DATABASE=test
MYSQL_USER=root
MYSQL_PASSWORD=1234
MYSQL_PORT=3306
MYSQL_ROOT_PASSWORD=1234
MYSQL_ENTRYPOINT_INITDB=./mysql/docker-entrypoint-initdb.d
and $ docker-compose ps
Name Command State Ports
------------------------------------------------------------------------------------------------------------
laradock_apache2_1 /opt/docker/bin/entrypoint ... Up 0.0.0.0:443->443/tcp, 0.0.0.0:80->80/tcp
laradock_applications_1 /true Exit 0
laradock_mysql_1 docker-entrypoint.sh mysqld Up 0.0.0.0:3306->3306/tcp
laradock_php-fpm_1 docker-php-entrypoint php-fpm Up 9000/tcp
laradock_phpmyadmin_1 /run.sh phpmyadmin Up 0.0.0.0:8080->80/tcp
laradock_workspace_1 /sbin/my_init Up 0.0.0.0:2222->22/tcp
Is that wrong with my login server position? or any idea?
Maybe phpmyadmin is trying to connect 3306 inside the phpmyadmin container?
If the image for phpmyadmin is the official one you need to link with host "db"
--link laradock_mysql_1:db
or stablish the host in env variable:
PMA_HOST