I have Ubuntu 20.04 and python 3.10.6 on WSL.
I have been trying to install airflow, and am getting 'airflow: command not found' when I'm trying to do 'airflow initdb' or 'airflow info'.
I have done
export AIRFLOW_HOME=~/airflow
and when I run
myname#LAPTOP-28BMMQV7:/root$ ls -l ~/.local/bin
I can see airflow in the list of files.
drwxrwxr-x 2 myname myname 4096 Nov 20 14:17 __pycache__
-rwxrwxr-x 1 myname myname 3472 Nov 20 14:17 activate-global-python-argcomplete
-rwxrwxr-x 1 myname myname 215 Nov 20 14:17 airflow
-rwxrwxr-x 1 myname myname 213 Nov 20 14:17 alembic
when I run this command to see where my python is, I can see this
myname#LAPTOP-28BMMQV7:/root$ ls -l /usr/bin/python*
lrwxrwxrwx 1 root root 10 Aug 18 11:39 /usr/bin/python3 -> python3.10
lrwxrwxrwx 1 root root 17 Aug 18 11:39 /usr/bin/python3-config -> python3.10-config
-rwxr-xr-x 1 root root 5912936 Nov 2 18:53 /usr/bin/python3.10
I also warnings similar to this:
WARNING: The script pygmentize is installed in '/home/myname/.local/bin' which is not on PATH.
So I need to find a way to add this directory to PATH.
I have found the following advice from the airflow documentation,
If the airflow command is not getting recognized (can happen on Windows when using WSL), then ensure that ~/.local/bin is in your PATH environment variable, and add it in if necessary:
PATH=$PATH:~/.local/bin
am not quite sure how to do it?
I also have a MySQL workbench/server 8.0.31 installed and want to connect it to airflow instead of SQLite. can anybody refer me to a good guide on how to install it correctly?
I have run 'pip install 'apache-airflow[mysql]'.
You were so close! I think your local python (and your terminal whenever you tried airflow db init ) was not able to see the airflow you installed on its path.
There is this video series I go to, whenever I need to install Airflow for a fellow coworker.
This video shows how to install Airflow locally. Also, in the second video it shows how to write a DAG.
And more importantly, on the third video it shows how to connect to a different database just like you wanted.
Related
For years, I'm running a perl script on my Synology NAS. This script writes to a MariaDB database. I never had any errors. Until the migration to DSM7. This is what happened ...
I never had any problem when using MariaDB5
I migrated my database from MariaDB5 to MariaDB10. I changed the database name and the credentials to make sure my perl script used the MariaDB10 database. Everything worked fine!
I upgraded my Synology NAS to DSM7, following the provided procedure, including the requirement to remove the MariaDB5 package.
Afterwards, I noticed I following error:
Can't load '/usr/local/lib/perl5/vendor_perl/auto/DBD/mysql/mysql.so' for module DBD::mysql: libmariadb.so.3: cannot open shared object file: No such file or directory at /usr/local/lib/perl5/core_perl/DynaLoader.pm line 193.
at ./mysql1_MDB10.pl line 2.
Compilation failed in require at ./mysql1_MDB10.pl line 2.
BEGIN failed--compilation aborted at ./mysql1_MDB10.pl line 2.
To make it easy to debug, I created a test script:
#!/usr/bin/perl
use DBD::mysql;
$solDBIDB = 'DBI:mysql:database=SOLAR2_10;host=127.0.0.1;port=3307';
$DBUser = 'mydbuser';
$DBPass = 'mydbpass';
$dbh = DBI->connect($solDBIDB, $DBUser, $DBPass) || die "Could not connect to database: $DBI::errstr";
$query = "SELECT * FROM SOLAR2_10.SUM_DATA_ITEMS";
$query_handle = $dbh->prepare($query);
# EXECUTE THE QUERY
$query_handle->execute();
I'm using a Synology NAS DS415P with DSM DSM 7.1-42661 Update 2 (latest updates)
Perl is installed as a package (via the Synology Package center)
perl -v provides the following version information: This is perl 5, version 28, subversion 3 (v5.28.3) built for i686-linux
The version of MariaDB10 is 10.3.32-1040
Update after comment #ikegami
I'm able to find the mysql.so file:
-r-xr-xr-x 1 root root 131923 Apr 21 2021 /usr/local/lib/perl5/vendor_perl/auto/DBD/mysql/mysql.so
I'm able to find the various libmariadb.so files:
ls -l in /volume1/#appstore/MariaDB10/usr/local/mariadb10/lib
lrwxrwxrwx 1 root root 15 Nov 23 2021 libmariadb.so -> libmariadb.so.3
-rwxr-xr-x 1 root root 275356 Nov 23 2021 libmariadb.so.3
lrwxrwxrwx 1 root root 15 Nov 23 2021 libmysqlclient_r.so -> libmariadb.so.3
lrwxrwxrwx 1 root root 15 Nov 23 2021 libmysqlclient.so -> libmariadb.so.3
lrwxrwxrwx 1 root root 17 Nov 23 2021 libmysqld.so -> libmariadbd.so.19
drwxr-xr-x 3 root root 4096 Nov 23 2021 mysql
drwxr-xr-x 2 root root 4096 Nov 23 2021 pkgconfig
Additional info:
cpan -l | egrep -i "mysql"
Bundle::DBD::mysql 4.048
DBD::mysql 4.048
DBD::mysql::GetInfo undef
To me, everything looks properly installed ...
What am I missing?
I've been trying to get gcloud to a usable state on Travis and I just can't seem to get passed the gcloud auth activate-service-account point.
When ever it runs I just get the following error:
ERROR: (gcloud.auth.activate-service-account) PyOpenSSL is not available.
See https://developers.google.com/cloud/sdk/crypto for details.
I've tried apt-get and pip installs both with the export CLOUDSDK_PYTHON_SITEPACKAGES=1 set and nothing seems to work.
Does anyone have any ideas or alternatives?
This is Travis version Ubuntu 14.04.
Update
If I run the command from the docs on travis I get the following error:
usage: gcloud auth activate-service-account ACCOUNT --key-file KEY_FILE [optional flags]
ERROR: (gcloud.auth.activate-service-account) too few arguments
This made me think I had to have an ACCOUNT parameter, but after running the command locally with the un-encrypted service account key, I know it's not needed (unless something has changed).
The only other thing I can think of is that the file isn't be decrypted correctly or the command itself isn't happy in Travis:
- gcloud auth activate-service-account --key-file client-secret.json
Update 2
Just dumped a load of logs to figure what is going on. (Massive shout out to #Vilas for his help)
It looks like gcloud is installed on the VM for node already, but it's a super old version.
$ which gcloud
/usr/bin/gcloud
$ gcloud --version
Google Cloud SDK 0.9.37
bq 2.0.18
bq-nix 2.0.18
compute 2014.11.25
core 2014.11.25
core-nix 2014.11.25
dns 2014.11.25
gcutil 1.16.5
gcutil-nix 1.16.5
gsutil 4.6
gsutil-nix 4.6
sql 2014.11.25
The next question is how can I get the path to find the right gcloud?
I've confirmed that the downloaded SDK installs to ${HOME}/google-cloud-sdk/bin by running this command.
$ ls -l ${HOME}/google-cloud-sdk/bin
total 24
drwxr-xr-x 2 travis travis 4096 Apr 27 21:44 bootstrapping
-rwxr-xr-x 1 travis travis 3107 Mar 28 14:53 bq
-rwxr-xr-x 1 travis travis 912 Apr 21 18:56 dev_appserver.py
-rwxr-xr-x 1 travis travis 3097 Mar 28 14:53 gcloud
-rwxr-xr-x 1 travis travis 3144 Mar 28 14:53 git-credential-gcloud.sh
-rwxr-xr-x 1 travis travis 3143 Mar 28 14:53 gsutil
I finally got a solution for it. Essentially Travis has a super old version of the gcloud SDK installed that was taking presidence over the downloaded SDK.
Steps to Help Diagnose
In your .travis.yml file add:
env:
global:
# Ensure the downloaded SDK is first on the PATH
- PATH=${HOME}/google-cloud-sdk/bin:$PATH
# Ensure the install happens without prompts
- CLOUDSDK_CORE_DISABLE_PROMPTS=1
Then in your install step add the following:
install:
# Make sure SDK is downloaded - cache once it's working
# NOTE: Note sure how to update the SDK if it's cached
- curl https://sdk.cloud.google.com | bash;
# List the SDK contents to ensure it's downloaded
- ls -l ${HOME}/google-cloud-sdk/bin
# Ensure the correct gcloud is being used
- which gcloud
# Print the gcloud version and make sure it's something
# Reasonably up to date compared with:
# https://cloud.google.com/sdk/downloads#versioned
- gcloud --version
I'm working on Mac OS X Yosemite (version: 10.10.5) and I want to use MySQL as database for my web app.
I downloaded MySQL -- "mysql-5.7.10-osx10.9-x86_64.dmg", and successfully installed it.
I'm following the steps given in:
http://geeksww.com/tutorials/database_management_systems/mysql/installation/how_to_download_and_install_mysql_on_mac_os_x.php
Where I'm at this step --
**"Install and setup auto start package for MySQL on OS X":**
you should now be able to start MySQL running the following command in OS X Terminal window
$ sudo /Library/StartupItems/MySQLCOM/MySQLCOM start
However I'm unable to locate MySQLCOM directory under /Library/StartupItems/ directory
[Terminal]:$ pwd
/Library/StartupItems
[Terminal]:$ ls -l
total 0
drwxr-xr-x 5 root wheel 170 Dec 7 22:55 HWNetMgr
drwxr-xr-x 6 root wheel 204 Dec 7 22:55 HWPortDetect
drwxr-xr-x 10 root wheel 340 Dec 7 22:55 StartOuc
drwxr-xr-x 3 root wheel 102 Sep 30 14:12 cma
I'm trying to debug a page in a web app that keeps crashing Chrome ("Aw, snap!" error). I've enabled/disabled automatic crash reporting, tried logging with google-chrome --enable-logging --v=1, (as well as various levels of verbosity), and all I get is a "crash dump ID" in the chrome_debug.log chrome://crashes Shows all of the dump IDs, but no actual dump file
I see other questions referring to reading the dump files, but I can't find the dump files themselves (just the ID).
Grepping for the crash ID in /tmp and ~/.config/google-chrome/ turns up nothing, but the ~/.config/google-chrome/chrome_debug.log shows that something was sent:
--2015-04-06 11:10:00-- https://clients2.google.com/cr/report
Resolving clients2.google.com (clients2.google.com)... 74.125.228.224, 74.125.228.225, 74.125.228.231, ...
Connecting to clients2.google.com (clients2.google.com)|74.125.228.224|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: ‘/dev/fd/3’
0K
Crash dump id: 7dac9d5d58258264
Any ideas on where to find the actual file/data that's sent?
Details:
Chrome version: 40.0.2214.111 (Official Build)
Linux Mint 16 (Petra)
Edit: Some extra info:
curtis#localhost:-$ tail -n 5 uploads.log && echo $(pwd)
1428584493,ddc357e4600a49e6
1428584497,7ac16455c152381a
1428589439,d00ad6f5e6426f3d
1428934450,66b3f722430511e8
1428939578,7a2efc2b681515d1
/home/curtis/.config/google-chrome/Crash Reports
curtis#localhost:-$ ll -a
total 12
drwx------ 2 curtis curtis 4096 Apr 6 11:32 .
drwx------ 9 curtis curtis 4096 Apr 13 11:43 ..
-rw------- 1 curtis curtis 3291 Apr 13 11:39 uploads.log
Automatic reporting is enabled...
Thanks!
The *.dmp files are stored in /tmp/, and this has nothing to do with the "Automatic crash reporting" checkbox. The file is also not related to the hash stored in ~/.config/google-chrome/
In ~/.config/google-chrome/Crash Reports/uploads.log:
1429189585,5bddea9f7433e3da
From using , the crash dump file for this particular report was:
chromium-renderer-minidump-2113a256de381bce.dmp
Solution:
root#localhost:-$ mkdir /tmp/misc && chmod 777 /tmp/misc
root#localhost:-$ cd /tmp
root#localhost:-$ watch -n 1 'find . -mmin -1 -exec cp {} /tmp/misc/ \;'
Then, as a regular user (not root):
google-chrome --enable-logging --v=1
Once you see files created by the watch command, run:
root#localhost:-$ ls -l
-rw------- 1 root root 230432 Apr 16 09:06 chromium-renderer-minidump-2113a256de381bce.dmp
-rw------- 1 root root 230264 Apr 16 09:12 chromium-renderer-minidump-95889ebac3d8ac81.dmp
-rw------- 1 root root 231264 Apr 16 09:13 chromium-renderer-minidump-da0752adcba4e7ca.dmp
-rw------- 1 root root 236246 Apr 16 09:12 chromium-upload-56dc27ccc3570a10
-rw------- 1 root root 237247 Apr 16 09:13 chromium-upload-5cebb028232dd944
Now you can use breakpad to work on the *.dmp files.
Google Chrome - Crash Dump Location
To generate the Crash Dump locally,
CHROME_HEADLESS=1 google-chrome
The .dmp files are then stored in ~/.config/google-chrome/Crash Reports
Produce Stack Trace
Check out and add depot_tools to your PATH (used to build breakpad)
git clone https://chromium.googlesource.com/chromium/tools/depot_tools
export PATH=`pwd`/depot_tools:"$PATH"
Check out and build breakpad (using fetch from depot_tools)
mkdir breakpad && cd breakpad
fetch breakpad
cd src
./config && make
To produce stack trace without symbols:
breakpad/src/processor/minidump_stackwalk -m /path/to/minidump
More here https://www.chromium.org/developers/decoding-crash-dumps
Personally Preferred Method
Enable crash reporting:
Chrome menu > Settings > Show advanced settings > Tick "Automatically send usage statistics and crash reports to Google"
Go to chrome://crashes > File bug > Takes you to crbug.com > Complete
report leaving the auto-added report_id field unchanged.
Someone from the Chrome/Chromium team will follow up. They can provide
you with your stack trace and aid at resolving the issue.
Trying to back out of a macports mysql installation and return to Snow Leopard Server's built-in MySQL server, but I cannot get it to work.
When I disable macports and enable the built-in service, mysql.sock cannot be found (locate mysql.sock returns nothing). When I re-enable the macport mysql, mysql.sock is found but now I cannot disable the built-in MySQL service.
Every time I try, it just re-enables it.
I have to run the following commands to get MacPorts MySQL to work upon reboots:
sudo launchctl unload -w /Library/LaunchDaemons/org.macports.mysql5.plist
sudo launchctl load -w /Library/LaunchDaemons/org.macports.mysql5.plist
ln -s /var/mysql/mysql.sock /tmp/mysql.sock
Permissions on /var/mysql are (which is where the built-in service is set to):
drwxr-xr-x 111 _mysql _mysql
Permissions on the macports datadir are:
drwxr-xr-x 116 _mysql _mysql
At one time, according to the access log file for the built-in mysql, it started up correctly (2010). Is there a way to manually disable this service from starting up when I reboot?
I realize how unclear my problem is, but somehow the previous admin got macports mysql tied in with the built-in mysql and I'm having a heck of a time untangling them.
In /Library/LaunchDaemons/ plist-files of the installed applications are located, here's what I have there:
$ ls -l /Library/LaunchDaemons/
-r--r--r-- 1 root wheel 573 Jan 10 18:33 at.obdev.littlesnitchd.plist
-rw-r--r-- 1 root wheel 567 Mar 5 19:02 com.parallels.desktop.launchdaemon.plist
lrwxr-xr-x 1 root admin 74 Jan 20 06:21 org.macports.mysql5.plist -> /opt/local/etc/LaunchDaemons/org.macports.mysql5/org.macports.mysql5.plist
lrwxr-xr-x 1 root admin 74 Oct 14 2011 org.macports.rsyncd.plist -> /opt/local/etc/LaunchDaemons/org.macports.rsyncd/org.macports.rsyncd.plist
And if you'd like to check the MacOS bundled services' config, take a look at /System/Library/LaunchDaemons/.