I am facing this error while starting logstash.
I have configured MYSQL db as input in logstash.
Pipeline_id:main
Plugin: <LogStash::Inputs::Jdbc jdbc_user=>"admin", jdbc_password=><password>, statement=>"SELECT * from table", jdbc_driver_library=>"/usr/s$
Error: Permission denied - /.logstash_jdbc_last_run
Exception: Errno::EACCES
Stack: org/jruby/RubyIO.java:1237:in `sysopen'
org/jruby/RubyIO.java:3800:in `write'
The last_run_metadata_path option defaults to "#{ENV['HOME']}/.logstash_jdbc_last_run", and it looks like that evaluates to /.logstash_jdbc_last_run. You are getting EACCES because you do not have write access to the root directory.
To fix it, set the last_run_metadata_path option to a file in a directory that you have write access to. If you have an input configuration that starts with
input {
jdbc {
jdbc_driver_library => "mysql-connector-java-5.1.36-bin.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
you can just add an additional line there
last_run_metadata_path => "/tmp/.logstash_jdbc_last_run"
using some path that you know you can write to and that nobody else will be using.
Related
I am trying to import data from MySQL to Elasticsearch using logstash version.
Versions of software used:
Java/JRE 1.8
Elasticsearch 6.1.0
Logstash 6.1.0
My conf contents are as follows:
file: simple-out.conf
input {
jdbc {
# MySQL jdbc connection string to our database, mydb
jdbc_connection_string => "jdbc:mysql://valid/validDBNAME?useSSL=false"
# The user we wish to execute our statement as
jdbc_user => "MY USER"
jdbc_password => "MY PWD"
# The path to our downloaded jdbc driver
jdbc_driver_library => "C:\JavaDevelopment\TomcatServer\apache-tomcat-8.5.20\lib\mysql-connector-java-5.1.45-bin.jar"
# The name of the driver class for Postgresql
jdbc_driver_class => "com.mysql.jdbc.Driver"
# our query
statement => "SELECT * from testtable"
jdbc_paging_enabled => "true"
jdbc_page_size => "50000"
}
}
output {
stdout { codec => json_lines }
elasticsearch {
hosts => "http://localhost:9200"
index => "test-migrate"
document_type => "data"
}
}
When I run logstash I get the following error:
[2017-12-19T16:50:08,055][ERROR][logstash.pipeline ] Pipeline aborted due to error {:pipeline_id=>"main", :exception=>#<NameError: SSLConnectionSocketFactory not found ERROR
Please suggest how to get past this.
Thanks
Try adding httpclient-VERSION.jar, httpcore-VERSION.jar files to LOGSTASH_HOME/vendor/jruby/lib/ folder.
I had same problem, the solution is:
add LOGSTASH_HOME variable path in environment variables in mycomputer properities to C:\some_path\logstash-6.3.0
then add these files to C:\some_path\logstash-6.3.0\logstash-core\lib\jars
files:
httpclient-4.5.2.jar
log4j-1.2.17.jar
httpcore-VERSION.jar
this is because logstash doesn't come with ssl jar file and you need to add this to jar folder and set the environment so as logstash can read those jar files
and try using gitbash rather than cmd or powershell!
and try to close and restart your logstash and elasticsearch
follow this link to see which index are comming to elasticsearch.
http://localhost:9200/_cat/indices?v
if you are running on diff. port please change port number
addidional tips
and it didn't work for me on jdk 10.x so use jdk 8.x
and maybe installer wont work even so just unzip file and run elasticsearch.bat and logstash -f cong_filepath\configfile.config in bin folders
windows user can type cmd in top of path
Please make sure MySQL is running, (doesn't mean open) on server
Learn # https://www.elastic.co/blog/logstash-jdbc-input-plugin
To resolve it yourself in future:
open logstash/bin folder and right click and gitbash (need git)
type "vi log" and press tab 3 times till all logstahsh files are shown up
a new line will appear, type "vi logstash" and press enter to open source file
you'll have code to understand flow,.. if you look carefully Logstash_home and javacmd (at bottom) are important that execute jar, javacmd should be set to your java_home variable in env variables to jdk8.x and Logstash_home has a path for jar directory, either change it or put you jar to this folder.
Hope this helps!
I am trying to set up a Mitaka OpenStack cloud. But when I try to execute:
# /usr/bin/nova-manage --debug api_db sync
And the I get the error message:
error: No sql_connection parameter is established
Yet I am able to access the nova database via mysql command line, using the values I am using for my I have in the /etc/nova/nova.conf:
[database]
connection=mysql://nova:nova#svl-os:3306/nova
In the Mitaka release they added a new DB schema, nova_api. So I needed to add ...
[api_database]
connection=mysql://nova_api_db_user:password#mydbhost:3306/nova_api
... to my /etc/nova/nova.conf file.
With Drush v8.0.3 in both sides (local is a Mac and desarrollo is a Linux Server with CPanel), I'm executing:
sql-sync #project.local #project.desarrollo
and getting this error at the end of the process (everything else works):
ERROR 1045 (28000): Access denied for user 'myremotehostuser'#'localhost' (using password: YES)
My alias file includes this:
$aliases['desarrollo'] = array (
'root' => '/home/myremotehostuser/subdomains/project/',
'uri' => 'http://project.myremotehost.com',
'remote-user' => 'myremotehostuser',
'remote-host' => 'myremotehost.com',
'path-aliases' => array(
'%dump-dir' => '/tmp',
),
'source-command-specific' => array (
'sql-sync' => array (
'no-cache' => TRUE,
'structure-tables-key' => 'common',
),
),
'command-specific' => array (
'sql-sync' => array (
'sanitize' => TRUE,
'no-ordered-dump' => TRUE,
'structure-tables' => array(
'common' => array('cache', 'cache_filter', 'cache_menu', 'cache_page', 'history', 'sessions', 'watchdog'),
),
),
),
);
And drush status on "desarrollo" returns:
Drupal version : 7.41
Site URI : http://project.myremotehost.com
Database driver : mysql
Database hostname : localhost
Database port :
Database username : databaseuser
Database name : databasename
PHP configuration : /usr/local/lib/php.ini
PHP OS : Linux
Drush script : /usr/local/bin/drush
Drush version : 8.0.3
Drush temp directory : /tmp
Drush configuration :
Drush alias files :
Drupal root : /home/myremotehostuser/subdomains/project/
Drupal Settings File : sites/default/settings.php
Site path : sites/default
As you can see, the "database owner" (a MySQL user) is not the same as "remote-host" (a CPanel user).
Drush is supossed to scan the setting files in both sides to figure out the right config, if I'm not wrong, you don't even have to add a $databases array or a db-url string since v7. Then, why is it trying to access the db with 'myremotehostuser'#'localhost' instead of 'databaseuser'#'localhost'?
In this case, there is no database user called 'myremotehostuser' and I guess that I could solve the problem by creating it and granting permissions for 'databasename' but I'm almost sure I'm missing something really dumb here and there must be a simple solution.
edit:
Trying the same alias file in the same server with host user name and database user name still being different, but this time in a freshly created account, seems to work prefectly. So I guess the problem it's being caused by some CPanel/WHM configuration issue rather than any Drush related thing. I will keep trying this a couple of times before closing the question.
I encountered this issue too. It turns out drush is first looking for the file ~/.my.cnf and if it's found, it'll use it to connect. The file contains the username and password.
[client]
user=myremotehostuser
Once I temporarily remove this file, drush can dump the sql successfully using the site's settings.php.
Repeating the rsync and sql-sync processes in new accounts work without problem. The issue only happens in this already created account, with multiple databases, users, and prefixes. I don't have full access to this account so I can't test it properly.
This issue was too specific and probably didn't deserve a question by itself since it's almost impossible to replicate. I'll consider it "solved" and will update it in the case I find out why it's not working.
As it is described here I'm trying to do the migration in order to let yii create the user authentication tables. However I get the error:
Exception 'yii\base\InvalidConfigException' with message 'You should
configure "authManager" component to use database before executing
this migration.'
The authManager is present in the configuration but I still get that error. Configuration:
'components' => [
...
'db' => $db,
'authManager' => [
'class' => 'yii\rbac\DbManager',
// 'defaultRoles' => ['guest'],
],
...
],
What's the problem?
Problem was quite simple: The configuration was not used. yii migrate is a console command and usually the configuration for such commands is placed in a seperated file.
In case of yii2-app-base template, from which I have created my application, there is a config/console.php configuration file where the authManager needs to be declared. It is not sufficient to have it in the config/web.php declared only.
In case of yii2-app-advanced the authManager should be declared in console/config/main.php and not in the backend or frontend configuration files.
I followed the steps trying to configure MySQL in WildFly. I have two questions for your help with:
1) I downloaded the mysql-connector-java-5.1.33-bin.jar and placed it under modules/system/layers/base/com/mysql/main/. Do I need to download the actual MySql? Or the connector jar is sufficient?
2) In creating a new data source in WildFly console, I was not able to create a new data source. Part of the information I need to fill in is a pair of user name and password to access the database. Where should I create this user name and password first? I am guessing this is where I got the problem from.
I got this error message when testing the connection in wildfly console:
Unexpected HTTP response: 500
Request
{
"address" => [
("subsystem" => "datasources"),
("data-source" => "mysqlDSPool")
],
"operation" => "test-connection-in-pool"
}
Response
Internal Server Error
{
"outcome" => "failed",
"failure-description" => "JBAS010440: failed to invoke operation: JBAS010447: Connection is not valid",
"rolled-back" => true
}
first you need to install Mysql server and a JDBC 4-compliant driver, normally all new JDBCs provided by Mysql.org are JDBC 4-compliant, find a platform independant one here, then you need to add a datasource here in this file standalone/configuration/standalone.xml or using this command
data-source add --name=myDataSource--jndi-name="java:jboss/datasources/myDataSource" \
--connection-url="jdbc:mysql://localhost:3306/myDB" \
--driver-name=h2 --user-name="myDB_Username" --password="myPassword"
username and password are those used to connect to Mysql database.
1) You need to download the jdbc-driver jar which, I think, is the connector jar. But please don't place it under modules/system/... but directly under modules since the system folder is reserved for internal modules that are delivered with the server.
2) Here is an example (configures an Oracle datasource):
/subsystem=datasources/jdbc-driver=OracleJdbcDriver:add(driver-module-name=oracle.jdbcaq,driver-name=OracleJdbcDriver)
/subsystem=datasources/data-source=OracleDS:add(jndi-name=java:jboss/datasources/OracleDS,enabled=true,jta=true,use-java-context=true,connection-url=jdbc:oracle:oci:#dbms:1523/DEV,driver-name=OracleJdbcDriver,min-pool-size=5,max-pool-size=100,user-name=username,password=password,prepared-statements-cache-size=100,exception-sorter-class-name=org.jboss.jca.adapters.jdbc.extensions.oracle.OracleExceptionSorter)