I'm building a NiFi dadaflow, and I need to get the data changes from a MySql database, so I want to use the CaptureChangeMySQL processor to do that.
I get the following error when I run the CaptureChangeMySQL processor and I don't see what's causing this :
Failed to process session due to Could not connect binlog client to any of the specified hosts due to: BinaryLogClient was unable to connect in 10000ms: org.apache.nifi.processor.exception.ProcessException: Could not connect binlog client to any of the specified hosts due to: BinaryLogClient was unable to connect in 10000ms
I have the following controller services enabled :
DistributedMapCacheClientService
DistributedMapCacheServer
But I'm not sure if they are properly configured :
DistributedMapCacheServer properties
DistributedMapCacheClientService properties
In MySql, I have enabled the log_bin variable, by default it wasn't. I checked and I have indeed some binlog files created when data change.
So I think the issue is with the controller services and how they connect, it's not clear to me.
I searched for tutorials about how to use this NiFi processor but I couldt not find how to fix this error. I looked mainly at this one : https://community.hortonworks.com/articles/113941/change-data-capture-cdc-with-apache-nifi-version-1-1.html but it did not helped me.
Does anyone have already use this processor to do CDC?
Thank you in advance.
I found what was wrong : I was trying to connect to the wrong port for the MySQL Host of the CaptureChangeMySQL processor :x
For others who are still facing similar issues, check if the firewall of the server is stopping the connection. Allow mysql 3306 in your firewall rules.
Related
Right now I am connecting to a cluster endpoint that I have set up for an Aurora DB-MySQL compatible cluster, and after I do a "failover" from the AWS console, my web application is unable to properly connect to the DB that should be writable.
My setup is like this:
Java Web App (tomcat8) with HikariCP as the connection pool, with ConnecterJ as the driver for MySQL. I am evaluating Aurora-MySQL to see if it will satisfy some of the needs the application has. The web app sits in an EC2 instance that is in the same VPC and SG as the Aurora-MySQL cluster. I am connecting through the cluster endpoint to get to the database.
After a failover, I would expect HikariCP to break connections (it does), and then attempt to reconnect (it does), however, the application must be connecting to the wrong server, because anytime a write is hit to the database, a SQL Exception is thrown that says:
The MySQL server is running with the --read-only option so it cannot execute this statement
What is the solution here? Should I rework my code to flush DNS after all connections go down, or after I start receiving this error, and then try to re-initiate connections after that? That doesn't seem right...
I don't know why I keep asking questions if I just answer them (I should really be more patient), but here's an answer in case anyone stumbles upon this in a Google search:
RDS uses DNS changes when working with the cluster endpoint to make it looks "seamless". Since the IP behind the hostname can change, if there is any sort of caching going on, then you can see pretty quickly how a change won't be reflected. Here's a page from AWS' docs that go into it a bit more: https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/java-dg-jvm-ttl.html
To resolve my issue, I went into the jvm's security file and then changed it to be 0 just to verify if what was happening was correct. Seems correct. Now I just need to figure out how to do it properly...
I am continuously getting below error while connecting to mysql-
Can\'t connect to MySQL server on /ip/address
but mysql is not generating any log for this error. I have below entries in my.cnf file
log=/var/log/mysql_err.log
log-error=/var/log/mysql/mysql_error.log
I am using correct credentials. No issue of conections.
Now how will I determine the exact issue If I'll not get any log ?
Any Idea, how to solve this ?
This is not surprising:
the server can only log an incident it is aware of. However your client does not even reach the server at all! So how should the server know some client has attempted to contact it?
The error message you get clearly indicates that you do have a connection issue.
You can easily make a test to check the most common problems: just open a telnet connection from the system trying to connect to the system the sql server runs on, connect to the mysql port: telnet <ip-of-mysql-server> mysql
On typical unixoid systems "mysql" will be substituted by the "well known port number of mysql, which is 3306. otherwise you have to specify it manually. Do you get a connection at all? I would guess not. This means either the mysql server is not listening where expected (not running or configured otherwise) or the connection is blocked on network level (firewall).
I got problem (#2006 Mysql server gone away) with mysql while connecting and performing some operations through web browser.
Operation Listed below:
When Executing big procedure
Importing database dump
When Access some particular tables It immediately throws "Server gone away".
Refer this question for Scenarios: Record Not Inserted - #2006 Mysql server gone away
Note : The above operations are works fine when I perform through terminal.
I tried some configuration as googing stated. That is set wait_timeout, max_allowed_packet. I checked for the bin_log but it is not available.
But the issues will not rectified.
What is the problem & How can I figure out & fix the issue?
what is the different between access phpmyadmin mysql server from web browser and terminal?
Where I can find the mysql server log file?
Note: If you know about any one of the above questions. Please post here. It would be helpful to trace.
Please help me to figure this out..
Thanks in advance...
Basically nothing except phpMyAdmin is limited by PHP's timeout and resource limits (limits to keep a runaway script from bogging down your entire machine for all eternity; see the docs for details of those values. In some cases, you might be authenticating through a different user account (for instance, root#localhost and root#127.0.0.1 aren't the same user), but as long as you're using a user with the same permissions the differences are minimal.
You can read more about logs in the MySQL manual, note that "By default, no logs are enabled (except the error log on Windows)".
Below are answer for question
From my research the problem is that browser have some limit to disconnect the connection i.e timeout connection. So that the above problem raised.
To resolve this problem
Go to /opt/lampp/phpmyadmin and open config.inc.php
add the command $cfg['ExecTimeLimit'] = 0;
Restart the xamp server. Now you can perform any operations.
`
2. Web client is differ from terminal because Terminal client will not getting timeout. Terminal client maintain the connection till the progress completed. I recommenced to use command prompt to import/export/run process by safe way.
Basically phpmyadmin will not have any log file. If you wanna see warnings and error you should configure the log file.
Configuration steps:
Go to /opt/lampp/etc/my.cnf
Add log_bin = /opt/lampp/var/mysql/filename.log
Restart the xamp server. You can get the log information.
I downloaded and installed the driver setup file, DB2OLEDB.exe, from here:
http://download.microsoft.com/mwg-internal/de5fs23hu73ds/progress?id=HYLbKUfGNl
Using the connection string that worked on another PC, I tried to create a Connection Object in an SSIS package. When I tested the connection I got this error:
Test connection failed because of an error in initializing provider. A TCPIP socket error has occurred (10057): A request to send or receive data was disallowed because the socket is not connected and (when sending on a datagram socket using a sendto call) no address was supplied.
Any suggestions on what the cause of this error is and how I might resolve this issue?
By the way, when I use the DB2 Configuration set up utility and test a connection from within that, I am able to successfully connect.
What other info can I provide to help you answer this question?
Thank you
Could this be related to a blocked port?
If you follow all the steps illustrated here: http://www.bidn.com/blogs/PatrickLeBlanc/ssis/700/connecting-to-db2-using-ssis do you still get the same result?
Maybe a silly question, did you restart the computer after the installation?
Are you an admin user on one machine and not on the other?
You could try to verify the port connectivity with a quick telnet command:
telnet your-db-host your-db-listening-port
If it connects, that one is off the list.
Doing some research I've found two possible fixes.
The first link suggests calling BeginReceive after the EndAccept logic is complete. Are you using script code, or just using the GUI without any scripting?
TCP async sockets throwing 10057
The second link points to drivers / software on the PC. It could be that you are missing a windows update or have faulty hardware / drivers.
I think this is less likely the case since you could connect to a different machine with the same connection string(?). Can you verify this is a valid statement?
http://social.msdn.microsoft.com/Forums/en-US/1bc3df95-c86d-4d25-aa20-30f61ed00c63/odd-socket-errors
If you could show the connection strings used for both the working and non working, and give a little more detail about The "Other PC" in comparison to the non-working PC... that would be helpful =]
If neither of the posts I've linked are the solution, this specific Google search has proven to yield some seemingly helpful results
"socket" "10057" "no address was supplied."
Actually I'm having more than one problem...but all of it has to do with connecting or working with the MySQL database...First you can see one of the errors on my website and no matter what you choose it seems to just say it can't connect to the database...I've configured a configure.php file to allow files to connect to the MySQL database...see the website here:
http://netsurfer123.byethost11.com/index.php
Then when I try to log-in to the Admin area of my MySQL database it just says:
phpMyAdmin - Error
Cannot start session without errors, please check errors given in your PHP and/or webserver log file and configure your PHP installation properly.
Please let me know what I can do to correct these errors...and thanks very much in advanced for any helpful response/s.
First you need to check if the MySQL server is actually running. Use a command such as
/etc/rc.d/init.d/mysqld status
Or similar for your system if possible.
Did you check the web server log file? Does it contain errors as indicated by phpMyAdmin?
Your site is simply unable to connect to the database to which it's trying to connect. This could be for any number of potential reasons:
Is the database running?
Is the database accessible by that machine? (Maybe there's a firewall in between the two servers, if they're separated?)
If the database is on that same machine, are you connecting to it properly? (Maybe it allows only certain connection methods, such as a socket vs. the loopback address?)
Is your connection string correct? Maybe the login is failing?
Check your PHP logs, your Apache logs, your MySQL logs. One of them is complaining about something and it's probably trying to tell you what.