I am trying to transfer data using sqoop from HDFS to the MSSQL server. But for some reasons, sqoop hangs at
tool.BaseSqoopTool: Enabled debug logging.
sqoop.ConnFactory: Added factory com.microsoft.sqoop.SqlServer.MSSQLServerManagerFactory specified by /usr/lib/sqoop/conf/managers.d/mssqoop-sqlserver
DEBUG sqoop.ConnFactory: Loaded manager factory: com.microsoft.sqoop.SqlServer.MSSQLServerManagerFactory
DEBUG sqoop.ConnFactory: Loaded manager factory: com.cloudera.sqoop.manager.DefaultManagerFactory
DEBUG sqoop.ConnFactory: Trying ManagerFactory: com.microsoft.sqoop.SqlServer.MSSQLServerManagerFactory
INFO SqlServer.MSSQLServerManagerFactory: Using Microsoft's SQL Server - Hadoop Connector
INFO manager.SqlManager: Using default fetchSize of 1000
DEBUG sqoop.ConnFactory: Instantiated ConnManager com.microsoft.sqoop.SqlServer.MSSQLServerManager#45db05b2
INFO tool.CodeGenTool: Beginning code generation
DEBUG manager.SqlManager: No connection paramenters specified. Using regular API for making connection.
I check the firewall and it is allowing connections without any restrictions. Sqoop gets authenticated but doesnt initiate a map reduce job after it gets authenticated. Any one has faced similar problems before?
Try using --verbose to print more information.
Is your SQL Server running on a Virtual Machine? I had a similar problem with Oracle. I was running Oracle on a VM with a static IP and a Bridged network adapter. Servers within the same network as the Oracle server could connect fine, but servers outside the network showed these same symptoms. The solution was to change from a Bridged Interface to a NAT'd interface. Then you need to set up a port forwarding rule on the host machine to your database server, and make your Sqoop connection to the host machine IP rather than the VM's IP. It took me several days to get this figured out. Hope it helps.
We has MsSQL server running on our machines. The problem was that the particular version of JVM (Java(TM) SE Runtime Environment (build 1.6.0_29-b11)) had bug and caused the client to hang in the getconnection method.
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7103725
We upgraded to a newer version and things worked fine.
Related
I have a server running Ovirt Node that I'm trying to manage remotely using libvirt. I have an SSH keypair installed and can ssh user#server -i ssh-privkey successfully. When I try to connect to qemu+ssh//user#host/system?keyfile=ssh-privkey, I get this error:
authentication failed: Failed to start SASL negotiation: -4 (SASL(-4): no mechanism available: No worthy mechs found)
That led me down the path of getting TLS keys and certificates installed on the client and the server mostly according to these instructions (the configuration is slightly different because I have only one host and am using Terraform to manage the certificates*). However, I still get the same error. When I look at the output of libvirt --listen --verbose on the server when a connection failed, the only useful output is this:
error : virNetSocketReadWire:1792 : End of file while reading data: Input/output error
I have checked every firewall between the client and the server and they should all be wide open. What else could be the cause of this error?
* The goal is ultimately to use Terraform to provision libvirt resources, however I get the same errors trying to connect with virsh and virt-manager.
UPDATE: It's easier to connect just via SSH; this question exists because I couldn't figure out how to turn off SASL. It turns out SASL is enabled for SSH connections due to vdsm setting auth_unix_rw="sasl" in /etc/libvirt/libvirtd.conf. Removing that config means I can just use my SSH private key as I intended. The TLS configuration was a wild goose chase that was further hindered by vdsm changing the configured location of all the PKI files.
You're likely missing a RPM package on your client host. First on the virtualization host check /etc/sasl2/libvirt.conf and see what 'mech_list' setting is uncommented.
Back on your client you'll need to install a 'cyrus-sasl-XXXX' RPM that provides the same mechanism that the server is set to use. For a modern libvirt install it will probably be using 'cyrus-sasl-scram' for plain username/password auth, but for older installs, it might still be using 'cyrus-sasl-md5'
I already have installed vernemq in my cloud server, everything seems running fine. But from any client, it is unable to connect the broker using username/password authentication. The authentication is based on MySQL, the Lua script is running fine. But strangely the authentication not working or something is wrong. I couldn't trace anything as nothing shows in Log. The only debug message I see is as follows:
[debug] <0.227.0>#plumtree_broadcast:schedule_lazy_tick:720 0ms mailbox traversal, schedule next lazy broadcast in 10000ms, the min interval is 10000ms
OS : Ubuntu 18.04
MySQL Version : 5.7.29
VerneMQ Version : 1.10
That debug log message is unrelated (it shows load information on the metadata exchange protocol).
Please make sure the password formats/hashing methods are compatible between VerneMQ and MySQL.
I'm Running Linux Ubuntu 16.04 LTS on google cloud compute engine. this server hosts a website which in turn uses the sql client to connect to a sql instance also running on google cloud platform. I connect to the webserver using ssh which all is fine until I connect to the sql instance.
once this connection is made and the website also can connect to the database my screen is flooded with messages like:
2017/07/12 16:56:30 New connection for
"snappy-gantry-xxxxxx:europe-west1:xxxxx" 2017/07/12 16:56:30 Client
closed local connection on
/cloudsql/snappy-gantry-xxxxxx:europe-west1:xxxxx
This makes it undo-able for me to continue working on this machine. How can i disable this so i can continue my work?
As it is the server is still in development. when it goes to production is don't mind it but even then i don't need those messages.
The problem doesnt occur anymore. What i first did was setting up the connection with sql under my own user account which then spit out the logging. I now started it under the service account and when i log in under my own name i don't see the logging anymore.
I've recently created a Mule application (3.7.0 CE) on a laptop. I'm connected to an AWS RDS instance when running locally in AnyPoint Studio using Maven. I started with a local MySQL DB and migrated it to AWS because my application "proofofconcept" is just that a proof of concept and I would like to show the application online (public url) instead of my laptop for a presentation. I added the database.url=... property to the application properties when I deployed to Anypoint Runtime Manager in the cloud. I'm currently getting a:
communications link failure
I've tried several things and nothing has worked. I tried a basic database connection first in the database config. And, then I created a JDBC datasource in Spring-beans. Both methods worked locally and in-communication with AWS (remote). When I deploy to Runtime Manager, the application deploys. And, I get the console that's generated runtime by the RAML. When I call a url e.g. api/v1/orders it runs and runs and after timeout provides the communication error.
Does anyone 1) know if the communication is allowed? 2) know how to fix this? I would like to demo the POC online for my client.
Thanks in advance
My issue was with Amazon VPC and the default security group assigned to my RDS instance. By default all outbound activity is set to any protocol and any port for any ip (0.0.0.0/0). Inbound routing, however was specifying only port 3306 but also a custom using-ip that was my home network public ip. I changed the ip specification to be 0.0.0.0/0. This now mean's that any ip can send a request though port 3306 to my Amazon MySQL instance.
I have mysql server installed on my machine. I would like to connect to the same local mysql instance using Google app script's jdbc service. I am getting connection failed error. So, I would like to confirm:
1. is it possible to connect to mysql instance running on private machine(laptop) using jdbc service.
2. If not is it always mandatory to connect to public IP. What is the alternative to connect to mysql instance running on private machine.
This definitely works, but you need to permit Google's machines to connect to your machine. That means opening a hole in your firewall, possibly setting up port forwarding on your router, etc.
There is an alternative to poking a hole in your security arrangement, it is called Secure Data Connector, and it lets your machine initiate the connection to Google - https://developers.google.com/secure-data-connector/
Unfortunately, I don't believe that SDC supports JDBC yet (I believe it only supports HTTP based communication).
(P.S. One very simple work around is to mirror your local database to some kind of cloud service, and then permit access to that replica from Google).