I am using IPFS version 0.4.4.
My goal is to connect two peers in order to prevent IPFS peer to halt on reading an IPFS-hash from the shared peer. In order to achieve it, I am using ipfs swarm connect to connect peer-A to peer-B, where peer-B can access ipfs-file on peer-A.
My question is related to:
ipfs swarm connect /ip4/x.x.x.x/tcp/4003/ipfs/QmXXXXXXXXXXXXXXXXXXX
When I try to connect my laptop to another IPFS-peer, I face with following error:
connect failure: dial attempt failed: context deadline exceeded.
But when I try on an Amazon AWS where all the ports are open, it works, hence swarm connect ended as success.
[Q] In order to make ipfs swarm connect work should API and Gateway port should be open? or should I do something else?
For example should: port 5001 and 8080 be open no matter what?
.ipfs/config file:
"API": "/ip4/127.0.0.1/tcp/5001",
"Gateway": "/ip4/127.0.0.1/tcp/8080",
I believe you don't have to open API and Gateway ports to be able to connect to your peer. Instead, just try checking your connectivity from the outside:
telnet x.x.x.x yyyy
#Trying x.x.x.x...
#Connected to x.x.x.x.
#Escape character is '^]'.
#/multistream/1.0.0
You can see the port in Addresses section of IPFS config, in my case it's 4001:
"Addresses": {
"Swarm": [
"/ip4/0.0.0.0/tcp/4001",
"/ip6/::/tcp/4001"
],
but since the host is behind NAT, the actual IP where it can be accessed can't be detected by IPFS daemon, so I had to put it to Announce section, like
"Announce": ["/ip4/z.z.z.z/tcp/4001"],
After finding the right IP and port, I was able to connect:
ipfs swarm connect /ip4/z.z.z.z/tcp/4001/ipfs/QmXXX_my_peer_id_XXX
#connect QmXXX_my_peer_id_XXX success
Related
I am facing difficulty in establishing remote database connection (MySQL / MariaDB) to AWS-RDS. On the server side (AWS RDS) everything looks fine and even Inbound rules are set to accept My IP ( TCP Protocol over Port 3306). I am working on Windows machine so the error might be related to Windows Firewall. For this reason, I even added new Outbound and Inbound rule in Firewall Manager and set Remote Port to 3306. But still I end with below connection error:
The following error occurred:
Can't connect to MySQL server on 'xxx.rds.amazonaws.com' (10060) QMYSQL: Unable to connect
Windows Firewall setting (similar for both Outbound & Inbound):
Amazon RDS is publicly accessible but still I am getting 10060 error. Is there something missing in Amazon (like in VPC) or is this caused by windows firewall?
I came across similar posts but those solutions didn't help. AWS RDS connection error
Can anyone please let me know what could be the issue and how to resolve it?
Update
After setting up VPC Flow logs to monitor the traffic, log messages are stored in S3 bucket with interval of 10mins. From 2 different logs, I could find:
version account-id interface-id srcaddr dstaddr srcport dstport protocol packets bytes start end action log-status
2 xx164 eni-08f88xxxx xx.xx.xxx.119 xxx.xx.2.17 34010 3306 6 5 260 1661944383 1661944424 REJECT OK
2 xx164 eni-08f88xxxx xx.xx.xxx.119 xx.xx.2.17 33953 3306 6 5 260 1661938359 1661938418 REJECT OK
The srcaddr, dstaddr and dstport is same in both cases but, srcport is different. Is this expected behavior? The srcaddr doesn't match with the IP addr set in security group (inbound rule). I don't understand which IP addr is taken as srcaddr.
Whereas doing telnet rdsendpoint 3306 results in Connecting To <rdsendpoint>...Could not open connection to the host, on port 3306: Connect failed
Thanks in advance.
P.S: please let me know if any info is missing here.
I deploy a docker container with Ghost inside, to the Google Cloud Run.
The Cloud Run service has a service account with Cloud SQL Client role.
I've added the SQL instance into the connections of the Cloud Run Service.
The Ghost's configuration file is the below property
"database": {
"client": "mysql",
"connection": {
"socketPath": "/cloudsql/xxxxxxx",
"user": "xxxxxxx",
"password": "xxxxxxx",
"database": "ghost1"
}
},
I have an Google Cloud SQL - MySQL instance up and running. I can connect to it through Public IP and using the same credentials.
After I deploy the container, I am getting a "We'll be right back" page on the Ghost.
When I look into the logs on each side, I see some errors which I do not understand the root cause.
Examples of the logs at Google Cloud SQL:
2021-11-14T06:40:37.183921Z 6971 [Warning] User 'mysql.session'#'localhost' was assigned access 0x8000 but was allowed to have only 0x0.
2021-11-14T06:49:09.008652Z 7002 [Note] Aborted connection 7002 to db: 'ghost1' user: 'xxxxxxx' host: 'cloudsqlproxy~107.178.207.100' (Got an error reading communication packets)
2021-11-14T06:50:29.721121Z 7471 [Note] Got timeout reading communication packets
Examples of the logs at Google Cloud Run
DatabaseError: Knex: Timeout acquiring a connection. The pool is probably full. Are you missing a .transacting(trx) call? at DatabaseError.KnexMigrateError
I have tried a lot of combinations like using VPC connector and Private IP but I keep getting the same network timeout errors all the time. I suspect that the Ghost mysql adaptor library (knex) is doing something wrong but I am not sure whether that is true and if there is something I can do about it.
Thanks for your help
For what it's worth, we have an example app that connects over Unix sockets.
As long as you're connecting to public IP, you won't need a Serverless VPC Access connector or a Private IP.
Also, you might double check that your socket path is correct. It should look like this: /cloudsql/project-name:region:instance-name.
There's a similar question here that might help:
How do I run Ghost with MySQL on GCP?
Make sure you allow SSL connections only and create a new certificate chain for your instance.
"ssl": {
"cert": "<database_cert>",
"ca": "<database_ca>",
"key": "<database_key>"
}
example here
Activating SSL connections only and adding the certificate chain did not work for me.
I am using the unix socketPath to connect from Ghost to a CloudSQL database like mentioned above: /cloudsql/project-name:region:instance-name.
When running Ghost version 5.x.x the db connection stopped working after some random time. The Cloud Run logs showed:
Error: connect ETIMEDOUT
and
Error: Connection lost: The server closed the connection.
I tested my configuration by running Ghost on localhost, using the cloudsql-proxy for authentication and it worked without problems.
I finally got it working on Cloud Run by patching the node_modules/knex/lib/client.js. Re-initialise the connection pool and try to reconnect to the database in case the pool has been destroyed.
The approach is also explained here in more detail: https://stackoverflow.com/a/69191743/2546954
I am having trouble connecting to my RDS MySQL Database.
My current Set up:
Django 1.9+
AWS - Launched an EC2 Instance (Amazon Linux). Launched an RDS instance of MySQL (5.6.7). I created a Virtual PC (VPC) for the EC2 instance and RDS instance, and in that group created several security rules so the EC2 instance could connect to the MySQL RDS. Below are the security rules I use and associate with the VPC.
Inbound Rules: (Type, Protocol, Port Range, Source)
1. Http , TCP, 80 , 0.0.0.0/0
2. SSH , TCP, 22 , <homePC address>/32
3. HTTPS, TCP, 443, 0.0.0.0/0
4. MYSQL/Aurora, TCP, 3306, source (my created security group)
Nginx - Serving my content redirecting from port 80 to port 443. I created SSL certificates in order to make my site secure. I correctly served my Django application, and the static files. Nginx serves everything great!
Previously I was able to connect to my RDS instance. I did not set up an elastic IP at that point in time, as it took some time to transfer my domain over to Route53 (amazons DNS). I successfully was able to transfer it, and associate it with my EC2 instance and even start my application
The problems -
First Problem I cannot connect to the RDS from my EC2 instance. I have checked my django settings file and the secret file (the one that feeds my settings file with sensitive information which I keep out of version control). Here are examples of my settings for the mysql connection - I changed them obviously
"DATABASE_NAME":"myDatabase",
"DATABASE_USERNAME": "myUsername",
"DATABASE_PASSWORD": "myPassword123",
I double checked the username and password in the RDS console, and even reset the password.
After I went ahead and activated my virtual environment and attempted to connect.
python manage.py dbshell
My Error was
ERROR 1045 (28000): Access denied for user 'myUsername'#'<private IP address from Elastic IP>' (using password: YES)
Notive the user name is not what I utilized in the settings. It changed. I feel this is the root cause of the problem, but am unsure why Django would change my username in such a manner.
Second Problem - I also try to connect to the database directly using mysql. I double check my endpoint, username and password are correct, but also run into trouble connecting.
Note - As I said, I was able to connect in both manners listed above before acquiring my IP address. Thus my troubles are configuring security rules to allow a permanent IP address, and being unable to use TCP connect to use the Mysql client to log in.
Thanks in advance.
I am trying to start sync_gateway from cmd with following command:
sync_gateway -url http://75.76.221.21:8091
I received following error:
20:33:23.014229 WARNING: Error installing Couchbase design doc: Put
http://192.168.2.102:8092/sync_gateway/_design/sync_gateway: dial tcp
192.168.2.102:8092: ConnectEx tcp: A connection attempt failed because the connected party did not properly respond after a period of time,
or established connection failed because connected host has failed to
respond. -- db.installViews() at database.go:29120:33:44.055739
WARNING: Error installing Couchbase design doc: Put
http:/_design/sync_housekeeping: dial tcp 192.168.2.102:8092:
ConnectExtcp: A connection attempt failed because the connected party
did not properly respond after a period of time, or established
connection failed because connected host has failed to respond. --
db.installViews() at database.go:30520:33:44.055739
FATAL: Error opening database: Put
192.168.2.102:8092/omnibazaar/_design/sync_housekeeping: dial tcp 192.168.2.102:8092: ConnectEx tcp: A connection attempt failed because the connected party did not properly respond after a period of time,
or established connection failed because connected host has failed to
respond. -- rest.RunServer() at config.go:415
How did you name your nodes in your Couchbase instance? If you look at the Server Nodes tab, is one of your nodes named "192.168.2.102". I see that from your sync gateway machine, you are trying to reach the Couchbase cluster using the 75.76.221.21 address. Have you checked network connectivity, from your sync gateway machine, try "telnet 192.168.2.102 8092" and see if it connects.
One other thing, it is more common to start sync gateway by redirecting in a configuration file, you can find them in the examples directory. For instance "sync_gateway < examples/config-server.json"
(I'm posting this question after the fact because of the time it took to find the root cause and solution. There's also a good chance other people will run into the same problem)
I have an RDS instance (in a VPC) that I'm trying to connect to from an application running on a classic EC2 instance, connected via ClassicLink. Security groups and DNS aren't an issue.
I am able to establish socket connections to the RDS instance, but cannot connect with CLI tools (psql, mysql, etc.) or DB GUI tools like toad or mysql workbench.
Direct socket connections with telnet or nc result in TCP connections in the "ESTABLISHED" state (output from netstat).
Connections from DB CLI, GUI tools, or applications result in timeouts and TCP connections that are stuck in the "SYN" state.
UPDATE: The root cause in my case was a problem with MTU size and EC2 ClassicLink. I've posted some general troubleshooting information below in an answer in case other people run into similar RDS connectivity issues.
Additional information for people who might run into similar issues trying to connect to RDS or RedShift:
1) Check security groups
Verify the security group for the RDS instance allows access from the security group your source server belongs to (or its IP added directly if external to AWS). The security group you should be looking at is the one specified in the RDS instance attributes from the RDS console UI (named "security group").
NOTE: Database security groups might be different from AWS EC2 security groups. If your RDS instance is in classic/public EC2, you should check in the "database security group" section of the RDS UI. For VPC users, the security group will be a normal VPC security group (the name sg-xxx will be listed in the RDS instance's attributes).
2) Confirm DNS isn't an issue.
Amazon uses split DNS, so a DNS lookup external to AWS will return the public IP while a lookup internal to AWS will return a private IP. If you suspect it is a DNS issue, have you confirmed different IPs are returned from different availability zones? If different AZs get different IPs, you will need to contact AWS support.
3) Confirm network connectivity by establishing a socket connection.
Tools like tracepath and traceroute likely won't help since RDS currently drops ICMP traffic.
Test port connectivity by trying to establish a socket connection to the RDS instance on port 3306 (mysql, or 5432 for postgres). Start by finding the IP of the RDS instance and using either telnet or nc (be sure to use the internal/private IP if connecting from within AWS):
telnet x.x.x.x 3306
nc -vz x.x.x.x 3306
a) If your connection attempt isn't successful and immediately fails, the port is likely blocked or the remote host isn't running a service on that port. you may need to engage AWS support to troubleshoot further. If connecting from outside of AWS, try to connect from another instance inside AWS first (as your firewall might be blocking those connections).
b) If your connection isn't successful and you get a timeout, packets are probably being dropped/ignored by a firewall or packets are returning on a different network path. You can confirm this by running netstat -an | grep SYN (from a different ssh session while waiting for the telnet/nc command to timeout).
Connections in the SYN state mean that you've sent a connection request, but haven't received anything back (SYN_ACK or reject/block). Usually this means a firewall or security group is ignoring or dropping packets.
It can also be a problem with NAT routing or multiple paths from multiple interfaces. Check to make sure you're not using iptables or a NAT gateway between your host and the RDS instance. If you're in a VPC, also make sure you allow egress/outbound traffic from the source host.
c) If your socket connection test was successful, but you can't connect with a mysql client (CLI, workbench, app, etc.), take a look at the output of netstat to see what state the connection is in (replace x.x.x.x with the actual IP address of the RDS instance):
netstat -an | grep x.x.x.x
If you were getting a connection established when using telnet or NC, but you see the 'SYN' state when using a mysql client, you might be running into an MTU issue.
RDS, at the time this is written, may not support ICMP packets used for PMTUD (https://en.wikipedia.org/wiki/Path_MTU_Discovery#Problems_with_PMTUD). This can be a problem if you're trying to access RDS or RedShift that's in a VPC from a classic ec2 instance via ClassicLink. Try lowering the MTU with the following, then testing again:
sudo ip link show
# take note of the current MTU (likely 1500 or 9001)
sudo ip link set dev eth0 mtu 1400
If the lower MTU worked, be sure to follow up with AWS customer support for help and mention that you are seeing an MTU issue while trying to connect to your RDS instance. This can happen if TCP packets are wrapped with encapsulation for tunneling, resulting in a lower usable MTU for packet data / payload. Lowering the MTU on the source server allows the wrapped packets to still fit under the MTU limit while passing through the tunneling gateway.
If it didn't work, set your MTU back to it's default and engage AWS support for further troubleshooting.