I ma trying to install redhat openshift using CRC by using this doc https://computingforgeeks.com/setup-local-openshift-cluster-with-codeready-containers/. But while strating container it is giving below exception
[crc#openshift ~]$ crc start
INFO Checking if oc binary is cached
INFO Checking if podman remote binary is cached
INFO Checking if goodhosts binary is cached
INFO Checking minimum RAM requirements
INFO Checking if running as non-root
INFO Checking if Virtualization is enabled
INFO Checking if KVM is enabled
INFO Checking if libvirt is installed
INFO Checking if user is part of libvirt group
INFO Checking if libvirt daemon is running
INFO Checking if a supported libvirt version is installed
INFO Checking if crc-driver-libvirt is installed
INFO Checking if libvirt 'crc' network is available
INFO Checking if libvirt 'crc' network is active
INFO Checking if NetworkManager is installed
INFO Checking if NetworkManager service is running
INFO Checking if /etc/NetworkManager/conf.d/crc-nm-dnsmasq.conf exists
INFO Checking if /etc/NetworkManager/dnsmasq.d/crc.conf exists
INFO Starting CodeReady Containers VM for OpenShift 4.5.9...
INFO CodeReady Containers VM is running
INFO Starting network time synchronization in CodeReady Containers VM
INFO Verifying validity of the cluster certificates ...
INFO Adding 8.8.8.8 as nameserver to the instance ...
INFO Check internal and public DNS query ...
INFO Check DNS query from host ...
WARN foo.apps-crc.testing resolved to [127.0.0.1] but 192.168.130.11 was expected
ERRO Failed to query DNS from host: Invalid IP for foo.apps-crc.testing
Failed to query DNS from host: Invalid IP for foo.apps-crc.testing
Os which I am using is RHEL8. I am not getting why it is giving the above error. If I run host -R 3 foo.apps-crc.testing I am geting below response:
[crc#openshift ~]$ host -R 3 foo.apps-crc.testing
foo.apps-crc.testing has address 127.0.0.1
/etc/resolv.conf:
# Generated by NetworkManager
search 8.8.4.4
nameserver 127.0.0.1
.
[crc#openshift ~]$ ping foo.apps-crc.testing
PING foo.apps-crc.testing (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost (127.0.0.1): icmp_seq=1 ttl=64 time=0.036 ms
64 bytes from localhost (127.0.0.1): icmp_seq=2 ttl=64 time=0.050 ms
64 bytes from localhost (127.0.0.1): icmp_seq=3 ttl=64 time=0.066 ms
64 bytes from localhost (127.0.0.1): icmp_seq=4 ttl=64 time=0.062 ms
64 bytes from localhost (127.0.0.1): icmp_seq=5 ttl=64 time=0.066 ms
64 bytes from localhost (127.0.0.1): icmp_seq=6 ttl=64 time=0.068 ms
64 bytes from localhost (127.0.0.1): icmp_seq=7 ttl=64 time=0.051 ms
64 bytes from localhost (127.0.0.1): icmp_seq=8 ttl=64 time=0.054 ms
64 bytes from localhost (127.0.0.1): icmp_seq=9 ttl=64 time=0.053 ms
Related
I compiled qemu-system-x86_64 on aarch64 host, and was able to run a x86_64 guest with a command like
qemu-system-x86_64 -m 4096 -drive file=vmimage.qcow2,if=virtio \
-boot once=c,menu=on -net nic,model=virtio-net-pci \
-net user,hostfwd=tcp::8080-:80,hostfwd=tcp::22222-:22
I could ssh into the guest using
ssh -p22222 user#localhost
Meanwhile, port 80 was not forwarded successfully.
For debugging, I used nc to listen to port 80 inside the guest
nc -l 80
Then in the host, I connected to the forwarded port
nc localhost 8080
However, it was unable to connect to guest nc .
I tried the monitor interface. When the host nc command is executed, info usernet shows following:
(qemu) info usernet
Hub 0 (#net162):
Protocol[State] FD Source Address Port Dest. Address Port RecvQ SendQ
TCP[SYN_SENT] 33 127.0.0.1 8080 10.0.2.15 80 0 0
TCP[ESTABLISHED] 21 127.0.0.1 22222 10.0.2.15 22 0 0
TCP[HOST_FORWARD] 12 * 8080 10.0.2.15 80 0 0
TCP[HOST_FORWARD] 11 * 22222 10.0.2.15 22 0 0
...
I believe the SYN_SENT (FD 33) corresponded to the host nc command, and this matched the HOST_FORWARD line (FD 12). However, it never became ESTABLISHED. And a few seconds later, nc died with Connection reset by peer. , and the FD 33 line disappeared.
If I nc localhost 22222, I can see the OpenSSH banner.
So it seems only port 22 forwarded. Any idea about the cause or how to debug?
Both host and guest had no firewalliptables configured, and SELinux is permissive.
Thanks
Edit:
As a temporary workaround, I configured a second nic, and used port 22 of the new interface for forwarding my service. I also switch to the newer -nic option, but hostfwd still worked for port 22 only.
qemu-system-x86_64 -m 4096 -drive file=vmimage.qcow2,if=virtio \
-boot once=c,menu=on \
-nic user,model=virtio-net-pci,hostfwd=tcp::60022-:22 \
-nic user,model=virtio-net-pci,net=10.0.3.0/24,hostfwd=tcp::8080-10.0.3.15:22
To forward successfully, I also need to
Configure sshd to listen to port 22 the first nic only.
Configure my service to listen to port 22 of the second nic.
Configure the second nic to use a different network. Otherwise, both nics were assigned the same IP (10.0.2.15. I may better hardcode the IP for both nics.)
The problem was actually about firewall. My VM (based on Oracle Linux 8.5 on Oracle Linux VM Templates) actually had firewall rules in both iptables and nft. After disabling both iptables and nft, the port forward worked.
I am trying to connect to a MySQL RDS instance in AWS with Apache NiFi on Linux Ubuntu.
The MySQL database is behind a SSH tunnel which is working in a dedicated terminal. The MySQL database is accessible through another Linux terminal.
$ ssh -i /home/bernardo/.ssh/sshbernardo3.pem -N \
-L 33061:<db_name>.<id>.eu-west-1.rds.amazonaws.com:3306 \
<user>#<ip_address> -p <port>
I have uploaded and unzipped the JDBC driver from https://downloads.mysql.com/archives/c-j/
The file java.sql.Driver is located in the directory /usr/share/java/META-INF/services in Linux
I have installed the jdbc connector in Linux and set up the user to use JDBC
$ sudo apt-get install libmysql-java
$ cd /home/bernardo
$ gedit .bashrc
Added the following lines at the end of the file:
# Setting up the user to use JDBC
CLASSPATH=$CLASSPATH:/usr/share/java/mysql-connector-java-8.0.26.jar
export CLASSPATH
Then source it:
$ source .bashrc
In NiFi I have used a DBCPConnectionPool1.15.2 controller service which is configured as follows:
• Database Connection URL: jbdc:mysql://127.0.0.1:33061/<db_name>
• Database Driver Class Name: java.sql.Driver
• Database Driver Location: /usr/share/java/META-INF/services
• Database User: <db_user>
• Password: <db_password>
When enabling the controller service, I get the following error:
StandardControllerServiceNode[service=DBCPConnectionPool[id=2440f624-017e-1000-9467-8bb12dda6ca6],
name=<controller_service_name>, active=true] Failed to invoke
#OnEnabled method due to java.lang.NoSuchMethodException:
java.sql.Driver.() causes: java.lang.InstantiationException:
java.sql.Driver causes:
org.apache.nifi.processor.exception.ProcessException: Creating driver
instance is failed: Creating driver instance is failed
Any hint?
This is the SW I have installed:
Linux Ubuntu 18.04.6 in VirtualBox 6.1 on top of Windows 10 Pro
openjdk version "11.0.13" 2021-10-19
javac 11.0.13
nifi-1.15.2
Thanks,
Bernardo
EDITED on 2022 January 6 08:50 UTC
I have made the following changes to the DBCPConnectionPool1.15.2 controller service configuration:
• Database Driver Class Name: com.mysql.cj.jdbc.Driver
• Database Driver Location: /usr/share/java/mysql-connector-java-8.0.26.jar
The other parameters are not changed:
• Database Connection URL: jbdc:mysql://127.0.0.1:33061/<db_name>
• Database User: <db_user>
• Password: <db_password>
The file mysql-connector-java-8.0.26.jar is in /usr/share/java
Also:
$ cat /usr/share/java/META-INF/services/java.sql.Driver
com.mysql.cj.jdbc.Driver
And:
$ netstat -nplt
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 127.0.0.1:33061 0.0.0.0:* LISTEN 4873/ssh
tcp6 0 0 ::1:33061 :::* LISTEN 4873/ssh
Still I get an error (a new error this time):
causes: org.apache.nifi.processor.exception.ProcessException: No suitable driver for the given Database Connection URL
Permissions to access the driver jar file:
bernardo#BDi-laptop:/usr/share/java$ ls -l
-rwxrwxr-- 1 bernardo bernardo 2462344 gen 4 19:01 mysql-connector-java-8.0.26.jar
EDITED on 2022 January 6 10:21 UTC
Added the CLASSPATH variable also in /etc/environment
bernardo#BDi-laptop:/usr/share/java$ echo $CLASSPATH
.:/usr/share/java/mysql-connector-java-8.0.26.jar:/usr/share/java/mysql-connector-java-8.0.26.jar
EDITED on 2022 January 7 09:08 UTC
I thought it could be an issue related to the driver version so I re-installed the jdbc driver. I have cleaned /home/bernardo/Download and /usr/share/java from the old sql related files. Then I have done the installation again.
Install the jdbc connector
$ sudo apt-get install libmysql-java
The .jar file is in /usr/share/java/mysql-connector-java-5.1.45.jar
Unpack the .jar file
$ sudo jar xvf /usr/share/java/mysql-connector-java-5.1.45.jar
To find the driver class name, open /usr/share/java/META-INF/services/java.sql.Driver
$ cat /usr/share/java/META-INF/services/java.sql.Driver
com.mysql.jdbc.Driver
com.mysql.fabric.jdbc.FabricMySQLDriver
The driver class name is: com.mysql.jdbc.Driver
This is the updated configuration in the DBCPConnectionPool1.15.2 controller service in Apache Ni-Fi:
• Database Driver Class Name: com.mysql.jdbc.Driver
• Database Driver Location: /usr/share/java/mysql-connector-java-5.1.45.jar
• Database Connection URL: jbdc:mysql://127.0.0.1:33061/<db_name>
• Database User: <db_user>
• Password: <db_password>
I have obtained the same result:
causes: org.apache.nifi.processor.exception.ProcessException: No suitable driver for the given Database Connection URL
Then I checked if the problem was in the SSH tunnel.
$ netstat -nplt
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 127.0.0.1:33061 0.0.0.0:* LISTEN 4497/ssh
tcp 0 0 127.0.0.1:3306 0.0.0.0:* LISTEN -
tcp6 0 0 ::1:33061 :::* LISTEN 4497/ssh
$ cat /proc/4497/status
Name: ssh
State: S (sleeping)
The SSH process is in sleeping state.
In a separate terminal I accessed the remote MySQL database:
$ mysql -u <db_user> -p -h 127.0.0.1 -P 33061
Finally, I tried to connect to my local mysql database in my local Linux instance by using the following Ni-Fi settings:
• Database Driver Class Name: com.mysql.jdbc.Driver
• Database Driver Location: /usr/share/java/mysql-connector-java-5.1.45.jar
• Database Connection URL: jbdc:mysql://127.0.0.1:3306/<local_db_name>
• Database User: <local_db_user>
• Password: <local_db_password>
I have obtained the same result:
causes: org.apache.nifi.processor.exception.ProcessException: No suitable driver for the given Database Connection URL
Still, in a separate Linux terminal I am able to access the local mysql database:
$ mysql -u <local_db_user> -p -h 127.0.0.1 -P 3306
EDITED on 2022 January 10 11:58 UTC
Corrected a typo in Ni-Fi controller service configuration:
• Database Connection URL: jdbc:mysql://127.0.0.1:33061/<db_name>
Controller service problem solved!
Thanks!
EDITED on 2022 January 12 08:26 UTC
I am still getting connectivity problems. I managed to enable the controller service but, when I run the processor QueryDatabaseTable, I get the following error: Cannot create PoolableConnectionFactory - Communications link failure
QueryDatabaseTable[id=017e1003-c2d8-14cf-4e34-feee76411595] Unable to execute SQL select query SELECT * FROM periodic_measurements_test due to java.sql.SQLException: Cannot create PoolableConnectionFactory (Communications link failure
The last packet successfully received from the server was 0 milliseconds ago. The last packet sent successfully to the server was 0 milliseconds ago.): javax.net.ssl.SSLHandshakeException: No appropriate protocol (protocol is disabled or cipher suites are inappropriate)
↳ causes: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet successfully received from the server was 0 milliseconds ago. The last packet sent successfully to the server was 0 milliseconds ago.
↳ causes: java.sql.SQLException: Cannot create PoolableConnectionFactory (Communications link failure
The last packet successfully received from the server was 0 milliseconds ago. The last packet sent successfully to the server was 0 milliseconds ago.)
↳ causes: org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException: Cannot create PoolableConnectionFactory (Communications link failure
The last packet successfully received from the server was 0 milliseconds ago. The last packet sent successfully to the server was 0 milliseconds ago.)
The processor configuration is default except for:
• Database Connection Pooling Service: < DBCPConnectionPool controller service name >
• Database Type: MySQL
• Table Name: periodic_measurements_test
The controller service configuration is still:
• Database Connection URL: jdbc:mysql://127.0.0.1:33061/<db_name>
• Database Driver Class Name: com.mysql.jdbc.Driver
• Database Driver Location: /usr/share/java/mysql-connector-java-5.1.45.jar
• Database User: <db_user>
• Password: <db_password>
The SSH tunnel seems to be fine:
netstat -nplt
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 127.0.0.1:33061 0.0.0.0:* LISTEN 29483/ssh
tcp 0 0 127.0.0.1:3306 0.0.0.0:* LISTEN -
tcp 0 0 127.0.0.1:8443 0.0.0.0:* LISTEN -
And in Linux terminal I can still access the MySQL database thorough CLI:
$ mysql -u <db_user> -p -h 127.0.0.1 -P 33061
And I get the same result when I try to fetch data from the MySQL database in my local Linux instance.
Finally, I managed to fix the issue. It was related to the version of the jdbc driver.
I did the following:
In Linux browser go to https://dev.mysql.com/downloads/connector/j/
Select
• Ubuntu Linux
• 18.04
Download
In a CLI terminal go to the Downloads directory
$ cd /home/bernardo/Downloads
Unpack the deb file
$ sudo dpkg -i mysql-connector-java_8.0.26-1ubuntu18.04_all.deb
Check where is the connector location in the file system
$ dpkg -L mysql-connector-java | fgrep jar
/usr/share/java/mysql-connector-java-8.0.26.jar
Clean /usr/share/java from the previous jdbc driver
$ sudo rm mysql-connector-java-5.1.45.jar
$ sudo rm -r com
$ sudo rm -r META-INF/
Install the connector: unpack the .jar file
$ sudo jar xvf /usr/share/java/mysql-connector-java-8.0.26.jar
To find the driver class name, open /usr/share/java/META-INF/services/java.sql.Driver
$ cat /usr/share/java/META-INF/services/java.sql.Driver
com.mysql.cj.jdbc.Driver
The driver class name is: com.mysql.jdbc.Driver
Restart Apache Ni-Fi
Modify the configuration of the controller service with:
• Database Driver Class Name: com.mysql.cj.jdbc.Driver
• Database Driver Location: /usr/share/java/mysql-connector-java-8.0.26.jar
I cannot build pods in OKD as Github cannot be resolved and the build fails.
The error:
Cloning "https://github.com/MeMyselfI/okd-hello-world " ...
error: fatal: unable to access 'https://github.com/MeMyselfI/okd-hello-world/': Could not resolve host: github.com; Unknown error
How OKD is started:
sudo oc cluster up --routing-suffix=192.168.2.20.xip.io --public-hostname=192.168.2.20
OC client version:
me#server:~$ sudo oc version
oc v3.11.0+0cbc58b
kubernetes v1.11.0+d4cacc0
features: Basic-Auth GSSAPI Kerberos SPNEGO
ping from SSH prompt:
me#server:~$ ping github.com
PING github.com (192.30.253.113) 56(84) bytes of data.
64 bytes from lb-192-30-253-113-iad.github.com (192.30.253.113): icmp_seq=1 ttl=52 time=33.4 ms
64 bytes from lb-192-30-253-113-iad.github.com (192.30.253.113): icmp_seq=2 ttl=52 time=34.0 ms
ping from local, manually launched, Docker container:
me#server:~$ sudo docker exec -it hwnodemongo_web_1 /bin/bash
root#7c2db2faacf7:/usr/src/app# ping github.com
PING github.com (192.30.253.112) 56(84) bytes of data.
64 bytes from lb-192-30-253-112-iad.github.com (192.30.253.112): icmp_seq=1 ttl=52 time=28.3 ms
64 bytes from lb-192-30-253-112-iad.github.com (192.30.253.112): icmp_seq=2 ttl=52 time=27.4 ms
Output of resolv.conf
me#server:~$ cat /etc/resolv.conf
nameserver 8.8.8.8
dnsmasq is running:
dnsmasq 1479 0.0 0.0 51580 376 ? S 12:07 0:00 /usr/sbin/dnsmasq -x /run/dnsmasq/dnsmasq.pid -u dnsmasq -7 /etc/dnsmasq.d,.dpkg-dist,.dpkg-old,.dpkg-new --local-service --trust-anchor=.,19036,8,2,49aac11d7b6f6446702e54a1607371607a1a41855200fd2ce1cdde32f24e8fb5 --trust-anchor=.,20326,8,2,e06d44b80b8f1d39a95c0b0d7c65d08458e880409bbc683457104237c7f8ec8d
dig from SSH:
me#server.com:~$ dig github.com
; <<>> DiG 9.11.3-1ubuntu1.5-Ubuntu <<>> github.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 20999
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;github.com. IN A
;; ANSWER SECTION:
github.com. 59 IN A 192.30.253.113
github.com. 59 IN A 192.30.253.112
;; Query time: 46 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Fri Apr 05 13:23:47 ADT 2019
;; MSG SIZE rcvd: 71
Update:
I redid the install on another machine. I used oc cluster up (without any other parameters) and got the same results (by accessing localhost:8443/console).
I am trying to setup zabbix proxy. My network is as below
Zabbix server IP: 192.168.101.11 (internal network)
Zabbix proxy server: 192.168.102.109 (internal network)
Zabbix agent: 172.1.16.2 (outside network but pingable from 102.109)
I can ping the zabbix agent IP from my proxy machine.
[root#102_109 ~]# ping 172.1.16.2
PING 172.1.16.2 (172.1.16.2) 56(84) bytes of data.
64 bytes from 172.1.16.2: icmp_seq=1 ttl=64 time=215 ms
64 bytes from 172.1.16.2: icmp_seq=2 ttl=64 time=214 ms
64 bytes from 172.1.16.2: icmp_seq=3 ttl=64 time=214 ms
64 bytes from 172.1.16.2: icmp_seq=4 ttl=64 time=214 ms
I can connect to the zabbix proxy from my zabbix server -
zabbix_get -k agent.ping -s 192.168.102.109
1
My zabbix_proxy.conf file (on 102.109) is as below
ProxyMode=0
Server=192.168.101.11
Hostname=CME_Proxy
LogFile=/tmp/zabbix_proxy.log
DBName=zabbix
DBUser=root
DBPassword=password
And on the zabbix agent machine (172.1.16.2) the configuration is as below.
EnableRemoteCommands=1
LogFile=/tmp/zabbix_agentd.log
Server=192.168.101.11,192.168.102.109
ServerActive=192.168.101.11,192.168.102.109
Hostname=172.1.16.2
AllowRoot=1
On my zabbix front end, I have configured the host as monitored by proxy (CME_Proxy) and there is only 1 item (agent.ping).
I am not able to get any data from the zabbix agent. From my proxy machine, when I run the following command, it returns a blank value.
zabbix_get -k agent.ping -s 172.1.16.2
<this is blank response>
Due to this, in the host configuration, zabbix shows error -
"Received empty response from Zabbix Agent at [172.1.16.2]. Assuming
that agent dropped connection because of access permissions."
Can someone please guide me if the way I have configured is correct? If not how to do this correctly. If you need additional data please let me know.
Thank you
Mukul
Figured it out:
In the agent config file, the following parameters
Server=192.168.101.11,192.168.102.109
ServerActive=192.168.101.11,192.168.102.109
should have been
Server=192.168.101.11,172.1.16.1
ServerActive=192.168.101.11,172.1.16.1
> server 172.1.16.2, some changes in zabbix_agentd.conf
you need specified who will have permission to request the data to agent (passive checks).
Server=192.168.102.109 # it will allow connections from proxy ip
# ServerActive=192.168.102.109 # comment ServerActive if you won't use active checks
at the web interface (set monitored by: CME_Proxy, or the same you defined in Hostname at zabbix_proxy.conf on 192.168.102.109)
> check communication: as you did before!
at the proxy terminal (192.168.102.109):
enter code here
zabbix_get -k agent.ping -s 172.1.16.2 # It should return 1.
PS: check Hostname in the zabbix_proxy.conf, it should be CME_Proxy, or the same you defined at web interface.
# FOR step-by-step guide of running latest zabbix version 5.0 follow these links
# https://blog.zabbix.com/zabbix-docker-containers/7150/
# https://techexpert.tips/zabbix/monitoring-docker-using-zabbix/
#it's simple just add all zabbix server IPs in zabbix host agent conf #file like below
Server=192.168.101.11,172.1.16.1
ServerActive=192.168.101.11,172.1.16.1
if you ur using zabbix server-agent model using docker containers then while deploying containers specify zabbix server Host/Container IPs which wants to connect to zabbix agent container
Assuming if you wanna deploy zabbix server and agent in the same server running docker containers just run below docker deploy commands
#Zabbix Server Container
sudo docker run --name zabbix-appliance -p 8080:80 -p 10051:10051 -d -h zabbix-server zabbix/zabbix-appliance
#Zabbix Agent container
sudo docker run --name=dockbix-agent-xxl --privileged -v /:/rootfs -v /var/run:/var/run -p 10050:10050 -e "ZA_Server=192.168.0.3,172.17.0.1" -e "ZA_ServerActive=192.168.0.3,172.17.0.1" -d monitoringartist/dockbix-agent-xxl-limited:latest
#Default username and password of zabbix server
#username: Admin password: zabbix
# For monitoring docker containers resources import a template from this cloned repository https://github.com/monitoringartist/zabbix-docker-monitoring
I'm getting the following error on my chrome console:
GET http://localhost/grunt/vendor/angular/angular.js net::ERR_CONTENT_LENGTH_MISMATCH
This only happens when a simultaneous requests are shot towards nginx e.g. when the browsers cache is empty and the whole app loads. Loading the resource above as a single requests succeeds.
Here are the headers to this requests, copied from Chrome:
Remote Address:127.0.0.1:80
Request URL:http://localhost/grunt/vendor/angular/angular.js
Request Method:GET
Status Code:200 OK
Request Headersview source
Accept:*/*
Accept-Encoding:gzip,deflate,sdch
Accept-Language:en-US,en;q=0.8,de;q=0.6,pl;q=0.4,es;q=0.2,he;q=0.2,gl;q=0.2
Cache-Control:no-cache
Connection:keep-alive
Cookie:gs_u_GSN-265185-D=1783247335:2567:5000:1377697930719
Host:localhost
Pragma:no-cache
Referer:http://localhost/grunt/
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.122 Safari/537.36
Response Headersview source
Accept-Ranges:bytes
Cache-Control:public, max-age=0
Connection:keep-alive
Content-Length:873444
Content-Type:application/javascript
Date:Tue, 23 Sep 2014 11:08:19 GMT
ETag:"873444-1411465226000"
Last-Modified:Tue, 23 Sep 2014 09:40:26 GMT
Server:nginx/1.6.0
the real size of the file:
$ ll vendor/angular/angular.js
-rw-rw-r-- 1 xxxx staff 873444 Aug 30 07:21 vendor/angular/angular.js
As you can see Content-Length and the real size of the file are the same, so that's weird
And the nginx configuration to this proxy:
location /grunt/ {
proxy_pass http://localhost:9000/;
}
Any ideas?
Thanks
EDIT: found more info on the error log:
2014/09/23 13:08:19 [crit] 15435#0: *8 open() "/usr/local/var/run/nginx/proxy_temp/1/00/0000000001" failed (13: Permission denied) while reading upstream, client: 127.0.0.1, server: localhost, request: "GET /grunt/vendor/angular/angular.js HTTP/1.1", upstream: "http://127.0.0.1:9000/vendor/angular/angular.js", host: "localhost", referrer: "http://localhost/grunt/"
Adding the following line to the nginx config was the only thing that fixed the net::ERR_CONTENT_LENGTH_MISMATCH error for me:
proxy_buffering off;
It seems that under pressure, nginx tried to pull angular.js from its cache and couldn't due to permission issues. Here's what solved this issue:
root#amac-2:/usr/local/var/run/nginx $ chown -R _www:admin proxy_temp
_www:admin might be different in your case, depending which user owns the nginx process. See more information on ServerFault:
https://serverfault.com/questions/534497/why-do-nginx-process-run-with-user-nobody
I tried all of the above and still couldn't get it to work. Even after resorting to chmod 777. The only thing that solved it for me was to disable caching entirely:
proxy_max_temp_file_size 0;
Whilst not a fix and no good for production use this was OK for me since I'm only using nginx as part of a local development setup.
For me the remedy were these two settings:
In the file:
/etc/nginx/nginx.conf
Add:
proxy_max_temp_file_size 0;
proxy_buffering off;
Between the lines client_max_body_size 128M; and server_names_hash_bucket_size 256;:
http {
client_max_body_size 128M;
proxy_max_temp_file_size 0;
proxy_buffering off;
server_names_hash_bucket_size 256;
ps aux | grep "nginx: worker process"
after executing above command you'll see the user through which nginx is running
eg.
www-data 25356 0.0 0.0 68576 4800 ? S 12:45 0:00 nginx: worker process
www-data 25357 0.0 0.0 68912 5060 ? S 12:45 0:00 nginx: worker process
now you have to run below command to give permission
chown -R www-data:www-data /var/lib/nginx/
Hope it will work
For us, it turned out to be that our server's rather small root (ie. /) was full.
It had mountains of logs and files from users in /home. Moving all that cruft out to another mounted drive solved things.
Just wanted to share as this can be another cause of the problem.
If somebody ran nginx as a different user in the past, ownership of cache folder may be twisted. I got
/var/cache/nginx# LANG=C ls -l proxy_temp/
total 40
drwx------ 18 nginx nginx 4096 Jul 14 2016 0
drwx------ 19 nginx nginx 4096 Jul 14 2016 1
drwx------ 19 nginx nginx 4096 Jul 14 2016 2
drwx------ 19 nginx nginx 4096 Jul 14 2016 3
drwx------ 19 nginx nginx 4096 Jul 14 2016 4
drwx------ 19 nginx nginx 4096 Jul 14 2016 5
drwx------ 19 nginx nginx 4096 Jul 14 2016 6
drwx------ 18 nginx nginx 4096 Jul 14 2016 7
drwx------ 18 nginx nginx 4096 Jul 14 2016 8
drwx------ 18 nginx nginx 4096 Jul 14 2016 9
while nginx was running as www-data. So the solution is to change ownership of nginx’s cache directory to the user nginx is running under. In the present case
/var/cache/nginx# chown -R www-data:www-data *
or, even simpler
# rm -r /var/cache/nginx/*
What worked for me was to change the proxy_temp_path to a folder with read/write permissions (777)
location / {
proxy_temp_path /data/tmp;
}
I had same issue.
Increasing the space of Directory or Folder where nginx is installed, solved the issue.
For macOS with nginx installed with homebrew, I used the following steps to track down and fix the issue.
Run nginx -h to find your error log location. Look for the following line:
-e filename : set error log file (default: /opt/homebrew/var/log/nginx/error.log)
Take your error log path and tail it to see what error it's reporting when you try to load the page.
tail -f /opt/homebrew/var/log/nginx/error.log
From that I saw that one of the lines showed a permission denied error:
open() "/opt/homebrew/var/run/nginx/proxy_temp/9/01/0000000019" failed (13: Permission denied) while reading upstream
Which means that your cached directories have incorrect permissions for the nginx user.
Stop nginx
brew services stop nginx
Delete all the temp folders (location from the permission error log line)
sudo rm -rf /opt/homebrew/var/run/nginx/*
Start nginx again
brew services start nginx
After doing this, nginx will recreate the temp folders with the correct permissions. At this point you should be good try and reload your page that was failing before.
When I tried the aforementioned solution it didn't fix the issue. I also changed the permission to write on the location but it didn't work. Then I realized I did something wrong in there. In the location to store the file, I had something like
"/storage" + fileName + ".csv"
. I was testing on the Windows environment and it was working great. But later when we moved the application to the Linux environment it stopped working. So later I had to change it to
"./storage" + fileName + ".csv"
and it started working normally.
For me, the solution was:
sudo chown -R nginx:nginx /var/cache/nginx/fastcgi_temp/
For anyone using HAProxy as proxy and getting these exact same symptoms, increasing the timeout values resolved the issue for me:
timeout connect 5000
timeout client 50000
timeout server 50000
The only thing that helped me was the following settings in nginx site .conf file:
proxy_read_timeout 720s;
proxy_connect_timeout 720s;
proxy_send_timeout 720s;
For me I had the same error except on a
different folder /var/lib/nginx/.
I changed the owner to nginx by
chown -R nginx:nginx /var/lib/nginx/. That did not work.
Then I checked who owned the nginx worker process by
ps aux| grep nginx
And it was running as nginx but when I looked through the nginx.conf file; I found that the user was nginx but it did not have any group. So, I added nginx to the user nginx; it turned out like this
user nginx nginx
Now I rebooted the system and the issue was fixed. I suppose I could have just used
chown -R nginx /var/lib/nginx/
That may have worked as well. So if anyone is facing this issue; firstly go into var/log/nginx and
check where the permission error occurred.