varnish error sess_workspace - configuration

Metadata: My system is debian 7 64bit
Hi, I am new to varnish and I have encountered an error that i cant seem to solve.
I have the following in my /etc/default/varnish configuration file:
DAEMON_OPTS="-a :80 \
-T localhost:6082 \
-f /etc/varnish/default.vcl \
-u www-data -g www-data \
-S /etc/varnish/secret \
-p thread_pools=2 \
-p thread_pool_min=25 \
-p thread_pool_max=250 \
-p thread_pool_add_delay=2 \
-p timeout_linger=50 \
-p sess_workspace=262144 \
-p cli_timeout=40 \
-s malloc,512m"
When I restarted he varnish service it failed with the error Unknown parameter "sess_workspace".
I checked the /var/log/varnish/ and no logs where generated.
I also checked syslog and the only logs that varnish wrote were logs that had to do with the startup and shutdown of varnish after he install and when I ran service varnish restart. Here ara all the relevant syslog entries (cat syslog | grep varnish):
Nov 6 00:56:27 HOSTNAME varnishd[7557]: Platform: Linux,3.2.0-4-amd64,x86_64,-smalloc,-smalloc,-hcritbit
Nov 6 00:56:27 HOSTNAME varnishd[7557]: child (7618) Started
Nov 6 00:56:27 HOSTNAME varnishd[7557]: Child (7618) said Child starts
Nov 6 01:04:58 HOSTNAME varnishd[7557]: Manager got SIGINT
Nov 6 01:04:58 HOSTNAME varnishd[7557]: Stopping Child
Nov 6 01:04:59 HOSTNAME varnishd[7557]: Child (7618) ended
Nov 6 01:04:59 HOSTNAME varnishd[7557]: Child (7618) said Child dies
Nov 6 01:04:59 HOSTNAME varnishd[7557]: Child cleanup complete
I have searched the vast seas of google but with no luck (I have even compared with example code from varnish site and still no luck).
Any ideas that may help me?

Bit delayed but in case anyone else has this issue I will post what I found.
Found this here: http://www.0550go.com/proxy/varnish/varnish-v4-config.html
sess_workspace
In 3.0 it was often necessary to increase
sess_workspace if a lot of VMODs, complex header operations or ESI
were in use.
This is no longer necessary, because ESI scratch space
happens elsewhere in 4.0.
If you are using a lot of VMODs, you may
need to increase either workspace_backend and workspace_client based
on where your VMOD is doing its work.

Related

systemd podman This usually indicates unclean termination of a previous run, or service implementation deficiencies

I am running container with systemd/pod, when I want to deploy new image tag. stopping service, updating the service file and starting. but container failed to start.
systemd file.
[Unit]
Description=hello_api Podman Container
After=network.target
[Service]
Restart=on-failure
RestartSec=3
ExecStartPre=/usr/bin/rm -f /%t/%n-pid /%t/%n-cid
ExecStartPre=-/usr/bin/podman rm hello_api
ExecStart=/usr/bin/podman run --conmon-pidfile /%t/%n-pid --cidfile /%t/%n-cid -d -h modelenv \
--name hello_api --rm --ulimit=host -p "8001:8001" -p "8443:8443" 7963-hello_api:7.8
ExecStop=/usr/bin/sh -c "/usr/bin/podman rm -f `cat /%t/%n-cid`"
KillMode=none
Type=forking
PIDFile=/%t/%n-pid
[Install]
WantedBy=default.target
here is error message.
May 21 10:41:43 webserver systemd[1471]: hello_api.service: Found left-over process 22912 (conmon) in control group while starting unit. Ignoring.
May 21 10:41:43 webserver systemd[1471]: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
May 21 10:41:43 webserver systemd[1471]: hello_api.service: Found left-over process 22922 (node) in control group while starting unit. Ignoring.
May 21 10:41:43 webserver systemd[1471]: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
May 21 10:41:43 webserver systemd[1471]: hello_api.service: Found left-over process 22960 (node) in control group while starting unit. Ignoring.
May 21 10:41:43 webserver systemd[1471]: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
May 21 10:41:44 webserver podman[24565]: 2020-05-21 10:41:44.586396547 -0400 EDT m=+1.090025069 container create 28eaf881f532339766cc96ec27a69d8ad588e07d4bfc70e65e7c54e8a5082933 (image=7963-hello_api:7.8, name=hello_api)
May 21 10:41:45 webserver podman[24565]: Error: error from slirp4netns while setting up port redirection: map[desc:bad request: add_hostfwd: slirp_add_hostfwd failed]
May 21 10:41:45 webserver systemd[1471]: hello_api.service: Control process exited, code=exited status=126
May 21 10:41:45 webserver systemd[1471]: hello_api.service: Failed with result 'exit-code'.
May 21 10:41:45 webserver systemd[1471]: Failed to start call_center_hello_api Podman Container.
why its giving this error, is there option to cleanly exit the old container?
I think we followed the same tutorial here: https://www.redhat.com/sysadmin/podman-shareable-systemd-services
"It’s important to set the kill mode to none. Otherwise, systemd will start competing with Podman to stop and kill the container processes. which can lead to various undesired side effects and invalid states"
I'm not sure if the behavior changed, but I removed the KillMode=none causing it to use the default KillMode=control-group. I have not had any problems managing the service since. Also, I removed the / from some of the commands because it was being duplicated:
ExecStartPre=/usr/bin/rm -f //run/user/1000/registry.service-pid //run/user/1000/registry.service-cid
It's now:
ExecStartPre=/usr/bin/rm -f /run/user/1000/registry.service-pid /run/user/1000/registry.service-cid
The full service file I use for running a docker registry:
[Unit]
Description=Image Registry
[Service]
Restart=on-failure
ExecStartPre=-/usr/bin/podman volume create registry
ExecStartPre=/usr/bin/rm -f /%t/%n-pid /%t/%n-cid
ExecStart=/usr/bin/podman run --conmon-pidfile %t/%n-pid --cidfile %t/%n-cid -d -p 5000:5000 -v registry:/var/lib/registry --name registry docker.io/library/registry
ExecStop=/usr/bin/sh -c "/usr/bin/podman rm -f `cat %t/%n-cid`"
Type=forking
PIDFile=/%t/%n-pid
[Install]
WantedBy=multi-user.target

gunicorn daemon (active: failed) / curl(56) Recv Failure: Connection reset by peer

First thing, I am not sure if this is better here or on ask ubuntu (ubuntu did not have a 'gunicorn' tag so I think i'm in the right place). If it is not appropriate here just drop it in the comments and I'll close it.
I am following a digitalocean tutorial on deployment(https://www.digitalocean.com/community/tutorials/how-to-set-up-django-with-postgres-nginx-and-gunicorn-on-ubuntu-18-04).I am up to gunicorn set up. I'm at my wits end trying to get this gunicorn to work, so I have come here. Anything in quotes is the name of the current section in the article. I got up to "Checking for the Gunicorn Socket File" and "check for the existence of the gunicorn.sock file within the /run directory:" before failure.
Check for socket file:
sudo systemctl status gunicorn.socket returns
Failed to dump process list, ignoring: No such file or directory
● gunicorn.socket - gunicorn socket
Loaded: loaded (/etc/systemd/system/gunicorn.socket; enabled; vendor pres
Active: active (listening) since Fri 2020-02-21 21:34:06 UTC; 1min 8s ago
Listen: /run/gunicorn.sock (Stream)
CGroup: /system.slice/gunicorn.socket
Check for existence of gunicorn.sock:
file /run/gunicorn.sock
output: /run/gunicorn.sock: socket
Upon "Testing socket activation", it fails:
sudo systemctl status gunicorn
output:
● gunicorn.service - gunicorn daemon
Loaded: loaded (/etc/systemd/system/gunicorn.service;
Active: failed (Result: exit-code) since Fri 2020-02-
Main PID: 15708 (code=exited, status=217/USER)
Feb 21 21:32:39 ubuntu-s-1vcpu-1gb-nyc3-01 systemd[1]: S
Feb 21 21:32:39 ubuntu-s-1vcpu-1gb-nyc3-01 systemd[15708
Feb 21 21:32:39 ubuntu-s-1vcpu-1gb-nyc3-01 systemd[15708
Feb 21 21:32:39 ubuntu-s-1vcpu-1gb-nyc3-01 systemd[1]: g
Feb 21 21:32:39 ubuntu-s-1vcpu-1gb-nyc3-01 systemd[1]: g
Feb 21 21:32:39 ubuntu-s-1vcpu-1gb-nyc3-01 systemd[1]: g
Feb 21 21:32:39 ubuntu-s-1vcpu-1gb-nyc3-01 systemd[1]: g
Feb 21 21:32:39 ubuntu-s-1vcpu-1gb-nyc3-01 systemd[1]: F
lines 1-13/13 (END)
It says to test socket activation, do the following:
curl --unix-socket /run/gunicorn.sock localhost
output(says I should see HTML):
curl: (56) Recv failure: Connection reset by peer
Not sure if I provided enough info. Below I will include my gunicorn.socket and gunicorn.service files as well as the layout of directories on my server.
gunicorn.socket:
[Unit]
Description=gunicorn socket
[Socket]
ListenStream=/run/gunicorn.sock
[Install]
WantedBy=sockets.target
gunicorn.service:
[Unit]
Description=gunicorn daemon
Requires=gunicorn.socket
After=network.target
[Service]
User=justin
Group=www-data
WorkingDirectory=/home/justin/project
ExecStart=/home/justin/project/env/bin/gunicorn \
--access-logfile - \
--workers 3 \
--bind unix:/run/gunicorn.sock \
jobzumo.wsgi:application
[Install]
WantedBy=multi-user.target
Layout of server/project:
home/justin/project/
This project/ folder contains env(env/bin/gunicorn does exist), jobzumo(django project), manage.py and static.
The only thing I am thinking is that I may have created these gunicorn files while using root and now I am trying to modify them using the user justin? Not really sure what is going on here. If I did not provide enough info or if you need me to run any type of debug commands please let me know. Thanks for any help.
I had the exact same propblem following this tutorial. OP's answer did not help in my case but I found a solution here. Maybe it helps others stubmling over this.
Many thanks to RussellMolimock for the following comment, which I found there!
"Go back into your virtualenv with source
[your_project_env]/bin/activate and enter which gunicorn That will
return the path to your gunicorn exectuable.
Paste that into the path section of the ‘ExecStart’ value inside the
’/etc/systemd/system/gunicorn.service’ file, and run the ‘sudo
systemctl daemon-reload’ and 'sudo systemctl restart gunicorn’
commands to restart your daemon and try curling again with curl
–unix-socket /run/gunicorn.sock localhost
I hope this helps!"
I had to run the following two commands:
sudo ufw delete allow 8000
sudo ufw allow 'Nginx Full'
and now everything is working. Apparently this opens my firewall up to port 80. Not sure why as I don't specify port 80 there, but it is working.
I faced this error because Gunicorn was not able to read the environment variables. This helped me in defining the environment variables for Gunicorn.
I deleted the whole project folder in Ubuntu (home/user/project) and restarted from the beginning, and it worked. I have tried multiple solutions on the Internet, restarting the daemon and changing the path of gunicorn, all fail.

Startup script doesn't seem to work

I've recently started using Google's Compute engine for some of my projects the problem is my startup script doesn't seem to work, For some reason my script just doesn't work, the VM has the startup-script metadata and it works fine when I run it manually with:
sudo google_metadata_script_runner --script-type startup
Here is what I am trying to run on startup:
#!/bin/bash
sudo apt-get update
sudo rm -f Eve.jar
sudo rm -f GameServerStatus.jar
wget <URL>/Eve.jar
wget <URL>/GameServerStatus.jar
sudo chmod 7777 Eve.jar
sudo chmod 7777 GameServerStatus.jar
screen -dmS Eve sh Eve.sh
screen -dmS PWISS sh GameServerStatus.sh
There are no errors in the log either, it just seems to stop at the chmod or screen commands, Any ideas?
Thanks!
To add to kangbu's answer:
Checking the logs in container-optimized OS by
sudo journalctl -u google-startup-scripts.service
showed that the script could not find the user. After a long time of debugging I finally added a delay before the sudo and now it works. Seems the user is not registered when the script runs.
#! /bin/bash
sleep 10 # wait...
cut -d: -f1 /etc/passwd > /home/user/users.txt # make sure the user exists
cd /home/user/project # cd does not work after sudo, do it before
sudo -u user bash -c '\
source /home/user/.bashrc && \
<your-task> && \
date > /home/user/startup.log'
I have the same problem #Brina mentioned. I set up metadata key startup-script and value like:
touch a
ls -al > test.txt
When I ran the script above sudo google_metadata_script_runner --script-type startup, it worked perfectly, However if I reset my VM instance the startup script didn't work. So, I checked startup script logs
...
Jul 3 04:30:37 kbot-6 ntpd[1514]: Listen normally on 5 eth0 fe80::4001:aff:fe8c:7 UDP 123
Jul 3 04:30:37 kbot-6 ntpd[1514]: peers refreshed
Jul 3 04:30:37 kbot-6 ntpd[1514]: Listening on routing socket on fd #22 for interface updates
Jul 3 04:30:38 kbot-6 startup-script: INFO Starting startup scripts.
Jul 3 04:30:38 kbot-6 startup-script: INFO Found startup-script in metadata.
Jul 3 04:30:38 kbot-6 startup-script: INFO startup-script: Return code 0.
Jul 3 04:30:38 kbot-6 startup-script: INFO Finished running startup scripts.
Yes. they found startup-script and ran it. I guessed it had executed as an another user. I changed my script like this:
pwd > /tmp/pwd.txt
whoami > /tmp/whoami.txt
The result is:
myuserid#kbot-6:/tmp$ cat pwd.txt whoami.txt
/
root
Yes. It was executed at the / diectory as root user. Finally, I changed my script to sudo -u myuserid bash -c ... which run it by specified userid.
Go to the VM instances page.
Click on the instance for which you want to add a startup script.
Click the Edit button at the top of the page.
Under Custom metadata, click Add item.
Add your startup script using one of the following keys:
startup-script: Supply the startup script contents directly with this key.
startup-script-URL: Supply a Google Cloud Storage URL to the start script file with this key.
It is working. The documentation for the new instance and existing instance as shown in GCE Start Up Script
Startup script output is written to the following log files:
CentOS and RHEL: /var/log/messages
Debian: /var/log/daemon.log
Ubuntu 14.04, 16.04, and 16.10: /var/log/syslog
On Ubuntu 12.04, SLES 11 and 12, and all images older than v20160606:
sudo /usr/share/google/run-startup-scripts
think that you do not need sudo, as well as the chmod 7777 should be 777
also a cd (or at least a pwd) at the beginning might be useful.
... log to text file, in order to know where the script may fail.

Orion Context Broker works half of the times

I've installed Orion Context Broker 0.23.0 and it behaves rare: it only works half of the times. For instance, when trying to retrieve the version I get this error message:
$ curl "http://localhost:1026/version"
curl: (52) Empty reply from server
$ curl "http://localhost:1026/version"
<orion>
<version>0.23.0</version>
<uptime>15 d, 22 h, 13 m, 18 s</uptime>
<git_hash>f5d76a6f11736d52402e63a4aa0ba990bff7f5eb</git_hash>
<compile_time>Fri Jul 10 13:21:42 CEST 2015</compile_time>
<compiled_by>fermin</compiled_by>
<compiled_in>centollo</compiled_in>
</orion>
$ curl "http://localhost:1026/version"
curl: (52) Empty reply from server
$ curl "http://localhost:1026/version"
<orion>
<version>0.23.0</version>
<uptime>15 d, 22 h, 13 m, 53 s</uptime>
<git_hash>f5d76a6f11736d52402e63a4aa0ba990bff7f5eb</git_hash>
<compile_time>Fri Jul 10 13:21:42 CEST 2015</compile_time>
<compiled_by>fermin</compiled_by>
<compiled_in>centollo</compiled_in>
</orion>
This behaviour is deterministic, I mean, after failing it always works, and after working it always fails. This occurs with all the operations within the REST API.
I've checked the listening ports and the process running them matches the Orion's one:
$ sudo netstat -ntlp | grep 1026
tcp 0 0 0.0.0.0:1026 0.0.0.0:* LISTEN 9944/contextBroker
tcp 0 0 :::1026 :::* LISTEN 9944/contextBroker
$ ps ax | grep contextBroker | grep -v grep
9944 ? Ssl 0:13 /usr/bin/contextBroker -port 1026 -logDir /var/log/contextBroker -pidpath /var/run/contextBroker/contextBroker.pid -dbhost localhost -db orion -multiservice
Any hints? Thanks!
Orion runs by default listening to IPv4 and IPv6. We have found that in cases similar to the ones you described disabling IPv6 solves the problem (we don't know yet the exact cause, maybe is related with the operating system or it is involved is some way...).
Tu run Orion in only-IPv4 mode you have to use the -ipv4 option either in the contextBroker command line or (if you are running Orion as a service) editing the /etc/sysconfig/contextBroker file to add -ipv4 to the BROKER_EXTRA_OPS variable (have a look to the documentation for more information about configuring Orion as a service). After modifying /etc/sysconfig/contextBroker you have to restart Orion using:
sudo /etc/init.d/contextBroker restart

"tcpdump -w 1.pcap" works, but "tcpdump -C 100 -w 1.pcap" - permission denied

I need to limit file size when I run "tcpdump -w 1.pcap". I try to do this with the key "-C", but when I add it I get error "permission denied". So:
> sudo tcpdump -w 1.pcap
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
^C821 packets captured
847 packets received by filter
24 packets dropped by kernel
But:
> sudo tcpdump -C 100 -w 1.pcap
tcpdump: 1.pcap: Permission denied
I run the command from my home directory and I tried to remove and create the file before running the command with different permissions, finally I have:
-rwxrwxrwx 1 root root 0 Aug 5 10:30 1.pcap
or
-rwxrwxrwx 1 fd8 users 0 Aug 5 10:30 1.pcap
Could you suggest why in the second case I can't write to the file?
You need to do -Z root. Read the man page:
-Z Drops privileges (if root) and changes user ID to user and the group ID to the primary group of user.
This behavior is enabled by default (-Z tcpdump), and can be disabled by -Z root.
I experienced similar issues on Ubuntu 12.04 LTS and my case was fixed as below procedures.
sudo apt-get install apparmor-utils
The aa-complain command which referred by user2704275 is included in this package.
If your environment is RedHat/CentOS distro, you can same command by yum.
sudo aa-complain /usr/sbin/tcpdump
This will change AppArmor mode of tcpdump from "enforce" to "complain".
You can check AppArmor status in /sys/kernel/security/apparmor/profiles.
Then I can success to get tcpdump with sudo.
After getting tcpdump, for security reason, you might revert apparmor status to previous mode as below command.
sudo aa-enforce /usr/sbin/tcpdump
Regards.
I experienced similar problems when I tried to read from file, like
tcpdump -r example.cap 'icmp[icmptype] = icmp-echo'
For me AppArmor caused the problem I had to switch from 'enforcement' mode to 'complain' mode on 'tcpdump'. Run the following command as root:
aa-complain /usr/sbin/tcpdump