So I came across a strange issue: as soon as I hit send Web Push notification in One Signal dashboard, the Mysql server dies - memory usage goes to 100% and the cpu goes nuts. Soon after it crashes and Mysql resumes normal operation.
I must mention that this happens even if images used in the notification are not hosted on the same server. Even if nobody actually clicks the notifications for the first 5minutes, it still crashes.
Our list has about 11.000 subscribers.
What could be the issue? I just dont know what to try anymore. Tried upping the max_connections and other my.cnf settings according to mysqltuner.pl. No luck.
This is happening on a Magento 1.9 store, with the following specs: 24gb RAM, 240gb SSD, 12-core 2ghz, CentOS7, running Apache with REDIS, php5.5.
UPDATE: Fixed by revising the OneSignal settings and the way notifications are sent + enabled skip-name-resolve in my.cnf.
I have alerted this issue to onesignal several times but each time they suggest to go for good hosting service.
I think the problem is with the onesignal wp plugin. I have a dedicated cloud server from vultr but still site goes down right after new push notification.
You will have no problem if you send push notification directly from onesignal dashboard.
Because all my request goes to deaf ears, now I do not use onesignal.
Related
To give some background, I'm an (unpaid) intern, and I'm unrelated to dealing with this kind of stuff. My employers wanted to update some pictures, and they did locally but didn't know how to upload the new version to the server.
I used the publish settings that were saved in Visual Studio from when the previous intern deployed the server (he was specialized in web site stuff) and it worked on deploy ... then I refreshed the page and I'm getting 502 server error.
Steps I have taken:
Connect to the VM and restart it - didn't solve it. it's using Microsoft server 2016.
Open the VM trough RDP, check if there are errors. There were 3 services not running, and I start them manually. One still isn't running, Downloaded Maps Manager. Ok... I google it and it's not a necessary service so I disable it. Now there are no errors and all services are running but I still am getting this error.
I tried pinging the IP of the server, and the URL itself and it works.
I believe it might be something to do with the load balancer, but I had one HTML class and nothing dealing with actually publishing stuff. If you could point me in the right direction I would appreciate it. The only reason why I'm trying to fix this myself is that I didn't make some kind of backup, and I feel so stupid having taken the site down.
Edit: I've gone to "load balancing" and it says service unhealthy. I tried going to the IPs there and it brings me to the same 502 server error page. From what I've gathered this is a configuration error, it's impossible they messed something with the site itself, right? It did work that first time, and if I run it from Visual Studio it works on the local machine ...
So I installed a single host Openshift OKD v3.11 cluster. I installed it on a VM running Centos 7.8.2003.
It seems to have installed ok except that it continually streams verbose logs to /var/log/messages. Around 5 logs per second and all seem to be about throttling requests. Example of a typical log message:
******Jun 13 15:49:13 centos7 journal: I0613 14:49:13.011402 1 request.go:485] Throttling request took 196.341689ms, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-service-cert-signer/serviceaccounts/service-serving-cert-signer-sa*****
The only reference I have managed to find is a question here but the access to the discussion is only available to those with deep pockets.
https://access.redhat.com/solutions/3348921
I assume these logs are nothing to worry about and so my main question is what is the "best"/cleanest/simplest/easiest way to ensure the Openshift cluster doesn't continue to fill up /var/log/messages but will still log any important messages there?
I would recommend looking at the root cause for this behavior. These messages indicate that there are a lot of requests coming to your API. Typically this is due to some application performing calls in a tight loop leading to this many messages. In your case check your openshift-service-cert-signer if you can see any warnings or an abnormal amount of log messages.
If you want to get rid of the throttling messages, you can increase the amount of Queries per second (QPS) for the API server: Recommended Practices for OKD Master Hosts (lower part).
The only reference I have managed to find is a question here but the access to the discussion is only available to those with deep pockets. https://access.redhat.com/solutions/3348921
I do not understand why you're saying that, as I can access that document with my free Red Hat account without any subscriptions. Have you tried with a free account as it says on the site?
Simon's answer was helpful but I've finally got to the bottom of this.
The problem was simply that the version of Docker I had installed was old. At the time of writing the latest version of Centos is 7.8.2003 and if you install that and then simply run "yum install docker" hoping that you'll get something at least reasonably new and certainly compatible with the rest of the linux installation, you'll probably be making a mistake.
The right thing to do is to follow the simple steps here:
https://docs.docker.com/engine/install/centos/
The reason I found the problem was because excessive logging of my openshift cluster wasn't the only issue. I started seeing strange behaviour of other containers. A process of trial and error narrowed down the issue to the default Centos version of docker. Once I followed the page above all my problem vanished including the original problem of /var/log/message getting hammered by openshift containers.
The main reason I decided to answer my own question was because surely someone else is going to be as impatient/thick as me and simply install Centos7 then try "yum install docker" without knowing they're about to enter a world of pain.
I was very cautious about adding a service worker to my PWA that would cache all my files. I tried to implement a system that would always call the server to get a "version" file so that when that "version" file updated, the cache would be cleared.
However, something didn't work correctly, and now the clients no longer call the server at all, since they have the files they need. This is perfect for offline use! But those clients will never call the server again so when I update the site to fix the problem (which I have done), they do not get the update!
Any suggestions on how I can connect with those clients again?
The easiest thing for you to do is deploy a change to your service worker code. In that version clear your cache and remove the buggy code.
Don't worry this happens a lot when you start working with service worker caching. :)
How does one keep OpenShift gears up-to-date? For example, updates to:
The Linux kernel
Important components/libraries like libc
Apache
Apache modules like mod_wsgi
Python
Python packages
Does OpenShift automatically update these and then restart the gear (or reboot the node)? Or does OpenShift send email notifications and the end-user can restart the gear during maintenance windows? What is the model?
What got me thinking about this was back in January there was a remote-code-execution bug in Ruby on Rails that everyone had to patch immediately.
This FAQ seems to suggest that some level of upgrades happen automatically, but it isn’t clear whether this only applies to the OpenShift-specific code, or also other components like the kernel, Apache, etc.
I can tell you from my experience that changes to the openshift system are not always automatic. They had a change about 10 days ago and I'm still tracking down what they did to make my app run correctly. As far as I know there was no email sent. I did find a blog post of some of the major changes, not all. Of course, they introduced at least one bug that I know of. YMMV
My experiences over the last few weeks have been the following:
Last week there seemed to be an unannounced reboot of the server. I detected this by logging from a custom action hook. I didn't receive any email about it and I didn't see any notice at https://twitter.com/openshift_ops or https://openshift.redhat.com/app/status.
This week, there was the Heartbleed OpenSSL vulnerability and it seems like some gears were restarted. I didn't receive any email about it, Twitter didn't show anything, but there was information on the status page.
I've seen this problem reported by other people whereby Chrome is incredibly slow to respond on local sites hosted in IIS. This is whether I'm using localhost or a specific host file entry (like mysite.local).
Existing fixes suggest removing the IPV6 entries from the host file, but that doesn't sort it out on my machine.
I'm running windows 8.1, vs2013 and IIS 8.5.
Can anyone help - its driving me crazy!
open 'worker processes' in features view of IIS panel and see if an earlier request is holding up rest of the queue. This can occur if the previous process waits for I/O to complete for instance.