ContextBroker Crashed - fiware

ContextBroker has been working fine for the past few days, I made a 12M subscription to orion a couple of days ago, and it decided to die this morning, noone even touched the computer at 6am when the last message was received. So when I checked the status it said:
contextBroker dead but pid file exists
So I just init.d started it again. Any solution to this occurrence?
Edit1 Here is the log file from that day (sorry for late response):
https://www.dropbox.com/s/7tl0n3u8kwzlyat/ContextBrokerLog?dl=0
It was stored as contextBroker.log.old inside the /var/log/contextBroker directory, maybe because I created new subscriptions since then.

Related

GDS Unable to connect to database: Communications link failure

Since this morning we're experiencing some problems in our Google Datastudio reports, connected through the MySQL connector. We are running MySQL8 through SSL and have not had any problems before. Unfortunately this morning some diagrams showed an error:
Unable to connect to database: Communications link failure The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
It appears to be random what diagrams will get the error. When I try to refresh the data for a specific diagram by, for example, changing the date, it works again. Also, manually refreshing the report results in random broken diagrams.
Connections etc are all tested with external tools and seem to work fine.
Hope anyone can help me.
some users complains similar issue for few days on community forum
There is no status page that can confirm this and there is no channel support existing for GDS
https://support.google.com/datastudio/threads?hl=en&thread_filter=(category%3Aconnect_to_data)&max_results=20

Spring Boot + Liquibase: Communications link failure

I'm using Liquibase in all my projects, I really love the way it handles db updates, but recently I'm having this issue:
liquibase : Successfully acquired change log lock
liquibase : Successfully released change log lock
liquibase : Could not release lock
liquibase.exception.LockException: liquibase.exception.DatabaseException: liquibase.exception.DatabaseException: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Communications link failure during rollback(). Transaction resolution unknown.
at liquibase.lockservice.StandardLockService.releaseLock(StandardLockService.java:283) ~[liquibase-core-3.5.5.jar!/:na]
at liquibase.Liquibase.update(Liquibase.java:218) [liquibase-core-3.5.5.jar!/:na]
at liquibase.Liquibase.update(Liquibase.java:192) [liquibase-core-3.5.5.jar!/:na]
.
.
.
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Communications link failure during rollback(). Transaction resolution unknown.
.
.
liquibase : Failed to restore the auto commit to true
.
.
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet successfully received from the server was 16,913 milliseconds ago. The last packet sent successfully to the server was 89 milliseconds ago.
.
.
Caused by: java.lang.NullPointerException: null
at com.mysql.jdbc.MysqlIO.clearInputStream(MysqlIO.java:899) ~[mysql-connector-java-5.1.46.jar!/:5.1.46]
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2477) ~[mysql-connector-java-5.1.46.jar!/:5.1.46]
I have 3 apps running on my server. One of them running for about 2years. The second running for 2 months and the third one is new.
They are all Spring Boot applications.
This started happening when I want to update the jar of the second app, I stopped the old jar, run the new one and I can not get it to start with the error above.
Then I run the old jar of the same app and it started just fine. I try stopping and restarting this old jar and a few times I get the error above but most of the times it started just fine.
I try restarting the server (all three apps start as services with system boot) but none of them successfully start because of the same error. Not even the one running for 2 years. I stop all other apps and try again with this long-lasting app, it kept failing until it started.
Could it be a memory issue? I'm using a DigitalOcean droplet with 1 Core and 2gb RAM.
Also, what I've notice is that when it successfully start the logs looks like:
liquibase : Successfully acquired change log lock
liquibase : Reading from myDataBase.DATABASECHANGELOG
liquibase : Successfully released change log lock
Note the statement between the acquisition and release of the lock. I also suspect about a timing issue, between acquiring the lock and reading the changelog. But don't know if I can increase that time or how can I do it.
UPDATE
I don't known the source of the issue, but after updating the liquibase-core to v3.7.0 all the apps started correctly
UPDATE 2
About the previous update, I was testing as is not that the problem is solved, is just less frequent. But it has happen
IMPORTANT UPDATE 3
After some digging I realize that by the same time this happen, the team was playing around with DB Connection Pool. The application.properties files contain this:
spring.datasource.tomcat.initial-size=5
spring.datasource.tomcat.max-wait=20000
spring.datasource.tomcat.max-active=5
spring.datasource.tomcat.max-idle=5
spring.datasource.tomcat.min-idle=5
spring.datasource.tomcat.default-auto-commit=true
Sadly I don't understand much about DB Pool. But what raise my attention was the last line about the auto-commit. The last time this error pops up. I disable all those lines and the app started fine. I don't know if it was just coincidence or not.
Thank you very much!
UPDATE 4
I'm back to square one... I don't know how to solve this any clue would be appreciatted
UPDATE 5
I updated mysql-connector-java to v8.0.15. My server has MySQL v5.6.33. I'm still having this issue and now I can not restart my app.
After the above change, now the exception ends with:
The last packet successfully received from the server was 10,036 milliseconds ago. The last packet sent successfully to the server was 10,036 milliseconds ago.
.
.
.
Caused by: java.io.IOException: Socket is closed
at com.mysql.cj.protocol.AbstractSocketConnection.getMysqlInput(AbstractSocketConnection.java:72) ~[mysql-connector-java-8.0.15.jar!/:8.0.15]
at com.mysql.cj.protocol.a.NativeProtocol.clearInputStream(NativeProtocol.java:833) ~[mysql-connector-java-8.0.15.jar!/:8.0.15]
... 92 common frames omitted
UPDATE 6 - WORKAROUND SOLUTION
In a moment of clarity I decided to disable liquibase from my project, by setting liquibase.enabled=false in the application.properties. Now I can start/stop/restart the app anytime with no problem. As I only use liquibase to mantain the DataBase, and the changes I made are executed (the problem is when releasing the lock), I will enable Liquibase just to update the schema and then disable it to run the app
UPDATE 7 - I think I've solve it
I kept stoping and starting the apps over and over and every time I did it I get the Socket is closed so I began to make my head around a timing issue... From the exceptions thrown I notice that every time it says that:
The last packet successfully received from the server was xxxx milliseconds ago.
The smallest value I get of those milliseconds ago was 10.085 and by chance I have some logs of the same app from time ago when this issue never pop-up. From those logs statements I meassure the time between acquiring the change-log-lock and releasing it. To my surprise it was around 9 seconds and never above that.
So I login to mysql console and issue:
show variables LIKE '%timeout%';
Which gave me a list of the defined timeouts. And there it was connect_timeout was the smallest and set to 10 seconds. It has to be it.
I googled how to set that value. I've try creating a file /etc/my.cnf and adding to it:
[mysqld]
connect_timeout=20
Then restart mysql with:
sudo /etc/init.d/mysql restart
But when I log back to the mysql console the value was still 10. So in the mysql console I issued this other command:
SET GLOBAL connect_timeout=20;
And the value was updated (I don't know if this will persist after a restart)
I restart mysql again and... voilà! Now the apps start normally with liquibase enabled. I'm happy! :-)
I kept stoping and starting the apps over and over and every time I did it I get the Socket is closed so I began to make my head around a timing issue... From the exceptions thrown I notice that every time it says that:
The last packet successfully received from the server was xxxx milliseconds ago.
The smallest value I get of those milliseconds ago was 10.085 and by chance I have some logs of the same app from time ago when this issue never pop-up. From those logs statements I meassure the time between acquiring the change-log-lock and releasing it. To my surprise it was around 9 seconds and never above that.
So I login to mysql console and issue:
show variables LIKE '%timeout%';
Which gave me a list of the defined timeouts. And there it was connect_timeout was the smallest and set to 10 seconds. It has to be it.
I googled how to set that value. I've try creating a file /etc/my.cnf and adding to it:
[mysqld]
connect_timeout=20
Then restart mysql with:
sudo /etc/init.d/mysql restart
But when I log back to the mysql console the value was still 10. So in the mysql console I issued this other command:
SET GLOBAL connect_timeout=20;
And the value was updated (I don't know if this will persist after a full server restart)
I restart mysql again and... voilà! Now the apps start normally with liquibase enabled. I'm happy! :-)
We were working with MySQL 8.0.19. For me, it were only a query which used to fail with the exception Caused by: java.io.IOException: Socket is closed.
Found that for MySQL 8.0.19, there was a bug reported with a similar stack trace of mine - https://bugs.mysql.com/bug.php?id=99234
After upgrading MySQL server to 8.0.21, it started working ! No other changes.
Hope it helps for folks who have spent quite some time with selected query(ies) not working and having the same error, and tried out various other options.
have you solved it yet?
I had a similar problem and solved it by forcing Liquibase to execute in synchronous mode.
Was your spring boot app generated by jhipster by any chance? If so, take a look at DatabaseConfiguration and change from AsyncSpringLiquibase to SpringLiquibase

MYSQL error, cannot start service and mysql.sock not found

Yesterday, my website crashed. Or, what appears to be a crash.
My host (JustHost) have disclosed that it was a SAN problem and have been rebuilding it. It's hard to get anymore information out of them because they are playing their cards very close to their chest for some reason.
I have suffered almost 24 hours downtime in 2 days and I'm getting fed up.
The error I am receiving on the frontend of the website is:
Database connection error (2): Could not connect to MySQL.Database connection error (2): Could not connect to MySQL.
Trying to access my phpMyadmin from cPanel gives me the following error:
Accessing SSH and trying to start the mysql service via putty I get:
Starting MySQL. ERROR! The server quit without updating PID file (/var/lib/mysql /mysqld.pid).
Can anyone please help me resolve this issue as my host is rather useless!
Thanks!
Yep, mysqld is down. Did you try running "mysqld_safe&"?
For anyone who wishes to know the conclusion of this issue.
I have been left with 2 choices:
1) Be patient and wait for Just Host to fix the issue and potentially lose my Google search rankings and a lot more future revenue
2) Set up a new hosting account, transfer my data over and wait for the domain to propagate.
I have chosen the latter of the two. My confidence in Just Host as a customer service hub and web hosting provider has thoroughly diminished to the point where all of my hosting accounts will now be switched over to another provider.
They have let me down by not fixing the issue in good time and have passed me from pillar to post in live chat, email support and phone support. The final nail in the coffin was my request for them to backup my databases and send them to me via email being met with the resoundingly passive "sorry, we do not provide that service" response.
I would do it myself if only I had access to phpMyadmin, which I do not have because of a server error that they have been incapable of fixing for the past 72 hours.
Goodbye Just Host.

SQL Server Agent job fails once a week with 'Login failed for user ...\$duplicate-1889'

I run a series of DTS packages via SQL Server (2000) Agent every night. They have been running ok for years until I added a couple of transform data tasks that pick up info from another local SQL Server (2008).
The odd thing is that the tasks run ok for about a week and then fail with the error 'Login failed for user domain\$duplicate-1889'. I don't run these packages using this user account, in fact I don't think it's a real account. I have a Windows account called dbadmin which has been set up on both servers to have access. This dbadmin account is used to start up the SQL Server Agent service and run the scheduled jobs.
When the jobs fail, all have to do to get them to run again is to stop and restart SQL Server Agent. But after a week they fail again with the same issue. Also, it's not the same day of the week each time.
I checked the Windows event log and although I can see the error, I can't figure out its cause. Any help or pointers would be appreciated.
The domain\$duplicate-1889 name looks like machine account. Every machine joined to a domain has such an account in the domain that the machine uses to authenticate to the domain.
The passwords for those accounts are automatically handled and the password is periodically changed automatically. By default that interval is 30 days but if is changed to once a week in your environment maybe there's a problem when the computer's password has been updated?
It is possible to disable the automatic changing maybe that's worth trying to see if the problem dissapears.

Made this vBulletin Forum years ago, now it's spamming me errors

So I made this vBulletin forum years ago and several months ago it started emailing me errors every now and then. Since I registered as the forum administrator when I was creating the site, it now sends me emails when there are bugs. The site owner and I have got together and tried to remove every mention of my email from his site. I didn't think much of it but it persisted. I emailed the site owner about it and he mentioned he switched hosting companies but other than that no big changes.
Last night I got 60+ bug reports from this forum. I tried to ask on vBulletin's official forum but you have to have a license and such to use it.
Long story short, I hate vBulletin. I hate it so badly.
Can anyone help me with this error?
Database error in vBulletin :
mysql_connect() [<a href='function.mysql-connect'>function.mysql-connect</a>]: Lost connection to MySQL server at 'reading initial communication packet', system error: 111
/home/drumline/public_html/forum/includes/class_core.php on line 317
MySQL Error :
Error Number :
Request Date : Tuesday, October 22nd 2013 # 06:25:41 AM
Error Date : Tuesday, October 22nd 2013 # 06:25:41 AM
Script : http://www.***.com/forum/showthread.php?162-Negative-Parents-how-to-convince-them-drumline-isn-t-evil&p=870&viewfull=1
Referrer :
IP Address : ***.**.*.***
Username :
Classname : vB_Database
MySQL Version :
Actually it's not a bug report, but an error report. The error message tells
Lost connection to MySQL server at 'reading initial communication packet'
The most probably case for this is that sometimes the MySQL server times out / doesn't respond in time and PHP stops waiting. Probably an issue of high load on your site.
[...] he mentioned he switched hosting companies [...]
This somewhat confirms my suspicions.
Agree with it being better suited on Server Fault. If you've got access to the my.cnf file try increasing the wait_timeout by 60 seconds.