Direct logback logs to graylog - logback

I have a third party java application which uses logback for logging.
I want to add a Socket Appender to the application's logback.xml as specified in the documentation
(https://logback.qos.ch/manual/appenders.html)
<configuration debug="true">
<appender name="SERVER"
class="ch.qos.logback.classic.net.server.ServerSocketAppender">
<port>${port}</port>
<includeCallerData>${includeCallerData}</includeCallerData>
</appender>
<root level="debug">
<appender-ref ref="SERVER" />
</root>
</configuration>
The purpose is to direct the logs to graylog using udp end point.
Is there a way to send the logs using the UDP protocol ?
Thanks for your time.

There are multiple GELF appenders for Logback listed on the Graylog Marketplace:
https://marketplace.graylog.org/addons?tag=logback
Personally, I'd recommend using logstash-gelf.

I created a TCP Socket Server application (say on port 1234 running on localhost).
I added a socket appender to the logback.xml of the third party application to send logs to the TCP Server application.
<appender name="SOCKET" class="ch.qos.logback.classic.net.SocketAppender">
<remoteHost>localhost</remoteHost>
<port>1234</port>
<reconnectionDelay>10000</reconnectionDelay>
<includeCallerData>true</includeCallerData>
</appender>
Now, on the Socket server application, I added the logstash-gelf dependency and appender that joschi recommended.
Now the logs from the third party application go thru a hop and then reach Graylog.
The one caveat that I had to get around was to determine the severity of the log from the log message before logging it again in the socket server application.

Related

Http requests to CEP

I installed the CEP ( Proton ) through the official documentation, https://forge.fiware.org/plugins/mediawiki/wiki/fiware/index.php/CEP_GE_-_IBM_Proactive_Technology_Online_Installation_and_Administration_Guide
After that, I watched this recommended video to learn more about CEP. https://edu.fiware.org/pluginfile.php/653/mod_resource/content/1/CEP-Tutorial.mp4
But I can't check engine instance state, because appears this error in response:Could not read instance state, message: Error activating jmx proxy:
It seems that JMX is not properly configured.
As described in the installation guide, in the Apache Tomcat users configuration file you need to add manager-jmx role, and add it to the manager user name:
<tomcat-users>
...
<role rolename="manager-jmx" />
<user username="manager" password="manager" roles="manager-gui,manager-status,manager-script,manager-jmx" />
...
</tomcat-users>
You need to enable JMX access on Apache Tomcat, by adding it to CATALINA_OPTS, as described in the installation guide.
You also need to specify the JMX service port in the ProtonAdmin.properties file, as described in the same installation guide.

Cant connect to Mysql master/slave using Entityframework

I have mysql master slave configuration and i cant connect to it when i specify two ips. I search for connection string and found http://www.connectionstrings.com/mysql-connector-net-mysqlconnection/
Server=serverAddress1, serverAddress2, serverAddress3;Database=myDataBase;
Uid=myUsername;Pwd=myPassword;
So its looks simple but when i specify my two ips i get error "No such host is known"
I also found https://blogs.oracle.com/MySqlOnWindows/entry/how_to_using_replication_load and tried it but strange thing is that then it much slower. So now i am lost i cant use my replication.
PS. I am not sure if it make sense but i am running that on Azure
You should specify server in specific config section
<configSections>
<section name="MySQL" type="MySql.Data.MySqlClient.MySqlConfiguration, MySql.Data, Version=6.7.4.0, Culture=neutral, PublicKeyToken=c5687fc88969c44d"/>
</configSections>
<MySQL>
<Replication>
<ServerGroups>
<Group name="MasterSlave">
<Servers>
<Server name="master" IsMaster="true" connectionstring=" server=<server>;port=<port>;uid=<user>;password=<password>;"/>
<Server name="slave" IsMaster="false" connectionstring=" server=<server>;port=<port>;uid=<user>;password=<password>;"/>
</Servers>
</Group>
</ServerGroups>
</Replication>
</MySQL>
Then use following as a connection string:
"server=MasterSlave;database=sakila;"
Source: https://blogs.oracle.com/MySqlOnWindows/entry/how_to_using_replication_load

WSO2 ESB: suspended anonymousendpoint

I am working on WSO2 ESB 4.8.1 and facing an issue regarding to suspended anonymous endpoint. The scenario is: TCP/IP transport receiver (custom one) -> Proxy -> invoke HTTP endpoint.
At the startup runtime, my custom transport receiver will connect to another machine by using TCP/IP protocol and start listener message event. When message event come, the transport receiver will format the message event as JSON format and send it to predefined Proxy (This step is working well even though suspended anonymous endpoint has been occurred). The implement in side the proxy is really simple, just send to HTTP endpoint:
<property name="messageType" value="application/json" scope="axis2" />
<property name="Accept" scope="transport" value="application/json" />
<payloadFactory media-type="json">
<format>$1</format>
<args>
<arg evaluator="json" expression="$.eventMessage" />
</args>
</payloadFactory>
<call>
<endpoint>
<http method="post" uri-template="http://localhost:9080/goal"></http>
</endpoint>
</call>
The process is working fine except the suspended anonymous endpoint has happened (Such as the HTTP endpoint is shutdown), then i am able to see a log message as below:
WARN {org.apache.synapse.endpoints.EndpointContext} - Suspending endpoint : AnonymousEndpoint - current suspend duration is : 30000ms - Next retry after : Tue Oct 28 17:38:23 ICT 2014
If i stop the request traffic at the transport receiver and restart the HTTP endpoint, after 30 ms the endpoint can back to service
In case request traffic at the transport receiver still running, i saw the log message above is displayed few times and then the ESB server don't log that message anymore. At this time, the HTTP endpoint is suspended and never come back to service except restarting ESB server
I also saw this log message:
WARN {org.apache.synapse.core.axis2.TimeoutHandler} - Expiring message ID : urn:uuid:5dcedb38-3f9a-4a75-a201-ae4e76428216; dropping message after global timeout of : 120 seconds
That is really strange for me and i wonder if i did something wrong here. I really need your suggestion guys
Best regards,
An
Use a named endpoint and configure suspension and timeout, have a look there : https://docs.wso2.com/display/ESB481/Endpoint+Error+Handling
If you don't want to loose any message and if your http endpoint can be down for a long time for exemple, you should consider using guaranteed delivery with message Store and message Processor, have a look there : https://docs.wso2.com/display/IntegrationPatterns/Guaranteed+Delivery

How to configure Logback to add host-name/IP to each log event?

I am using Logback for logging. Scribe appenders send the logs in real time to a central Scribe aggregator. But I don't know how to add source machine IP in the logs for each log events. Looking at the aggregated central Scribe logs, it is almost impossible to know which machine is sending the logs. Hence, appending the IP of source machine to each log event will be helpful, and will be really great if we can control that through logback configuration.
It's possible to pass down hostname to remote receiver thru contextName.
Add following to logback.xml on all appenders:
<contextName>${HOSTNAME}</contextName>
Then, on aggregator instance, it will be available for inclusion in the pattern:
<pattern>%contextName %d %-5level %logger{35} - %msg %n</pattern>
According to the Logback docs, there's now a CanonicalHostNamePropertyDefiner expressly to add a hostname to your logs. Add a define to your project:
<define name="hostname"
class="ch.qos.logback.core.property.CanonicalHostNamePropertyDefiner"/>
and access it as ${hostname}
well if you are working on a client server project then u can use MDC feature of slf4j/logback full document here and in this case you can have a well structured log file that you can identify which log is for which client
hope this helps!

The HTTP request is unauthorized with client authentication scheme 'Anonymous'. The authentication header received from the server was 'NTLM’

I have an ASP.NET web application written in VB.NET. One part of the application makes an AJAX call to an internal ASMX file which in turn makes a call to a remote web service which is just a single ASMX file. Normally this works fine and has been deployed a couple of times and works fine. One client however, is getting the message from the AJAX call:
The HTTP request is unauthorized with client authentication scheme
'Anonymous'. The authentication header received from the server was
'NTLM’.
I have scoured a large amount of websites trying to fix this but I can’t seem to find any answer that works for me.
I have been unable to replicate the error on my test server, which is the same as the client, Win2003 IIS6.
The remote web service is deployed on Windows 2008 r2 – IIS7.5. The remote service is deployed using ‘Anonymous’ authentication only. The client deployment is set up with Anonymous and ‘Integrated Windows Authentication’. I have tried changing the authentication levels on both implementations but cannot replicate the issue. The closest I have come is when I set the remote service IIS authentication to
The HTTP request is unauthorized with client authentication scheme
'Ntlm'. The authentication header received from the server was ''.
In the web.config file the reference to the remote service is:
<system.serviceModel>
<bindings>
<basicHttpBinding>
<binding name="SVCMappingSoap" closeTimeout="00:01:00" openTimeout="00:01:00" receiveTimeout="00:10:00" sendTimeout="00:01:00" allowCookies="false" bypassProxyOnLocal="false" hostNameComparisonMode="StrongWildcard" maxBufferSize="65536" maxBufferPoolSize="524288" maxReceivedMessageSize="65536" messageEncoding="Text" textEncoding="utf-8" transferMode="Buffered" useDefaultWebProxy="true">
<readerQuotas maxDepth="32" maxStringContentLength="8192" maxArrayLength="16384" maxBytesPerRead="4096" maxNameTableCharCount="16384"/>
<security mode="None">
<transport clientCredentialType="None" proxyCredentialType="None" realm=""/>
<message clientCredentialType="UserName" algorithmSuite="Default"/>
</security>
</binding>
</basicHttpBinding>
</bindings>
<client>
<endpoint address="http://svc.website.com/services/myService.asmx" binding="basicHttpBinding" bindingConfiguration="SVCMappingSoap" contract="SVCMappingService.SVCMappingSoap" name="SVCMappingSoap"/>
</client>
</system.serviceModel>
I have tried changing a number of the setting in the <security> section but still unable to replicate.
I am not sure of your total server setup.
<security mode="None">
<transport clientCredentialType="None" proxyCredentialType="None" realm=""/>
</security>
instead of above one please try with below configuration
<security mode="TransportCredentialOnly">
<transport clientCredentialType="Ntlm"/>
<message clientCredentialType="UserName" algorithmSuite="Default"/>
</security>
please go through below links, you can more idea on those and you can change the config based on your requirements:
http://blogs.msdn.com/b/publicsector/archive/2005/10/19/482833.aspx
http://fczaja.blogspot.com/2009/10/http-request-is-unauthorized-with.html
http://ddkonline.blogspot.com/2009/11/fix-http-request-is-unauthorized-with.html
I had to change the default generated
<security mode="Transport"/>
into
<security mode="Transport" >
<transport clientCredentialType="Ntlm"/>
</security>
One more comment for this problem:
If you are not using HTTPS,
<security mode="Transport"/>
is not supported. You can use
<security mode="TransportCredentialOnly">
instead.