I want to use a Logback TurboFilter to modify the contents of logged messages. Specifically, I want to use Spring Security to prepend the username of the logged in user to the message (that way, I can use a simple grep on the logs to find log messages referring to a single logged-in user).
It looks like a custom TurboFilter is the way to go, but the documentation is somewhat unclear relating to what I want to do.
Here's a skeleton of what I want to do:
public class LoggingTurboFilter extends TurboFilter {
#Override
public FilterReply decide(Marker marker, Logger logger, Level level,
String format, Object[] params, Throwable t) {
final String userId = SecurityContextHolder.getContext().getAuthentication().getName();
// get message
final String output = String.format("[%s] %s", userid, message); // or just use +
// ?
return FilterReply.NEUTRAL;
}
}
This way, a log statement in the application log will look like this:
[smithjohn] entered methodName()
Can anyone assist?
Related
We recently migrated from Splunk to ELK. We wanted to log our message as json for better searchability in Kibana.
Our application was using vert.x 3.9. I came across https://reactiverse.io/reactiverse-contextual-logging but that requires vertx-core to be updated to 4.x. This will be a major change for us and I looked for other options. Also, I joined this team recently and new to Vert.x
I came across net.logstash.logback:logstash-logback-encoder and able to log the messages as json. I used io.vertx.core.json.JsonObject to convert my message and key values to convert to json. I created a wrapper class that returns the string for given message and key/values as shown below.
public class KeyValueLogger {
public static String getLog(final String message) {
return new JsonObject().put("message", message).encode();
}
public static String getLog(final String message, final Map<String, Object> params) {
return new JsonObject().put("message", message).mergeIn(new JsonObject(params)).encode();
}
}
Every log message will call the above KeyValueLogger.getLog to get json message. Since Vert.x is a reactive application, is there a better solution to convert the log messages to json? Most of the log messages are in worker threads. I am afraid if any is in event loop thread then it might impact the performance.
Thanks in advance!
I am able to use logstash-logback-encoder by referring to https://github.com/logfellow/logstash-logback-encoder/tree/logstash-logback-encoder-6.6#readme in section Examples using Markers: . This eliminated custom code to convert the message and key values conversion to json. I tested the performance and it is really good and no need of reactive/async for my use case.
Sample Code
import java.util.Map;
import static net.logstash.logback.marker.Markers.appendEntries;
#Test
public void logTest(){
Map<String, String> map = ImmutableMap.of("p1", "v1", "p2", "v2");
log.info(appendEntries(map), "startTest 100");
log.info("startTest {} , key1={}, key2= {}", 101, "v1", "v2");
log.info(appendEntries(map), "startTest {} , key1={}, key2= {}", 101, "v1", "v2");
log.info("startTest 102");
log.error(appendEntries(map), "error occured", new RuntimeException());
}
Sample logback configuration
Note: <logstashMarkers/> is required to log the Map as key values in the json log output.
<appender name="console" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder">
<providers>
<timestamp/>
<logLevel/>
<contextName/>
<threadName>
<fieldName>thread</fieldName>
</threadName>
<callerData>
<fieldName>src</fieldName>
<classFieldName>class</classFieldName>
<methodFieldName>method</methodFieldName>
<fileFieldName>file</fileFieldName>
<lineFieldName>line</lineFieldName>
</callerData>
<logstashMarkers/>
<pattern>
<pattern>
{
"message": "%message"
}
</pattern>
</pattern>
<stackTrace>
<throwableConverter class="net.logstash.logback.stacktrace.ShortenedThrowableConverter">
<maxDepthPerThrowable>30</maxDepthPerThrowable>
<maxLength>2048</maxLength>
<shortenedClassNameLength>20</shortenedClassNameLength>
<exclude>^sun\.reflect\..*\.invoke</exclude>
<exclude>^net\.sf\.cglib\.proxy\.MethodProxy\.invoke</exclude>
<rootCauseFirst>true</rootCauseFirst>
</throwableConverter>
</stackTrace>
</providers>
</encoder>
</appender>
Performance test
Added the following performance test and it took ~ 0.08286483 milli seconds = 82864.8304 nano seconds per log message on average.
private static Map<String, Object> getMap(){
int count = 5;
Map<String, Object> map = new HashMap<String, Object>();
for(int i=0; i<count; ++i){
map.put("key"+i, UUID.randomUUID());
}
return map;
}
#Test
public void logPerformanceTest(){
long startTime = System.nanoTime();
final int COUNT = 10000;
for(int i=0; i<COUNT; i++) {
log.info(appendEntries(getMap()), "startTest 100");
}
long time = System.nanoTime() - startTime;
System.out.println("###### TOOK " + time + " (ns) ");
}
There might be a better solution to convert the log messages to JSON in Vert.x. One option could be using the Vert.x event bus to send log messages from worker threads to a dedicated logger verticle. This way, the logger verticle can handle the JSON conversion and logging, freeing up the worker threads from performing additional tasks.
This approach also ensures that the logging process does not impact the performance of the event loop thread, as it's running in a separate verticle.
First, you need to create a logger verticle that will handle the JSON conversion and logging. This verticle can subscribe to a specific address on the event bus to receive log messages. The logger verticle can look like this:
public class LoggerVerticle extends AbstractVerticle {
#Override
public void start() {
EventBus eventBus = vertx.eventBus();
eventBus.consumer("logger.address", message -> {
JsonObject logMessage = (JsonObject) message.body();
// Convert log message to JSON format and log it
System.out.println(logMessage.encodePrettily());
});
}
}
Next, in your worker threads, you can send log messages to the logger verticle using the event bus. To send a log message, you can create a JsonObject with the log message and other key-value pairs, and then send it to the logger.address on the event bus.
Map<String, Object> params = new HashMap<>();
params.put("timestamp", System.currentTimeMillis());
params.put("worker-thread", Thread.currentThread().getName());
JsonObject logMessage = new JsonObject(params);
logMessage.put("message", "Log message from worker thread");
EventBus eventBus = vertx.eventBus();
eventBus.send("logger.address", logMessage);
Finally, you need to deploy the logger verticle in your Vert.x application, so that it starts receiving log messages.
Vertx vertx = Vertx.vertx();
vertx.deployVerticle(new LoggerVerticle());
This way, you can ensure that log messages are converted to JSON format before logging and that the logging process does not impact the performance of the event loop thread.
It's like having a dedicated chef to cook the dishes and serve them, freeing up the waiter to focus on taking orders and managing the restaurant.
We are using the forge-api-java-client. There is an issue in Model Derivatives getManifest call.
The response fails mapping with a single Message String being returned instead of the expected String Array.
Have switched to using local build of the jar, change in file Message.java to include an alternative constructor for the class setMessage
public void setMessage(String message) {
List<String> messages = new ArrayList<>();
messages.add(message);
setMessage(messages);
}
Could this change be merged into the project.
We'll check it, but as of today, that package is just under maintenance. You are welcome to submit a PR.
Castle Windsor passes the registered concrete type to Controller's constructors. A typical implementation (no pun intended) is:
private readonly IDepartmentRepository _deptsRepository;
public DepartmentsController(IDepartmentRepository deptsRepository)
{
if (deptsRepository == null)
{
throw new ArgumentNullException("deptsRepository");
}
_deptsRepository = deptsRepository;
}
I need to pass the ctor a second parameter, if possible, so that I can pass that val on to the Repository constructor (I know: tramp data alert, but I don't know if there's a straightforward way around it:
public DepartmentsController(IDepartmentRepository deptsRepository, int DBInstance)
{
if (deptsRepository == null)
{
throw new ArgumentNullException("deptsRepository");
}
_deptsRepository = deptsRepository(DBInstance);
}
REPOSITORY
public DepartmentRepository(int dbInst)
{
string connStr = string.Format("Phoo{0}Bar", dbInst);
using (var conn = new OleDbConnection(connStr))
{
using (var cmd = conn.CreateCommand())
{
. . .
Is it possible to tweak what Castle Windsor sends to the Controller constructor this way? If so, how?
AND/BUT: For this to be of any value (to me, anyway), I need to be able to get the int val (that will be passed to the Controller) from the URL the client sends. IOW, if the client asks the server for data via:
http://locohost:4242/Platypus/GetAll/1
I need to pass a "1" as the second argument to PlatypusController.
If the user asks the server for data via:
http://locohost:4242/Platypus/GetAll/42
I need to pass a "42" as the second argument to PlatypusController.
etc.
This is what I did to solve my Controller/Repository data context Dilemma:
0) Added a database context argument to the Controller's routing attribute. IOW, this:
[Route("api/HHSUsers/GetAll")]
...got changed to this:
[Route("api/HHSUsers/GetAll/{dbContext=03}")]
1) Passed that database context arg to the Repository. To wit, this:
return _hhsusersrepository.GetAll();
...got changed to this:
return _hhsusersrepository.GetAll(dbContext);
...so that the Controller method is now:
[Route("api/HHSUsers/GetAll/{dbContext=03}")]
public IEnumerable<HHSUsers> GetAllHHSUsersRecords(int dbContext)
{
return _hhsusersrepository.GetAll(dbContext);
}
2) Changed the corresponding method in the Repository interface from:
IEnumerable<HHSUsers> GetAll();
...to this:
IEnumerable<HHSUsers> GetAll(string dbContext);
3) Changed the Repository method from this:
public HHSUsersRepository()
{
// All the data is loaded here in the ctor
}
public IEnumerable<HHSUsers> GetAll()
{
return hhsusers;
}
....to this:
public IEnumerable<HHSUsers> GetAll(string dbContext)
{
LoadHHSUsers(dbContext);
return hhsusers;
}
private void LoadHHSUsers(int dbContext)
{
string connStr = string.Format("Foo{0}Bar", dbContext);
// The same as previously from this point on, except that this:
// using (var conn = new OleDbConnection(#"Foo Bar Phoo Bar etc"...
// becomes:
// using (var conn = new OleDbConnection(connStr))
4) Tack the dbcontext val to the end of the URL when calling the method, so that it is this:
http://localhost:28642/api/HHSUsers/GetAll/42
...instead of this:
http://localhost:28642/api/HHSUsers/GetAll
If the data context to use is "03" I can omit the dbcontext arg from the URL, as 03 is the default value I set when I appended "=03" to the Controller's "dbContext" routing attribute arg.
I know some fancy-pants propeller-heads will find fault with this for some reason (for one reason because of the tramp data going here and there and everywhere like a hobo on steroids), but my response is the same as that of an athlete who is getting trash-talked by an opposing player and yet whose team is winning: just point at the scoreboard. IOW, this works for me, so that's pretty much all I care about. Style points are for runway models and, again, fancy-pants propeller-heads (AKA Star-Bellied Sneeches (as opposed to us plain
cats with the unstarred bellies)); see "The perfect is the enemy of the good."
This simple way has that self-same benefit -- of being (relatively) simple to grok and, thus, modify/refactor as necessary. Inelegant? Sure, but so was Joe Kapp.
I'm using Google Http Client and Jackson to query data to backend (JSON API).
I need to pass parameters (one Java bean object). The object might have few or lot of field. Initially I attempt to pass it as content as follow:
HttpRequest request = requestFactory.buildGetRequest(getUrl(api)).setContent(new JsonCContent(jsonFactory, params));
However, I'm not allowed to set the HTTP content in GET operation.
Any suggestion how can I pass these parameters?
Under one condition:
I don't want to write a util method to convert this object into string of URL parameters. But if there's already reusable API to do it, that would be fine.
I need generic solution if possible. Because I'm going to apply this to 600 JSON API calls.
My last alternative would be to change backend to expect POST request instead of GET, then I perform POST operation on the client side.
Thanks
Instead of extends GenericUrl, you can use GenericUrl.put (inherit from GenericData) to set query parameters. For example:
GenericUrl genericUrl = new GenericUrl("http://yourapi.com/request");
genericUrl.put("user", "user name");
genericUrl.put("token", "token values");
HttpRequest request = requestFactory.buildGetRequest(genericUrl);
It seems like the expected usage is to extend the URL class you are using for your buildGetRequest() call. For instance, let's say you wanted to provide two extra query parameters called "user" and "token". You could do this with the following:
HttpRequest request = requestFactory.buildGetRequest(
new CustomUrl("http://www.yourserver.com").setUser(userId).setToken(token));
where the CustomUrl class is defined as:
public class CustomUrl extends GenericUrl {
public CustomUrl(String encodedUrl) {
super(encodedUrl);
}
#Key("user")
private String mUserId;
#Key("token")
private String mToken;
public CustomUrl setUser(String userId) {
mUserId = userId;
return this;
}
public CustomUrl setToken(String token) {
mToken = token;
return this;
}
}
The values are not necessary for the #Key annotations, but will be used as the name of the respective query parameters if provided. If omitted, the name of the variable will be used instead (see example)
Check google-http-client's javadoc for more info.
I am getting time out from using JsonpRequestBuilder.
The entry point code goes like this:
// private static final String SERVER_URL = "http://localhost:8094/data/view/";
private static final String SERVER_URL = "http://www.google.com/calendar/feeds/developer-calendar#google.com/public/full?alt=json-in-script&callback=insertAgenda&orderby=starttime&max-results=15&singleevents=true&sortorder=ascending&futureevents=true";
private static final String SERVER_ERROR = "An error occurred while "
+ "attempting to contact the server. Please check your network "
+ "connection and try again.";
/**
* This is the entry point method.
*/
public void onModuleLoad() {
JsonpRequestBuilder requestBuilder = new JsonpRequestBuilder();
// requestBuilder.setTimeout(10000);
requestBuilder.requestObject(SERVER_URL, new Jazz10RequestCallback());
}
class Jazz10RequestCallback implements AsyncCallback<Article> {
#Override
public void onFailure(Throwable caught) {
Window.alert("Failed to send the message: " + caught.getMessage());
}
#Override
public void onSuccess(Article result) {
// TODO Auto-generated method stub
Window.alert(result.toString());
}
The article class is simply:
import com.google.gwt.core.client.JavaScriptObject;
public class Article extends JavaScriptObject {
protected Article() {};
}
The gwt page, however, always hit the onFailure() callback and show this alert:
Failed to send the message. Timeout while calling <url>.
Fail to see anything on the Eclipse plugin console. I tried the url and it works perfectly.
Would appreciate any tip on debugging technique or suggestion
Maybe you should set the callback function explicitly via setCallbackParam, since you have callback=insertAgenda in your url - I presume that informs the server what should be the name of the callback function that wraps the JSON.
Also, it's worth checking Firebug's console (or a similar tool for your browser) - even if GWT doesn't report any exceptions, Firebug still might.
PS: It's useful to use a tool like Firebug to see if the application does in fact receive the response from the server (that would mean that, for example, you do need the setCallbackParam call) or maybe there's something wrong on the server side (for whatever reason).
You have to read the callback request-Parameter (default callback, value something like __gwt_jsonp__.P0.onSuccess) on serversite and have to modify the output to
<callback>(<json>);
In this case:
__gwt_jsonp__.P0.onSuccess(<json>);
Both of these guys are absolutely correct, but here is a concrete example to help you understand exactly what they are referring too.
This is a public JSON api. Take a look at the results:
http://ws.geonames.org/postalCodeLookupJSON?postalcode=M1&country=GB&maxRows=4
This public API supports JSONP through the predefined parameter 'callback'. Basically whatever value you pass into callback, will be used as the function name to wrap around the JSON data you desire. Take a look at the results of these few requests:
http://ws.geonames.org/postalCodeLookupJSON?postalcode=M1&country=GB&maxRows=4&callback=totallyMadeUp
http://ws.geonames.org/postalCodeLookupJSON?postalcode=M1&country=GB&maxRows=4&callback=trollingWithJSONP
It could be happening because of another reason, that the webservice call is returning a JSON object and but the callback is expecting JSONP object (note there is a difference).
So if you are dealing with google maps api, and you are seeing this exception, you need to change it to api provide by maps api, something like
final GeocoderRequest request = GeocoderRequest.create();
request.setAddress(query);
try {
GWT.log("sending GeoCoderRequest");
if (m_geocoder == null) {
m_geocoder = Geocoder.create();
}
m_geocoder.geocode(request, new Geocoder.Callback() {
#Override
public void handle(final JsArray<GeocoderResult> results,
final GeocoderStatus status) {
handleSuccess(results, status);
}
});
} catch (final Exception ex) {
GWT.log("GeoCoder", ex);
}
Or else you could use RequestBuilder as in gwt library.