I am trying to construct a custom receiver to get input stream from a MySQL database, but a NullPointerException appears.
class CustomReceiver(sqlContext: SQLContext, url: String) extends Receiver[DataFrame](StorageLevel.MEMORY_AND_DISK_2) {
val connectionProperties = new Properties()
connectionProperties.setProperty("user", "XXX")
connectionProperties.setProperty("password", "XXX")
connectionProperties.setProperty("driver", "com.mysql.jdbc.Driver")
def onStart() {
// Start the thread that receives data over a connection
new Thread("Database Receiver") {
override def run() { receive() }
}.start()
}
def onStop() {
// There is nothing much to do as the thread calling receive()
// is designed to stop by itself if isStopped() returns false
}
/** Create a database connection and receive data until receiver is stopped */
private def receive() {
var data:DataFrame = null
try {
val jdbcQuery = "SQL Query String"
data = sqlContext.read.jdbc(url, jdbcQuery, connectionProperties)
if (data != null) {
store(data)
}
restart("Trying to connect again")
} catch {
case e: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException =>
restart("MySQLSyntaxError", e)
case n: java.lang.NullPointerException =>
restart("NullPointer", n)
}
}
}
Here is the error log:
java.lang.NullPointerException
at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:112)
at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:110)
at org.apache.spark.sql.DataFrameReader.<init>(DataFrameReader.scala:547)
at org.apache.spark.sql.SparkSession.read(SparkSession.scala:595)
at org.apache.spark.sql.SQLContext.read(SQLContext.scala:504)
at CustomReceiver.CustomReceiver$$receive(receiver.scala:44)
at CustomReceiver$$anon$1.run(receiver.scala:27
The error is from here:
data = sqlContext.read.jdbc(url, jdbcQuery, connectionProperties)
I think it is because the SQL Context can only be executed in the driver side. I can't find any answer to this question but I found some articles which said that it's impossible to stream from the database. Is this the reason the null pointer exception occurs?
Related
I am trying to figure out how to properly send response with ResponseEntity as JSON from Netty Reactor HTTP Server.
My current implementation reacts on request from WebClient and should send back response with some ResponseEntity status (let's assume just HTTP OK).
Unfortunately I'm still getting InvalidDefinitionException on the client side saying that it not possible to construct instance due to no default constructor.
I know what it means but for example Spring Webflux can have return type of rest endpoint Mono as well and no issues on client side will appear.
So is it somehow possible to properly serialize entity as JSON on server side and deserialize it at the client side?
This is my client
import org.springframework.web.reactive.function.client.WebClient;
public Mono<ResponseEntity> postRequest(final Object body, final String uri) {
return webClient.post()
.uri(uri)
.contentType(MediaType.APPLICATION_JSON)
.body(BodyInserters.fromValue(body))
.exchange()
.flatMap(clientResponse -> clientResponse.toEntity(ResponseEntity.class));
}
This is my Server
public void runWithPost(final String endpointPath, final ServerCallback callback) {
server = HttpServer.create()
.host(this.host)
.port(this.port)
.route(routes ->
routes.post(endpointPath, (request, response) ->
response.addHeader(CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE)
.sendString(Mono.just(getJSON(callback.handleCallback())))))
.wiretap(true)
.bindNow();
System.out.println("Starting server...");
}
private String getJSON(final ResponseEntity responseEntity) {
String json = StringUtils.EMPTY;
try {
json = objectMapper.writeValueAsString(responseEntity);
System.out.println("Serialized JSON: " + json);
} catch (final JsonProcessingException ex) {
System.err.println("JSON serializer error: " + ex.getMessage());
}
return json;
}
This is callback
public interface ServerCallback {
ResponseEntity handleCallback();
}
and usage
reactiveRestServer.runWithPost("/transaction", () -> ResponseEntity.ok().build());
Unfortunately on the client side I do not get HTTP status OK but deserialization exception:
2020-04-28 16:09:35.345 ERROR 15136 --- [ctor-http-nio-2] c.a.t.t.inbound.ArpMessageServiceImpl : Type definition error: [simple type, class org.springframework.http.ResponseEntity]; nested exception is com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of `org.springframework.http.ResponseEntity` (no Creators, like default construct, exist): cannot deserialize from Object value (no delegate- or property-based Creator)
at [Source: (io.netty.buffer.ByteBufInputStream); line: 1, column: 2]
2020-04-28 16:09:35.349 WARN 15136 --- [ctor-http-nio-2] io.netty.util.ReferenceCountUtil : Failed to release a message: DefaultLastHttpContent(data: PooledSlicedByteBuf(freed), decoderResult: success)
io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1
at io.netty.util.internal.ReferenceCountUpdater.toLiveRealRefCnt(ReferenceCountUpdater.java:74) ~[netty-common-4.1.45.Final.jar:4.1.45.Final]
What I am missing?
So I finally resolved that issue. For those who would be solving similar issue here is the answer.
The problem is that Spring Webflux converts ResponseEntity into DefaultFullHttpResponse, so that DefaultFullHttpResponse contains headers, status and also body. I resolved that issue by doing exactly the same approach.
public void runWithPost(final String endpointPath, final ServerCallback callback) {
if (server == null || server.isDisposed()) {
server = HttpServer.create()
.host(this.host)
.port(this.port)
.route(routes ->
routes.post(endpointPath, (request, response) -> processResponse(response, callback)))
.wiretap(true)
.bindNow();
logger.info("Starting server...");
} else {
logger.info("Couldn't start server because one is already running!");
}
}
and conversion is here
private NettyOutbound processResponse(final HttpServerResponse response, final ServerCallback callback) {
final ResponseEntity responseEntity = callback.handleCallback();
// set status
response.status(responseEntity.getStatusCodeValue());
// set headers
final HttpHeaders entityHeaders = responseEntity.getHeaders();
if (!entityHeaders.isEmpty()) {
entityHeaders.entrySet().stream()
.forEach(entry -> response.addHeader(entry.getKey(), buildValue(entry.getValue())));
}
if (responseEntity.hasBody()) {
try {
final Object body = responseEntity.getBody();
if (body instanceof String) {
return response.sendString(Mono.just((String) body));
} else {
return response.send(Mono.just(Unpooled.wrappedBuffer(getBytesFromObject(body))));
}
} catch (final IOException ex) {
response.status(HttpResponseStatus.INTERNAL_SERVER_ERROR);
return response.sendString(Mono.just(ex.getMessage()));
}
}
// set body
return response.send(Mono.empty());
}
Usage is as follows:
mockReactiveRestServer.runWithPost("/transaction", () -> ResponseEntity.ok().build());
I am running dotnet core 2.* and as the title mentions I have trouble getting my try catch to work when calling from API. And before anyone comments I am also running middle-ware to catch any exceptions. It too doesn't perform as expected
Addinional Information:
The Two Classes are in different namespaces/projects
Queries.Authentication is static.
They are both in the same solution
Controller:
[AllowAnonymous]
[HttpPost]
public string Login([FromBody] AuthRequest req)
{
// See if the user exists
if (Authenticate(req.username, req.password))
{
try {
// Should Fail Below
UserDetails ud = Queries.Authentication.GetUser(req.username);
} catch (RetrievalException e){ }
catch (Exception e){ } // Exception Still Comes Through
}
}
Queries.Authentication.GetUser Code:
public static class Authentication {
public static UserDetails GetUser (string username)
{
// Some Code
if (details.success)
{
// Some Code
}
else
{
throw new RetrievalException(details.errorMessage); // This is not caught propperly
}
}
}
Retrieval Exception:
public class RetrievalException : Exception
{
public RetrievalException()
{
}
public RetrievalException(String message)
: base(message)
{
}
public RetrievalException(String message, Exception inner)
: base(message, inner)
{
}
}
EDIT: Adding Middleware Code Here as per request:
public class CustomExceptionFilter : IExceptionFilter
{
public void OnException(ExceptionContext context)
{
HttpStatusCode status = HttpStatusCode.InternalServerError;
String message = String.Empty;
var exceptionType = context.Exception.GetType();
if (exceptionType == typeof(UnauthorizedAccessException))
{
message = "Unauthorized Access";
status = HttpStatusCode.Unauthorized;
}
else if (exceptionType == typeof(NullReferenceException))
{
message = "Null Reference Exception";
status = HttpStatusCode.InternalServerError;
}
else if (exceptionType == typeof(NotImplementedException))
{
message = "A server error occurred.";
status = HttpStatusCode.NotImplemented;
}
else if (exceptionType == typeof(RSClientCore.RetrievalException))
{
message = " The User could not be found.";
status = HttpStatusCode.NotFound;
}
else
{
message = context.Exception.Message;
status = HttpStatusCode.NotFound;
}
context.ExceptionHandled = true;
HttpResponse response = context.HttpContext.Response;
response.StatusCode = (int)status;
response.ContentType = "application/json";
var err = "{\"message\":\"" + message + "\",\"code\" :\""+ (int)status + "\"}";
response.WriteAsync(err);
}
}
App Config:
public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
if (env.IsDevelopment())
{
app.UseDeveloperExceptionPage();
} else
{
app.UseExceptionHandler();
}
...
}
Service Config:
public void ConfigureServices(IServiceCollection services)
{
// Add Model View Controller Support
services.AddMvc( config =>
config.Filters.Add(typeof (CustomExceptionFilter))
);
UPDATE: After playing around with it I noticed that even though my program throws the exception, if I press continue the API controller then handles it as if the exception was never thrown (as in it catches it and does what I want). So I turned off the break on Exception setting, this fixed it in debugger mode. However this the break doesn't seem to be an issue when I build/publish the program. This makes me think it is definitely a issue with visual studio itself rather than the code.
When you set ExceptionHandled to true that means you have handled the exception and there is kind of no error anymore. So try to set it to false.
context.ExceptionHandled = false;
I agree it looks a bit confusing, but should do the trick you need.
Relevant notes:
For those who deal with different MVC and API controller make sure you implemented appropriate IExceptionFilter as there are two of them - System.Web.Mvc.IExceptionFilter (for MVC) and System.Web.Http.Filters.IExceptionFilter (for API).
There is a nice article about Error Handling and ExceptionFilter Dependency Injection for ASP.NET Core APIs you could use as a guide for implementing exception filters.
Also have a look at documentation: Filters in ASP.NET Core (note selector above the left page menu to select ASP.NET Core 1.0, ASP.NET Core 1.1,ASP.NET Core 2.0, or ASP.NET Core 2.1 RC1). It has many important notes and explanations why it works as it does.
I'm working in an Android application and I'm having some trouble using the BroadcastReceiver with MvvmCross. I have the broadcast to receive messages from GCM (Push Notification) and then i create an IntentService that would do something when a message is received.
In my IntentService I use a Mvx.Resolve to make some calls to the Core project. Since the app is not running it throws an exception that it was not found (when trying to use Mvx.Resolve).
I tried finding a solution in the web and the closest i got was: How do I initialize the MvvmCross framework without a splash Activity? .
So I tried using it but I received another exception.
Android.Content.ReceiverCallNotAllowedException: Exception of type 'Android.Content.ReceiverCallNotAllowedException' was thrown
I saw this was happening when trying to create a receiver inside another receiver.
My broadcast is the following (using the code from the stack above):
[BroadcastReceiver(Permission= "com.google.android.c2dm.permission.SEND")]
[IntentFilter(new string[] { "com.google.android.c2dm.intent.RECEIVE" }, Categories = new string[] { "MyPackage" }, Priority = (int)IntentFilterPriority.HighPriority)]
[IntentFilter(new string[] { "com.google.android.c2dm.intent.REGISTRATION" }, Categories = new string[] { "MyPackage" }, Priority = (int)IntentFilterPriority.HighPriority)]
[IntentFilter(new string[] { "com.google.android.gcm.intent.RETRY" }, Categories = new string[] { "MyPackage" }, Priority = (int)IntentFilterPriority.HighPriority)]
public class GCMBroadcastReceiver : BroadcastReceiver
{
public override void OnReceive(Context context, Intent intent)
{
var setup = MvxAndroidSetupSingleton.EnsureSingletonAvailable(context);
setup.EnsureInitialized();
GCMIntentService.RunIntentInService(context, intent);
SetResult(Result.Ok, null, null);
}
}
My IntentService:
[Service]
public class GCMIntentService : IntentService
{
static PowerManager.WakeLock sWakeLock;
static object LOCK = new object();
public static void RunIntentInService(Context context, Intent intent)
{
lock (LOCK)
{
if (sWakeLock == null)
{
// This is called from BroadcastReceiver, there is no init.
var pm = PowerManager.FromContext(context);
sWakeLock = pm.NewWakeLock(WakeLockFlags.Partial, "GCM Broadcast Receiver");
}
}
sWakeLock.Acquire();
intent.SetClass(context, typeof(GCMIntentService));
context.StartService(intent);
}
protected override void OnHandleIntent(Intent intent)
{
try
{
var context = this.ApplicationContext;
var action = intent.Action;
if (action.Equals("com.google.android.c2dm.intent.REGISTRATION"))
{
HandleRegistration(context, intent);
}
else if (action.Equals("com.google.android.c2dm.intent.RECEIVE"))
{
HandleMessage(context, intent);
}
}
finally
{
lock (LOCK)
{
//Sanity check for null as this is a public method
if(sWakeLock != null)
{
sWakeLock.Release();
}
}
}
}
private void HandleRegistration(Context context, Intent intent)
{
var registration = intent.GetStringExtra("registration_id");
if(intent.GetStringExtra("error") != null)
{
// Registration failed, should try again later.
}
else if(intent.GetStringExtra("unregistered") != null)
{
// Unregistered
}
else if(registration != null)
{
//Do registration
}
}
private void HandleMessage(Context context, Intent intent)
{
Task.Factory.StartNew(() =>
{
Mvx.Resolve<MyInterface>().DoSomething();
});
}
}
I read in that same stack overflow that if I was using only a simple Mvx.Resolve it would be easier to instantiate it by myself. The problem is that inside my method "DoSomething" it uses more Mvx.Resolves and I didn't want to change the core application because of the Android.
So I was wondering if there is any way to start the framework so that I would not have to change the Core or if the best way to deal with this would be changing the core.
Edit:
The Exception Trace is (I got this from the logcat since i can't debug an app that is not running):
Android.Content.ReceiverCallNotAllowedException: Exception of type 'Android.Content.ReceiverCallNotAllowedException' was thrown.
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () <0x00028>
at Android.Runtime.JNIEnv.CallObjectMethod (intptr,intptr,Android.Runtime.JValue[]) <0x000c3>
at Android.Content.ContextWrapper.RegisterReceiver (Android.Content.BroadcastReceiver,Android.Content.IntentFilter) <0x0018f>
at Thrust.Plugins.Network.Droid.NetworkService.OnNetworkChanged (System.Action`1<Thrust.Plugins.Network.NetworkStatus>) <0x00093>
at Chat.Core.App.Initialize () <0x00533>
at Cirrious.MvvmCross.Platform.MvxSetup.CreateAndInitializeApp (Cirrious.CrossCore.Plugins.IMvxPluginManager) <0x0005b>
at Cirrious.MvvmCross.Platform.MvxSetup.InitializeApp (Cirrious.CrossCore.Plugins.IMvxPluginManager) <0x00027>
at Cirrious.MvvmCross.Platform.MvxSetup.InitializeSecondary () <0x00133>
at Cirrious.MvvmCross.Platform.MvxSetup.Initialize () <0x0002b>
at Cirrious.MvvmCross.Droid.Platform.MvxAndroidSetupSingleton.EnsureInitialized () <0x0011b>
at Chat.Droid.GCM.GCMBroadcastReceiver.OnReceive (Android.Content.Context,Android.Content.Intent) <0x0002b>
at Android.Content.BroadcastReceiver.n_OnReceive_Landroid_content_Context_Landroid_content_Intent_ (intptr,intptr,intptr,intptr) <0x0007b>
at (wrapper dynamic-method) object.0daa077c-c2f5-4cd5-bcbc-bc391ce0b2c3 (intptr,intptr,intptr,intptr) <0x0004b>
--- End of managed exception stack trace ---
android.content.ReceiverCallNotAllowedException: BroadcastReceiver components are not allowed to register to receive intents
at android.app.ReceiverRestrictedContext.registerReceiver(ContextImpl.java:246)
at android.app.ReceiverRestrictedContext.registerReceiver(ContextImpl.java:235)
at chat.droid.gcm.GCMBroadcastReceiver.n_onReceive(Native Method)
at chat.droid.gcm.GCMBroadcastReceiver.onReceive(GCMBroadcastReceiver.java:28)
at android.app.ActivityThread.handleReceiver(ActivityThread.java:2659)
at android.app.ActivityThread.access$1800(ActivityThread.java:174)
at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1383)
at android.os.Handler.dispatchMessage(Handler.java:102)
at android.os.Looper.loop(Looper.java:146)
at android.app.ActivityThread.main(ActivityThread.java:5593)
at java.lang.reflect.Method.invokeNative(Native Method)
at java.lang.reflect.Method.invoke(Method.java:515)
at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1283)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1099)
at dalvik.system.NativeStart.main(Native Method)
[ERROR] FATAL UNHANDLED EXCEPTION: Android.Content.ReceiverCallNotAllowedException: Exception of type 'Android.Content.ReceiverCallNotAllowedException' was thrown.
I need to build a very simple netty TCP server handling data sending from client. The server need to send back the result but it's not necessary for client to receive. So it may throw an exception caused by sending results after client close the connection. The channel seems stall then, but what I need server to do is continue to finish all data reading & processing.
public class TCPServerHandler extends ChannelHandlerAdapter {
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg) {
ByteBuf in = (ByteBuf) msg;
String str = new String();
String retstr;
try {
while (in.isReadable()) {
char c = (char) in.readByte();
if (c != '\n' && c != '\r' && c != SEP_BYTE) {
str += c;
}
}
retstr = dosomething(str);
ByteBuf ok = Unpooled.copiedBuffer(retstr, CharsetUtil.UTF_8);
ctx.writeAndFlush(ok);//May trigger exceptionCaught when flushing
} catch (Throwable e) {
// TODO Auto-generated catch block
e.printStackTrace();
} finally {
ReferenceCountUtil.release(msg);
}
}
#Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
// Close the connection when an exception is raised.
System.out.println("ExceptionCaught: sending channel closed by client");
//cause.printStackTrace();
//ctx.read();//not working
//ctx.close();
}
}
So how can I ask netty to continue "channelRead" all the messages already in the buffer when exception caught?
I am trying to implement jsr-179 APi into Nokia Symbian phone for periodic location update using setLocationListener through J2me. In emulator it is working fine. While I installed Midlet on the device nokia 5230, it is given NullPointerException and the application is automatically terminating. What might be possible causes?
Below is my class, I am instantiating object for this class on a form in netbeans
class MovementTracker implements LocationListener {
LocationProvider provider;
Location lastValidLocation;
UpdateHandler handler;
boolean done;
public MovementTracker() throws LocationException
{
done = false;
handler = new UpdateHandler();
new Thread(handler).start();
//Defining Criteria for Location Provider
/*
Criteria cr = new Criteria();
cr.setHorizontalAccuracy(500);
*/
//you can place cr inside getInstance
provider = LocationProvider.getInstance(null);
//listener,interval,timeout,int maxAge
//Passing -1 selects default interval
// provider.setLocationListener(MovementTracker.this, -1, -1, -1);
provider.setLocationListener(MovementTracker.this, -1, 30000, 30000);
}
public void locationUpdated(LocationProvider provider, Location location)
{
handler.handleUpdate(location);
batteryLevel = System.getProperty("com.nokia.mid.batterylevel");
sn = System.getProperty("com.nokia.mid.networksignal");
localTime = System.currentTimeMillis();
Send_Location();
}
public void providerStateChanged(LocationProvider provider, int newState)
{
}
class UpdateHandler implements Runnable
{
private Location updatedLocation = null;
// The run method performs the actual processing of the location
public void run()
{
Location locationToBeHandled = null;
while (!done)
{
synchronized(this)
{
if (updatedLocation == null)
{
try
{
wait();
}
catch (Exception e)
{
// Handle interruption
}
}
locationToBeHandled = updatedLocation;
updatedLocation = null;
}
// The benefit of the MessageListener is here.
// This thread could via similar triggers be
// handling other kind of events as well in
// addition to just receiving the location updates.
if (locationToBeHandled != null)
processUpdate(locationToBeHandled);
}
try
{
Thread.sleep(10000); //Sleeps for 10 sec & then sends the data
}
catch (InterruptedException ex)
{
}
}
public synchronized void handleUpdate(Location update)
{
updatedLocation = update;
notify();
}
private void processUpdate(Location update)
{
latitude = update.getQualifiedCoordinates().getLatitude();
longitude = update.getQualifiedCoordinates().getLongitude();
altitude = update.getQualifiedCoordinates().getAltitude();
}
}
}
public MovementTracker() throws LocationException
...
I have not written any code for handling LocationException.
No code is very dangerous practice, just search the web for something like "java swallow exceptions".
It is quite possible that because of implementation specifics Nokia throws LocationException where emulator does not throw it. Since you don't handle exception this may indeed crash you midlet at Nokia - and you wouldn't know the reason for that because, again, you have written no code to handle it.
How can I catch that exception?
The simplest thing you can do is to display an Alert with exception message and exit the midlet after user reads and dismisses alert