We are using a Class that implements Cache<K, V>. The cached stored in a Hazelcast IMap<InternalKey<K, C>, V>. There's a "remove" logic that removes by an InternalKey of <String, String>.
K, C and V are all Strings.
public class HazelcastMapJCacheAdapter<K, C, V> implements Cache<K, V> {
private IMap<InternalKey<K, C>, V> cachedData;
public void removeEntriesByKeyAndContext(BiPredicate<K, C> removeCondition) {
Predicate<InternalKey<K, C>, V> predicate = (Predicate<InternalKey<K, C>, V> & Serializable)
mapEntry -> (removeCondition.test(mapEntry.getKey().getOriginalKey(), mapEntry.getKey().getContext()));
cachedData.removeAll(predicate);
}
}
I keep getting a HazelcastSerializationException which I can't understand. What can go wrong with String serialisation? How can I properly log this scenario in order to get more data (the argument I have at hand is a Predicate which is a function interface...)? I fail to reproduce this scenario on my development environment, so debugging it is a problem.
Thanks
2020-03-23 02:07:34 WARN | [10.212.179.245]:5701 [MyApp] [3.12.5] Error while logging processing event
com.hazelcast.nio.serialization.HazelcastSerializationException: Failed to serialize 'com.hazelcast.spi.impl.operationservice.impl.operations.PartitionIteratingOperation'
at com.hazelcast.internal.serialization.impl.SerializationUtil.handleSerializeException(SerializationUtil.java:82) ~[hazelcast-3.12.5.jar:3.12.5]
at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toBytes(AbstractSerializationService.java:157) ~[hazelcast-3.12.5.jar:3.12.5]
at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toBytes(AbstractSerializationService.java:133) ~[hazelcast-3.12.5.jar:3.12.5]
at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toBytes(AbstractSerializationService.java:124) ~[hazelcast-3.12.5.jar:3.12.5]
at com.hazelcast.spi.impl.operationservice.impl.OutboundOperationHandler.send(OutboundOperationHandler.java:56) ~[hazelcast-3.12.5.jar:3.12.5]
at com.hazelcast.spi.impl.operationservice.impl.Invocation.doInvokeRemote(Invocation.java:656) ~[hazelcast-3.12.5.jar:3.12.5]
at com.hazelcast.spi.impl.operationservice.impl.Invocation.doInvoke(Invocation.java:631) ~[hazelcast-3.12.5.jar:3.12.5]
at com.hazelcast.spi.impl.operationservice.impl.Invocation.invoke0(Invocation.java:592) ~[hazelcast-3.12.5.jar:3.12.5]
at com.hazelcast.spi.impl.operationservice.impl.Invocation.invoke(Invocation.java:256) ~[hazelcast-3.12.5.jar:3.12.5]
at com.hazelcast.spi.impl.operationservice.impl.InvocationBuilderImpl.invoke(InvocationBuilderImpl.java:61) ~[hazelcast-3.12.5.jar:3.12.5]
at com.hazelcast.spi.impl.operationservice.impl.InvokeOnPartitions.invokeOnAllPartitions(InvokeOnPartitions.java:121) ~[hazelcast-3.12.5.jar:3.12.5]
at com.hazelcast.spi.impl.operationservice.impl.InvokeOnPartitions.invokeAsync(InvokeOnPartitions.java:99) ~[hazelcast-3.12.5.jar:3.12.5]
at com.hazelcast.spi.impl.operationservice.impl.InvokeOnPartitions.invoke(InvokeOnPartitions.java:88) ~[hazelcast-3.12.5.jar:3.12.5]
at com.hazelcast.spi.impl.operationservice.impl.OperationServiceImpl.invokeOnAllPartitions(OperationServiceImpl.java:385) ~[hazelcast-3.12.5.jar:3.12.5]
at com.hazelcast.map.impl.proxy.MapProxySupport.removeAllInternal(MapProxySupport.java:618) ~[hazelcast-3.12.5.jar:3.12.5]
at com.hazelcast.map.impl.proxy.NearCachedMapProxyImpl.removeAllInternal(NearCachedMapProxyImpl.java:330) ~[hazelcast-3.12.5.jar:3.12.5]
at com.hazelcast.map.impl.proxy.MapProxyImpl.removeAll(MapProxyImpl.java:285) ~[hazelcast-3.12.5.jar:3.12.5]
at com.myapp.cache.impl.HazelcastMapJCacheAdapter.removeEntriesByKeyAndContext(HazelcastMapJCacheAdapter.java:578) ~[myapp-distributed-cache-impl.jar:?]
Here is a summary of requirements for removeAll (and any other kind of server-side processing like executeOnKey etc):
Key and value must be serializable and their classes must be available on server classpath
Predicate (or EntryProcessor etc) must be serializable and available on server classpath
My guess is that in your case the BiPredicate<K, C> removeCondition is not serializable; however it has to be serialized as part of the lambda you pass to removeAll. For example consider this code:
class Scratch {
public static void main(String[] args) {
remove((x, y) -> true);
}
private static void remove(BiPredicate<String, String> predicate) {
HazelcastInstance hz = Hazelcast.newHazelcastInstance();
// put 100 items in the map
IMap<String, String> map = hz.getMap("map");
for (int i = 0; i < 100; i++) {
map.put("" + i, "" + i);
}
map.removeAll((Predicate<String, String> & Serializable)
entry -> predicate.test(entry.getKey(), entry.getValue()));
// now size is 0
System.out.println("Map size after removeAll " + map.size());
}
}
The above code will be executed without any issues. The moment you add another HazelcastInstance in the cluster (just add Hazelcast.newHazelcastInstance() in remove method body), the Predicate instance must be serialized and sent over the network between the 2 cluster members. This fails because the predicate argument is not Serializable but it is part of the lambda Predicate passed as argument to removeAll. The solution is to make sure everything referenced in your lambda is serializable. So in the above example the fix is to update main method like this:
public static void main(String[] args) {
remove((BiPredicate<String, String> & Serializable) (x, y) -> true);
}
On a general note, exercise caution when using lambdas because it is easy to accidentally capture outer class fields and in that case you will need to address serializability of containing class and your lambda's serialized form size will balloon unexpectedly. Quoting https://docs.oracle.com/javase/tutorial/java/javaOO/lambdaexpressions.html#serialization "You can serialize a lambda expression if its target type and its captured arguments are serializable. However, like inner classes, the serialization of lambda expressions is strongly discouraged."
You need to set class definition of InternalKey on Server’s classpath as Entries are deserialized for predicates.
Slightly out of context, you can also try setting in-memory-format to OBJECT for performance gains in predicate execution.
Related
Would appreciate some help with hamcrest and junit matchers... :)
I'm using junit-4.11.jar and hamcrest-core-1.3.jar on Eclipse Kepler with sun's jdk 1.6.0_30.
I have a class that holds an instance of any unknown type like so:
class UnknownClassHolder {
private Class<?> clazz;
public Class<?> getClazz() {
return clazz;
}
public void setClazz(Class<?> clazz) {
this.clazz = clazz;
}
}
clazz can be any class.
I want to my junit test to be something like this:
class UnknownClassHolderTest {
#Test
public void test() {
ArrayList<UnknownClassHolder> list = new ArrayList<UnknownClassHolder>();
UnknownClassHolder x = new UnknownClassHolder();
//lets add an Integer
x.setClazz(Integer.class);
list.add(x);
UnknownClassHolder y = new UnknownClassHolder();
//lets add a vector
y.setClazz(Vector.class);
list.add(y);
//now check that we added an Integer or a Vector using assertThat
for (UnknownClassHolder u: list) {
assertThat(u.getClazz(), anyOf(isA(Integer.class), isA(Vector.class))));
}
}
}
Junit's assertThat doesn't like this. It doesn't compile due to Integer & Vector Types not being related to each other via sub/super classes:
The method assertThat(T, Matcher<? super T>) in the type Assert is not applicable for the arguments (Class<capture#1-of ?>, AnyOf<Vector>)
Is there a more succinct way to do this other than:
assertThat(u.getClazz().getName(), either(is(Integer.class.getName())).or(is(Vector.class.getName())));
Is there a particular reason for using Matcher<? super T> rather than Matcher<?> in the org.hamcrest.MatcherAssert.assertThat(...) method?
Thanks.
First, you should be using is instead of isA since you're asserting that one class equals another. isA is for testing that an object is an instance of some class. Second, the only thing I can make work is forcing the compiler to see these as raw Objects.
assertThat(u.getClazz(), anyOf(is((Object) Integer.class), is((Object) Vector.class)));
There is a local instance of a class being instantiated inside a method.
eg
class TestMe{
public void foo()
{
A a = new A();
a.setState(this);
}
}
class A
{
private B b;
public void setState(TestMe tm)
{
b.doSomething(); //returns void
b.doSomethingAdditional(); //returns void
}
}
When testing foo, using powerMock, I wanted to do a verify to ensure that methods doSomething() and doSomethingAdditional() are called. I was looking something along the lines of the Mockito.verify(ObjectName).functionName() to do. Any suggestions??
PowerMock works in combination with Mockito. You should still have all the functionality to verify using Mockito calls.
Once you mock B, when you mock its functions and then pass it into A you can use the Mockito.verify.
verify(mockBObject).doSomething();
verify(mockBObject).doSomethingAdditional();
This may need a few extra steps to hook your mock object into A, since you don't have a constructor where you can define B passed in, or have some constructor to hook into. If you are just having A initialize B at the creation of A, you can use some PowerMockito tools to tell it what to do. Just mocking B will not do, because when your A is setup, it doesn't know to use your mocked object in its code as the B internal.
PowerMockito.whenNew(B.class).withNoArguments().thenReturn(mockBObject);
As of right now B is never setup or initialized, so it could cause some issues. If B is a static singleton object that you are assuming is created somewhere else, you can do similar mocking of the static getInstance() call to return a mock of B.
I'm experimenting with interception in Castle Windsor and notice that interceptors seem to be created as decorators of my service interface.
In other words, if I have an interface "ISomethingDoer" and a concrete "ConcreteSomethingDoer", the proxy implements ISomethingDoer but does not inherit from ConcreteSomethingDoer.
This is fine, and no doubt by design, but what I'm wondering is whether I can intercept protected virtual methods in my concrete classes that wouldn't be known by the public interface. I am doing this in order to add logging support, but I might want to log some of the specific internal details of a class.
In my slightly unimaginative test case I have this:
public interface ISomethingDoer
{
void DoSomething(int Count);
}
[Loggable]
public class ConcreteSomethingDoer : ISomethingDoer
{
public void DoSomething(int Count)
{
for (var A = 0; A < Count; A++)
{
DoThisThing(A);
}
}
[Loggable]
protected virtual void DoThisThing(int A)
{
("Doing a thing with " + A.ToString()).Dump();
}
}
So what I want to do is log calls to "DoThisThing" even though it's not part of the interface.
I've managed to get this working in Autofac. (I've created a Linqpad script here: http://share.linqpad.net/frn5a2.linq) but am struggling with Castle Windsor (see http://share.linqpad.net/wn7877.linq)
In both cases my interceptor is the same and looks like this:
public class Logger : IInterceptor
{
public void Intercept(IInvocation Invocation)
{
String.Format("Calling method {0} on type {1} with parameters {2}",
Invocation.Method.Name,
Invocation.InvocationTarget.GetType().Name,
String.Join(", ", Invocation.Arguments.Select(a => (a ?? "*null*").ToString()).ToArray())).Dump();
Invocation.Proceed();
"Done".Dump();
}
}
What I really want to do is say "any classes with a [Loggable] attribute, should use the logging interceptor". In the Autofac example I've specifically attached a logger to the registration, whereas with Castle I'm using an IModelInterceptorsSelector which looks like this:
public class LoggerInterceptorSelector : IModelInterceptorsSelector
{
public bool HasInterceptors(ComponentModel Model)
{
return Model.Implementation.IsDefined(typeof(LoggableAttribute), true);
}
public InterceptorReference[] SelectInterceptors(ComponentModel Model, InterceptorReference[] Interceptors)
{
return new[]
{
InterceptorReference.ForType<Logger>()
};
}
}
Finally, the code to execute all this is:
var Container = new WindsorContainer();
Container.Register(
Component.For<Logger>().LifeStyle.Transient
);
Container.Kernel.ProxyFactory.AddInterceptorSelector(new LoggerInterceptorSelector());
Container.Register(
Component.For<ISomethingDoer>()
.ImplementedBy<ConcreteSomethingDoer>()
.LifeStyle.Transient
);
var Doer = Container.Resolve<ISomethingDoer>();
Doer.DoSomething(5);
When run I would expect to see "Calling method DoThisThing with parameters x" for each time the method is called. Instead I only get the call to DoSomething logged.
I can see why Castle Windsor is doing this, but I'm wondering if there is a way to tweak the behaviour?
(As a side-note I don't want to use Windsor's own interceptor attributes as I don't want to introduce dependencies to Castle outside of my composition root.)
I have tried resolving the ConcreteSomethingDoer specifically and this works, but not if I'm resolving the ISomethingDoer.
Apologies for the long post, and also apologies because I am pretty new to Castle Windsor!
I you could register like:
Container.Register(
Component.For<ISomethingDoer, ConcreteSomethingDoer>()
.ImplementedBy<ConcreteSomethingDoer>()
.LifeStyle.Transient
);
This should create a class proxy by deriving from ConcreteSomethingDoer. However this won't work with dynamic interceptors. However you probably can work around that by creating a facility which registers the interceptor when needed.
How to register two services with one instance of implementation? I used:
_container.Register(Component.For(new [] { typeof(IHomeViewModel), typeof(IPageViewModel) }).
ImplementedBy(typeof(HomeViewModel)).Named("IHomeViewModel").LifeStyle.Singleton)
But upper code registers two instances of HomeViewModel.
That's exactly the way to do it. See "Type Forwarding" in the docs. It registers one logical component accessible via IHomeViewModel or IPageViewModel. The following test passes:
public interface IHomeViewModel {}
public interface IPageViewModel {}
public class HomeViewModel: IHomeViewModel, IPageViewModel {}
[Test]
public void Forward() {
var container = new WindsorContainer();
container.Register(Component.For(new[] {typeof (IHomeViewModel), typeof (IPageViewModel)})
.ImplementedBy(typeof(HomeViewModel)).Named("IHomeViewModel").LifeStyle.Singleton);
Assert.AreSame(container.Resolve<IHomeViewModel>(), container.Resolve<IPageViewModel>());
}
BTW you might want to use generics instead of all those typeof, and also remove the lifestyle declaration, since singleton is the default:
container.Register(Component.For<IHomeViewModel, IPageViewModel>()
.ImplementedBy<HomeViewModel>());
I've got my MVC application wired up so that the Repository Layer queries the LINQ to SQL classes, the Service Layer queries the Repository Layer, and the Controllers call the Service layer.
Basically I have code as follows
Repository
Public Function GetRegions() As IQueryable(Of Region) Implements IRegionRepository.GetRegions
Dim region = (From r In dc.Regions
Select r)
Return region.AsQueryable
End Function
Service
Public Function GetRegionById(ByVal id As Integer) As Region Implements IRegionService.GetRegionById
Return _RegionRepository.GetRegions() _
.Where(Function(r) (r.ID = id _
And r.isActive)) _
.FirstOrDefault()
End Function
Public Function GetRegionByNameAndParentID(ByVal region As String, ByVal parentid As Integer) As Region Implements IRegionService.GetRegionByNameAndParentID
Return _RegionRepository.GetRegions() _
.Where(Function(r) (r.Region = region _
And r.ParentID = parentid _
And r.isActive)) _
.FirstOrDefault()
End Function
Public Function GetActiveRegions() As List(Of Region) Implements IRegionService.GetActiveRegions
Return _RegionRepository.GetRegions() _
.Where(Function(r) r.isActive) _
.ToList
End Function
Public Function GetAllRegions() As List(Of Region) Implements IRegionService.GetAllRegions
Return _RegionRepository.GetRegions().ToList
End Function
I'm wondering if there's a nice/efficient way to add Caching to the Service layer so that it doesn't always have to be calling the REPO if the calls are the same.
As caching is a cross cutting concern (do a search in Wikipedia), you can use policy injection to implement caching on your repository layer, but the constraint is that you use a DI framework like Castle, Unity, ... Advantage of this concept is that you keep clean code in your repository layer.
I'll start with It Depends, but in simple scenario's where no interaction with other service agents is required, it is only recommended to cache the access to the database, as database access is the slowest of all. That's why I would recommend not to cache the access to the service layer, but rather the repository layer. This is also what Martin Fowler describes in his data mapper pattern.
If you are in a distributed scenario, whereby your controller and service are running on different servers, you might opt to cache on your controller as well to prevent the serialization of reference data every time you load e.g. your countrylist dropdown or tax code values.
In your scenario, I would attach a CachingHandler to your repository GetRegions(), and make a CacheKey which combines e.g. the method and parameters (if any). In a simplistic approach, save the CacheKey and the list of results to an Hashtable (in real life, use Patterns & Practices Caching application block or System.Web.Cache), and to every request to your repository, see if the cache key is in your Hashtable, then return the cached list.
A quick search in google gives you this to get started:
http://entlib.codeplex.com/Thread/View.aspx?ThreadId=34190
rockinthesixstring - yes, you can add an http cache into that layer using an anonymous function to either pull from the repo or pull from cache. basically, you'd do it allong the following lines (this is from an app that i'm working on just now that uses subsonic, but the premise of what you're after is identical.
/// <summary>
/// Returns an IQueryable based on the passed-in Expression Database
/// </summary>
IQueryable<T> IRepository<T>.Find(Expression<Func<T, bool>> expression)
{
// set up our object cacheKey
string keyValue = ParseExpression(expression);
if(keyValue==null)
{
return _repository.Find(expression);
}
string cacheKey = string.Format(EntityrootList, _className, "Find", keyValue, DateTime.UtcNow.Ticks.ToString(), string.Empty);
// try to populate from the cache
// rockinthesixstring - this is the part that is most relevant to you
var result = Cache.Get(cacheKey,
() => _repository.Find(expression),
CacheDuration);
return result;
}
[edit] in the controller, you'd call it like so (the controller _repository is set as:
readonly IRepository<Booking> _repository;
in the example):
[Authorize]
[AcceptVerbs(HttpVerbs.Post)]
public ContentResult ListBookings(int shareholderid)
{
Expression<Func<Booking, bool>> exprTree = x => x.FundShareholderEntity.ShareholderID == shareholderid;
var bookings = _repository.Find(exprTree).OrderByDescending(x => x.BookingDetailEntity.ActualDateFrom).OrderBy(x => x.BookingTypeID);
return Content(this.RenderPartialToString("BookingListNoPaging", bookings));
}
In the above example, Cache (i.e. Cache.Get()) is a class that wraps the httpcontext cache in a more user friendly way.
hope this helps...
jim
[edit] - added cache interface to add to the 'debate' :)
public interface ISessionCache
{
T Get<T>(string key);
T Get<T>(string key, Func<T> getUncachedItem, int cacheDuration);
void Insert(string key, object obj, int cacheDuration, CacheDependency arg0, TimeSpan arg2);
void Remove(string key);
object this[string key] { get; } // default indexer
IDictionaryEnumerator GetEnumerator();
}
in the injectable class would be used along the lines of:
public class FakeCache : ISessionCache
{... all inteface members implemented here etc..}
or for httpcache:
public class HttpContextCache : ISessionCache
{... all inteface members implemented here etc..}
etc, etc..
cheers again - jim