Issues with a UDF - junit

I have a UDF that accepts a bag as input and converts it to a map. Each key of the map consists of the distinct elements in the bag and the values corresponding to their count
But it's failing the junit tests

It seems you except to use a bag of two tuples, but you are indeed creating a bag that contains a tuple with two fields.
This code :
DataBag dataBag = bagFactory.newDefaultBag();
Tuple nestedTuple = tupleFactory.newTuple(2);
nestedTuple.set(0, "12345");
nestedTuple.set(1, "12345");
dataBag.add(nestedTuple);
should be transformed to :
DataBag dataBag = bagFactory.newDefaultBag();
Tuple tupleA = tupleFactory.newTuple(1);
tupleA.set(0, "12345");
dataBag.add(tupleA);
Tuple tupleB = tupleFactory.newTuple(1);
tupleB.set(0, "12345");
dataBag.add(tupleB);
Or you may iterate on all your tuple's fields.

The output of 1 is correct: in your UDF you are counting the number of tuples that have the same value for the first field, but in the test you are adding only one tuple with two values.
If what you want is to count the number of tuples with the same value as "key" (where key is the first value in your tuple), then what you are doing is correct, but you would have to change your test:
public void testExecWithSimpleMap() throws Exception {
Tuple inputTuple = tupleFactory.newTuple(1);
DataBag dataBag = bagFactory.newDefaultBag();
Tuple nestedTuple = tupleFactory.newTuple(2);
nestedTuple.set(0, "12345");
nestedTuple.set(1, "another value");
dataBag.add(nestedTuple);
// Add a second tuple
nestedTuple.set(0, "12345");
nestedTuple.set(1, "and another value");
dataBag.add(nestedTuple);
inputTuple.set(0,dataBag);
Map output = testClass.exec(inputTuple);
assertEquals(output.size(), 1);
System.out.println(output.get("12345"));
assertEquals(output.get("12345"),2);
}
However, if what you wanted was to count how many times a value is repeated in the whole Bag, whether it's on different Tuples or on the same Tuple, (it is not very clear in your question), then you need to change your UDF:
public class BagToMap extends EvalFunc<Map> {
public Map exec(Tuple input) throws IOException {
if(input == null) {
return null;
}
DataBag values = (DataBag)input.get(0);
Map<String, Integer> m = new HashMap<String, Integer>();
for (Iterator<Tuple> it = values.iterator(); it.hasNext();) {
Tuple t = it.next();
// Iterate through the Tuple as well
for (Iterator<Object> it2 = t.iterator(); it2.hasNext();) {
Object o = it2.next();
String key = o.toString();
if(m.containsKey(key)) {
m.put(key, m.get(key)+1);
} else {
m.put(key, 1);
}
}
}
return m;
}
}
In this case, your test should pass.

Related

Displaying data from SQLite table's columns, one of which holds an array

I managed to retrieve the SQLite table with only the first item of the array and put it in the UI's TextView. Couldn't get all the of the array's items. From each of the rest of the columns, a single value is returned successfully.
The JSON is parsed and passed as a parcelable ArrayList to a Fragment where it's presented in a list. Clicking on a list item directs to another Fragment where all the of item's details are presented.
I've been trying to write a for loop that returns the Strings in the array into the TextView, but the condition i < genresList.size() is always false. I tried using a while loop, but it returns only the first item of the list.
Various ways I've found on the internet didn't work.
Thanks.
Parsing and insertion to SQLite
private void parseJsonAndInsertToSQLIte(SQLiteDatabase db) throws JSONException {
// parsing the json
String jsonString = getJsonFileData();
JSONArray moviesArray = new JSONArray(jsonString);
ContentValues insertValues;
for (int i = 0; i < moviesArray.length(); i++) {
JSONObject jsonObject = moviesArray.getJSONObject(i);
String title = jsonObject.getString("title");
String imageUrl = jsonObject.getString("image");
String rating = jsonObject.getString("rating");
String releaseYear = jsonObject.getString("releaseYear");
JSONArray genresArray = jsonObject.getJSONArray("genre");
List<String> genres = new ArrayList<>();
for (int k = 0; k < genresArray.length(); k++) {
genres.add(genresArray.getString(k));
}
insertValues = new ContentValues();
insertValues.put(Movie.TITLE, title);
insertValues.put(Movie.IMAGE_URL, imageUrl);
insertValues.put(Movie.RATING, rating);
insertValues.put(Movie.RELEASE_YEAR, releaseYear);
for (int k = 0; k < genresArray.length(); k++) {
insertValues.put(Movie.GENRE, genres.get(k));
}
Log.i(TAG, "insertValues: " + genresArray);
long res = db.insert(TABLE_NAME, null, insertValues);
Log.i(TAG, "parsed and inserted to sql - row: " + res);
}
}
The item's details Fragment
public class MovieDetailsFragment extends Fragment{
... variables declarations come here...
#Nullable
#Override
public View onCreateView(#NotNull LayoutInflater inflater, #Nullable ViewGroup container, Bundle savedInstanceState) {
View rootView = inflater.inflate(R.layout.fragment_details_movie, container, false);
Context context = getActivity();
Bundle idBundle = getArguments();
if (idBundle != null) {
movieId = getArguments().getInt("id");
}
getDatabase = new GetDatabase(context);
getDatabase.open();
Cursor cursor = getDatabase.getMovieDetails(movieId);
... more irelevant code comes here...
titleView = rootView.findViewById(R.id.movieTtlId);
ratingView = rootView.findViewById(R.id.ratingId);
releaseYearView = rootView.findViewById(R.id.releaseYearId);
genreView = rootView.findViewById(R.id.genreID);
String titleFromSQLite = cursor.getString(cursor.getColumnIndex(Movie.TITLE));
String ratingFromSQLite = cursor.getString(cursor.getColumnIndex(Movie.RATING));
String releaseYearFromSQLite = cursor.getString(cursor.getColumnIndex(Movie.RELEASE_YEAR));
String genreFromSQLite;
if(cursor.moveToFirst()) {
do {
genreFromSQLite = cursor.getString(cursor.getColumnIndex(Movie.GENRE));
genres.add(genreFromSQLite);
} while (cursor.moveToNext());
}
else{
genreFromSQLite = cursor.getString(cursor.getColumnIndex(Movie.RELEASE_YEAR));
}
getDatabase.close();
//more irelevant code comes here
genreView.setText(genreFromSQLite);
genreView.setFocusable(false);
genreView.setClickable(false);
return rootView;
}
}
The method that returns the table from SQLite:
public ArrayList<Movie> getMovies() {
String[] columns = {
Movie.ID,
Movie.TITLE,
Movie.IMAGE_URL,
Movie.RATING,
Movie.RELEASE_YEAR,
Movie.GENRE
};
// sorting orders
String sortOrder =
Movie.RELEASE_YEAR + " ASC";
ArrayList<Movie> moviesList = new ArrayList<>();
Cursor cursor = db.query(TABLE_NAME, //Table to query
columns,
null,
null,
null,
null,
sortOrder);
if (cursor.moveToFirst()) {
do {
Movie movie = new Movie();
movie.setMovieId(Integer.parseInt(cursor.getString(cursor.getColumnIndex(Movie.ID))));
movie.setTitle(cursor.getString(cursor.getColumnIndex(Movie.TITLE)));
movie.setImageUrl(cursor.getString(cursor.getColumnIndex(Movie.IMAGE_URL)));
movie.setRating(cursor.getDouble(cursor.getColumnIndex(Movie.RATING)));
movie.setReleaseYear(cursor.getInt(cursor.getColumnIndex(Movie.RELEASE_YEAR)));
List<String> genreArray = new ArrayList<>();
while(cursor.moveToNext()){
String genre = cursor.getString(cursor.getColumnIndex(Movie.GENRE));
genreArray.add(genre);
}
movie.setGenre(Collections.singletonList(String.valueOf(genreArray)));
// Adding a movie to the list
moviesList.add(movie);
} while (cursor.moveToNext());
}
Log.d(TAG, "The movies list from sqlite: " + moviesList);
cursor.close();
db.close();
return moviesList;
}
I believe your issue is with :-
for (int k = 0; k < genresArray.length(); k++) {
insertValues.put(Movie.GENRE, genres.get(k));
}
That will result in just the last value in the loop being inserted as the key/column name (first parameter of the put) does not change (and probably can't as you only have the one column).
You could use :-
StringBuilder sb = new StringBuilder();
for (int k = 0; k < genresArray.length(); k++) {
if (k > 0) {
sb.append(",");
}
sb.append(genres.get(k));
}
insertValues.put(Movie.GENRE, sb.toString());
Note the above code is in-principle code. It has not been tested or run and may therefore contains errors.
This would insert all the data as a CSV into the GENRE column.
BUT that is not a very good way as far as utilising databases. It would be far better if the Genre's were a separate table and probably that a mapping table were used (but that should be another question).
This is going to cause you issues as well :-
if (cursor.moveToFirst()) {
do {
Movie movie = new Movie();
movie.setMovieId(Integer.parseInt(cursor.getString(cursor.getColumnIndex(Movie.ID))));
movie.setTitle(cursor.getString(cursor.getColumnIndex(Movie.TITLE)));
movie.setImageUrl(cursor.getString(cursor.getColumnIndex(Movie.IMAGE_URL)));
movie.setRating(cursor.getDouble(cursor.getColumnIndex(Movie.RATING)));
movie.setReleaseYear(cursor.getInt(cursor.getColumnIndex(Movie.RELEASE_YEAR)));
List<String> genreArray = new ArrayList<>();
while(cursor.moveToNext()){
String genre = cursor.getString(cursor.getColumnIndex(Movie.GENRE));
genreArray.add(genre);
}
movie.setGenre(Collections.singletonList(String.valueOf(genreArray)));
// Adding a movie to the list
moviesList.add(movie);
} while (cursor.moveToNext());
That is you move to the first row of the Cursor, extract some data MoveieId,Title ... ReleaseYear.
Then
a) if there any other rows you move to the next (which would be for a different Movie) and the next until you finally reached the last row adding elements to the genreArray.
or
b) If there is only the one row in the Cursor genreArray is empty.
You then add the 1 and only movie to the movieList and return.
1 move (row) in the Cursor will exist per movie and there is only the 1 GENRE column per movie. You have to extract the data in that column and then split the data into the genreArray without moving (see the previous fix that will create a CSV (note that would be messed up if the data contained commas)).
IF you used the previous fix and store the multiple genres as a CSV, then you could use :-
if (cursor.moveToFirst()) {
do {
Movie movie = new Movie();
movie.setMovieId(Integer.parseInt(cursor.getString(cursor.getColumnIndex(Movie.ID))));
movie.setTitle(cursor.getString(cursor.getColumnIndex(Movie.TITLE)));
movie.setImageUrl(cursor.getString(cursor.getColumnIndex(Movie.IMAGE_URL)));
movie.setRating(cursor.getDouble(cursor.getColumnIndex(Movie.RATING)));
movie.setReleaseYear(cursor.getInt(cursor.getColumnIndex(Movie.RELEASE_YEAR)));
List<String> genreArray = new List<>(Arrays.asList((cursor.getString(cursor.getColumnIndex(Movie.GENRE))).split(",",0)));
movie.setGenre(Collections.singletonList(String.valueOf(genreArray)));
// Adding a movie to the list
moviesList.add(movie);
} while (cursor.moveToNext());
Note the above code is in-principle code. It has not been tested or run and may therefore contains errors.

How to extract the a value from a JSON object by matching the key in a Map using Java 8?

I'm using Java 8 and jackson to try and get an int value from a json object. Here is some similar code I used to verify the structure.
HashMap<String, Object> myMap = new HashMap<String, Object>();
myMap
.entrySet()
.stream()
.forEach(x -> System.out.println(x.getKey() + " => " + x.getValue()));
and the result:
myKey1 => {"number":1}
myKey2 => {"number":1}
myKey3 => {"number":2}
What I'm trying to do is use the key, like myKey1 to find the json value i.e. {"number":1} and pull out the actual number, which for that key is 1.
However, I don't all know the values of the key's, just the key I want to match up. So I have to do this dynamically. I know the structure of the values and that is always the same, except the number can be different.
I think because I don't know the keys and their order, plus I'm using entrySet, that is forcing me into using Optional, which is making this more difficult.
Here is my code where I'm actually using the key to pull the json value:
Optional<Object> object = myMap.entrySet().stream()
.filter(e -> e.getKey().equals(myKey))
.map(Map.Entry::getValue)
.findFirst();
However, the stream pulls back
Optional[{"number":1}]
and I can't seem to get the number out so I can return it from a method. I don't actually
have a Java class for the object, so I assume that is why it's returning Optional as I was getting a compile error without it, but I'm not sure.
Any ideas as to the right way to do this?
Why iterating over all the entries of the map and do a linear search by key, to get the value of the entry?
Just get the value directly:
Object value = myMap.get(myKey);
Now, with regard to the number inside value, as you say you don't have a class that represents the values of the map, the most likely thing is that the JSON library you're using is creating a Map for each value. So from now on, let's assume that the values are actually some implementation of Map:
Integer = null;
if (value != null) {
// What's the type of the value? Maybe a Map?
if (value instanceof Map) {
Map<String, Object> valueAsMap = (Map<String, Object>) value;
number = (Integer) valueAsMap.get("number");
}
}
I'm assuming the numbers are Integers, but they can perfectly be Long instances or even Doubles or BigDecimals. Just be sure of the exact type, so the second cast doesn't fail.
EDIT: For completeness, here's the code for when the values are not maps, but of some class that represents a json node. Here I'm using JsonNode from Jackson library, but the approach is very similar for other libs:
Integer = null;
if (value != null) {
if (value instanceof JsonNode) {
JsonNode valueAsNode = (JsonNode) value;
number = (Integer) valueAsNode.get("number").numberValue();
}
}
findFirst() returns an Optional. You need to call .get() or .orElse() or .orElseThrow() after you call findFirst().
You can then cast the Object to JsonNode and retrieve the value. Here is the full code:-
public static void main(String[] args) throws Exception {
ObjectMapper mapper = new ObjectMapper();
HashMap<String, Object> myMap = new HashMap<String, Object>();
myMap.put("myKey1", mapper.readTree("{\"number\":1}"));
myMap.put("myKey2", mapper.readTree("{\"number\":1}"));
myMap.put("myKey3", mapper.readTree("{\"number\":2}"));
System.out.println(getNumberUsingMapKey(myMap, "myKey3"));
}
private static int getNumberUsingMapKey(Map<String, Object> map, String key) throws Exception {
return Optional.of(map.get(key))
.map(o -> ((JsonNode) o).get("number").asInt())
.get(); //or use one of the following depending on your needs
// .orElse(-1);
// .orElseThrow(Exception::new);
}
//or use this method
private static int getNumberUsingMapKeyWithoutOptional(Map<String, Object> map, String key) throws Exception {
Object o = map.get(key);
return ((JsonNode) o).get("number").asInt();
}
Output
2
Unfortunately I didn't describe my problem statement well from the start, but this is the final working code that I was able to piece from the other answers.
This is what the data structure looked like. A key that had a value which was a JSON object that had a key and an int, in other words a nested JSON object.
Key: myKey1 Value:{"number":1}
Key: myKey2 Value:{"number":1}
Key: myKey3 Value:{"number":2}
I'm posting it in case some else runs into this use case. This may not be the best way to do it, but it works.
Object value = myMap.get(keyName);
ObjectMapper mapper = new ObjectMapper();
String jsonString = (String) value;
JsonNode rootNode = mapper.readTree(jsonString);
JsonNode numberNode = rootNode.path("number");
System.out.println("number: " + numberNode.intValue());
and the result:
number: 1

Mapping to a Variable and retrieving once they match

I have a list of check boxes that when selected the getEventName method should reurn the key that matches the label. For example if the label is "NEW", the key should be returned when map[key] = "new". These have been defined in the LABEL_EVENTTYPE function below. It is always returning an empty string and can't seem to figure out why.
public static const LABEL_EVENTTYPE_MAP:Object = {
"CANCEL":["cancelled","expired", "doneForDay"],
"NEW":["new"],
"TRADE":["trade"],
"AMEND":["replaced"],
}
private function getEventName(label:String):String{
var map:Object = ReplayConstants.LABEL_EVENTTYPE_MAP;
for each(var key:String in map){
if (map[key] == label){
return key;
}
}
return "";
}
Iterating through object properties requires for..in loop instead of for each.. in
for (var key:String in map){
if (map[key] == label){
return key;
}
}
Also take into account, that objects in your map are arrays, that's why your comparison map[key] == label will always return false.

Lazy loading with JPA criterea

I implemented a generic solution for using lazy loading primefaces datatables using JPA Criterea.
However I am still having some doubts with thie implemented solution whenever we deal with several Joins (say for example an entity User that has relation with other entities like Account, Address, Department.....in addition to raw type properties like: String username, Date birthdate...etc).
I tested this solution but I am having some delays while loading huge number of data (however the solution is supposed to load only a limited number of rows specified by PageSize coming from datatable), so:
How to improve the performance of this solution?
How to be sure the number of loaded data is the one specified in the Pagesize?
Can you check the count() method and tell if it counts the number of result rows without loading all the data?
And most importantly how to use this solution in order to be generic with filters coming from Search forms (I mean how to use this sae generic method and give search critereas from a search form with multi search fields)?
Please I need your answer on the above mentioned questions especially the last one.
Here is the code:
public <T extends Object> List<T> search(Class<T> type, int first, int pageSize, String sortField, SortOrder sortOrder, Map<String, String> filters){
CriteriaBuilder cb = entityManager.getCriteriaBuilder();
CriteriaQuery<T> q = cb.createQuery(type);
Root<T> root=q.from(type);
q.select(root);
//Sorting
if (sortField != null && !sortField.isEmpty()) {
String[] sortingField = sortField.split("\\.", 2);
Path path = sortingField.length == 1 ? root.get(sortingField[0]): root.join(sortingField[0]).get(sortingField[1]);
if (sortOrder.equals(SortOrder.ASCENDING)) {
q.orderBy(cb.asc(path));
} else if (sortOrder.equals(SortOrder.DESCENDING)) {
q.orderBy(cb.desc(path));
}
}
// Filtering
Predicate filterCondition = cb.conjunction();
String wildCard = "%";
for (Map.Entry<String, String> filter : filters.entrySet()) {
String[] filterField = filter.getKey().split("\\.", 2);
Path path = filterField.length == 1 ? root.get(filterField[0]): root.join(filterField[0]).get(filterField[1]);
filterCondition = cb.and(filterCondition, filter.getValue().matches("[0-9]+")
? cb.equal(path, Long.valueOf(filter.getValue()))
: cb.like(path, wildCard + filter.getValue() + wildCard));
}q.where(filterCondition);
//Pagination
TypedQuery<T> s = entityManager.createQuery(q);
if (pageSize >= 0){
s.setMaxResults(pageSize);
}
if (first >= 0){
s.setFirstResult(first);
}
log.info("\n\n\n");
log.info("XXXXXXXXXxX");
log.info("=> CommonRepository - Total number of rows returned: ");
log.info("XXXXXXXXXXX");
log.info("\n\n\n");
return s.getResultList();
}
public <T extends Object> int count(Class<T> type, Map<String, String> filters){
CriteriaBuilder cb = entityManager.getCriteriaBuilder();
CriteriaQuery<Long> cq = cb.createQuery(Long.class);
Root<T> root=cq.from(type);
// Filtering
Predicate filterCondition = cb.conjunction();
String wildCard = "%";
for (Map.Entry<String, String> filter : filters.entrySet()) {
String[] filterField = filter.getKey().split("\\.", 2);
Path path = filterField.length == 1 ? root.get(filterField[0]): root.join(filterField[0]).get(filterField[1]);
filterCondition = cb.and(filterCondition, filter.getValue().matches("[0-9]+")
? cb.equal(path, Long.valueOf(filter.getValue()))
: cb.like(path, wildCard + filter.getValue() + wildCard));
}cq.where(filterCondition);
cq.select(cb.count(root));
return entityManager.createQuery(cq).getSingleResult().intValue();
}

Large Numbers as keys in Dictionaries (ActionScript 3)

I am using a Dictionary where the keys are Number objects, but I run into unexpected problems if the keys are large integers.
Here is some example code that inserts two key-value pairs in a Dictionary, one with a small key and one with a large one:
var dictionary:Dictionary = new Dictionary();
var smallNumberKey:Number = 1;
dictionary[smallNumberKey] = "some value";
var largeNumberKey:Number = 0x10000000;
dictionary[largeNumberKey] = "some value";
for (var key:Object in dictionary) {
trace("Key: " + key);
trace("Key type: " + flash.utils.getQualifiedClassName(key));
trace("Key as Number: " + (key as Number));
}
This gives the following output:
Key: 1
Key type: int
Key as Number: 1
Key: 268435456
Key type: String
Key as Number: null
None of the keys seems to be stored as a Number. Why is that? The first one is stored as an int, which can be converted to a Number. The second one however seems to be stored as a String, which makes no sense to me. What is going on here?
You can't use a Number as a key, because binary floating-point numbers do not lend themselves well to exact comparison, which is exactly what the Dictionary class does. (is "uses strict equality (===) for key comparison"). Read up on floating point numbers to understand why.
So when you assign a Number as a key to a Dictionary, the Flash engine must convert it to some other thing which can be represented exactly. String and int values can both be represented exactly by their contents, so Flash picks one to convert the supplied Number to. Doing some experimenting, it seems flash always converts an whole number value (even if supplied as a string) to an int if less than or equal to 0xFFFFFFF, and always converts to a String any whole number greater than that, probably due to some internal optimization in the fastest way to compare keys:
import flash.utils.Dictionary;
import flash.utils.getQualifiedClassName;
var dictionary:Dictionary = new Dictionary();
var stringKey:String = "123456789";
var intKey:int = 0xFFFFFFF;
var intKey2:int = intKey + 1;
dictionary[stringKey] = "Value 1";
dictionary[intKey] = "Value 2";
dictionary[intKey2] = "Value 3";
for (var key:Object in dictionary) {
trace( key + " [" + getQualifiedClassName(key) + "] = " + dictionary[key]);
}
Prints:
123456789 [int] = Value 1
268435455 [int] = Value 2
268435456 [String] = Value 3
From the Adobe doc:
The Dictionary class lets you create a dynamic collection of properties, which uses strict equality (===) for key comparison. When an object is used as a key, the object's identity is used to look up the object, and not the value returned from calling toString() on it.
"Using the object's identity" means using a reference to the object, a pointer to the place in memory where the actual object is stored. The problem when using primitive values as dictionary keys is that they are never passed by reference, and thus the values are not stored in the actual dictionary (i.e. they are not references to an object), but rather treated as "regular" dynamic properties, just like the ones you would expect from an Object. When you set
object[key] = "some value";
the key values are automatically converted either to int(just like the index of an Array) or String.
You can get around this, and use the actual performance benefits of Dictionary, by using a simple NumberKey wrapper class:
package
{
import flash.display.Sprite;
import flash.utils.Dictionary;
public class SimpleTest extends Sprite
{
var dictionary:Dictionary = new Dictionary();
public function SimpleTest() {
var smallNumberKey:NumberKey = new NumberKey(1);
dictionary[smallNumberKey] = "some value";
var largeNumberKey:NumberKey = new NumberKey(0x10000000);
dictionary[largeNumberKey] = "some value";
for(var key:Object in dictionary) {
trace( "Key: "+key );
trace( "Key type: "+flash.utils.getQualifiedClassName( key ) );
trace( "Key as Number: "+(key.numberValue) );
}
}
}
}
package {
public class NumberKey {
public var numberValue : Number;
public function NumberKey( n:Number ) {
numberValue = n;
}
}
}