Code that I writed below stopping compliling before contructor or #Before (depend of hiding). There is no errors and It can't run even one time.
I did it with tutorial:
https://www.tutorialspoint.com/junit/junit_parameterized_test.htm
Can somebody have idea what is wrong with this code?
import static org.junit.Assert.*;
import java.io.File;
import java.io.FileNotFoundException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
import java.util.Scanner;
import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
#RunWith(Parameterized.class)
public class ParametryzowaneTestyKarty {
private ArrayList<Karta> talia;
private String wynik;
private karty kartyy;
#Before
public void initialize() {
kartyy = new karty();
}
public ParametryzowaneTestyKarty(ArrayList<Karta> talia, String wynik) {
this.talia = talia;
this.wynik = wynik;
}
#Parameterized.Parameters
public static Collection wyniki() throws FileNotFoundException {
File plik22 = new File("...");
Scanner test = new Scanner(plik22);
while(test.hasNextLine()) {
ArrayList<Karta> talia = new ArrayList<>();
String wiersz = test.nextLine();
String[] parts = wiersz.split(",");
for(int i=0;i<10;i+=2) {
String part0 = parts[i];
String part1 = parts[i+1];
int kol=Integer.parseInt(part0);
int fig=Integer.parseInt(part1);
Kolor[] k = Kolor.values();
Kolor ko=k[kol];
Figura[] f = Figura.values();
Figura fi = f[fig];
talia.add(new Karta(ko, fi));
String w = parts[10];
Arrays.asList(new Object[][] {
{ talia, w },
});
}
}
return Arrays.asList();
}
#Test
public void TestParametryzowaneKarty() {
System.out.println("1");
System.out.println("Karty : " + talia);
assertEquals(wynik,
karty.check(talia));
}
}
It would help to know the exact error you are getting.
There are some issues with your code, as Arrays.asList() doesn't do what you are expecting, and as thus the method public static Collection wyniki() is returning a empty list.
The following code might fix the issue, but I doubt it as the talia list is reused for each row in the file that is being read.
#Parameterized.Parameters
public static Collection wyniki() throws FileNotFoundException {
File plik22 = new File("...");
Scanner test = new Scanner(plik22);
while(test.hasNextLine()) {
ArrayList<Karta> talia = new ArrayList<>();
ArrayList<Object[]> rows = new ArrayList<>();
String wiersz = test.nextLine();
String[] parts = wiersz.split(",");
for(int i=0;i<10;i+=2) {
String part0 = parts[i];
String part1 = parts[i+1];
int kol=Integer.parseInt(part0);
int fig=Integer.parseInt(part1);
Kolor[] k = Kolor.values();
Kolor ko=k[kol];
Figura[] f = Figura.values();
Figura fi = f[fig];
talia.add(new Karta(ko, fi));
String w = parts[10];
// new code
rows.add(new Object[]{talia, w} );
}
}
return rows;
}
i have some problems during writing my mapreduce funtions.
I want to solve the following problem:
I have a JSON file with 1mio JSONObject like this:
{"_id":3951,"title":"Two Family House (2000)","genres":["Drama"],"ratings":[{"userId":173,"rating":5},{"userId":195,"rating":5},{"userId":411,"rating":4},{"userId":593,"rating":2},{"userId":629,"rating":3},{"userId":830,"rating":3},{"userId":838,"rating":5},{"userId":850,"rating":4},{"userId":856,"rating":4},{"userId":862,"rating":5},{"userId":889,"rating":1},{"userId":928,"rating":5},{"userId":986,"rating":4},{"userId":1001,"rating":5},{"userId":1069,"rating":3},{"userId":1168,"rating":3},{"userId":1173,"rating":2},{"userId":1242,"rating":3},{"userId":1266,"rating":5},{"userId":1331,"rating":5},{"userId":1417,"rating":5},{"userId":1470,"rating":4},{"userId":1474,"rating":5},{"userId":1615,"rating":3},{"userId":1625,"rating":4},{"userId":1733,"rating":4},{"userId":1799,"rating":4},{"userId":1865,"rating":5},{"userId":1877,"rating":5},{"userId":1897,"rating":5},{"userId":1946,"rating":4},{"userId":2031,"rating":4},{"userId":2129,"rating":2},{"userId":2353,"rating":4},{"userId":2986,"rating":4},{"userId":3940,"rating":4},{"userId":3985,"rating":3},{"userId":4025,"rating":5},{"userId":4727,"rating":3},{"userId":5333,"rating":3}]}
and more....
One JSON Object is a Movie, which contains a array ratings. I want to count all ratings in the JSON File.
I created a Maven Proct in IntelliJ with the dependencys for Hadoop and JSON Parser. My MapReduce Class is this:
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapred.OutputCollector;
import org.apache.hadoop.mapred.Reporter;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.Mapper;
import org.apache.hadoop.mapreduce.Reducer;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
import org.json.simple.JSONArray;
import org.json.simple.JSONObject;
import org.json.simple.parser.JSONParser;
import org.json.simple.parser.ParseException;
import java.io.IOException;
import java.util.Iterator;
public class RatingCounter {
public static class RatingMapper extends Mapper<JSONObject, Text, Text, Text>{
private Text id = new Text();
private Text ratingAnzahl = new Text();
public void map(LongWritable key, Text value, OutputCollector<Text, Text> output, Reporter reporter) throws IOException{
JSONParser parser = new JSONParser();
try {
Object obj = parser.parse(value.toString());
JSONObject jsonObject = (JSONObject) obj;
String movieId = (String) jsonObject.get("_id");
int count = 0;
// loop array
JSONArray ratings = (JSONArray) jsonObject.get("ratings");
Iterator<String> iterator = ratings.iterator();
while (iterator.hasNext()) {
count++;
}
} catch (ParseException e) {
e.printStackTrace();
}
}
}
public static class RatingReducer extends Reducer<Text, Text, Text, Text> {
public void reduce(Text key, Iterable<Text> values, Context context) throws IOException, InterruptedException {
Text resultValue = new Text();
int allRatings = 0;
while (values.hasNext()){
allRatings += Integer.parseInt(values.toString());
}
resultValue.set(""+allRatings);
context.write(key, resultValue);
}
}
public static void main (String[] args) throws Exception {
Configuration conf = new Configuration();
Job job = new Job(conf, "ratings count");
job.setJarByClass(RatingCounter.class);
job.setMapperClass(RatingMapper.class);
job.setReducerClass(RatingReducer.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(Text.class);
FileInputFormat.addInputPath(job, new Path(args[0]));
FileOutputFormat.setOutputPath(job, new Path(args[1]));
System.exit(job.waitForCompletion(true) ? 0 : 1);
}
}
I have no idea, how I can write the functions in Mapper and Reducer. Can someone help me pls?
I've made a few changes to your mapper and reducer.
First, for your mapper, you are not writing the output anywhere and your syntax while extending the Mapper class is also wrong(arguably). The first input to any mapper is a LongWritable (or Object type) offset of line. You can notice the changes below
public static class RatingMapper extends Mapper<LongWritable, Text, Text, IntWritable>{
public void map(LongWritable key, Text value, Context context) throws IOException, ParseException{
JSONParser parser = new JSONParser();
Object obj = parser.parse(value.toString());
JSONObject jsonObject = (JSONObject) obj;
String movieId = (String) jsonObject.get("_id");
JSONArray ratings = (JSONArray) jsonObject.get("ratings");
context.write(new Text(movieId), new IntWritable(ratings.size()) );
}
}
Notice here, the output of map is written using context.write
Now, coming onto your Reducer some things will change because of the changes I made in the mapper. Also, since your Number of Ratings will always be an integer, you don't need to convert it to Text, use parseInt and then convert to Text again.
public static class RatingReducer extends Reducer<Text, IntWritable, Text, IntWritable> {
public void reduce(Text key, Iterable<IntWritable> values, Context context) throws IOException, InterruptedException {
int allRatings = 0;
while (values.hasNext()){
allRatings += value.get();
}
context.write(key, new IntWritable(resultValue));
}
}
I created a restful service witch generates JSON with the GSON api, but i need table name in front of the JSON structure, and i can't show this, let me show the codes
package webService;
import java.util.ArrayList;
import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import com.google.gson.Gson;
import model.AccessManager;
import dto.Usuarios;
#Path("/UsuariosService")
public class UsuariosService
{
#GET
#Path("/usuarios")
#Produces("application/json")
public String usuarios()
{
String usuarios = null;
ArrayList<Usuarios> usuariosList = new ArrayList<Usuarios>();
try
{
usuariosList = new AccessManager().getUsuarios();
Gson gson = new Gson();
usuarios = gson.toJson(usuariosList);
} catch (Exception e)
{
e.printStackTrace();
}
return usuarios;
}
}
The return I need is:
{
"usuarios" : [
{"usr_id":1,"usr_login":"teste#gmail.com","usr_pwd":"123456"},
{"usr_id":2,"usr_login":"teste#teste.com.br","usr_pwd":"123456"}
]
}
But the return I get is:
[
{"usr_id":1,"usr_login":"teste#gmail.com","usr_pwd":"123456"},
{"usr_id":2,"usr_login":"teste#teste.com.br","usr_pwd":"123456"}
]
i.e. without the table's name, but that name is needed in my SAPUI5 Application
What stops you from just adding that as String
"{\"usuarios\" :" + gson.toJson(usuariosList) + "}"
Although you need to escape "
Otherwise you have to introduce a different return object
class ReturnObject {
List<Usuarios> usuarios;
public ReturnObject(List<Usuarios> usuarios) {
this.usuarios = usuarios;
}
}
And use that
gson.toJson(new ReturnObject(usuariosList));
My class:
class ExampleBean {
private String _firstField;
private String _secondField;
// respective getters and setters
}
I want to appear as follows:
{
"FirstField":"value",
"SecondField":"value"
}
And not like this
{
"_FirstField":"value",
"_SecondField":"value"
}
I initialize the parser as follows:
GsonBuilder builder = new GsonBuilder();
builder.setDateFormat(DateFormat.LONG);
builder.setFieldNamingPolicy(FieldNamingPolicy.UPPER_CAMEL_CASE);
builder.setPrettyPrinting();
set_defaultParser(builder.create());
I could see the API and in the documentation of "FieldNamePolicy" but I am surprised that not give the option to skip "_"
I also know I can use the annotation...
# SerializedName (" custom_naming ")
...but do not want to have to write this for alllllll my fields ...
It's very useful for me to distinguish between local variables and fields of a class. :( Any Idea?
EDIT: There would be many obvious solutions, (inheritance, gson overwriting methods, regular expresions). My question is more focused on whether there is a native solution of gson or a less intrusive fix?
Maybe we could propose as new FieldNamePolicy?
GsonBuilder provides a method setFieldNamingStrategy() that allows you to pass your own FieldNamingStrategy implementation.
Note that this replaces the call to setFieldNamingPolicy() - if you look at the source for GsonBuilder these two methods are mutually exclusive as they set the same internal field (The FieldNamingPolicy enum is a FieldNamingStrategy).
public class App
{
public static void main(String[] args)
{
Gson gson = new GsonBuilder()
.setFieldNamingStrategy(new MyFieldNamingStrategy())
.setPrettyPrinting()
.create();
System.out.println(gson.toJson(new ExampleBean()));
}
}
class ExampleBean
{
private String _firstField = "first field value";
private String _secondField = "second field value";
// respective getters and setters
}
class MyFieldNamingStrategy implements FieldNamingStrategy
{
public String translateName(Field field)
{
String fieldName =
FieldNamingPolicy.UPPER_CAMEL_CASE.translateName(field);
if (fieldName.startsWith("_"))
{
fieldName = fieldName.substring(1);
}
return fieldName;
}
}
Output:
{
"FirstField": "first field value",
"SecondField": "second field value"
}
What you want is
import java.lang.reflect.Field;
import java.text.DateFormat;
import com.google.gson.FieldNamingStrategy;
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
public class GsonExample {
public static void main(String... args) throws Exception {
final GsonBuilder builder = new GsonBuilder();
builder.setDateFormat(DateFormat.LONG);
builder.setPrettyPrinting();
builder.setFieldNamingStrategy(new FieldNamingStrategy() {
#Override
public String translateName(Field f) {
String fieldName = f.getName();
if(fieldName.startsWith("_") && fieldName.length() > 1) {
fieldName = fieldName.substring(1, 2).toUpperCase() + fieldName.substring(2);
}
return fieldName;
}
});
final Gson gson = builder.create();
System.out.println(gson.toJson(new ExampleBean("example", "bean")));
}
private static class ExampleBean {
private final String _firstField;
private final String _secondField;
private ExampleBean(String _firstField, String _secondField) {
this._firstField = _firstField;
this._secondField = _secondField;
}
}
}
which generates
{"FirstField":"example","SecondField":"bean"}
Is there a way to register a global property filter in ObjectMapper?
Global means that it will be applied to all serialized beans. I can't use annotations (I can't modify serialized beans) and don't know what properties the beans have.
The filtering should be name based.
My first idea was to write a custom serializer, but I don't know what should I pass to the constructor.
I'd make use of a FilterProvider. It's a little involved, but not too unwieldy.
import org.codehaus.jackson.annotate.JsonAutoDetect.Visibility;
import org.codehaus.jackson.annotate.JsonMethod;
import org.codehaus.jackson.map.ObjectMapper;
import org.codehaus.jackson.map.ObjectWriter;
import org.codehaus.jackson.map.annotate.JsonFilter;
import org.codehaus.jackson.map.ser.FilterProvider;
import org.codehaus.jackson.map.ser.impl.SimpleBeanPropertyFilter;
import org.codehaus.jackson.map.ser.impl.SimpleFilterProvider;
public class JacksonFoo
{
public static void main(String[] args) throws Exception
{
Bar bar = new Bar();
bar.id = "42";
bar.name = "James";
bar.color = "blue";
bar.foo = new Foo();
bar.foo.id = "7";
bar.foo.size = "big";
bar.foo.height = "tall";
ObjectMapper mapper = new ObjectMapper().setVisibility(JsonMethod.FIELD, Visibility.ANY);
System.out.println(mapper.writeValueAsString(bar));
// output:
// {"id":"42","name":"James","color":"blue","foo":{"id":"7","size":"big","height":"tall"}}
String[] ignorableFieldNames = { "id", "color" };
FilterProvider filters = new SimpleFilterProvider().addFilter("filter properties by name", SimpleBeanPropertyFilter.serializeAllExcept(ignorableFieldNames));
mapper = new ObjectMapper().setVisibility(JsonMethod.FIELD, Visibility.ANY);
mapper.getSerializationConfig().addMixInAnnotations(Object.class, PropertyFilterMixIn.class);
ObjectWriter writer = mapper.writer(filters);
System.out.println(writer.writeValueAsString(bar));
// output:
// {"name":"James","foo":{"size":"big","height":"tall"}}
}
}
#JsonFilter("filter properties by name")
class PropertyFilterMixIn
{
}
class Bar
{
String id;
String name;
String color;
Foo foo;
}
class Foo
{
String id;
String size;
String height;
}
For other approaches and more information, I recommend the following resources.
http://wiki.fasterxml.com/JacksonJsonViews
http://www.cowtowncoder.com/blog/archives/2011/02/entry_443.html
http://wiki.fasterxml.com/JacksonFeatureJsonFilter
http://www.cowtowncoder.com/blog/archives/2011/09/entry_461.html