JSONObject and Streams/Lambda - json

I'm trying to get more familiar with Java lambda, can do some streams and such but still a lot to learn.
Got this simple code using JSONObject and JSONArray (org.json.simple with this exact library and no other because Gson is too easy :P) is there a way to simplify the code with java lambda/streams? (I tried with no luck)
JSONArray jsonArray = (JSONArray) jsonObject.get("someData");
Iterator<JSONObject> iterator = jsonArray.iterator();
double total = 0;
while(iterator.hasNext()) {
JSONObject iteratedJson = iterator.next();
// iteratedJson.get("ip") = "101.99.99.101" example values
String ip = (String) iteratedJson.get("ip");
// Need only first octet
ip = ip.substring(0, ip.indexOf("."));
if (Integer.valueOf(ip) >= 1 && Integer.valueOf(ip) <= 100) {
// Another object inside the array object
JSONObject locationObject = (JSONObject) iteratedJson.get("location");
// Id is int but JSONObject don't let me parse int...
long locationId = (Long) locationObject.get("id");
if (locationId == 8) {
// iteratedJson.get("amount") = "$1,999.10" example values
Number number = NumberFormat.getCurrencyInstance(Locale.US).parse((String)iteratedJson.get("amount"));
// Don't need a lot of precission
total = total + number.doubleValue();
}
}
}

You can do like this:
first of all to extract data from JsonObject I've created a class. this class takes a JosonObject as an argument and extract its values as bellow.
class ExtractData {
Integer ip;
long id;
double amount;
public ExtractData(JSONObject jsonObject) {
this.ip = Integer.valueOf(jsonObject.get("ip").toString().split("\\.")[0]);
this.id = Long.parseLong(((JSONObject) jsonObject.get("location")).get("id").toString());
try {
this.amount = NumberFormat.getCurrencyInstance(Locale.US)
.parse((String) jsonObject.get("amount")).doubleValue();
} catch (ParseException e) {
this.amount = 0d;
}
}
// getter&setter
}
then you can use stream API to calculate the sum of the amount property.
jsonArray.stream()
.map(obj -> new ExtractData((JSONObject) obj))
.filter(predicate)
.mapToDouble(value -> ((ExtractData) value).getAmount())
.sum();
for simplifying I've extracted filter operation.
Predicate<ExtractData> predicate = extractData ->
extractData.getIp()>=1 && extractData.getIp()<=100 && extractData.getId() == 8;

Related

Accessing nested JSON in Processing

I need to extract 3 values from a JSON file. I just managed to get all data but I can't separate them.
The file looks like this:
{
"motionAndDeviceRelated":{
"mOrientation":[0,0,0],
"mLocalVelocity":[0,0,0],
"mWorldVelocity":[0,0,0],
"mAngularVelocity":[0,0,0],
"mLocalAcceleration":[0,0,0],
"mWorldAcceleration":[0,0,0],
"mExtentsCentre":[0,0,0]
}
}
I need to extract in float variables data from mLocalAcceleration.
import http.requests.*;
JSONObject json;
void setup() {
size(600,600);
}
void draw() {
// actually returns something instead of throwing an error, which is progress
long t = System.currentTimeMillis();
GetRequest get = new GetRequest("http://" + "localhost:8180/crest2/v1/api?motionDeviceRelated=true&formatted=true");
get.send();
JSONObject json = parseJSONObject(get.getContent());
println(json);
println(System.currentTimeMillis() - t);
loop();
}
Here is the code that works:
`import http.requests.*;
JSONObject json;
float x;
float y;
float z;
void setup() {
size(800,800);
}
void draw() {
long t = System.currentTimeMillis();
GetRequest get = new GetRequest("http://" + "localhost:8180/crest2/v1/api?motionDeviceRelated=true&formatted=true");
get.send();
JSONObject json = parseJSONObject(get.getContent());
JSONObject motion = json.getJSONObject("motionAndDeviceRelated");
JSONArray acc = motion.getJSONArray("mWorldAcceleration");
x = acc.getFloat(0);
z = acc.getFloat(1);
y = acc.getFloat(2);
println(System.currentTimeMillis() - t);
loop();
}`

Displaying data from SQLite table's columns, one of which holds an array

I managed to retrieve the SQLite table with only the first item of the array and put it in the UI's TextView. Couldn't get all the of the array's items. From each of the rest of the columns, a single value is returned successfully.
The JSON is parsed and passed as a parcelable ArrayList to a Fragment where it's presented in a list. Clicking on a list item directs to another Fragment where all the of item's details are presented.
I've been trying to write a for loop that returns the Strings in the array into the TextView, but the condition i < genresList.size() is always false. I tried using a while loop, but it returns only the first item of the list.
Various ways I've found on the internet didn't work.
Thanks.
Parsing and insertion to SQLite
private void parseJsonAndInsertToSQLIte(SQLiteDatabase db) throws JSONException {
// parsing the json
String jsonString = getJsonFileData();
JSONArray moviesArray = new JSONArray(jsonString);
ContentValues insertValues;
for (int i = 0; i < moviesArray.length(); i++) {
JSONObject jsonObject = moviesArray.getJSONObject(i);
String title = jsonObject.getString("title");
String imageUrl = jsonObject.getString("image");
String rating = jsonObject.getString("rating");
String releaseYear = jsonObject.getString("releaseYear");
JSONArray genresArray = jsonObject.getJSONArray("genre");
List<String> genres = new ArrayList<>();
for (int k = 0; k < genresArray.length(); k++) {
genres.add(genresArray.getString(k));
}
insertValues = new ContentValues();
insertValues.put(Movie.TITLE, title);
insertValues.put(Movie.IMAGE_URL, imageUrl);
insertValues.put(Movie.RATING, rating);
insertValues.put(Movie.RELEASE_YEAR, releaseYear);
for (int k = 0; k < genresArray.length(); k++) {
insertValues.put(Movie.GENRE, genres.get(k));
}
Log.i(TAG, "insertValues: " + genresArray);
long res = db.insert(TABLE_NAME, null, insertValues);
Log.i(TAG, "parsed and inserted to sql - row: " + res);
}
}
The item's details Fragment
public class MovieDetailsFragment extends Fragment{
... variables declarations come here...
#Nullable
#Override
public View onCreateView(#NotNull LayoutInflater inflater, #Nullable ViewGroup container, Bundle savedInstanceState) {
View rootView = inflater.inflate(R.layout.fragment_details_movie, container, false);
Context context = getActivity();
Bundle idBundle = getArguments();
if (idBundle != null) {
movieId = getArguments().getInt("id");
}
getDatabase = new GetDatabase(context);
getDatabase.open();
Cursor cursor = getDatabase.getMovieDetails(movieId);
... more irelevant code comes here...
titleView = rootView.findViewById(R.id.movieTtlId);
ratingView = rootView.findViewById(R.id.ratingId);
releaseYearView = rootView.findViewById(R.id.releaseYearId);
genreView = rootView.findViewById(R.id.genreID);
String titleFromSQLite = cursor.getString(cursor.getColumnIndex(Movie.TITLE));
String ratingFromSQLite = cursor.getString(cursor.getColumnIndex(Movie.RATING));
String releaseYearFromSQLite = cursor.getString(cursor.getColumnIndex(Movie.RELEASE_YEAR));
String genreFromSQLite;
if(cursor.moveToFirst()) {
do {
genreFromSQLite = cursor.getString(cursor.getColumnIndex(Movie.GENRE));
genres.add(genreFromSQLite);
} while (cursor.moveToNext());
}
else{
genreFromSQLite = cursor.getString(cursor.getColumnIndex(Movie.RELEASE_YEAR));
}
getDatabase.close();
//more irelevant code comes here
genreView.setText(genreFromSQLite);
genreView.setFocusable(false);
genreView.setClickable(false);
return rootView;
}
}
The method that returns the table from SQLite:
public ArrayList<Movie> getMovies() {
String[] columns = {
Movie.ID,
Movie.TITLE,
Movie.IMAGE_URL,
Movie.RATING,
Movie.RELEASE_YEAR,
Movie.GENRE
};
// sorting orders
String sortOrder =
Movie.RELEASE_YEAR + " ASC";
ArrayList<Movie> moviesList = new ArrayList<>();
Cursor cursor = db.query(TABLE_NAME, //Table to query
columns,
null,
null,
null,
null,
sortOrder);
if (cursor.moveToFirst()) {
do {
Movie movie = new Movie();
movie.setMovieId(Integer.parseInt(cursor.getString(cursor.getColumnIndex(Movie.ID))));
movie.setTitle(cursor.getString(cursor.getColumnIndex(Movie.TITLE)));
movie.setImageUrl(cursor.getString(cursor.getColumnIndex(Movie.IMAGE_URL)));
movie.setRating(cursor.getDouble(cursor.getColumnIndex(Movie.RATING)));
movie.setReleaseYear(cursor.getInt(cursor.getColumnIndex(Movie.RELEASE_YEAR)));
List<String> genreArray = new ArrayList<>();
while(cursor.moveToNext()){
String genre = cursor.getString(cursor.getColumnIndex(Movie.GENRE));
genreArray.add(genre);
}
movie.setGenre(Collections.singletonList(String.valueOf(genreArray)));
// Adding a movie to the list
moviesList.add(movie);
} while (cursor.moveToNext());
}
Log.d(TAG, "The movies list from sqlite: " + moviesList);
cursor.close();
db.close();
return moviesList;
}
I believe your issue is with :-
for (int k = 0; k < genresArray.length(); k++) {
insertValues.put(Movie.GENRE, genres.get(k));
}
That will result in just the last value in the loop being inserted as the key/column name (first parameter of the put) does not change (and probably can't as you only have the one column).
You could use :-
StringBuilder sb = new StringBuilder();
for (int k = 0; k < genresArray.length(); k++) {
if (k > 0) {
sb.append(",");
}
sb.append(genres.get(k));
}
insertValues.put(Movie.GENRE, sb.toString());
Note the above code is in-principle code. It has not been tested or run and may therefore contains errors.
This would insert all the data as a CSV into the GENRE column.
BUT that is not a very good way as far as utilising databases. It would be far better if the Genre's were a separate table and probably that a mapping table were used (but that should be another question).
This is going to cause you issues as well :-
if (cursor.moveToFirst()) {
do {
Movie movie = new Movie();
movie.setMovieId(Integer.parseInt(cursor.getString(cursor.getColumnIndex(Movie.ID))));
movie.setTitle(cursor.getString(cursor.getColumnIndex(Movie.TITLE)));
movie.setImageUrl(cursor.getString(cursor.getColumnIndex(Movie.IMAGE_URL)));
movie.setRating(cursor.getDouble(cursor.getColumnIndex(Movie.RATING)));
movie.setReleaseYear(cursor.getInt(cursor.getColumnIndex(Movie.RELEASE_YEAR)));
List<String> genreArray = new ArrayList<>();
while(cursor.moveToNext()){
String genre = cursor.getString(cursor.getColumnIndex(Movie.GENRE));
genreArray.add(genre);
}
movie.setGenre(Collections.singletonList(String.valueOf(genreArray)));
// Adding a movie to the list
moviesList.add(movie);
} while (cursor.moveToNext());
That is you move to the first row of the Cursor, extract some data MoveieId,Title ... ReleaseYear.
Then
a) if there any other rows you move to the next (which would be for a different Movie) and the next until you finally reached the last row adding elements to the genreArray.
or
b) If there is only the one row in the Cursor genreArray is empty.
You then add the 1 and only movie to the movieList and return.
1 move (row) in the Cursor will exist per movie and there is only the 1 GENRE column per movie. You have to extract the data in that column and then split the data into the genreArray without moving (see the previous fix that will create a CSV (note that would be messed up if the data contained commas)).
IF you used the previous fix and store the multiple genres as a CSV, then you could use :-
if (cursor.moveToFirst()) {
do {
Movie movie = new Movie();
movie.setMovieId(Integer.parseInt(cursor.getString(cursor.getColumnIndex(Movie.ID))));
movie.setTitle(cursor.getString(cursor.getColumnIndex(Movie.TITLE)));
movie.setImageUrl(cursor.getString(cursor.getColumnIndex(Movie.IMAGE_URL)));
movie.setRating(cursor.getDouble(cursor.getColumnIndex(Movie.RATING)));
movie.setReleaseYear(cursor.getInt(cursor.getColumnIndex(Movie.RELEASE_YEAR)));
List<String> genreArray = new List<>(Arrays.asList((cursor.getString(cursor.getColumnIndex(Movie.GENRE))).split(",",0)));
movie.setGenre(Collections.singletonList(String.valueOf(genreArray)));
// Adding a movie to the list
moviesList.add(movie);
} while (cursor.moveToNext());
Note the above code is in-principle code. It has not been tested or run and may therefore contains errors.

How the performance of a JavaFx-MySQL application can be enhanced

In my JavaFx application, i'm loading an ObservableList when a button is clicked and then display the list in a table.
the controller code:
#FXML
private void initialize() throws SQLException, ParseException, ClassNotFoundException {
searchChoice.setItems(criteriaList);
searchChoice.getSelectionModel().selectFirst();
productIdColumn.setCellValueFactory(cellData -> cellData.getValue().productIdProperty());
unitColumn.setCellValueFactory(cellData -> cellData.getValue().unitProperty());
productTitleColumn.setCellValueFactory(cellData -> cellData.getValue().titleProperty());
productTypeColumn.setCellValueFactory(cellData -> cellData.getValue().typeProperty());
productUnitPriceColumn.setCellValueFactory(cellData -> Bindings.format("%.2f", cellData.getValue().unitPriceProperty().asObject()));
productQuantityColumn.setCellValueFactory(cellData -> cellData.getValue().quantityProperty().asObject());
productStatusColumn.setCellValueFactory(cellData -> cellData.getValue().productStatusProperty());
descriptionColumn.setCellValueFactory(cellData -> cellData.getValue().descriptionProperty());
reorderPointColumn.setCellValueFactory(cellData -> cellData.getValue().reOrderPointProperty().asObject());
surplusPointColumn.setCellValueFactory(cellData -> cellData.getValue().surplusPointProperty().asObject());
productIdColumn.setSortType(TableColumn.SortType.DESCENDING);
productTable.getSortOrder().add(productIdColumn);
productTable.setRowFactory(tv -> new TableRow<Product>() {
#Override
public void updateItem(Product item, boolean empty) {
super.updateItem(item, empty);
if (item == null) {
setStyle("");
} else if (item.getQuantity() < item.getReOrderPoint()) {
setStyle("-fx-background-color: tomato;");
} else if (item.getQuantity() > item.getSurplusPoint()) {
setStyle("-fx-background-color: darkorange;");
} else {
setStyle("-fx-background-color: skyblue;");
}
}
});
try {
ObservableList<Product> productData = ProductDAO.searchProducts();
populateProducts(productData);
String[] expireDate = new String[productData.size()];
String[] id = new String[productData.size()];
String[] existingStatus = new String[productData.size()];
for (int i = 0; i < productData.size(); i++) {
expireDate[i] = productData.get(i).getExpireDate();
id[i] = productData.get(i).getProductId();
existingStatus[i] = productData.get(i).getProductStatus();
DateFormat format = new SimpleDateFormat(app.values.getProperty("DATE_FORMAT_PATTERN"), Locale.ENGLISH);
Date expireDateString = format.parse(expireDate[i]);
Date in = new Date();
LocalDateTime ldt = LocalDateTime.ofInstant(in.toInstant(), ZoneId.systemDefault());
Date today = Date.from(ldt.atZone(ZoneId.systemDefault()).toInstant());
if (expireDateString.before(today) && !existingStatus[i].equals(app.values.getProperty("STATUS_TYPE2"))) {
ProductDAO.updateProductStatus(id[i], app.values.getProperty("STATUS_TYPE3"));
}
if (expireDateString.after(today) && !existingStatus[i].equals(app.values.getProperty("STATUS_TYPE2"))) {
ProductDAO.updateProductStatus(id[i], app.values.getProperty("STATUS_TYPE1"));
}
}
ObservableList<Product> productDataRefreshed = ProductDAO.searchProducts();
populateProducts(productDataRefreshed);
ObservableList<Product> productCodesData = ProductDAO.getProductCodes();
ObservableList<Product> productTitlesData = ProductDAO.getProductTitles();
ObservableList<Product> productTypesData = ProductDAO.getProductTypes();
ObservableList<Product> productStatusData = ProductDAO.getProductStatus();
String possibleProducts1[] = new String[productCodesData.size()];
for (int k = 0; k < productCodesData.size(); k++) {
possibleProducts1[k] = productCodesData.get(k).getProductId();
}
String possibleProducts2[] = new String[productTitlesData.size()];
for (int k = 0; k < productTitlesData.size(); k++) {
possibleProducts2[k] = productTitlesData.get(k).getTitle();
}
String possibleProducts3[] = new String[productTypesData.size()];
for (int k = 0; k < productTypesData.size(); k++) {
possibleProducts3[k] = productTypesData.get(k).getType();
}
String possibleProducts4[] = new String[productStatusData.size()];
for (int k = 0; k < productStatusData.size(); k++) {
possibleProducts4[k] = productStatusData.get(k).getProductStatus();
}
TextFields.bindAutoCompletion(searchField, possibleProducts1);
TextFields.bindAutoCompletion(searchField, possibleProducts2);
TextFields.bindAutoCompletion(searchField, possibleProducts3);
TextFields.bindAutoCompletion(searchField, possibleProducts4);
} catch (SQLException e) {
Alert alert = new Alert(Alert.AlertType.ERROR);
alert.setTitle(app.values.getProperty("ERROR_TITLE"));
alert.setHeaderText(app.values.getProperty("FAILURE_MESSAGE"));
alert.setHeaderText(app.values.getProperty("ERROR_GETTING_INFORMATION_FROM_DATABASE_MESSAGE"));
alert.showAndWait();
throw e;
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
}
the service mysql query :
public static ObservableList<Product> searchProducts() throws SQLException, ClassNotFoundException {
String selectStmt = "SELECT * FROM product";
ResultSet rsPrdcts = DbUtil.dbExecuteQuery(selectStmt);
ObservableList<Product> productList = getProductList(rsPrdcts);
return productList;
}
The issue here is, when there are more than 200-300 items in the list the scene gets really slow to load. What countermeasures can I take regarding this matter? Any idea will be very much appreciated.
Thanks in advance.
You need to implement an ObservableList which only retrieves the data which is rqeusted by the TableView. Currently you retrive all elements in the table and cast the retrieved list to an ObservableList.
The TableView uses the .get(int idx) method of the OL to retrieve all items which should be displayed and the .size() method for determining the size of the scrollbar. When you scroll the TableView will discard all items which are not displayed and call the get method again.
To solve your problem need to create a class which implements ObservableList<E>. First you need to implement the .get(int idx) and the .size() method, for all other methods I would throw new UnsupportedOperationException() and later on see which other method is needed. So the .size() method needs to execute the following query
SELECT count(*) FROM product
and the get(int idx) something like this
int numItems = 30;
int offset = idx - (idx % numItems)
SELECT * FROM product LIMIT offset, numItems
you can create an internal list which only holds e.g. 30 items from your db and whenever the requested idx < offset || idx > offset + numItems you issue a new db request.
I used this this approach with database tables with millions of rows and had a very performant GUI. You can also add paging to the TableView because with to many rows the scrollbar gets useless, but this is a different discussion.
edit:
I forgot to mention that this is called Lazy Loading

How to access nested JSON with array in firebase

I want to access a JSON of this structure in firebase
The structure
{
"questions":{
"English":{
"English_2002":[
{
"correct_ans":"A",
"OptionA":"a coder",
"OptionB":"a hacker",
"OptionC":"a writer",
"OptionD":"a programmer",
"Question":"Who build software"
},
{},
{}
],
"English_2003":[],
}
}
}
I want this structure. In the subject structure, other subjects will come after I exhaust 9 years of English.
My confusion is how to logically get each subject since firebase will only accept the root name questions.
Please I may sound dumb, but I have a very long questions thread almost 55000 lines. Because firebase accept one JSON tree.
Sorry i wasn't very clear i was asking from the stack phone app:
I have a question json tag of the structure above; my question is how will i be able to access the object subject like "english":{
// then accessing the first english array "english":[]
//since am now using firebase.
}
initially each array was individual json file, i have to recreate them into one for firebase sake. this is how i was parsing it then.
public class QuestionParser {
Context context;
public QuestionParser(Context c) {
this.context = c;
}
public ArrayList<Question> getJsonFromUrl(String url, String arrayName)
{
ArrayList<Question> arrayofQuestion = new ArrayList<>();
return arrayofQuestion;
}
// Processing question from JSon file in res > raw folder
public ArrayList<Question> parseQuestionJson(int rawJsonFileId, String arrayName) {
ArrayList<Question> questionList = new ArrayList<>();
String jsonstr = null;
try {
InputStream in = context.getResources().openRawResource(rawJsonFileId);
BufferedReader br = new BufferedReader(new InputStreamReader(in));
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = br.readLine()) != null) {
sb.append(line + "\n");
}
jsonstr = sb.toString();
Log.d("REEEEADDD" + this.toString(), jsonstr);
//System.out.println(jsonstr);
} catch (Exception e) {
Log.e("Buffer Error", "Error converting result " + e.toString());
}
// If the JSON string is empty or null, then return early.
if (TextUtils.isEmpty(jsonstr)) {
return null;
}
try {
JSONObject jsonObject = new JSONObject(jsonstr);
JSONArray jsonArray = jsonObject.getJSONArray(arrayName);
JSONObject jobject;
for (int i = 0; i < jsonArray.length(); i++) {
// TEST
jobject = jsonArray.getJSONObject(i);
String ans = jobject.getString("correct_answer");
String graphic_name = jobject.getString("question_image");
String optionA = jobject.getString("optiona");
String optionB = jobject.getString("optionb");
String optionC = jobject.getString("optionc");
String optionD = jobject.getString("optiond");
String questionNo = jobject.getString("question_number");
String question = jobject.getString("question");
questionList.add(new Question(questionNo, graphic_name, question, optionA, optionB, optionC, optionD, ans));
Log.d("DDD" + this.toString(), String.valueOf(questionList.get(i)));
}
Log.i("ONE QUESTION", questionList.get(50).getQuestion());
} catch (Exception e) {
Log.e("JSON Parser", "Error parsing data " + e.toString());
}
return questionList;
}
}
So how can i parse it from firebase because initially, if a student chooses question and year i passes those value as parameter and use them for parsing. but in firebase now i have access to only root firebase name in the get reference e method
To access for example "correct_ans":"A" you would query your firebase like so:
your.firebase.domain/questions/English/English_2002/0/correct_ans
Notice that each level in the json object is represented by a / and the key you want to access whereas in case of an array you simple add the array index. JSON's simple structure also allows simple REST like access

Google end point returns JSON for long data type in quotes

I am using Google cloud end point for my rest service. I am consuming this data in a GWT web client using RestyGWT.
I noticed that cloud end point is automatically enclosing a long datatype in double quotes which is causing an exception in RestyGWT when I try to convert JSON to POJO.
Here is my sample code.
#Api(name = "test")
public class EndpointAPI {
#ApiMethod(httpMethod = HttpMethod.GET, path = "test")
public Container test() {
Container container = new Container();
container.testLong = (long)3234345;
container.testDate = new Date();
container.testString = "sathya";
container.testDouble = 123.98;
container.testInt = 123;
return container;
}
public class Container {
public long testLong;
public Date testDate;
public String testString;
public double testDouble;
public int testInt;
}
}
This is what is returned as JSON by cloud end point. You can see that testLong is serialized as "3234345" rather than 3234345.
I have the following questions.
(1) How can I remove double quotes in long values ?
(2) How can I change the string format to "yyyy-MMM-dd hh:mm:ss" ?
Regards,
Sathya
What version of restyGWT are you using ? Did you try 1.4 snapshot ?
I think this is the code (1.4) responsible for parsing a long in restygwt, it might help you :
public static final AbstractJsonEncoderDecoder<Long> LONG = new AbstractJsonEncoderDecoder<Long>() {
public Long decode(JSONValue value) throws DecodingException {
if (value == null || value.isNull() != null) {
return null;
}
return (long) toDouble(value);
}
public JSONValue encode(Long value) throws EncodingException {
return (value == null) ? getNullType() : new JSONNumber(value);
}
};
static public double toDouble(JSONValue value) {
JSONNumber number = value.isNumber();
if (number == null) {
JSONString val = value.isString();
if (val != null){
try {
return Double.parseDouble(val.stringValue());
}
catch(NumberFormatException e){
// just through exception below
}
}
throw new DecodingException("Expected a json number, but was given: " + value);
}
return number.doubleValue();
}