I want to save some data about tweets in a persistent json file. I'm using org.codehaus.jackson.JsonGenerator to write the tweets to a file, which works fine when they are inserted all at once. My code:
JsonGenerator g = new JsonFactory().createJsonGenerator(jsonFile, JsonEncoding.UTF8);
for(Tweet t : list){
g.writeStartObject();
g.writeStringField(ID_FIELD, t.getTweetID());
g.writeStringField(DATE_FIELD, dateFormat.format(t.getDate()));
if(t.isRetweet())
g.writeStringField(RETWEET_FIELD, t.getOriginalTweet());
g.writeArrayFieldStart(HASHTAG_FIELD);
for(String s : t.getHashtags())
g.writeString(s);
g.writeEndArray();
g.writeEndObject();
}
g.close();
Problem is, if I want to add another bunch of tweets to the same file, this function will overwrite the previous ones. I've been trying to keep the generator open instead of closing it every time, but it screwed up the process of writing in the first place. also, the API doesn't suggest any way of "continuing where I left off". No luck with other json streamers either so far. Suggestions?
Not sure if I understand correctly, but if the problem is that you can't append multiple JSON objects to your output - then just open a FileWriter and make sure that the target isn't auto-closed after each object write.
For instance:
FileWriter output = new FileWriter("/path/to/file");
ObjectWriter jsonWriter = new ObjectMapper()
.configure(JsonGenerator.Feature.AUTO_CLOSE_JSON_CONTENT, false)
.writer();
jsonWriter.writeValue(output, object1); output.write('\n');
jsonWriter.writeValue(output, object2); output.write('\n');
jsonWriter.writeValue(output, object3); output.write('\n');
Related
I was trying to use the method found here (see most up-voted answer):
Google Apps Script Fastest way to find a row?
I currently use this while it does work I wanted to try the above linked method yet when I replace the below code
function AutoPopulate (evalue)
{
//uses google drive file irretator reads in JSON file and parses it to a Javascript object that we can work with
var iter = DriveApp.getFilesByName("units.json");
// iterate through all the files named units.json
while (iter.hasNext()) {
// define a File object variable and set the Media Tyep
var file = iter.next();
var jsonFile = file.getBlob().getDataAsString();
// log the contents of the file
//Logger.log(jsonFile);
}
var UnitDatabase = JSON.parse(jsonFile);
//Logger.log(UnitDatabase);
//Logger.log(UnitDatabase[1027]);
return UnitDatabase[evalue];
}
WITH THIS CODE:
function AutoPopulate (evalue)
{
//this method did not work for me but should have according to stackflow answer linked above I am trying to understand why or how I can find out why it may have thrown an error
var jsonFile = DriveApp.getFilesByName("units.json").next(),
UnitDatabase = UnitDatabase.getBlob().getDataAsString();
return UnitDatabase[evalue];
}
I get an error in the excecution indicating that there is a % at postion 0 in the JSON, between the methods I dont alter the JSON file in anyway so I dont understand why does the top method work but the bottom one does not?
For further information the idea behind the code is that I have a list of Unit numbers and model numbers that are in a spreadsheet. I then convert this to a JSON file, this however is only done when a new unit is added to the fleet. As I learned one can parse a whole JSON file into a javascript object which makes working with the data set much faster. This javascript object is used so that when a user enters a UNIT# the MODEL# is auto populated based on the JSON file.
I cannot share the JSON file as it contains client information.
Your code does not work for two reasons:
You have a typo in the line UnitDatabase = UnitDatabase.getBlob()... - it should be UnitDatabase = jsonFile.getBlob()...
If you want to retrieve a nested object from a json file - you need to parse the JSOn - otherwise it is considered a string and you can not access the nested structure
Modified working code:
function AutoPopulate2 (evalue)
{
var jsonFile = DriveApp.getFilesByName("units.json").next();
var UnitDatabase = JSON.parse(jsonFile.getBlob().getDataAsString());
return UnitDatabase[evalue];
}
Mind that this code will only work if you have a "units.json" file on your drive and if evalue is a valid 1st-level nested object of this json.
So im making this memory game and im trying to add a scoreboard and i want to write the data to a txt file using JSON. I got all of that to work but now i have a small issue, everytime i run my program the existing data in my txt file gets overwritten by the new data.
Here is the code that i use:
public static void Score(String gamescore, string loginname)
{
List<Highscore> Myhighscores = new List<Highscore>();
Myhighscores.Add(new Highscore { Score = gamescore, Name = loginname });
string Jstr = JsonConvert.SerializeObject(Myhighscores);
File.WriteAllText(#"c:\temp\hs.txt", Jstr);
}
does anyone know how i can keep the existing data and also write the new data into the txt file?
I figured it out my self all i needed to do is read the existing data like this:
string hs = File.ReadAllText(#"c:\temp\hs.txt");
and put it back into my list "Myhighscores" like this:
Myhighscores = JsonConvert.DeserializeObject<List<Highscore>>(hs);
No thanks to Sagar V who just bitched that i put the jquery tag in.
We are building a service. It has to read config from a file. We are currently using YAML and Jackson for deserializing the YAML. We have a situation where our YAML file needs to inherit/extend another YAML file(s). E.g., something like:
extends: base.yaml
appName: my-awesome-app
...
thus part of the config is stored in base.yaml. Is there any library that has support for this? Bonus points if it allows to inherit from more than one file. We could change to using JSON instead of YAML.
Neither JSON nor YAML have the ability to include files. Whatever you do will be a pre-processing step where you will be putting the base.yaml and your actual file together.
A crude way of doing this would be:
#include base.yaml
appName: my-awesome-app
Let this be your file. Upon loading, you first read the first line, and if it starts with #include, you replace it with the content of the included file. You need to do this recursively. This is basically what the C preprocessor does with C files and includes.
Drawbacks are:
even if both files are valid YAML, the result may not.
if either files includes a directive end or document end marker (--- or ...), you will end up with two separate documents in one file.
you cannot replace any values from base.yaml inside your file.
So an alternative would be to actually operate on the YAML structure. For this, you need the API of the YAML parser (SnakeYAML in your case) and parse your file with that. You should use the compose API:
private Node preprocess(final Reader myInput) {
final Yaml yaml = new Yaml();
final Node node = yaml.compose(myInput);
processIncludes(node);
return node;
}
private void processIncludes(final Node node) {
if (node instanceof MappingNode) {
final List<NodeTuple> values = ((MappingNode) node).getValue();
for (final NodeTuple tuple: values) {
if ("!include".equals(tuple.getKeyNode().getTag().getValue())) {
final String includedFilePath =
((ScalarNode) tuple.getValueNode()).getValue();
final Node content = preprocess(new FileReader(includedFilePath));
// now merge the content in your preferred way into the values list.
// that will change the content of the node.
}
}
}
}
public String executePreprocessor(final Reader source) {
final Node node = preprocess(source);
final StringWriter writer = new StringWriter();
final DumperOptions dOptions = new DumperOptions()
Serializer ser = new Serializer(new Emitter(writer, dOptions),
new Resolver(), dOptions, null);
ser.open();
ser.serialize(node);
ser.close();
return writer.toString();
}
This code would parse includes like this:
!include : base.yaml
appName: my-awesome-app
I used the private tag !include so that there will not be name clashes with any normal mapping key. Mind the space behind !include. I didn't give code to merge the included file because I did not know how you want to handle duplicate mapping keys. It should not be hard to implement though. Be aware of bugs, I have not tested this code.
The resulting String can be the input to Jackson.
Probably for the same desire, I have created this tool: jq-front.
You can do it by following syntax and combinating with yq command.
extends: [ base.yaml ]
appName: my-awesome-app
...
$ yq -j . your.yaml | jq-front | yq -y .
Note that you need to place file names to be extended in an array since the tool supports multiple inheritance.
Points potentially you don't like are
It's quite a bit slow. (But for configuration information, it might be ok since you can convert it to an expanded file once and you will never not the original one after that for your system)
Objects inside an array cannot behave as expected since the tool relies on * operator of jq.
Any built-in ready-to-use solution in Kendo UI to parse JSON data according to schema.model?
Maybe something like kendo.parseData(json, model), which will return array of objects?
I was searching for something like that and couldn't find anything built-in. However, using Model.set apparently uses each field's parse logic, so I ended up writing this function which works pretty good:
function parse(model, json) {
// I initialize the model with the json data as a quick fix since
// setting the id field doesn't seem to work.
var parsed = new model(json);
var fields = Object.keys(model.fields);
for (var i=0; i<fields.length; i++) {
parsed.set(fields[i], json[fields[i]]);
}
return parsed;
}
Where model is the kendo.data.Model definition (or simply datasource.schema.model), and json is the raw object. Using or modifying it to accept and return arrays shouldn't be too hard, but for my use case I only needed a single object to be parsed at a time.
I actually saw your post the day you posted it but did not have the answer. I just needed to solve this problem myself as part of a refactoring. My solution is for DataSources, not for models directly.
kendo.data.DataSource.prototype.parse = function (data) {
return this.reader.data(data);
// Note that the original data will be modified. If that is not what you want, change to the following commented line
// return this.reader.data($.extend({}, data));
}
// ...
someGrid.dataSource.parse(myData);
If you want to do it directly with a model, you will need to look at the DataReader class in kendo.data.js and use a similar logic. Unfortunately, the DataReader takes a schema instead of a model and the part dealing with the model is not extracted in it's own method.
I'm using SOAPUI to mock out a web-api service, I'm reading the contents of a static json response file, but changing the contents of a couple of the nodes based on what the user has passed through in the request.
I can't create multiple responses as surcharge is calculated from the amount passed.
The toString() method of the object that gets return by the slurper is replacing { with [ with invalidates my JsonResponse. I've included the important bits of the code below, has anyone got a way around this or is JsonSlurper not the right thing to use here?
def json=slurper.parseText(new File(path).text)
// set the surcharge to the two credit card nodes
// these are being set fine
json.AvailableCardTypeResponse.PaymentCards[0].Surcharge="${sur_charge}"
json.AvailableCardTypeResponse.PaymentCards[1].Surcharge="${sur_charge}"
response.setContentType("application/json;charset=utf-8" );
response.setContentLength(length);
Tools.readAndWrite( new ByteArrayInputStream(json.toString().getBytes("UTF-8")), length,response.getOutputStream() )
return new com.eviware.soapui.impl.wsdl.mock.WsdlMockResult(mockRequest)
You're slurping the json into a List/Map structure, then writing this List/Map structure out.
You need to convert your lists and maps back to json.
Change the line:
Tools.readAndWrite( new ByteArrayInputStream(json.toString().getBytes("UTF-8")), length,response.getOutputStream() )
to
Tools.readAndWrite( new ByteArrayInputStream( new JsonBuilder( json ).toString().getBytes("UTF-8")), length,response.getOutputStream() )