Hi below is my code to extract particular metadata tags and write those tags to a json file. And i imported json.lib.jar and tika-app.jar into my build path.
File dir = new File("C:/pdffiles");
File listDir[] = dir.listFiles();
for (int i = 0; i < listDir.length; i++)
{
System.out.println("files"+listDir.length);
String file=listDir[i].toString();
File file1 = new File(file);
InputStream input = new FileInputStream(file1);
Metadata metadata = new Metadata();
BodyContentHandler handler = new BodyContentHandler(10*1024*1024);
AutoDetectParser parser = new AutoDetectParser();
parser.parse(input, handler, metadata);
Map<String, String> map = new HashMap<String, String>();
map.put("File name: ", listDir[i].getName());
map.put("Title: " , metadata.get("title"));
map.put("Author: " , metadata.get("Author"));
map.put("Content type: " , metadata.get("Content-Type"));
JSONObject json = new JSONObject();
json.accumulateAll(map);
FileWriter file2;
file2 = new FileWriter("C:\\test.json");
file2.write(json.toString());
file2.flush();
}
But it is writing only single file metadata to the json file. Is there any problem with my code, please suggest me.
may be you should use-
file2.write(json.toJSONString());
instead of this line -
file2.write(json.toString());
Related
The following Groovy code can create a new CSV file (in this example testfile.csv) and write JSON data in the in CSV. I do not want to create a new CSV file but just want to add (Append) few more lines to the existing testfile.csv file without overwriting the file. May someone please help what to change in the following code to force it to append the file instead of writing a new one? I heard about StandardOpenOption.APPEND but no idea where to put that. Thanks
import groovy.json.JsonSlurper;
import org.apache.commons.csv.CSVFormat;
import org.apache.commons.csv.CSVPrinter;
import com.oracle.e1.common.OrchestrationAttributes;
import java.text.SimpleDateFormat;
HashMap < String, Object > main(OrchestrationAttributes orchAttr, HashMap inputMap) {
HashMap < String, Object > returnMap = new HashMap < String, Object > ();
returnMap.put("CSVComplete", "false");
// Write the view number after jsonIn.fs_DATABROWSE_
def jsonIn = new JsonSlurper().parseText(inputMap.get("Vendor Data"));
def jsonData = jsonIn.fs_DATABROWSE_GettingJsonDataFromSomewhere.data.gridData.rowset;
if (jsonData.size() == 0) {
returnMap.put("CSVComplete", "empty");
return returnMap;
}
def fileName = orchAttr.getTempFileName("testfile.csv");
returnMap.put("CSVOutFileName", fileName);
//class writer to write file def
def sw = new StringWriter();
//build the CSV writer with a header
//def csv = new CSVPrinter(sw, CSVFormat.DEFAULT.withHeader("Business Unit", "Document Number", "LT", "SUB","Amount","HardcodedTHREAD","ApprovedBudget","fromview003"));
def csv = new CSVPrinter(sw, CSVFormat.DEFAULT); //No header
// create output file
fileCsvOut = new File(fileName);
def count=0;
// build the CSV
def an8Map = new ArrayList();
for (int i = 0; i < jsonData.size(); i++) {
def businessunit = jsonData[i].table1_column1;
if (an8Map.contains(businessunit)) {
continue;
}
an8Map.add(businessunit);
count++;
csv.printRecord(businessunit, jsonData[i].table_column,
jsonData[i].table1_column1, jsonData[i]. table1_column2, jsonData[i]. table1_column3, "Fixed text1 "Fixed text2", "Fixedtext3");
}
csv.close();
//writing csv to file
fileCsvOut.withWriter('UTF-8') {
writer ->
writer.write(sw.toString())
}
orchAttr.writeDebug(sw.toString());
returnMap.put("csv", sw.toString());
returnMap.put("CSVComplete", "true");
returnMap.put("CSVcount", Integer.toString(count));
return returnMap;
}
use withWriterAppend instead of withWriter
https://docs.groovy-lang.org/latest/html/groovy-jdk/java/io/File.html#withWriterAppend(java.lang.String,%20groovy.lang.Closure)
I created an ArrayList with the json values from an Rest API.
This is the code to read the Rest API:
#RestController
public class exemploclass {
#RequestMapping(value="/vectors")
//#Scheduled(fixedRate = 5000)
public ArrayList<StateVector> getStateVectors() throws Exception {
ArrayList<StateVector> vectors = new ArrayList<>();
String url = "https://opensky-network.org/api/states/all?lamin=41.1&lomin=6.1&lamax=43.1&lomax=8.1";
//String url = "https://opensky-network.org/api/states/all?lamin=45.8389&lomin=5.9962&lamax=47.8229&lomax=10.5226";
URL obj = new URL(url);
HttpURLConnection con = (HttpURLConnection) obj.openConnection();
// optional default is GET
con.setRequestMethod("GET");
//add request header
con.setRequestProperty("User-Agent", "Mozilla/5.0");
int responseCode = con.getResponseCode();
System.out.println("\nSending 'GET' request to URL : " + url);
System.out.println("Response Code : " + responseCode);
BufferedReader in = new BufferedReader(
new InputStreamReader(con.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
JSONObject myResponse = new JSONObject(response.toString());
JSONArray states = myResponse.getJSONArray("states");
System.out.println("result after Reading JSON Response");
for (int i = 0; i < states.length(); i++) {
JSONArray jsonVector = states.getJSONArray(i);
String icao24 = jsonVector.optString(0);
String callsign = jsonVector.optString(1);
String origin_country = jsonVector.optString(2);
Boolean on_ground = jsonVector.optBoolean(8);
//System.out.println("icao24: " + icao24 + "| callsign: " + callsign + "| origin_country: " + origin_country + "| on_ground: " + on_ground);
//System.out.println("\n");
StateVector sv = new StateVector(icao24, callsign, origin_country, on_ground);
vectors.add(sv);
}
System.out.println("Size of data: " + vectors.size());
return vectors;
}
}
The last line " return vectors;" returns a list with the values i parsed and returns it like this:
But i want this more "pretty", i want it to be one Array in each line, how can i achieve this?
P.S. Its on the .html page, not on console
Your return value seems a valid Json Object. If you want it more pretty so you can read it clearly then pass it through an application that makes that json pretty.
If you call your API from Postman, it will give you a pretty Json Object which will be better formatted. This will be because you have annotated your controller with #RestController so it will deliver an application/json response which Postman will know and then it will try to make it prettier.
P.S. Its on the .html page, not on console
So you hit your API from a browser. Most browsers don't expect a Json object to be returned, so they will not make it pretty. You can't force that from your Service either.
Just hit your API from Postman, it will understand it and make it pretty.
Please keep in mind this is a open question and I am not looking for a specific answer but just approaches and routes I can take.
Essentially I am getting a csv file from my aws s3 bucket. I am able to get it successfully using
AmazonS3 s3Client = new AmazonS3Client(new ProfileCredentialsProvider());
S3Object object = s3Client.getObject(
new GetObjectRequest(bucketName, key));
Now I want to populate a dynamodb table using this JSON file.
I was confused as i found all sorts of stuff online.
Here is one suggestion - This approach is however only reading the file it is not inserting anything to the dynamodb table.
Here is another suggestion - This approach is lot closer to what i am looking for , it is populating a table from a JSON file.
However i was wondering is there a generic way to ready any json file and populate a dynamodb table based on that ? Also for my case what approach is the best?
Since i originally asked the question I did more work.
What I have done so far
I have a csv file sitting in s3 that looks like this
name,position,points,assists,rebounds
Lebron James,SF,41,12,11
Kyrie Irving,PG,41,7,5
Stephen Curry,PG,29,8,4
Klay Thompson,SG,31,5,5
I am able to sucessfully pick it up as a s3object doing the following
AmazonS3 s3client = new AmazonS3Client(/**new ProfileCredentialsProvider()*/);
S3Object object = s3client.getObject(
new GetObjectRequest("lambda-function-bucket-blah-blah", "nba.json"));
InputStream objectData = object.getObjectContent();
Now I want to insert this in to my dynamodb table so i am attempting the following.
AmazonDynamoDBClient dbClient = new AmazonDynamoDBClient();
dbClient.setRegion(Region.getRegion(Regions.US_BLAH_1));
DynamoDB dynamoDB = new DynamoDB(dbClient);
//DynamoDB dynamoDB = new DynamoDB(client);
Table table = dynamoDB.getTable("MyTable");
//after this point i have tried many json parsers etc and did table.put(item) etc but nothing has worked. I would appreciate kind help
For CSV parsing, you can use plain reader as your file looks quite simple
AmazonS3 s3client = new AmazonS3Client(/**new ProfileCredentialsProvider()*/);
S3Object object = s3client.getObject(
new GetObjectRequest("lambda-function-bucket-blah-blah", "nba.json"));
InputStream objectData = object.getObjectContent();
AmazonDynamoDBClient dbClient = new AmazonDynamoDBClient();
dbClient.setRegion(Region.getRegion(Regions.US_BLAH_1));
DynamoDB dynamoDB = new DynamoDB(dbClient);
//DynamoDB dynamoDB = new DynamoDB(client);
Table table = dynamoDB.getTable("MyTable");
String line = "";
String cvsSplitBy = ",";
try (BufferedReader br = new BufferedReader(
new InputStreamReader(objectData, "UTF-8"));
while ((line = br.readLine()) != null) {
// use comma as separator
String[] elements = line.split(cvsSplitBy);
try {
table.putItem(new Item()
.withPrimaryKey("name", elements[0])
.withString("position", elements[1])
.withInt("points", elements[2])
.....);
System.out.println("PutItem succeeded: " + elements[0]);
} catch (Exception e) {
System.err.println("Unable to add user: " + elements);
System.err.println(e.getMessage());
break;
}
}
} catch (IOException e) {
e.printStackTrace();
}
Depending the complexity of your CSV, you can use 3rd party libraries like Apache CSV Parser or open CSV
I leave the original answer for parsing JSon
I would use the Jackson library and following your code do the following
AmazonS3 s3client = new AmazonS3Client(/**new ProfileCredentialsProvider()*/);
S3Object object = s3client.getObject(
new GetObjectRequest("lambda-function-bucket-blah-blah", "nba.json"));
InputStream objectData = object.getObjectContent();
AmazonDynamoDBClient dbClient = new AmazonDynamoDBClient();
dbClient.setRegion(Region.getRegion(Regions.US_BLAH_1));
DynamoDB dynamoDB = new DynamoDB(dbClient);
//DynamoDB dynamoDB = new DynamoDB(client);
Table table = dynamoDB.getTable("MyTable");
JsonParser parser = new JsonFactory()
.createParser(objectData);
JsonNode rootNode = new ObjectMapper().readTree(parser);
Iterator<JsonNode> iter = rootNode.iterator();
ObjectNode currentNode;
while (iter.hasNext()) {
currentNode = (ObjectNode) iter.next();
String lastName = currentNode.path("lastName").asText();
String firstName = currentNode.path("firstName").asText();
int minutes = currentNode.path("minutes").asInt();
// read all attributes from your JSon file
try {
table.putItem(new Item()
.withPrimaryKey("lastName", lastName, "firstName", firstName)
.withInt("minutes", minutes));
System.out.println("PutItem succeeded: " + lastName + " " + firstName);
} catch (Exception e) {
System.err.println("Unable to add user: " + lastName + " " + firstName);
System.err.println(e.getMessage());
break;
}
}
parser.close();
Inserting the records in your table will depend of your schema, I just put an arbitrary example, but anyway this will get you the reading of your file and the way to insert into the dynamoDB table
As you talked about the different approaches, another possibility is to setup a AWS Pipeline
I would like to as if it is possible/supported by commons-configuration of apache to get from a properties file a property as a map
Up to now I have managed to do this indirectly with the following code snippet
Map<String, T> map = new LinkedHashMap<>();
Configuration subset = config.subset(key);
if (!subset.isEmpty()) {
Iterator it = subset.getKeys();
while (it.hasNext()) {
String k = (String) it.next();
//noinspection unchecked
T v = (T) subset.getProperty(k);
map.put(k, v);
}
}
return map;
Does anyone knows a more straight forward way than this?
Thank you very much
I prefer how you did it but if you like: ConfigurationMap
Map<Object,Object> config = new ConfigurationMap(subset);
to get all properties as Map with apache commons confuguration2
Parameters params = new Parameters();
File propertiesFile = new File("properties.properties");
FileBasedConfigurationBuilder<FileBasedConfiguration> builder =
new FileBasedConfigurationBuilder<FileBasedConfiguration>(PropertiesConfiguration.class)
.configure(params.fileBased()
.setFile(propertiesFile)
.setEncoding("UTF-8"));
Configuration config = builder.getConfiguration();
Map<Object,Object> cfg = new ConfigurationMap(config);
cfg.entrySet();
to check out:
for (Map.Entry entry : cfg.entrySet()) {
System.out.println(entry.getKey() + ", " + entry.getValue());
}
I am creating a windows 8 phone application, in which i am reading a xml file called User and add want to add the attributes id and name to the user element of the xaml using XDocument.
But I am not getting how to save it back to the xml file.
XDocument doc = XDocument.Load(#"XDocument.Load(#"Assets\User.xml");
XElement element = doc.Element("user");
XAttribute idAtt = new XAttribute("id", userDetails.UserId);
element.Add(idAtt);
XAttribute nameAtt = new XAttribute("name", userDetails.UserName);
element.Add(nameAtt);
Please help.
That's how I save my XML files:
XmlWriterSettings xmlWriterSettings = new XmlWriterSettings();
xmlWriterSettings.Indent = true;
using (IsolatedStorageFile myIsolatedStorage = IsolatedStorageFile.GetUserStoreForApplication())
{
using (IsolatedStorageFileStream stream = myIsolatedStorage.OpenFile("User.xml", FileMode.Create))
{
XmlSerializer serializer = new XmlSerializer(typeof(PrivacyDataClass));
using (XmlWriter xmlWriter = XmlWriter.Create(stream, xmlWriterSettings))
{
serializer.Serialize(xmlWriter, data);
}
}
}