I'm practicing some USACO past released problems but whenever I submit my code for grading I receive the error:
Your output file (FILENAME.out):
[File missing!]
I tested every problem using this simple code, but still receive the same error:
import java.util.*;
import java.io.*;
public class Test
{
public static void main (String [] args) throws IOException
{
PrintWriter out = new PrintWriter(new BufferedWriter(new FileWriter(FILENAME)));
out.println("Hello world.");
out.close();
System.exit(0);
}
}
Why would this code not create an output file?
The USACO grading system has the output file already made in the same directory as your java solution, so all you need to do is just write to it.
In your line
PrintWriter out = new PrintWriter(new BufferedWriter(new FileWriter(FILENAME)));
you should change this to
PrintWriter out = new PrintWriter(new BufferedWriter(new FileWriter(FILENAME.out)));
since this is the name of the file. This does not create an actual file, but just writes to the existing one on the USACO grading system.
Related
I am new to apache flink and trying to learn data streams. I am reading student data which has 3 columns(Name,Subject and Marks) from a csv file. I have applied filter on marks and only selecting those records where marks >40.
I am trying to write this data to csv file but program runs successfully and csv file remains empty. No data gets written to csv file.
I tried with different syntax for writing csv file but none of them worked for me. I am running this locally through eclipse. Write to text file works fine.
DataStream<String> text = env.readFile(format, params.get("input"),
FileProcessingMode.PROCESS_CONTINUOUSLY,100);
DataStream<String> filtered = text.filter(new FilterFunction<String>(){
public boolean filter(String value) {
String[] tokens = value.split(",");
return Integer.parseInt(tokens[2]) >= 40;
}
});
filtered.writeAsText("testFilter",WriteMode.OVERWRITE);
DataStream<Tuple2<String, Integer>> tokenized = filtered
.map(new MapFunction<String, Tuple2<String, Integer>>(){
public Tuple2<String, Integer> map(String value) throws Exception {
return new Tuple2("Test", Integer.valueOf(1));
}
});
tokenized.print();
tokenized.writeAsCsv("file:///home/Test/Desktop/output.csv",
WriteMode.OVERWRITE, "/n", ",");
try {
env.execute();
} catch (Exception e1) {
e1.printStackTrace();
}
}
}
Below is my input CSV format:
Name1,Subj1,30
Name1,Subj2,40
Name1,Subj3,40
Name1,Subj4,40
Tokenized.print() prints all correct records.
I did a little experimenting, and found that this job works just fine:
import org.apache.flink.api.java.tuple.Tuple2;
import org.apache.flink.core.fs.FileSystem;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
public class WriteCSV {
public static void main(String[] args) throws Exception {
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
env.setParallelism(1);
env.fromElements(new Tuple2<>("abc", 1), new Tuple2<>("def", 2))
.writeAsCsv("file:///tmp/test.csv", FileSystem.WriteMode.OVERWRITE, "\n", ",");
env.execute();
}
}
If I don't set the parallelism to 1, then the results are different. In that case, test.csv is a directory containing four files, each written by one of the four parallel subtasks.
I'm not sure what's wrong in your case, but maybe you can work backwards from this example (assuming it works for you).
You should remove tokenized.print(); before tokenized.writeAsCsv();.
It will consume the data the print();.
I am trying to convert a JSON file into CSV.
I have a JAVA code which is able to do it perfectly on UNIX file system and on local file system.
I have written below main class to perform this conversion on HDFS.
public class ClassMain {
public static void main(String[] args) throws IOException {
String uri = args[1];
String uri1 = args[2];
Configuration conf = new Configuration();
FileSystem fs = FileSystem.get(URI.create(uri), conf);
FSDataInputStream in = null;
FSDataOutputStream out = fs.create(new Path(uri1));
try{
in = fs.open(new Path(uri));
JsonToCSV toCSV = new JsonToCSV(uri);
toCSV.json2Sheet().write2csv(uri1);
IOUtils.copyBytes(in, out, 4096, false);
}
finally{
IOUtils.closeStream(in);
IOUtils.closeStream(out);
}
}
}
json2sheet and write2csv are methods which perform the conversion and write operation.
I am running this jar using below command:
hadoop jar json-csv-hdfs.jar com.nishant.ClassMain /nishant/large.json /nishant/output
The problem is, it does not write anything at /nishant/output. It creates a 0 sized /nishant/output file.
Maybe the usage of copyBytes is not a good idea here.
How to achieve this on HDFS if it is working OK on unix FS and local FS.
Here I am trying to convert JSON file to CSV and not trying to map JSON objects to their values
FileSystem needs only one configuration key to successfully connect to HDFS.
conf.set(key, "hdfs://host:port"); // where key="fs.default.name"|"fs.defaultFS"
I'm getting an NullPointerException from uploadedFile.getName() in the following code that utilizes the google api. If I change that to a string fileName the code works fine.
public static void downloadFile(boolean useDirectDownload, File uploadedFile, java.io.File parentDir)throws IOException {
OutputStream out = new FileOutputStream(new java.io.File(parentDir, uploadedFile.getName()));
DRIVE.files().get(uploadedFile.getId()).executeMediaAndDownloadTo(out);
}
I understand what a null pointer is but not sure why I'm getting one in this case because prior to calling the download file I set the filename in code earlier and the file is uploaded with the name I set.
public static File uploadFile(boolean useDirectUpload, String uploadFilePath, String fileType, String fileName) throws IOException {
File fileMetadata = new File();
fileMetadata.setName(fileName);
fileMetadata.setMimeType(fileType);
java.io.File filePath = new java.io.File(uploadFilePath);
FileContent mediaContent = new FileContent(fileType, filePath);
return DRIVE.files().create(fileMetadata, mediaContent)
.setFields("id")
.execute();
}
The above two sets of code are called by
java.io.File parentDir = P.createDirectory("C:\\DIR");
File uploadedFile = P.uploadFile(true, "C:\\DIR\AA.pdf", "application/pdf", "BB.pdf");
P.downloadFile(true, uploadedFile, parentDir);
For a specific question, why does the File returned by the following statement have null for the name?
return DRIVE.files().create(fileMetadata, mediaContent)
.setFields("id")
.execute();
Maven dependency for drive services:
<dependency>
<groupId>com.google.apis</groupId>
<artifactId>google-api-services-drive</artifactId>
<version>v3-rev40-1.22.0</version>
</dependency>
You're only getting id as part of the partial response when the create method is called since its the only one specified in the setFields - https://developers.google.com/resources/api-libraries/documentation/drive/v3/java/latest/com/google/api/services/drive/Drive.Files.Create.html.
Try to add other fields (name, and other related fields necessary) and the call should work
why does the File returned by the following statement have null for the name?
Because you haven't requested the file name to be included in the REST response. Change .setFields("id") to .setFields("id, name")
I have my Data in a CSV file. I want to read the CSV file which is in HDFS.
Can anyone help me with the code??
I'm new to hadoop. Thanks in Advance.
The classes required for this are FileSystem, FSDataInputStream and Path. Client should be something like this :
public static void main(String[] args) throws IOException {
// TODO Auto-generated method stub
Configuration conf = new Configuration();
conf.addResource(new Path("/hadoop/projects/hadoop-1.0.4/conf/core-site.xml"));
conf.addResource(new Path("/hadoop/projects/hadoop-1.0.4/conf/hdfs-site.xml"));
FileSystem fs = FileSystem.get(conf);
FSDataInputStream inputStream = fs.open(new Path("/path/to/input/file"));
System.out.println(inputStream.readChar());
}
FSDataInputStream has several read methods. Choose the one which suits your needs.
If it is MR, it's even easier :
public static class YourMapper extends
Mapper<LongWritable, Text, Your_Wish, Your_Wish> {
public void map(LongWritable key, Text value, Context context)
throws IOException, InterruptedException {
//Framework does the reading for you...
String line = value.toString(); //line contains one line of your csv file.
//do your processing here
....................
....................
context.write(Your_Wish, Your_Wish);
}
}
}
If you want to use mapreduce you can use TextInputFormat to read line by line and parse each line in mapper's map function.
Other option is to develop (or find developed) CSV input format for reading data from file.
There is one old tutorial here http://hadoop.apache.org/docs/r0.18.3/mapred_tutorial.html but logic is same in new versions
If you are using single process for reading data from file it is same as reading file from any other file system. There is nice example here https://sites.google.com/site/hadoopandhive/home/hadoop-how-to-read-a-file-from-hdfs
HTH
Im pretty new to Java and Im searching the Internet for a simple way to load an external csv into JavaFX TableView.
I was able to parse the CSV into an array but I dont know how I have to handle it now. Then I was playing with the DataFX library. But again wasnt able to pass the parsed csv into my table.
I think I dont really understand ObservableLists here which I believe is kind of necessary? Do you know a good tutorial or could you explain what the next steps would be after parsing the file?
thx
Edit: That's what I did
import javafx.application.Application;
import javafx.scene.SceneBuilder;
import javafx.scene.control.TableColumn;
import javafx.scene.control.TableView;
import javafx.stage.Stage;
import org.javafxdata.datasources.reader.FileSource;
import org.javafxdata.datasources.provider.CSVDataSource;
public class CSVTableSample extends Application {
#SuppressWarnings("unchecked")
#Override
public void start(Stage stage) throws Exception {
stage.setTitle("Test App");
// Just loading the file...
FileSource fs = new FileSource("test.csv");
// Now creating my datasource
CSVDataSource dataSource = new CSVDataSource(
fs, "order-id", "order-item-id");
#SuppressWarnings("rawtypes")
TableView table1 = new TableView();
TableColumn<?, ?> orderCol = dataSource.getNamedColumn("order-id");
TableColumn<?, ?> itemCol = dataSource.getNamedColumn("order-item-id");
table1.getColumns().addAll(orderCol, itemCol);
table1.setItems(dataSource);
stage.setScene(SceneBuilder.create().root(table1).build());
stage.show();
}
public static void main(String[] args) {
Application.launch(args);
}
}
eclipse says for table1.setItems(dataSource);
The method setItems(ObservableList) in the type TableView is not applicable for the arguments (CSVDataSource)
There is a sample solution here for a tab delimited file.
A csv file could handled similarly.
The sample works by declaring the type of the the TableView as TableView<ObservableList<StringProperty>> such that each row in the TableView is an ObservableList of string properties where each property represents a field in the csv file. The TableView's items list is a list of such lists. cellValueFactorys set for each column extract the correct cell value for that column from the ObservableList<StringProperty> backing that cell's row.
The method setItems(ObservableList) in the type TableView is not
applicable for the arguments (CSVDataSource)
change your line
table1.setItems(dataSource);
to
table1.setItems(dataSource.getData());
Example Code Using DataFX :
DataSourceReader dsr1 = new FileSource("your csv file path");
String[] columnsArray // create array of column names you want to display
CSVDataSource ds1 = new CSVDataSource(dsr1,columnsArray);
TableView tableView = new TableView();
tableView.setItems(ds1.getData());
tableView.getColumns().addAll(ds1.getColumns());
if you want to do it in standard javafx way : Look Here