Hi i am using [box][1]
[1]: https://www.box.com/ to store my csv file. Size of file is 2GB. Now i want to process each record of file and do some operation based on file content.
what i did:
public class BoxConnector {
public static void main(String[] args) throws IOException {
BoxAPIConnection api = new BoxAPIConnection("My access token");
BoxFolder rootFolder = BoxFolder.getRootFolder(api);
for (BoxItem.Info itemInfo : rootFolder) {
System.out.format("[%s] %s\n", itemInfo.getID(), itemInfo.getName());
BoxFile file = new BoxFile(api, itemInfo.getID());
BoxFile.Info info = file.getInfo();
try {
System.out.println(info.getSize());
File tmpFile = File.createTempFile("file", "temp");
FileOutputStream fsTmpStrem = new FileOutputStream(tmpFile);
long blockSize = 1000;
long roundChunks = info.getSize() / blockSize;
long startByteRange = 0;
long endByteRange = blockSize;
for (long start = 0; start < roundChunks; start++) {
file.downloadRange(fsTmpStrem, startByteRange, endByteRange);
processFile(tmpFile);
startByteRange = endByteRange;
endByteRange = endByteRange + blockSize;
}
//last download block
file.downloadRange(fsTmpStrem, blockSize * roundChunks, info.getSize());
processFile(tmpFile);
} finally {
}
}
}
private static void processFile(File tmpFile) throws IOException {
BufferedReader br = new BufferedReader(new InputStreamReader(new FileInputStream(tmpFile)));
String line = null;
while ((line = br.readLine()) != null) {
System.out.println("Process line record");
}
br.close();
//after each process lets delete the temp file
tmpFile.delete();
}
}
with this i am able to get the file name which i have uploaded to box.com. Now i want to read each record and process.
However i need an API which allows me file chunk access .
with this files are getting downloaded as per chunk defined by start and end byte range flag.however due to chunk download i am getting few records broken . Meaning i am not getting complete line say below is my record
16F11C78-D004-4600-8D28-445C087D2A7D
31C99F3D-D4C7-418A-9ACC-D9A382BCD53A
30C1AA92-B5B7-4ABF-A631-A8C150D90C4F
D9FC1DBF-B309-4BB1-8955-D9F48F643E97
i am getting
16F11C78-D004-4600-8D28-445C087D2A7D
31C99F3D-D4C7-418A-9ACC-D9A382BCD53A
30C1AA92-B5B7-4ABF-A631-A8C150D90C4F
D9FC1DBF-
i.e. B309-4BB1-8955-D9F48F643E97 part from last line is missing.
How should i manage this with download API?
This is not currently possible with the Box API. You can only download an entire file.
Related
I have an app using windows forms that goes through all the resource files in a new release writes the name value and comment to a csv file. I am ResXResourceReader and the method GetEnumerator to create a IDictionaryEnumerator and go through each resource and add the key and value to a dictionary. Im getting exceptions on some files, for example:
{[System.ArgumentException: ResX file Could not find a part of the path 'C:\ibml\SoftTrac CaptureSuite\Utilities\TranslationUtility\TranslationUtility\bin\resources\accept.png'. Line 12017, position 5. cannot be parsed. ---> System.Xml.XmlException: Could not find a part of the path 'C:\ibml\SoftTrac CaptureSuite\Utilities\TranslationUtility\TranslationUtility\bin\resources\accept.png'. Line 12017, position 5. ---> System.IO.DirectoryNotFoundException: Could not find a part of the path 'C:\ibml\SoftTrac CaptureSuite\Utilities\TranslationUtility\TranslationUtility\bin\resources\accept.png'.
My code for this is below:
private List<object> GetNewResxStrings(List<object> NewlyAddedResxFiles)
{
bool breaker = true;
Dictionary<string, int> Exceptionslist = new Dictionary<string, int>();
foreach (var resxfile in NewlyAddedResxFiles)
{
breaker = true;
while (breaker)
{
using (FileStream fsNew = File.Open(resxfile.ToString(), FileMode.Open, FileAccess.ReadWrite, FileShare.None))
{
ResXResourceReader readerNew = new ResXResourceReader(fsNew);
try
{
IDictionaryEnumerator dictNew = readerNew.GetEnumerator();
while (dictNew.MoveNext())
{
NewFilesAddedToCurrentVersion.Add(dictNew.Key, dictNew.Value);
}
Dictionary<object, object> temp = new Dictionary<object, object>(NewFilesAddedToCurrentVersion);
NewUtilityResxDictionary.Add(fsNew.Name.ToString(), temp);
NameOfChangeNew.Clear();
NameOfChangeOld.Clear();
NewFilesAddedToCurrentVersion.Clear();
breaker = false;
}
catch (Exception E)
{
Exceptionslist.Add(E.ToString(), 1);
breaker = false;
break;
}
}
}
}
return NewlyAddedResxFiles;
}
I once had this error after I c/p a path from explorer. Turned out there was a hidden char in the path. Between C and : if I recall. Could that be the problem? Check in excel with MID or LEN.
I have this error when I Run the code but no when I Compile it.
Exception in thread "main" java.lang.NullPointerException
at java.util.StringTokenizer.<init>(StringTokenizer.java:199)
at java.util.StringTokenizer.<init>(StringTokenizer.java:221)
at WillinsLaMarkusFileInputOutput.main(WillinsLaMarkusFileInputOutput.java:23)
Here is my complete code
import java.io.*;
public class WillinsLaMarkusFileInputOutput {
public static void main(String[] args) throws IOException {
/* open the files */
// Scanner sc = new Scanner(System.in);
BufferedReader r = new BufferedReader(new FileReader("input.txt"));
BufferedWriter w = new BufferedWriter(new FileWriter("output.txt"));
float[] values = new float[10];
String str = r.readLine();
int i = 0;
float sum = 0.0f, avg = 0.0f;
/* tokenize the string into floats separated by spaces */
java.util.StringTokenizer tk = new java.util.StringTokenizer(str, " ");
while (tk.hasMoreElements()) {
values[i] = Float.valueOf(tk.nextToken()).floatValue();
/* compute sum */
sum += values[i];
i++;
}
/* calculate average */
avg = sum / 10.0f;
/* write results to output.txt */
w.write("Sum: " + sum);
w.newLine();
w.write("Average: " + avg);
w.flush();
/* close the files */
r.close();
w.close();
}
}
Any suggestions on how to fix this ?
The exception goes from this line:
java.util.StringTokenizer tk = new java.util.StringTokenizer(str, " ");
So first let's read the documentation when this constructor may throw NullPointerException:
public StringTokenizer(String str, String delim)
Throws:
NullPointerException - if str is null
Fine. It means that str is null. The str is assigned in this line:
String str = r.readLine();
Now you should read the documentation when r.readLine() may return null:
Returns:
A String containing the contents of the line, not including any line-termination characters, or null if the end of the stream has been reached
So it signals that end of file is reached. As it's the very first attempt to read something from this file it seems that you are reading an empty file.
In general careful documentation reading may greatly help to understand why your program doesn't work.
I am working with csv file having very large dataset. while reading file i had extracted 4th place(BALANCE) ';' separated numeric value from each rows through while loop iteration. and make a list of Double after some mathematical calculation(here incremented).
now I want to store this list of Double in reverse order(from end to beginning).as its original position(here 4th place).example
public static void main(String[] args) throws IOException {
String filename = "abc.csv";
List<Double> list = new ArrayList<Double>();
File file = new File(filename);
Scanner inputStream = new Scanner(file);
inputStream.next();
while (inputStream.hasNext()) {
String data = inputStream.next();
String[] values = data.split(";");
double BALANCE = Double.parseDouble(values[1]);
BALANCE = BALANCE + 1;
ListIterator li = list.listIterator(list.size());
while (li.hasPrevious()) {
values[1] = String.valueOf(li.previous()); }
inputStream.close();
}
} }
You can use Collections.reverse. Example Collections.reverse(list);
I'm working on a Windows Phone 8 app.
I'm having issue appending to my JSON file.
It works fine if I keep the app open but once I close it and come back in it starts back writing from the beginning of the file.
Relevant code:
private async void btnSave_Click(object sender, RoutedEventArgs e)
{
// Create a entry and intialize some values from textbox...
GasInfoEntries _entry = null;
_entry = new GasInfoEntries();
_entry.Gallons = TxtBoxGas.Text;
_entry.Price = TxtBoxPrice.Text;
_GasList.Add(_entry);
//TxtBlockPricePerGallon.Text = (double.Parse(TxtBoxGas.Text) / double.Parse(TxtBoxPrice.Text)).ToString();
// Serialize our Product class into a string
string jsonContents = JsonConvert.SerializeObject(_GasList);
// Get the app data folder and create or open the file we are storing the JSON in.
StorageFolder localFolder = ApplicationData.Current.LocalFolder;
StorageFile textfile = await localFolder.CreateFileAsync("gasinfo.json", CreationCollisionOption.OpenIfExists); //if get await operator error add async to class (btnsave)
//open file
using (IRandomAccessStream textstream = await textfile.OpenAsync(FileAccessMode.ReadWrite))
{
//write JSON string
using (DataWriter textwriter = new DataWriter(textstream))
//using (DataWriter textwriter = new DataWriter(textstream))
{
textwriter.WriteString(jsonContents);
await textwriter.StoreAsync(); //writes buffer to store
}
}
}
private async void btnShow_Click(object sender, RoutedEventArgs e)
{
StorageFolder localfolder = ApplicationData.Current.LocalFolder;
try
{
// Getting JSON from file if it exists, or file not found exception if it does not
StorageFile textfile = await localfolder.GetFileAsync("gasinfo.json");
using (IRandomAccessStream textstream = await textfile.OpenReadAsync())
{
//read text stream
using (DataReader textreader = new DataReader(textstream))
{
//get size ...not sure what for think check the file size (lenght) then based on next 2 commands waits until its all read
uint textlength = (uint)textstream.Size;
await textreader.LoadAsync(textlength);
//read it
string jsonContents = textreader.ReadString(textlength);
// deserialize back to gas info
_GasList = JsonConvert.DeserializeObject<List<GasInfoEntries>>(jsonContents) as List<GasInfoEntries>;
displayGasInfoEntries();
}
}
}
catch
{
txtShow.Text = "something went wrong";
}
}
private void displayGasInfoEntries()
{
txtShow.Text = "";
StringBuilder GasString = new StringBuilder();
foreach (GasInfoEntries _entry in _GasList)
{
GasString.AppendFormat("Gallons: {0} \r\n Price: ${1} \r\n", _entry.Gallons, _entry.Price); // i think /r/n means Return and New line...{0} and {1} calls "variables" in json file
}
txtShow.Text = GasString.ToString();
}
Thanks
Do you call the btnShow_Click each time you've started the app? Because otherwise the _GasList will be empty; if you now call the btnSave_Click all previous made changes will be lost.
So please make sure, that you restore the previously saved json data before you add items to the _GasList.
I was having a hard time loading training set from Ms Access database in to the main form that does the Face Recognition. I saved the training sets with their names and ID in to the database as a binary data with an OLE Object format.The method i used to change, save and read the data from the database and in to the training sets is
private static byte[] ConvertImageToBytes(Image InputImage)
{
using (Bitmap BmpImage = new Bitmap(InputImage))
{
using (MemoryStream MyStream = new MemoryStream())
{
BmpImage.Save(MyStream, System.Drawing.Imaging.ImageFormat.Jpeg);
byte[] ImageAsBytes = MyStream.ToArray();
return ImageAsBytes;
}
}
}
The method that i use to store the converted byte data to the database is the following:
private void StoreData(byte[] ImageAsBytes,String NameStudent,String IDStudent)
{
if (DBConnection.State.Equals(ConnectionState.Closed))
DBConnection.Open();
try
{
//MessageBox.Show("Saving image at index : " + rowPosition);
using (OleDbCommand insert = new OleDbCommand(String.Format("Insert INTO
TrainingSet(rowPosition,StudentName,StudentID,StudentFace) values ('
{0}','{1}','{2}',#StudentFace)", rowPosition, NameStudent, IDStudent),
DBConnection))
{
OleDbParameter imageParameter = insert.Parameters.AddWithValue(#"StudentFace",
SqlDbType.Binary);
imageParameter.Value = ImageAsBytes;
imageParameter.Size = ImageAsBytes.Length;
int rowsAffected = insert.ExecuteNonQuery();
MessageBox.Show(String.Format("Data stored successfully in {0}
Row",rowsAffected));
}
rowPosition++;
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
MessageBox.Show(ex.Message);
}
finally
{
RefreshDBConnection();
}
}
The method that i use to Read this binary data is as follows:
private Image ReadImageFromDB()
{
Image FetchedImg;
if (rowNumber >= 0)
{
byte[] FetchedImgBytes = (byte[])LocalDataTable.Rows[rowNumber]["StudentFace"];
MemoryStream stream = new MemoryStream(FetchedImgBytes);
FetchedImg = Image.FromStream(stream);
return FetchedImg;
}
else
{
MessageBox.Show("There are no images in the database yet.Please reconnect
or add some pictures.");
return null;
}
}
I have successfully saved the training sets/images as a binary data in to the database.The problem is when i load these training sets for Recognition.
// Declaring the variables=====trainingImages is where the training sets are
// loaded from the database NameLabels and IDLabels are text in the database
// and where name and Id of subject
//is saved.
List<Image<Gray,byte>> trainingImages = new List<Image<Gray,byte>>();
List<string> NameLables= new List<string>();
List<string> IDLables = new List<string>();
int ContTrain, NumNameLabels,NumIDLabels, t;
//The training sets from the database are loaded in to the facerecognizer code as
// follows
public FaceRecognizer()
{
InitializeComponent();
try
{
//Load previous trained and labels for each image from the database Here
RefreshDBConnection();
String[] NameLabels = (String[])LocalDataTable.Rows[rowNumber]["StudentName"];
NumNameLabels = Convert.ToInt16(NameLabels[0]);
String[] IDLabels = (String[])LocalDataTable.Rows[rowNumber]["StudentID"];
NumIDLabels = Convert.ToInt16(IDLabels[0]);
if (NumNameLabels == NumIDLabels)
{
ContTrain = NumNameLabels;
string LoadFaces;
// Converting the master image to a bitmap
Image imageFromDB;
Bitmap imageChangedToBitmap;
// Normalizing it to grayscale
Image<Gray, Byte> normalizedMasterImage;
for (int tf = 1; tf < NumNameLabels + 1; tf++)
{
imageFromDB = ReadImageFromDB();
//image loaded from the database is converted in to Bitmap and then
//convert the bitmap image in to Image<Gray,byte> for input to
//EigenObjectRecognizer(,,,)
imageChangedToBitmap = new Bitmap(imageFromDB);
normalizedMasterImage = new Image<Gray, Byte>(imageChangedToBitmap);
LoadFaces = String.Format("face{0}.bmp", tf);
trainingImages.Add(normalizedMasterImage);
//trainingImages.Add(new Image<Gray, byte>());
NameLables.Add(NameLabels[tf]);
IDLables.Add(IDLabels[tf]);
rowNumber = rowNumber + 1;
}
}
else
MessageBox.Show("There's a conflict between Name labels and id labels");
}
catch (Exception e)
{
MessageBox.Show("Nothing in the database, please add at least a
face.Train the database","Triained faces load",MessageBoxButtons.OK,
MessageBoxIcon.Exclamation);
}
}
I am only getting the message in the catch when the the form loads even if there are faces saved in the database. I have used EigenObjectRecognizer and i will post the code if necessary.
at the part of loading face, you did not save by face1, face2, face3 etc. So you can not load using;
LoadFaces = String.Format("face{0}.bmp", tf);