Detecting folders/directories in javascript FileList objects - html

I have recently contributed some code to Moodle which uses some of the capabilities of HTML5 to allow files to be uploaded in forms via drag and drop from the desktop (the core part of the code is here: https://github.com/moodle/moodle/blob/master/lib/form/dndupload.js for reference).
This is working well, except for when a user drags a folder / directory instead of a real file. Garbage is then uploaded to the server, but with the filename matching the folder.
What I am looking for is an easy and reliable way to detect the presence of a folder in the FileList object, so I can skip it (and probably return a friendly error message as well).
I've looked through the documentation on MDN, as well as a more general web search, but not turned up anything. I've also looked through the data in the Chrome developer tools and it appears that the 'type' of the File object is consistently set to "" for folders. However, I'm not quite convinced this is the most reliable, cross-browser detection method.
Does anyone have any better suggestions?

You cannot rely on file.type. A file without an extension will have a type of "". Save a text file with a .jpg extension and load it into a file control, and its type will display as image/jpeg. And, a folder named "someFolder.jpg" will also have its type as image/jpeg.
Instead, try to read the first byte of the file. If you are able to read the first byte, you have a file. If an error is thrown, you probably have a directory:
try {
await file.slice(0, 1).arrayBuffer();
// it's a file!
}
catch (err) {
// it's a directory!
}
If you are in the unfortunate position of supporting IE11, The file will not have the arrayBuffer method. You have to resort to the FileReader object:
// use this code if you support IE11
var reader = new FileReader();
reader.onload = function (e) {
// it's a file!
};
reader.onerror = function (e) {
// it's a directory!
};
reader.readAsArrayBuffer(file.slice(0, 1));

I also ran into this problem and below is my solution. Basically, I took have a two pronged approach:
(1) check whether the File object's size is large, and consider it to be a genuine file if it is over 1MB (I'm assuming folders themselves are never that large).
(2) If the File object is smaller than 1MB, then I read it using FileReader's 'readAsArrayBuffer' method. Successful reads call 'onload' and I believe this indicates the file object is a genuine file. Failed reads call 'onerror' and I consider it a directory. Here is the code:
var isLikelyFile = null;
if (f.size > 1048576){ isLikelyFile = false; }
else{
var reader = new FileReader();
reader.onload = function (result) { isLikelyFile = true; };
reader.onerror = function(){ isLikelyFile = false; };
reader.readAsArrayBuffer(f);
}
//wait for reader to finish : should be quick as file size is < 1MB ;-)
var interval = setInterval(function() {
if (isLikelyFile != null){
clearInterval(interval);
console.log('finished checking File object. isLikelyFile = ' + isLikelyFile);
}
}, 100);
I tested this in FF 26, Chrome 31, and Safari 6 and three browsers call 'onerror' when attempting to read directories. Let me know if anyone can think of a use case where this fails.

I proposing calling FileReader.readAsBinaryString on the File object. In Firefox, this will raise an Exception when the File is a Directory. I only do this if the File meets the conditions proposed by gilly3.
Please see my blog post at http://hs2n.wordpress.com/2012/08/13/detecting-folders-in-html-drop-area/ for more details.
Also, version 21 of Google Chrome now supports dropping folders. You can easily check if the dropped items are folders, and also read their contents.
Unfortunately, I donĀ“t have any (client-side) solution for older Chrome versions.

One other note is that type is "" for any file that has an unknown extension. Try uploading a file named test.blah and the type will be empty. AND... try dragging and dropping a folder named test.jpg - type will be set to "image/jpeg". To be 100% correct, you can't depend on type solely (or if at all, really).
In my testing, folders have always been of size 0 (on FF and Chrome on 64-bit Windows 7 and under Linux Mint (Ubuntu essentially). So, my folder check is just checking if size is 0 and it seems to work for me in our environment. We also don't want 0-byte files uploaded either so if it's 0 byte the message comes back as "Skipped - 0 bytes (or folder)"

FYI, this post will tell you how to use dataTransfer API in Chrome to detect file type: http://updates.html5rocks.com/2012/07/Drag-and-drop-a-folder-onto-Chrome-now-available

The best option is to use both the 'progress' and 'load' events on a FileReader instance.
var fr = new FileReader();
var type = '';
// Early terminate reading files.
fr.addEventListener('progress', function(e) {
console.log('progress - valid file');
fr.abort();
type = 'file';
});
// The whole file loads before a progress event happens.
fr.addEventListener('load', function(e) {
console.log('load - valid file');
type = 'file';
});
// Not a file. Possibly a directory.
fr.addEventListener('error', function(e) {
console.log('error - not a file or is not readable by the web browser');
});
fr.readAsArrayBuffer(thefile);
This fires the error handler when presented with a directory and most files will fire the progress handler after reading just a few KB. I've seen both events fire. Triggering abort() in the progress handler stops the FileReader from reading more data off disk into RAM. That allows for really large files to be dropped without reading all of the data of such files into RAM just to determine that they are files.
It may be tempting to say that if an error happens that the File is a directory. However, a number of scenarios exist where the File is unreadable by the web browser. It is safest to just report the error to the user and ignore the item.

An easy method is the following:
Check if the file's type is an empty string: type === ""
Check if the file's size is 0, 4096, or a multiple of it: size % 4096 === 0.
if (file.type === "" && file.size % 4096 === 0) {
// The file is a folder
} else {
// The file is not a folder
}
Note: Just by chance, there could be files without a file extension that have the size of some multiple of 4096. Even though this will not happen very often, be aware of it.
For reference, please see the great answer from user Marco Bonelli to a similar topic. This is just a short summary of it.

Related

firebase Google Cloud storage download URL has folder name which becomes file name

We are using Firebase Google Cloud Storage Bucket to store our files.
When the logged in user wants the download the file kept inside certain folder
Eg: 123/admin/1469611803143/123.xlsx
The url generated will be
https://firebasestorage.googleapis.com/v0/b/MYWEBSITE.appspot.com/o/123%2Fadmin%2F1469611803143%2F123.xlsx?alt=media&token=whatever_alpa_numeric_token
As I download this file the file name will be 123%2Fadmin%2F1469611803143%2F123.xlsx
and not 123.xlsx
We have tried using download attribute to change the file name
but this did not change the file name to 123.xlsx
Please HELP
I'm pretty new with firebase but I achieved this with the following code :
var storageRef = firebase.storage().ref();
var child = storageRef.child("your path");
var uploadTask = child.put(<file>);
uploadTask.on(firebase.storage.TaskEvent.STATE_CHANGED,
function(snapshot){
// HANDLE TASK PROGRESS
},
function(error){
// HANDLE ERROR
},
function(){
// UPLOAD SUCCESSFULL
var newMetadata = {
contentDisposition : "attachment; filename=" + fileName
}
child.updateMetadata(newMetadata)
})
This is (fortunately or unfortunately) intended behavior. Technically, files in Firebase Storage are stored with the full path (so 123%2Fadmin%2F1469611803143%2F123.xlsx is actually the file name--the slashes and percent escaping are part of the name, and are only represented as path separators in the UI), which is how we get this behavior.
We're likely to modify how downloads work in the future (in that we'll truncate the name), but we've been busy fixing other bugs and polishing higher priority pieces.

How to programmatically read-write scripts for offline usage in chrome extension?

I need to have predefined scripts, accessible from chrome content_script, that could be updated automatically from given URL.
Exactly what i do:
I have content_script.js. Inside it, i`d like to create iframe for current page from predefined html+css+js.Sometimes html or css or js can be changed. I want to avoid updating extension, instead, each time user have internet, he could load fresh html+css+js for further offline usage.
So, how to read and write some internal files within extension from content script (or delegate this task to background script)?
You can use HTML5 Filesystem to have a read/write place for files, or just store it as strings in chrome.storage (with "unlimitedStorage" permission as needed) for later reuse.
This code can then be executed in a content script using executeScript, or, if you enable 'unsafe-eval' for the extension CSP, in the main script (which is dangerous, and should be avoided in most cases).
Note that this Filesystem API has a warning that's it's only supported in Chrome, but that shouldn't be a problem (Firefox / WebExtensions platform explicitly reject self-update mechanisms).
You can do read extension file contents, but you can't write to extension folder since it is sandboxed.
To read an extension file, you can just send Ajax call using chrome.runtime.getURL("filepath") as url
var xhr = new XMLHttpRequest();
xhr.open('GET', chrome.runtime.getURL('your file path'), true);
xhr.onreadystatechange = function() {
if (chr.readyState === XMLHttpRequest.DONE && xhr.status === 200) {
var text = xhr.responseText;
// Do what you want using text
}
};
xhr.send();

Read user directory in HTML 5 and load images in it

I've been toying around with the FileSystem and File API, in Chrome, to try to implement a transient "instant gallery". The user chooses a directory, and all the images in it are then displayed in the webpage.
But I'm having a hard time, it seems Chrome requires some extra launching arguments to allow file access, FileSystem and File API are not W3C and not portable, I cannot instantiate certain objects...
I cannot even get the directory absolute path to open files in it (though maybe I don't need the absolute path, but I feel like it lacks a good documentation).
Anyway, I wanted to know how to implement this? Is there another API? A simpler way? Do I absolutely need to use FileSystem and File, and set Chrome's arguments?
In order to read the files in the directory you will need to create a DirectoryReader object, and use the readEntries() method to read the content of the directory:
fs.root.getDirectory('Documents', {}, function(dirEntry){<br>
var dirReader = dirEntry.createReader();
dirReader.readEntries(function(entries) {<br>
for(var i = 0; i < entries.length; i++) {
var entry = entries[i];
if (entry.isDirectory){
console.log('Directory: ' + entry.fullPath);
}
else if (entry.isFile){
console.log('File: ' + entry.fullPath);
}
}
}, errorHandler);
}, errorHandler);
Please take a look here: http://code.tutsplus.com/tutorials/toying-with-the-html5-filesystem-api--net-24719
But I think that Chrome will not be able to access an entire directory that the user has selected from his computer, only if the user has selected multiple files in an input field. If that is ok and suits your needs there is a good tutorial here:
http://www.html5rocks.com/en/tutorials/file/dndfiles/

Is it possible to easy get normal (deobfuscated) access to all files in a sandbox written using FileSystem API?

I used Filesystems API to write to a new file in a sandboxed storage of Chrome:
preparing the FS:
window.requestFileSystem = window.requestFileSystem || window.webkitRequestFileSystem;
function errorHandler(e) {
var msg = '';
switch (e.code) {
case FileError.QUOTA_EXCEEDED_ERR:
msg = 'QUOTA_EXCEEDED_ERR';
break;
case FileError.NOT_FOUND_ERR:
msg = 'NOT_FOUND_ERR';
break;
case FileError.SECURITY_ERR:
msg = 'SECURITY_ERR';
break;
case FileError.INVALID_MODIFICATION_ERR:
msg = 'INVALID_MODIFICATION_ERR';
break;
case FileError.INVALID_STATE_ERR:
msg = 'INVALID_STATE_ERR';
break;
default:
msg = 'Unknown Error';
break;
};
console.log('Error: ' + msg);
}
var fileSystem;
function onInitFs(fs) {
console.log('Opened file system: ' + fs.name);
fileSystem = fs;
}
navigator.webkitPersistentStorage.requestQuota(1024*1024,
function(gB){
window.requestFileSystem(PERSISTENT, gB, onInitFs, errorHandler);
}, function(e){
console.log('Error', e);
})
writing the file:
fileSystem.root.getFile('log.txt', {create: true}, function(fileEntry) {
// Create a FileWriter object for our FileEntry (log.txt).
fileEntry.createWriter(function(fileWriter) {
fileWriter.onwriteend = function(e) {
console.log('Write completed.');
};
fileWriter.onerror = function(e) {
console.log('Write failed: ' + e.toString());
};
// Create a new Blob and write it to log.txt.
var blob = new Blob(['Lorem Ipsum'], {type: 'text/plain'});
fileWriter.write(blob);
}, errorHandler);
}, errorHandler);
So afterwards I found a new File in the ./ChromeFolder/FileSystem/003/p/00/00000000 with the Lorem Ipsum content (reading it with a hex-editor).
I thought that I could access the sandboxed FS as a normal mounted FS, so that I have normal File and Directories names. Instead I see some obfuscated file names (00000000 instead of expected log.txt) and not the structure I expected.
Like this:
Is it possible to access this Sandboxed FS as a normal FS, so I could manage all files as I create them in the Chrome using the FileSystems API (I mean structure and file names) or is it impossible and it stays obfuscated for the outside of the Chrome?
Are there any tricks, any flag changes in Chrome to get what I expected?
As with many "How come I can't ______?" sort of questions, the answer is "Security." And the short answer to your question is "no." The File System API was specifically designed solely as a method for web clients (e.g., browsers) to give developers a file system-like storage structure only through the API not from the outside.
Section "4.3 Security Considerations" of the API specification addresses exactly the behavior you're attempting. If the client were to store the raw files with their actual file names (e.g., "FinanceReport.doc") then it would make things much easier for malicious software on a compromised machine to locate and exploit sensitive data stored through the File System API. Also, if the actual file name were used, then it could make those files executable, such as storing "EvilActions.exe" on the local file system with that name. (Note: Some clients, such as Chrome, won't even let you store executables.) These are some of the reasons you see the obfuscation of file and storage. In fact, the API does not explicitly specify how the client should store the data, only that local storage raises security concerns that clients should address.
I recently completed a full scale security assessment of HTML5, including the File System API. I assure you that as the API and client implementation if the API both mature, you will almost certainly see further measures to block or at least obfuscate locally stored data vis-a-vis access from outside the client. To further bolster local storage security, clients may even shift toward storing the whole thing as one big file, similar to how MS Access uses a .mdb file for storage. Again, it's entirely up to the client. Because you're attempting a sort of backdoor access to the data/files apart from the API, and doing so in an manner specifically called out as a "security concern" in the API, it's likely that any "solution" you employ today may fail tomorrow as client security matures. If you can do it for legitimate purposes, malware authors can do is for malicious purposes, and client manufacturers will do what they can to prevent that.

html5 - How to save binary data to html5 file system

i want to save a dropped image into the html5 filesystem. The problem is that i do not know how to write binary data into the file system with the file api. Below are parts of my code:
var reader = new FileReader();
reader.onloadend = function(e)
{
var data = e.target.result;
// Open File System code (Codes Omitted)
// Create file (Codes Omitted)
fileEntry.createWriter(function(fileWriter)
{
// This is the problem, the file is create, something is written to the file
// because the size is exact same with the dragged file (image). But when i
// try to view the file (image), it displays nothing.
var bb = new WebKitBlobBuilder
bb.append(data);
fileWriter.write(bb.getBlob(""));
}
}
reader.readAsBinaryString(cur_file);
I intentionally left out some codes like the file system and file creation. I only need help with saving binary data.
Thanks.
I've encounter a similar issue
Since you didn't parse your code here,I assume that you read the image in a wrong way,
maybe
You read the cur_file as binary string,Did your fileReader read it by readAsText method
Since the file stores binaryString,If your fileReader read it by another way,e.g readAsDataURL,that means you encode it again,so you couldn't get the right answer