I was wondering if it was ok to encode json data as a blob (using atob) alongside some other binary data to the server, and then reload the json file file into memory to save it in the database?
I am doing this in order to prevent multiple posts to the server, I could for example, send the json data, validate it, and when it's OK, I could send the files with some Ids, and then link the both of them server side.
but this method seems broken, because if the user leaves the browser while he is sending the data, and part of the files have been sent, then i'd have alot of uncomplete posts.
Thanks for your answers in advance.
Related
I'm working on a back-end feature that involves processing CSV files uploaded by users. Most tutorials I've found so far suggest that I should read that CSV file through a multipart request.
https://www.appcoda.com/restful-api-tutorial-how-to-upload-files-to-server/
How does HTTP file upload work?
However, as far as I know, multipart requests involve a boundary and only make sense when we need to send different kinds of payload over the same request. For CSV file uploading, all I need is to send a byte stream over the request body (with the appropriate text/csv content type).
I'm not sure if there is any specific reason that people use and suggest multipart requests for uploading files?
if you use text/csv your request body should be raw text (like with json) like
"1,2,3
4,5,6"
But if you want upload file, content-type should say about it.
so, in that case i recomend to use multipart/form-data when uploading any type of files
I am trying to upload files(.txt, .pdf) in Elasticsearch. Elasticsearch receives only content in json format. Is there any way that I send parsed content(.pdf or .txt to String) directly or I have to parse String into json document to send it to Elasticsearch.
You can only send JSON when indexing a Document, so basically, a base64 encoded version of the file in some field of that JSON will do just fine. If you do not wish to search inside this content, then all you have to do is disabling indexing on that "binary data" field (option index:false in your mapping).
If you wish to send a PDF file and have the textual content extracted and indexed / searchable, you should have a look at the Ingest Attachment Plugin.
You can look at this https://github.com/dadoonet/fscrawler for your use case.
Basically, This crawler helps to index binary documents such as PDF, Open Office, MS Office and will give you following feature
Local file system (or a mounted drive) crawling and index new files,
update existing ones and removes old ones.
Remote file system over SSH crawling.
REST interface to let you "upload" your binary documents to
elasticsearch.
There does not appear to be a dedicated field type in Airtable for "meta" data blobs and/or a JSON string.
Is the "Attachment" type my best bet?
I could store it either as a json attachment, or on a String type column.
Since a full json on a text column would likely not be readable, I would store it as attachments.
However, it seems that at least for now, uploading attachments require the file to be already hosted somewhere first, so this route might not be the easiest one:
https://community.airtable.com/t/is-it-possible-to-upload-attachments/188
Right now this isn’t possible with the Airtable API alone. It’s
something we’ll think about for future API versions though. A
workaround for now is to use a different service
(e.g. Filestack90, imgur52, etc.) to process the upload before then
sending the url to Airtable. When Airtable processes the attachment,
it will copy the file to Airtable’s own (S3) server for safekeeping,
so it’s OK if the original uploaded file url is just temporary
i want to handle a requirement in polymer webcomponents where user can upload csv file from ui and csv file can be parsed to json and sent to server ,i searched and found for vaadin upload,looked over the api but i am not sure how to receive the csv file and convert to json and sent to server,can anyone show a jsfiddle of vaadin upload or any other web component to handle this scenario?
First of all, I am wondering why you would not simply do the conversion on the server side.
In this case, you would be able to use the vaadin-upload directly indeed.
Here is a snippet that would upload all files to the example.com server, and only allow CSV files.
<vaadin-upload target="https://example.com/upload" method="POST" accept="text/csv">
</vaadin-upload>
There are plenty of resources on how to convert CSV files to JSON.
Here is a snippet
And here is a node library
If you really wanted to do the conversion client side, then I would suggest to create an element that would embed a vaadin-upload, and convert the Files array to Json before manually calling the uploadFiles method.
I have a form where I simultaneously send several (let's say 10) files to the backend, along with some other regular input fields.
When I find a validation problem in the backend I display the form again and fill the regular inputs (dropdowns, text inputs, etc.), but I cannot fill the file fields, forcing the user to select the files from the file directory again.
The solution I think of is sending the base64 encoded representation of the files, and put them back in the form in case there is a validation error, but I would like to know if there is a simpler way.
No matter what you do, you can not skip server-side validation. For security reasons (someone sending raw requests for example). As far as showing the files in case of error (or simple reloading) I would send it as Base64, but via AJAX if the files are large.
Client-side validation: most validations (if not all) can be done using plain JS. Be it filesize, content type... and more specific validations according to filetypes (image, document...)
Server-side validation: Once sent, fully validate the file and content.
Send files back:In case of error, send the files with the request if small. If big send them via AJAX to avoid blocking and keep page loading times fast.
Hope it helps!
The simplest thing to do would be :
1) Validate all fields on the client side (javascript/jquery)
2) Once all other inputs are validated, then you send in Files.
NB: If you need to validate some values on the server side, use ajax for both steps (1) and (2). Also, try to give a bit more details to get quality answers
I would create a temporary store for uploaded files and would send back only a unique-hard-to-guess ID got from the store instead than the whole file.
The form section for a single file upload would then include also an hidden input carrying the unique file-ID.
When processing the request: if there is a file in the request get the file (and store it if form not valid) otherwise if there is a file-ID get the file from store.