Saving image blob by decoding base64 with sequelize v5 - mysql

I want to save received base64 string in mysql table in a BLOB field. I have used sequelize version 5 as my ORM and the model is defined as follows.
sequelize.define('PLAYER', {
player_id: {
primaryKey: true,
type: DataTypes.INTEGER,
allowNull: false,
autoIncrement: true
},
player_name: DataTypes.STRING,
player_image: {
type: DataTypes.BLOB('medium'),
get () {
let data = this.getDataValue('player_image');
return data ? data.toString('base64') : '';
}
}
}
In my put method,
db.PLAYER.findByPk(req.params.playerId).then((player)=>{
if (req.body.player_image) {
const base64 = req.body.player_image.replace(/^data:image\/[a-z]+;base64,/, "");
const blob = b64toBlob(base64, 'image/jpeg');
player.player_image = blob;
}
player.player_name = req.body.player_name;
player.save().then(() => {
res.status(200).json(player);
}).catch(function (err) {
return next(err);
});
});
For Base64 to Blob conversion.
const b64toBlob = (b64Data, contentType = '', sliceSize = 512) => {
const byteCharacters = atob(b64Data);
const byteArrays = [];
for (let offset = 0; offset < byteCharacters.length; offset += sliceSize) {
const slice = byteCharacters.slice(offset, offset + sliceSize);
const byteNumbers = new Array(slice.length);
for (let i = 0; i < slice.length; i++) {
byteNumbers[i] = slice.charCodeAt(i);
}
const byteArray = new Uint8Array(byteNumbers);
byteArrays.push(byteArray);
}
const blob = new Blob(byteArrays, { type: contentType });
return blob;
}
A blob is saving in the table but my response body is as follows. I have converted BLOB to base64 before sending the response.
{
"player_id": 1032,
"player_name": "Oliver Driscoll",
"player_image": "[object Blob]"
}
Which means the saved blob is actually "[object Blob]". Not the image.
The received base64 (req.body.player_image) for the end point is confirmed as correct. I am running out of time and don't know what to do as this is the requirement (Saving the image as blob).

I Know, this is a old issue. But i had this same problem, and after so many tries, i was able to solve it. So, if anyone have this problem in the future, i will let my solution here:
In my case i was able to get a Blob already, from a fetch response.
So i have in my code like this:
await fetch(imageUrl)
.then(async (response) => {
// GET Blob Object from response; can use response.arrayBuffer() directly too
let myBlobImg = await response.blob();
// Convert Blob.ArrayBUffer to a Buffer
let myBufferImg = Buffer.from(await myBlobImg.arrayBuffer());
})
Then i just save myBufferImg on DB, as a Buffer. In your case #silent_27 could be
player.player_image = Buffer.from(await myBlobImg.arrayBuffer());
I'm using SQlite3 DataBase on NodeJS with sqlite3^5.0.1 package.
I save like this on my dataBase and works fine. I've tried save like myBlobImg.text();. But this away, i was not able to load on a <img src={}> html tag in my application.
If anyone need to know, to show this BLOB on a <img> HTML, i just do a get on my DB to get my ProductObject and convert to a base64 image:
// imageData is the Buffer saved on BD. imageData.data is an Array from Buffer
let imgBase64 = Buffer.from(product.productImage.imageData.data).toString('base64');
imgBase64 = `data:image/jpeg;base64,${imgBase64}`;
//....
<img src={imgBase64}>
I Had to use Buffer.from again otherwhise if i just use like
let imgBase64 = product.productImage.imageData.toString('base64');
the imgBase64 would be "[Object ojbect]

Related

Convert JavaScript array data to json file

So I have this array with different links
const urls = ['myurl.com', 'localhost.com'... etc]
In the for loop I want to create an object like so, it should basically create a new object for every URL, and then pass the looped URL to the userUrl section
for(let url of urls) {
[
{
user: 1,
isVerified: true
userUrl: url
}
]
}
After the loop finishes, this data should be readable in a JSON file
it should look something like this
[
{
user: 1,
isVerified: true,
userUrl: myUrl.com
},
{
user: 2,
isVerified: true,
userUrl: localhost.com
}
...etc
]
I tried this code on Chrome and it works correctly plus it correctly format the json data now, instructing JSON.stringify to use 4 spaces indentation.
It won't work in the snippet, but it will if you save it in your own file. I did it inside the chrome developer tools in the Sources tab as a snippet and as soon as executed the download queue was fed with the json file.
I left the live snippet here because there's the chance to see the jsonData on console anyway.
The way to programmatically send a string to a download file was inspired by this question:
How do I save JSON to local text file
function download(content, fileName, contentType) {
var a = document.createElement("a");
var file = new Blob([content], {type: contentType});
a.href = URL.createObjectURL(file);
a.download = fileName;
a.click();
}
const urls = ['myurl.com', 'localhost.com'];
const data = factory(urls);
const jsonData = JSON.stringify(data, null, 4);
console.log(jsonData);
download(jsonData, 'json.txt', 'text/plain');
function factory(urls){
const output = [];
for(let url of urls) {
output.push({
user: 1,
isVerified: true,
userUrl: url
});
}
return output;
}

How to save & retrive image to/from mysql?

Two part quersion.
Part 1:
Im uploading an image to my server and want to save it to my database.
So far:
table:
resolver:
registerPhoto: inSequence([
async (obj, { file }) => {
const { filename, mimetype, createReadStream } = await file;
const stream = createReadStream();
const t = await db.images.create({
Name: 'test',
imageData: stream ,
});
},
])
executing query:
Executing (default): INSERT INTO `images` (`Id`,`imageData`,`Name`) VALUES (DEFAULT,?,?);
But nothing is saved.
Im new to this and im probably missing something but dont know what.
Part2:
This is followed by part 1, lets say I manage to save the image, how do I read it and send it back to my FE?
An edit: Ive read alot of guides saving the an image name to the db and then tha actuall image in a folder. This is NOT what im after, want to save the image to the DB and then be able to fetch it from the DB abd present it.
This took me some time but I finaly figured it out.
First step (saving to the db):
Have to get the entire stream data and read it like this:
export const readStream = async (stream, encoding = 'utf8') => {
stream.setEncoding('base64');
return new Promise((resolve, reject) => {
let data = '';
// eslint-disable-next-line no-return-assign
stream.on('data', chunk => (data += chunk));
stream.on('end', () => resolve(data));
stream.on('error', error => reject(error));
});
};
use like this:
const streamData = await readStream(stream);
Before saving I tur the stream into a buffer:
const buff = Buffer.from(streamData);
Finaly the save part:
db.images.create(
{
Name: filename,
imageData: buff,
Length: stream.bytesRead,
Type: mimetype,
},
{ transaction: param }
);
Note that I added Length and Type parameter, this is needed if you like to return a stream when you return the image.
Step 2 (Retrieving the image).
As #xadm said multiple times you can not return an image from GRAPHQL and after some time I had to accept that fact, hopefully graphql will remedy this in the future.
S What I needed to do is set up a route on my fastify backend, send a image Id to this route, fetch the image and then return it.
I had a few diffirent approaches to this but in the end I simpy returned a binary and on the fronted I encoded it to base64.
Backend part:
const handler = async (req, reply) => {
const p: postParams = req.params;
const parser = uuIdParserT();
const img = await db.images.findByPk(parser.setValueAsBIN(p.id));
const binary = img.dataValues.imageData.toString('binary');
const b = Buffer.from(binary);
const myStream = new Readable({
read() {
this.push(Buffer.from(binary));
this.push(null);
},
});
reply.send(myStream);
};
export default (server: FastifyInstance) =>
server.get<null, any>('/:id', opts, handler);
Frontend part:
useEffect(() => {
// axiosState is the obj that holds the image
if (!axiosState.loading && axiosState.data) {
// #ts-ignore
const b64toBlob = (b64Data, contentType = '', sliceSize = 512) => {
const byteCharacters = atob(b64Data);
const byteArrays = [];
for (let offset = 0; offset < byteCharacters.length; offset += sliceSize) {
const slice = byteCharacters.slice(offset, offset + sliceSize);
const byteNumbers = new Array(slice.length);
// #ts-ignore
// eslint-disable-next-line no-plusplus
for (let i = 0; i < slice.length; i++) {
byteNumbers[i] = slice.charCodeAt(i);
}
const byteArray = new Uint8Array(byteNumbers);
byteArrays.push(byteArray);
}
const blob = new Blob(byteArrays, { type: contentType });
return blob;
};
const blob = b64toBlob(axiosState.data, 'image/jpg');
const urlCreator = window.URL || window.webkitURL;
const imageUrl = urlCreator.createObjectURL(blob);
setimgUpl(imageUrl);
}
}, [axiosState]);
and finaly in the html:
<img src={imgUpl} alt="NO" className="imageUpload" />
OTHER:
For anyone who is attempting the same NOTE that this is not a best practice thing to do.
Almost every article I found saved the images on the sever and save an image Id and other metadata in the datbase. For the exact pros and cons for this I have found the following helpful:
Storing Images in DB - Yea or Nay?
I was focusing on finding out how to do it if for some reason I want to save an image in the datbase and finaly solved it.
There are two ways to store images in your SQL database. You either store the actual image on your server and save the image path inside your mySql db OR you create a BLOB using the image and store it in db.
Here is a handy read https://www.technicalkeeda.com/nodejs-tutorials/nodejs-store-image-into-mysql-database
you should save the image in a directory and save the link of this image in the database

Ionic 4 : Recording Audio and Send to Server (File has been corrupted)

I would like to record the audio file in mobile application(iOS & Android) and tranfer to server as a formData in ionic 4. I have used the "cordova-plugin-media" to capture the audio using below logics
if (this.platform.is('ios')) {
this.filePaths = this.file.documentsDirectory;
this.fileExtension = '.m4a';
} else if (this.platform.is('android')) {
this.filePaths = this.file.externalDataDirectory;
this.fileExtension = '.3gp';
}
this.fileName = 'recording'+new Date().getHours()+new Date().getMinutes()+new Date().getSeconds()+this.fileExtension;
if(this.filePaths) {
this.file.createFile(this.filePaths,this.fileName,true).then((entry:FileEntry)=> {
this.audio = this.media.create(entry.toInternalURL());
this.audio.startRecord();
});
}
Even I have tried to create the media directly without "File Creation"
I can record and play the audio, but If I am trying to send this file
to server using below logics It won't send properly(corrupted data)
and also web application unable to play .m4a extensions
.
Please correct me If I am doing anything wrong in my code
Upload logic:
let formData:FormData = new FormData();
formData.append('recordID' , feedbackID);
that.file.readAsDataURL(filePath,file.name).then((data)=>{
const audioBlob = new Blob([data], { type: file.type });
formData.append('files', audioBlob, file.name);
that.uploadFormData(formData,feedbackID); //POST Logics -
})
;
I have used the soultion as suggested by Ankush and it works fine.
Used readAsArrayBuffer instead of readAsDataURL.
The .m4a format has supported both ios and android. Also I can
download the the same file from web application.
I am using below code to upload the image to the server. I assume that only a few modifications will be required in this code to transfer media instead of the image file.
private uploadPicture(imagePath: string, apiUrl: string): Observable<ApiResponse<ImageUploadResponseModel>> {
return this.convertFileFromFilePathToBlob(imagePath).pipe(
switchMap(item => this.convertBlobToFormData(item)),
switchMap(formData => this.postImageToServer(formData, apiUrl))
);
}
Rest functions used in above code:
private postImageToServer(formData: FormData, apiUrl: string): Observable<ApiResponse<ImageUploadResponseModel>> {
const requestHeaders = new HttpHeaders({ enctype: 'multipart/form-data' });
return this.http.post<ApiResponse<ImageUploadResponseModel>>(apiUrl, formData, { headers: requestHeaders });
}
private convertBlobToFormData(blob: Blob): Observable<FormData> {
return new Observable<FormData>(subscriber => {
// A Blob() is almost a File() - it's just missing the two properties below which we will add
// tslint:disable-next-line: no-string-literal
blob['lastModifiedDate'] = new Date();
// tslint:disable-next-line: no-string-literal
blob['name'] = 'sample.jpeg';
const formData = new FormData();
formData.append('file', blob as Blob, 'sample.jpeg');
subscriber.next(formData);
subscriber.complete();
});
}
private convertFileFromFilePathToBlob(imagePath: string): Observable<Blob> {
return new Observable<Blob>(subscriber => {
const directoryPath = imagePath.substr(0, imagePath.lastIndexOf('/'));
let fileName = imagePath.split('/').pop();
fileName = fileName.split('?')[0];
this.file.readAsArrayBuffer(directoryPath, fileName).then(fileEntry => {
const imgBlob: any = new Blob([fileEntry], { type: 'image/jpeg' });
imgBlob.name = 'sample.jpeg';
subscriber.next(imgBlob);
subscriber.complete();
}, () => {
subscriber.error('Some error occured while reading image from the filepath.');
});
});
}

nodejs piping stream after modifying data

I am learning about streaming with nodejs, I understand the examples shown in the request npm module;
request(url).pipe(fs.createWriteStream('./filename.json'))
But there are two parts of my problem.
Case 1:
function fetchSitemaps() {
return requestAsync(url).then(data => {
const $ = cheerio.load(data);
let urls = [];
$("loc").each((i, e) => urls.push($(e).text()));
fs.writeFileSync('./sitemaps.json', JSON.stringify(urls))
})
}
I want to convert the above from writeFileSync to createWriteStream, but how do I keep appending data to an array which is in JSON format?
Case 2:
function fetchLyricUrls() {
let sitemaps = JSON.parse(fs.readFileSync('./sitemaps.json'));
sitemaps.forEach((sitemap, i) => {
let fileName = i + '.json';
if(url_pat.exec(sitemap)) {
fileName = url_pat.exec(sitemap)[1] + '.json';
}
requestAsync(url).then(data => {
const $ = cheerio.load(data);
let urls = [];
$("loc").each((i, e) => urls.push($(e).text()));
return urls;
}).then(urls => {
let allUrls = [];
urls.map(u => {
return requestAsync(u).then(sm => {
const $ = cheerio.load(sm);
$("loc").each((i, e) => allUrls.push($(e).text()))
fs.writeFileSync('./lyrics.json', JSON.stringify(allUrls))
return allUrls;
});
});
});
});
}
The first part of the problem is same, appending to a json data using writeStream, but this time, I want to parse the the html data and get some text, which I want to send using stream, not the html data as a whole.
So let's split up the answers
Case 1
First of all I'd try to keep the data as a stream and try not to accumulate it. So in essence, instead of loading the whole sitemap and then parsing it, I'd use something like the xml-nodes so that the nodes are a separate stream. Then my module scramjet would come to transform
const request = require('request');
const xmlNodes = require('xml-nodes');
const writable = fs.createWritableStream('./sitemaps.json');
const cheerio = require('cheerio');
const scramjet = require('scramjet');
writable.write('[');
let first = 0;
request('http://example.com/sitemap.xml')
// this fetches your sitemap
.on('end', () => writable.end("]"))
// when the stream ends, this will end the sitemaps.json
.pipe(xmlNodes('loc'))
// this extracts your "loc" nodes
.pipe(new scramjet.DataStream())
// this creates a mappable stream
.map((nodeString) => cheerio('loc', nodeString).text())
// this extracts the text as in your question
.map((url) => (first++ ? ',' : '') + JSON.stringify(url))
// this makes sure that strings are nicely escaped
// and prepends them with a comma on every node, but first one
.pipe(writable, {end: false})
// and this will push all your entries to the writable stream
Case 2
Here you'll need to do something similar, although if case 1 is an immediate step, then I'd suggest to store the files in lines of JSONs, not an array. It'd make easier to stream that way.

extract data from json

I have a json data coming from wcf servicein jquery like this
GetBedTypeList1Result1 is function in wcf
{
"GetBedTypeList1Result":[
{"Code":23,"CompanyCode":null,"Decode":"1 Class New Born Bed","DivisionCode":0,"LocationCode":0,"admDueDepAmt":0,"bedTypeCode":0,"caseTypeCode":0,"caseTypeDecode":null,"ptnClassCode":0,"ptnClassDecode":null,"rsvDueDepAmt":0},
{"Code":22,"CompanyCode":null,"Decode":"1st Class Bed","DivisionCode":0,"LocationCode":0,"admDueDepAmt":0,"bedTypeCode":0,"caseTypeCode":0,"caseTypeDecode":null,"ptnClassCode":0,"ptnClassDecode":null,"rsvDueDepAmt":0},
{"Code":5,"CompanyCode":null,"Decode":"Classique Bed","DivisionCode":0,"LocationCode":0,"admDueDepAmt":0,"bedTypeCode":0,"caseTypeCode":0,"caseTypeDecode":null,"ptnClassCode":0,"ptnClassDecode":null,"rsvDueDepAmt":0}
],
"strErrMsg":"Y",
"chrErrFlg":"c"
}
I am calling service like below
function CallWcfService() {
//alert("CallWcfServicexxxx");
jQuery.ajax
(
{
type: Type,
url: Url,
data: parameters,
contentType: ContentType, // content type sent to server
dataType: DataType, //Expected data format from server
cache: "false",
crossDomain: true, //Same result if i remove this line
processdata: ProcessData, //True or False
success: function (msg)
{
ServiceSucceeded(msg);
},
error: ServiceFailed// When Service call fails
}
);
}
function callService()
{
DataType = "json";
Type = "GET";
var par = 4;
parameters = null;
Url = "http://192.168.2.42/CWSERVERWCF/bedtypemasterService.svc/GetBedTypeList?callback=?";
parameters = "{'strErrMsg':'1'},{'chrErrFlg':'A'},{'pcompanycode':'0'},{'pdiv':'1'},{'ploc':'1'}";
// alert(parameters);
ContentType = "application/json; charset=utf-8";
ProcessData = true;
//alert("sssssasasadsds");
CallWcfService();
}
I am trying to fetch data but not getting lke below
function ServiceSucceeded(result)
{
if (DataType == "json")
{
var obj = jQuery.parseJSON(JSON.stringify(JSON.stringify(result)));
for (var x = 0; x < obj.length; x++)
{
}
}
}
In obj.length count of characters is coming and jQuery.parseJSON(result) is not working
Please help
If the result is json there is no need to parse it in this way. jQuery.ajax will return a javascript object if the datatype is set to json.
So in your ServiceSucceeded function you may operate on the result variable directly. If you are trying to iterate over the bed types change your for loop to something like this:
for (var i = 0; i < result.GetBedTypeList1Result.length; i++) {
// ...do stuff
// var bed = result.GetBedTypeList1Result[i]]
}
Try using JSON.parse(result) instead of: var obj = jQuery.parseJSON(JSON.stringify(JSON.stringify(result)));
Also, since you've mentioned the dataType as 'json' in your $.ajax call, your response should already be in JSON format with no parsing required.