I'm currently using Crypto to encrypt/ decrypt data, but, if the server restarts, the decrypt won't work anymore. That's what i'm currently using =>
const crypto = require("crypto");
const algorithm = "aes-256-cbc";
const initVector = crypto.randomBytes(16);
const Securitykey = crypto.randomBytes(32);
function encrypt(text){
const cipher = crypto.createCipheriv(algorithm, Securitykey, initVector);
let encryptedData = cipher.update(text, "utf-8", "hex");
encryptedData += cipher.final("hex");
return encryptedData;
}
function decrypt(text){
const decipher = crypto.createDecipheriv(algorithm, Securitykey, initVector);
let decryptedData = decipher.update(text, "hex", "utf-8");
decryptedData += decipher.final("utf8");
return decryptedData;
}
And this is the error I get if i want to decrypt something after server restart
Error: error:06065064:digital envelope routines:EVP_DecryptFinal_ex:bad decrypt
So as I can see from the code your IV and Key are randomly generated and I am assuming that you are not saving them anywhere.
const initVector = crypto.randomBytes(16);
const Securitykey = crypto.randomBytes(32);
So basically on server restart you are getting a new pair of IV and key, so when you are decrypting it is not matching with the Key and IV used at the time of encryption.
My suggested solution :
const crypto = require("crypto");
const algorithm = "aes-256-cbc";
const initVectorString = "Any random hex string of 16bytes"; // You can store this into a env file
const SecuritykeyString = "Random security hex string of 32bytes"; // You can store this into a env file
const initVector = Buffer.from(initVectorString, "hex");
const Securitykey = Buffer.from(SecurityKeyString, "hex");
function encrypt(text){
const cipher = crypto.createCipheriv(algorithm, Securitykey, initVector);
let encryptedData = cipher.update(text, "utf-8", "hex");
encryptedData += cipher.final("hex");
return encryptedData;
}
function decrypt(text){
const decipher = crypto.createDecipheriv(algorithm, Securitykey, initVector);
let decryptedData = decipher.update(text, "hex", "utf-8");
decryptedData += decipher.final("utf8");
return decryptedData;
}
Update:-
So if you are using a utf-8 string for IV then the string length should be 16 characters only (if you are using only 1 byte characters a-zA-Z0-9 all are 1 byte characters) and you need to change the encoding type in Buffer.from() function from "Hex" to "utf-8".
Similar for the security key length of the string should be 32 characters only and you need to change the encoding type in Buffer.from() function from "Hex" to "utf-8".
Related
On the front end of my app I wanted to parse some data related to a CSV they upload. Through the file upload tool, I first get a FileList object and then pull the 1 file out of it.
I want to turn it into a json object which I could then iterate. I was thinking to user csv-parser from node, but I dont see a way to leverage a File object stored in memory.
How Can I accomplish this?
At first I was doing:
let f = fileList.item(0);
let decoder = new window.TextDecoder('utf-8');
f.arrayBuffer().then( data => {
let _data = decoder.decode(data)
console.log("Dataset", data, _data)
});
And that was passing the array buffer, and decoding the string. While I Could write a generic tool which process this string data based on \n and ',' I wanted this to be a bit more easier to read.
I wanted to do something like:
let json = csvParser(f)
is there a way to user csv-parser from node, (3.0.0) or is there another tool i should leverage? I was thinking that levering modules based on the browser ( new window.TextDecoder(...) ) is poor form since it has the opportunity to fail.
Is there a tool that does this? im trying to create some sample data and given a File picked from an input type="file" i would want to have this be simple and straight forward.
This example below works, but i feel the window dependancy and a gut feeling makes me think this is naive.
const f : File = fileList.item(0)
console.log("[FOO] File", f)
let decoder = new window.TextDecoder('utf-8');
f.arrayBuffer().then( data => {
let _data = decoder.decode(data)
console.log("Dataset", data, _data)
let lines = _data.split("\n")
let headers = lines[0].split(',')
let results = []
for ( let i = 1; i < lines.length; i++) {
let line = lines[i]
let row = {}
line.split(",").forEach( (item, idx) => {
row[headers[idx]] = item;
})
results.push(row)
}
console.log("JSON ARRAY", results)
})
The issue i run when i stop and do: ng serve is that it does not like using the arrayBuffer function and accessing TextDecoder from window, since that thost functions/classes are not a part of File and window respectively during build.
Any thoughts?
This is what I ended up doing, given the file input being passed into this function:
updateTranscoders(project: Project, fileList: FileList, choice: string = 'replace') {
const f: File = fileList.item(0)
//Reads a File into a string.
function readToString(file) : Promise<any> {
const reader = new FileReader();
const future = new Promise( (resolve,reject) => {
reader.addEventListener("load", () => {
resolve(reader.result);
}, false)
reader.addEventListener("error", (event) => {
console.error("ERROR", event)
reject(event)
}, false)
reader.readAsText(file)
});
return future;
}
readToString(f).then( data => {
let lines = data.split("\n")
let headers = lines[0].split(',')
let results = []
for (let i = 1; i < lines.length; i++) {
let line = lines[i]
let row = {}
line.split(",").forEach((item, idx) => {
row[headers[idx]] = item;
})
results.push(row)
}
if (choice.toLowerCase() === 'replace'){
let rows = project.csvListContents.toJson().rows.filter( row => row.isDeployed)
rows.push( ...results)
project.csvListContents = CsvDataset.fromJson({ rows: rows })
}else if (choice.toLowerCase() === 'append') {
let r = project.csvListContents.toJson();
r.rows.push(...results);
project.csvListContents = CsvDataset.fromJson(r);
}else {
alert("Invalid option for Choice.")
}
this.saveProject(project)
})
}
Now the CHOICE portion of the code is where I have a binary option to do a hard replace on CSV contents or just append to it. I would then save the project accordingly. This is also understanding that the first row contains column headers.
I am trying to insert array in my firebase collection from cloud function. I need to have multiple lines in one document so for each line i am inserting an array. Please check my attached screenshot where you can see line0 , same way i need to have Line1,Line2,Line3..,Line n in the same document.
for line0 i am passing array from code like below and its working fine.
admin.firestore().collection("qbContestWinners").add(
{
'cmpientryid': context.params.processId,
'qbid': '',
'qbsyncdate': '',
'qbsyncstatus': 'pending',
'Line0':
{
id: "0",
description: 'PRIZE AMOUNT',
amount: 1000,
accountrefid: contestresultData.qbcontestid,
accountrefname: contestresultData.qbcontestname,
contestresultId: context.params.processId,
},
})
when i am looping through data i am getting from another table , i am not able to generate proper JSON to insert.
below is how i am looping and creating JSON after getting data from another table.
i = 1;
admin.firestore().collection("results").where('cid', '==', 'LKRRk2XXXXXXXX')
.orderBy("rank", "asc").get().then(snapshots =>
{
snapshots.forEach(doc =>
{
const contestresultId = doc.id;
const prizeAmount = doc.data().prizeamt;
const userId = doc.data().userid;
const lineNum = "Line" + i;
console.log("new line numner is: ", lineNum);
console.log(`lineNum? ${lineNum}`);
const linetxt = "Line" + String(i);
const insertData = "{"+linetxt +
":{id:'" + i +
"', description: 'PRIZE AMOUNT'"+
", amount:" + prizeAmount + "," +
"accountrefid:"+ contestresultData.qbcontestid +","+
"accountrefname:'" +contestresultData.qbcontestname +"',"+
"contestresultId:'" + contestresultId +"'," +
"},}"
const finalInsert = JSON.stringify(insertData);
const finalJSON = JSON.parse(finalInsert);
admin.firestore().collection("qbContestWinners").doc(mainID).set(
finalInsert.toJSON(),
{
merge: true
});
i= i+1;
});
});
using this code i am getting error
finalInsert.toJSON is not a function
Actually, the Line0 field is a map and not an Array, see this doc for more details.
So, if you want to create similar fields (Line1, Line2, ...), you simply need to pass a JavaScript Object to the set() method, as follows:
snapshots.forEach(doc => {
const contestresultId = doc.id;
const prizeAmount = doc.data().prizeamt;
const userId = doc.data().userid;
const lineNum = "Line" + i;
console.log("new line numner is: ", lineNum);
console.log(`lineNum? ${lineNum}`);
const lineObj = {
id: i,
description: 'PRIZE AMOUNT',
accountrefid: contestresultData.qbcontestid, //Not sure if you have defined contestresultData somewhere...
//...
}
const dataObj = {};
dataObj["Line" + i] = lineObj // See https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Property_accessors
admin.firestore().collection("qbContestWinners").doc(mainID).set(dataObj, {merge: true});
i= i+1;
});
HOWEVER, note that you must return a promise that resolves when all the asynchronous work in your Cloud Function is complete (i.e. call to the Firestore set() method).
This is explained in the official Firebase video series, watch in particular the three videos titled "Learn JavaScript Promises".
Since you are calling several times the set() method in a forEach loop, you need to use Promise.all() in order to return a Promise when all these parallel calls to the set() method are completed.
The following should do the trick:
let i = 1;
return admin.firestore().collection("results") // <-- See the return here
.where('cid', '==', 'LKRRk2XXXXXXXX')
.orderBy("rank", "asc").get()
.then(snapshots => {
const promises = [];
snapshots.forEach(doc => {
const contestresultId = doc.id;
const prizeAmount = doc.data().prizeamt;
const userId = doc.data().userid;
const lineNum = "Line" + i;
const lineObj = {
id: i,
description: 'PRIZE AMOUNT',
accountrefid: contestresultData.qbcontestid,
//...
}
const dataObj = {};
dataObj[lineNum] = lineObj;
promises.push(admin.firestore().collection("qbContestWinners").doc(mainID).set(dataObj, {merge: true}));
i= i+1;
});
return Promise.all(promises) // <-- See the return here
});
A last remark: if mainID keeps the same value in the snapshots.forEach loop, you may adopt a totally different approach, consisting in building a JavaScript object with several LineXX properties and call the set() method only once. Since you didn't share the entire code of your Cloud Function it is impossible to say if this approach should be used or not.
first to the error
You stringify and parse a string. The problem here seems to be the order. You have to parse a "String" and to stringify an "Object". The result won't have a toJSON Method as well, but u can just stringify the Object to get a json.
the second thing
Why do you use a string to create your object? You shouldn't. Just use an object.
the third thing
You should not use Objects as Arrays. Not even in firebase.
Just use arrays. Example:
[Line0Object, Line1Object, ...]
Hint: If your array can work as its own collection. Just use a SubCollection. This might fit your needs.
I have a text file. I need to read the file inside a function and return it as a JSON object. The following is throwing an error "Unexpected token V in JSON at position 0" .
Server.js
fs.readfile('result.txt', 'utf8', function(err,data) {
if(err) throw err;
obj = JSON.parse(data);
console.log(obj);
});
result.txt looks like the following
VO1: 10 5 2
VO2: 5 3 2
I think I cannot use JSON.parse directly. How do I proceed?
Assuming the following:
Every line is separated by a newline character (\n)
Every line is separated by a : where the part in front of it is the key and the part behind it is a (space) separated string that should indicate the keys values as an array.
Below should work for your format:
fs.readfile('result.txt', 'utf8', function(err,data) {
if(err) throw err;
let obj = {};
let splitted = data.toString().split("\n");
for (let i = 0; i<splitted.length; i++) {
let splitLine = splitted[i].split(":");
obj[splitLine[0]] = splitLine[1].trim();
}
console.log(obj);
});
It could be issue with UTF-8 string format, Tried below code and it works
const resultBuffer = fs.readFileSync('result.txt');
const resultData = JSON.parse(resultBuffer.toString().trim());
Thanks to Baao for providing that answer.
As another flavor of solution, if you don't have any ":" for perhaps a list of files you could always code in a key like so:
var data = fs.readFileSync(pathAndFilename);
var testData = {};
var splitList = data.toString().split('\r\n');
for (var i = 0; i < splitList.length; i++) {
testData['fileNumber' + i.toString()] = splitList[i];
}
You need to parse the text file by yourself. You can use RegExp or some other means to extract the values, create an object out of that and then JSON.stringify it.
improving upon #baao answer:
const fs = require("fs")
fs.readFile('.czrc', 'utf8', function (err, data) {
if (err) {
console.error(err)
throw "unable to read .czrc file.";
}
const obj = JSON.parse(data)
});
Your result.txt is not valid json.
Valid json would look like this.
{
"VO1": [10, 5, 2],
"VO2": [5, 3, 2]
}
I'm having some trouble here while trying to decode some encrypted text.
CheckpswdBasedKey is always returning false, because of the BadPaddingException at c.doFInal
I'm using AES, basicaly the encryption:
public static String generatePswdBasedKey(String password){
String finalKey = null;
SecretKey sk = null;
KeySpec keySpec = new PBEKeySpec(password.toCharArray(), salt, IT, KEY_LENGTH);
SecretKeyFactory keyFactory = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA1");
byte[] keyBytes = keyFactory.generateSecret(keySpec).getEncoded();
sk = new SecretKeySpec(keyBytes, "AES");
Cipher cipher = Cipher.getInstance(Cifrador.AES_MODE);//AES_MODE = AES/CBC/PKCS5Padding
IvParameterSpec ivParams = new IvParameterSpec(iv);//IV already initialized
cipher.init(Cipher.ENCRYPT_MODE, sk, ivParams);
byte pwdbytes[] = password.getBytes();//I also tried using Base64 to decode... without success
byte cc[] = cipher.doFinal(pwdbytes);
finalKey = Base64.encodeToString(cc, false); //.encodeToString(byte[] sArr, boolean lineSep)
return finalKey;
Now decrypt mode:
//This method compares a password received from keyboard with the decrypted password (decrypting output from generatePswdBasedKey(String password))
public static boolean checkPswdBasedKey(String password, String passwordInput){
byte bufferBytes[] = Base64.decode(password);
SecretKey sk = new SecretKeySpec(bufferBytes, 0, bufferBytes.length, "AES"); //Also tried new SecretKeySPec(bufferBytes, "AES");...
Cipher c = Cipher.getInstance(Cifrador.AES_MODE);//AES_MODE = AES/CBC/PKCS5Padding
IvParameterSpec ivParams = new IvParameterSpec(iv);//IV already initialized
c.init(Cipher.DECRYPT_MODE, sk, ivParams);
byte result[] = c.doFinal(bufferBytes);
String resultStr = Base64.encodeToString(result, false); //.encodeToString(byte[] sArr, boolean lineSep)
if(passwordInput.equalsIgnoreCase(resultStr)){
return true;
}
return false;
}
I compared bytes from iv #checkPswdBasedKey and iv #generatePswdBasedKey and they are all the same. Same happens to the secretkey #checkPswdBasedKey (i get those bytes with: sk.getEncoded() ) and secretkey #generatePswdBasedKey... they are all equal.
So basically when i decrypt i know i'm using the same key, same IV and same message... and an appropiate length (16 bytes key, 16 bytes msg, 16 bytes iv, using AES 128) Any idea?
As Joachim Isaksson commented, if you want to implement a password check, you ought to use a secure hash representation of the password, that is not reversible. This way, the password can't be obtained by decryption even if the hash + key is compromised.
Anyway, in your generatePswdBasedKey you use the PBKDF2WithHmacSHA1 algorithm to generate a SecretKey, and then use that key to encrypt the password. Now you have two options to verify the password in checkPswdBasedKey. Either you:
encrypt the password the same way as in generatePswdBasedKey and compare that they give the same encrypted string
or you
decrypt the encrypted version and compare the result with the password in clear.
I presume that you try the later approach as you init your cipher for decrypt with:
c.init(Cipher.DECRYPT_MODE, sk, ivParams);
Hovewer, for that approach to work you need to instantiate your SecretKey the same way
as you did in generatePswdBasedKey - currently you end up with two different keys.
In generatePswdBasedKey:
SecretKey sk = null;
KeySpec keySpec = new PBEKeySpec(password.toCharArray(), salt, IT, KEY_LENGTH);
SecretKeyFactory keyFactory = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA1");
byte[] keyBytes = keyFactory.generateSecret(keySpec).getEncoded();
sk = new SecretKeySpec(keyBytes, "AES");
In checkPswdBasedKey:
byte bufferBytes[] = Base64.decode(password);
SecretKey sk = new SecretKeySpec(bufferBytes, 0, bufferBytes.length, "AES");
When that is fixed, you also need to look at your compare logic. You should not do a Base64 encoding of your result before the compare - and the compare ought to be case sensitive.
Don't use:
byte result[] = c.doFinal(bufferBytes);
String resultStr = Base64.encodeToString(result, false);
if (passwordInput.equalsIgnoreCase(resultStr)) {
return true;
}
But instead use:
byte result[] = c.doFinal(bufferBytes);
String resultStr = new String(result);
if (passwordInput.equals(resultStr)) {
return true;
}
somebody know what is encoding type of message that client sends to websocket server?
I'm studying this site
http://blogs.claritycon.com/blog/2012/01/18/websockets-with-rfc-6455/
and from what this site teaches me, below are decoding code to get message from client in server side!!!
public string HybiDecode(byte[] data)
{
byte firstByte = data[0];
byte secondByte = data[1];
int opcode = firstByte & 0x0F;
bool isMasked = ((firstByte & 128) == 128);
int payloadLength = secondByte & 0x7F;
if (!isMasked) { return null; } // not masked
if (opcode != 1) { return null; } // not text
List<int> mask = new List<int>();
for (int i = 2; i < 6; i++)
{
mask.Add(data[i]);
}
int payloadOffset = 6;
int dataLength = payloadLength + payloadOffset;
List<int> unmaskedPayload = new List<int>();
for (int i = payloadOffset; i < dataLength; i++)
{
int j = i - payloadOffset;
unmaskedPayload.Add(data[i] ^ mask[j % 4]);
}
return ToAscii(unmaskedPayload.Select(e => (byte)e).ToArray());
}
public string ToAscii(byte[] data)
{
System.Text.ASCIIEncoding decoder = new System.Text.ASCIIEncoding();
return decoder.GetString(data, 0, data.Length);
}
but now I'm studying in C language so I have to convert ToAscii() to C language!
but... from what? from unicode to ASCII or from utf-8 to ASCII???
could you let me know if you know about this???
Messages are transmitted as utf-8. See section 5.6 of the spec for details.
Its up to you what encoding you use within your server. UTF-8 is easy to handle in C (it is still terminated by single nul character so all string functions work) so you might find it easiest to use UTF-8 in your code and not convert to ascii/unicode.