I have recently started using Q.spawn function to run generators that yield promises. This works well in browsers where the support for streams is yet to land but in case of node we have streams. If you're using streams inside a generator function and would like to yield once writer stream is done then your code becomes not so clean.
Q.spawn(function* () {
yield new Promise(resolve => {
let fromStream = fs.createReadStream('x.txt');
let toStream = fs.createWriteStream('y.txt');
toStream.on('end', () => resolve());
fromStream.pipe(toStream);
});
});
It works but as soon as I start dealing with a lot streams the code becomes really ugly. Can this be made as simple as following snippet?.
someGeneratorFuncRunner(function* () {
yield fs.createReadStream('x.txt')
.pipe(fs.createWriteStream('y.txt'));
});
You don't need to put most of that code inside the Promise constructor:
Q.spawn(function* () {
let fromStream = fs.createReadStream('x.txt');
let toStream = fs.createWriteStream('y.txt');
fromStream.pipe(toStream);
yield new Promise(resolve => {
toStream.on('end', resolve);
});
});
And of course if you're waiting for lots of streams, it would make sense to factor out the promise constructor call into a helper function:
function endOf(stream) {
return new Promise(resolve => {
stream.on('end', resolve);
});
}
Related
I am writing unit tests for my code and wish to use transactions to prevent any stray data between tests.
The code uses Sequelize ORM for all interactions with the database. Since changing the actual code is not an option, I would be using cls-hooked to maintain transaction context instead of passing transaction to all the queries. There is a problem, however. On reading the official documentation and trying to go about it, the above approach seems to only work for managed transactions.
So far, the test code looks somewhat like:
test("Test decription", async () => {
try {
await sequelize.transaction(async (t) => {
//Actual test code
});
} catch (error) {
//Do nothing if query rolled back
}
});
What I intend to achieve (for obvious reasons):
let t;
beforeEach(async () => {
t = await sequelize.transaction();
});
test("Test decription", async () => {
//Actual test code
});
afterEach(async () => {
await t.rollback();
});
Is this possible? If yes, any help in implementing this would be appreciated.
I'm having the same problem -- after much Googling, I found this closed issue where they indicate that unmanaged transactions aren't supported by design 🥲
It's true that Sequelize doesn't automatically pass transactions to queries when you're using unmanaged transactions. But you can manually set the transaction property on the CLS namespace, just like Sequelize does on a managed transaction:
https://github.com/sequelize/sequelize/blob/v6.9.0/lib/transaction.js#L135
namespace.run(() => {
namespace.set('transaction', transaction);
/* run code that does DB operations */
});
This is tricky for tests because describe() calls can be nested and each can have their own beforeAll()/afterAll() and beforeEach()/afterEach() hooks. To do this right, each before hook needs to set up a nested transaction and the corresponding after hook should roll it back. In addition, the test case itself needs to run in a nested transaction so that its DB operations don't leak into other tests.
For anyone from the future:
I was facing the above problem and found a way to fix it with a helper function and cls-hooked.
const transaction = async (namespace: string, fn: (transaction: Transaction) => unknown) => {
const nameSpace = createNamespace(namespace);
db.Sequelize.useCLS(nameSpace);
const sequelize = db.sequelize;
const promise = sequelize.transaction(async (transaction: Transaction) => {
try {
await fn(transaction);
} catch (error) {
console.error(error);
throw error;
}
throw new TransactionError();
});
await expect(promise).rejects.toThrow(TransactionError);
destroyNamespace(namespace);
};
What the above code does is creating a cls namespace and transaction that will be discarded after the test run, the TransactionError is thrown to ensure the entire transaction is always rolled back on each test run.
Usage on tests would be:
describe('Transaction', () => {
test('Test', async () => {
await transaction('test', async () => {
// test logic here
});
});
});
So backstop.js provides ability to run custom script against underlying engine. I use puppeteer as an engine so I try to mock Date.now with 'onReadyScript':
page.evaluate('window.Date.now = () => 0; Date.now = () => 0;');
...
page.addScriptTag({
// btw `console.log` here is not executed, do I use it in wrong way?
content: 'Date.now = () => 0;'
});
...
page.evaluate(() => {
window.Date.now = () => 0;
Date.now = () => 0;
});
Last one, I think, is modifying Date in context of Node, not inside the puppeteer, but anyway tried that as well.
Nothing worked, script under the test still output real Date.now. Also I checked Override the browser date with puppeteer but it did not help me.
Yes, I know I'm able to skip particular selectors, but it does not always make sense(think about clock with arrows).
After trying onBeforeScript with evaluateOnNewDocument() it works for me. Complete script:
module.exports = async function (page, scenario) {
if (!page.dateIsMocked) {
page.dateIsMocked = true
await page.evaluateOnNewDocument(() => {
const referenceTime = '2010-05-05 10:10:10.000';
const oldDate = Date;
Date = function(...args) {
if (args.length) {
return new oldDate(...args);
} else {
return new oldDate(referenceTime);
}
}
Date.now = function() {
return new oldDate(referenceTime).valueOf();
}
Date.prototype = oldDate.prototype;
})
}
};
Reason: onReadyScript is executed when page under testing has already been loaded and executed. So code is bound to original Date by closure, not the mocked version.
What I am trying to do is create a chrome extension that creates new, nested, bookmark folders, using promises.
The function to do this is chrome.bookmarks.create(). However I cannot just
loop this function, because chrome.bookmarks.create is asynchronous. I need to wait until the folder is created, and get its new ID, before going on to its children.
Promises seem to be the way to go. Unfortunately I cannot find a minimal working example using an asynchronous call with its own callback like chrome.bookmarks.create.
I have read some tutorials 1, 2, 3, 4. I have searched stackOverflow but all the questions do not seem to be about plain vanilla promises with the chrome extension library.
I do not want to use a plugin or library: no node.js or jquery or Q or whatever.
I have tried following the examples in the tutorials but many things do not make sense. For example, the tutorial states:
The promise constructor takes one argument—a callback with two
parameters: resolve and reject.
But then I see examples like this:
const wait = ms => new Promise(resolve => setTimeout(resolve, ms));
How this works is a mystery to me.
Also, how can you call resolve() when its never been defined? No example in the tutorials seem to match real life code. Another example is:
function isUserTooYoung(id) {
return openDatabase() // returns a promise
.then(function(col) {return find(col, {'id': id});})
How do I pass in col, or get any results!
So if anyone can give me a minimal working example of promises with an asynchronous function with its own callback, it would be greatly appreciated.
SO wants code, so here is my non-working attempt:
//loop through all
function createBookmarks(nodes, parentid){
var jlen = nodes.length;
var i;
var node;
for(var i = 0; i < nodes.length; i++){
var node = nodes[i];
createBookmark(node, parentid);
}
}
//singular create
function createBookmark(node, parentid){
var bookmark = {
parentId : parentid,
index : node['index'],
title : node['title'],
url : node['url']
}
var callback = function(result){
console.log("creation callback happened.");
return result.id; //pass ID to the callback, too
}
var promise = new Promise(function(resolve, reject) {
var newid = chrome.bookmarks.create(bookmark, callback)
if (newid){
console.log("Creating children with new id: " + newid);
resolve( createBookmarks(bookmark.children, newid));
}
});
}
//allnodes already exists
createBookmarks(allnodes[0],"0");
Just doesn't work. The result from the callback is always undefined, which it should be, and I do not see how a promise object changes anything. I am equally mystified when I try to use promise.then().
var newid = promise.then( //wait for a response?
function(result){
return chrome.bookmarks.create(bookmark, callback);
}
).catch(function(error){
console.log("error " + error);
});
if (node.children) createBookmarks(node.children, newid);
Again, newid is always undefined, because of course bookmarks.create() is asynchronous.
Thank you for any help you can offer.
Honestly, you should just use the web extension polyfill. Manually promisifying the chrome APIs is a waste of time and error prone.
If you're absolutely insistent, this is an example of how you'd promisify chrome.bookmarks.create. For other chrome.* APIs, you also have to reject the callback's error argument.
function createBookmark(bookmark) {
return new Promise(function(resolve, reject) {
try {
chrome.bookmarks.create(bookmark, function (result) {
if (chrome.runtime.lastError) reject(chrome.runtime.lastError)
else resolve(result)
})
} catch (error) {
reject(error)
}
})
}
createBookmark({})
.then(function (result) {
console.log(result)
}).catch(function (error) {
console.log(error)
})
To create multiple bookmarks, you could then:
function createBookmarks(bookmarks) {
return Promise.all(
bookmarks.map(function (bookmark) {
return createBookmark(bookmark)
})
)
}
createBookmarks([{}, {}, {}, {}])
.catch(function (error) {
console.log(error)
})
Take the advantage of the convention that the callback function always be the last argument, I use a simple helper function to promisify the chrome API:
function toPromise(api) {
return (...args) => {
return new Promise((resolve) => {
api(...args, resolve);
});
};
}
and use it like:
toPromise(chrome.bookmarks.create)(bookmark).then(...);
In my use case, it just works most of the time.
Is there a way to delay the evaluation of an array of promises using Promise.all()?
Does it make sense to manually add a delay function to the end of each promise before adding them to the array?
Promise.all([p1,p2,p3]).then(res => console.log(res))
I would like to add a delay because my server can't handle too many requests at once.
Promise.all is intended to resolve when the promises are fulfilled, but existing promises are evaluated regardless of Promise.all.
In order to do this, promises should be initially created to produce a delay:
const delayIncrement = 500;
let delay = 0;
const p1 = new Promise(resolve => setTimeout(resolve, delay)).then(() => fetch(...));
delay += delayIncrement;
const p2 = new Promise(resolve => setTimeout(resolve, delay)).then(() => fetch(...));
delay += delayIncrement;
...
Promise.all([p1,p2,p3]).then(...);
The same solution can be used for creating request promises in batch inside a loop.
The recipes for delayed promises can be found in this answer.
I needed to create the calls dynamically, so based on the answer from #estus-flask, managed to come up with:
let delay = 0; const delayIncrement = 1000;
const promises = items.map(item => {
delay += delayIncrement;
return new Promise(resolve => setTimeout(resolve, delay)).then(() =>
fetch(...);
})
let results = await Promise.all(promises);
Yes, you can delay promises using Promise.all to create staggered execution and it's quite easy to do:
// Promise.all() with delays for each promise
let tasks = [];
for (let i = 0; i < 10; i++) {
const delay = 500 * i;
tasks.push(new Promise(async function(resolve) {
// the timer/delay
await new Promise(res => setTimeout(res, delay));
// the promise you want delayed
// (for example):
// let result = await axios.get(...);
let result = await new Promise(r => {
console.log("I'm the delayed promise...maybe an API call!");
r(delay); //result is delay ms for demo purposes
});
//resolve outer/original promise with result
resolve(result);
}));
}
let results = Promise.all(tasks).then(results => {
console.log('results: ' + results);
});
You can run it here too.
Rather than a delay between the chain, which can be done with .then() as shown in other answers, this is a delay that differs for each Promise so that when you call Promise.all() they will be staggered. This is useful when, say, you are calling an API with a rate limit that you'd breach by firing all the calls in parallel.
Peace
The simplest solution for me seemed to be to just take the current index of the map function that produces the promises, and use that index to base a delay on:
const sleep = ms => new Promise(resolve => setTimeout(resolve, ms))
await Promise.all(
dataPoints.map(async (dataPoint, index) => {
await sleep(index * 1000)
...
This makes each of the operations wait index * 1 second to be fired, effectively placing a 1s delay between each operation.
Is there a way to delay the evaluation of an array of promises using
Promise.all()?
No. Promises are not "evaluated", they just resolve. When this happens is determined by their creator and nothing else. When Promise.all is called, the promises p1, p2 and p3 have already been created (and their asynchronous tasks probably already have been started).
Another way you can do this is by hijacking the way loops are transpiled:
async function doABatchOfAsyncWork(workItems) {
for (const item of workItems) {
await workTask(item)
await delay(1000) // not built-in but easily implemented with setTimeout + promise
}
}
You can also save the values of course and return them at the end, exactly as you usually can in for-loops. You can't do this with map since the await would have to be in the async context of the map-functor passed in. If you used map it would execute everything at ~the same time, with a delay of 1s at the end.
I came across this syntax which I believe is ES6 or ES7. What do these lines of code do?
module.exports = async (taskData) => {
// do stuff
}
It exports an asynchronous arrow function which takes in an argument taskData. The asynchronous function is new syntax that is set to come with this year's release, ES2017, called the async function, though the rest of the code is ES6 (ES2015) and ES5 (ES2011). It goes hand-in-hand with await and returns a Promise.
It's primarily useful to clean up promise chains, where the code can get really messy. Consider this example using promises, found here:
function loadStory() {
return getJSON('story.json').then(function(story) {
addHtmlToPage(story.heading);
return story.chapterURLs.map(getJSON)
.reduce(function(chain, chapterPromise) {
return chain.then(function() {
return chapterPromise;
}).then(function(chapter) {
addHtmlToPage(chapter.html);
});
}, Promise.resolve());
}).then(function() {
addTextToPage("All done");
}).catch(function(err) {
addTextToPage("Argh, broken: " + err.message);
}).then(function() {
document.querySelector('.spinner').style.display = 'none';
});
}
The example above fetches a story, and iterates through all the chapters and adds them to the HTML. It works, but it can be very messy and hard to follow if you've got lots of stuff to do. Instead, you can use async and await, which is just syntactic sugar, but makes it much more clean:
async function loadStory() {
try {
let story = await getJSON('story.json');
addHtmlToPage(story.heading);
for (let chapter of story.chapterURLs.map(getJSON)) {
addHtmlToPage((await chapter).html);
}
addTextToPage("All done");
} catch (err) {
addTextToPage("Argh, broken: " + err.message);
}
document.querySelector('.spinner').style.display = 'none';
}
The above is (in my opinion) way more clean and easy to follow, versus the messy promise chain in the first example.
This is an async function which is set to return Promise as #Andrew says.
It is using Arrow Function syntax.
()=>{}
This function is in ES6 and does not define its own this context and any use of this will refer to the outer scope.
Async await syntax is set to be supported when ES2017 is released. Your code is essentially a function that returns a promise. Additionally, the async keyword allows for syntactic sugar when working with promises by use of the await keyword.
The => part is called an arrow function which again is a syntactic sugar for the most part. It is part of ES6.
Your function
module.exports = async (taskData) => {
// do stuff
}
... is almost the same as:
module.exports = function(taskData) {
return new Promise(function() {
// do stuff
if (err) reject(err)
resolve(your return value)
}
}
Note: the main difference is just the binding of the this keyword via the arrow function.