When using node v16.16.0, redis-cli 7.0.0 & "redis": "^4.2.0"
getting such exeption below:
Caught exception: TypeError: listener is not a function Exception
origin: uncaughtException[2022-07-18T07:39:30.386Z] process.on
uncaughtException ERRORCODE 105199 TypeError: listener is not a
function
at Function._RedisCommandsQueue_emitPubSubMessage (/mnt/c/Projects/konnectcore/app/sse/sse/node_modules/#redis/client/dist/lib/client/commands-queue.js:241:9)
at RedisCommandsQueue._RedisCommandsQueue_handlePubSubReply (/mnt/c/Projects/konnectcore/app/sse/sse/node_modules/#redis/client/dist/lib/client/commands-queue.js:283:100)
It's working fine while using node redis "redis": "^2.8.0".
I had a similar issue where I was using probably some old way of subscribing and receiving the messages. Something like this:
sub.on('message', (channel, message) => {
redisClient.hSet('values', message, someFunction);
});
sub.subscribe('channel');
I hope you are using the right way of publishing and subscribing to a channel in redis client. Here is one example from their documentation:
// This is how you create the client
import { createClient } from 'redis';
const client = createClient();
// This is the subscriber part
const subscriber = client.duplicate();
await subscriber.connect();
await subscriber.subscribe('channel', (message) => {
console.log(message); // 'message'
});
// This is an example of how to publish a message to the same channel
await publisher.publish('channel', 'message');
Here is the link if you would like to see some more details about publishing and subscribing of the messages using node-redis client: https://github.com/redis/node-redis#pubsub
I am currently building a data connector but would like to throw and error out to the user if the date range they have provided is not supported by my API endpoint (we don't have data for more than 90 days). I looked through the documentation and found this: https://developers.google.com/datastudio/connector/error-handling#user-facing-errors
And copied the code example exactly and tried to run it but my project still isn't showing the error dialog box back to the user.
I've also taken a look at how other people implement this in this repository (https://github.com/googledatastudio/community-connectors) but still can't see an issue with what I wrote.
function getData(request) {
try {
var dataSchema = getDataSchema(request);
var data = lookupRequestData(request, dataSchema);
} catch (e) {
console.log('pre throw');
// throw Error('some error!');
cc.newUserError()
.setDebugText('Error fetching data from API. Exception details: ' + e)
.setText('There was an error communicating with the service. Try again later, or file an issue if this error persists.')
.throwException();
console.log('post throw');
}
return {
schema: dataSchema,
rows: data
};
}
I can see both the pre throw and post throw strings in my log but there is still no error message being displayed. Just wondering if someone might be able to offer a bit of advice for other things to try.
Thanks
I am writing a private plugin for nodebb (open forum software). In the nodebb's webserver.js file there is a line that seems to be hogging all incoming json data.
app.use(bodyParser.json(jsonOpts));
I am trying to convert all incoming json data for one of my end-points into raw data. However the challenge is I cannot remove or modify the line above.
The following code works ONLY if I temporarily remove the line above.
var rawBodySaver = function (req, res, buf, encoding) {
if (buf && buf.length) {
req.rawBody = buf.toString(encoding || 'utf8');
}
}
app.use(bodyParser.json({ verify: rawBodySaver }));
However as soon as I put the app.use(bodyParser.json(jsonOpts)); middleware back into the webserver.js file it stops working. So it seems like body-parser only processes the first parser that matches the incoming data type and then skips all the rest?
How can I get around that? I could not find any information in their official documentation.
Any help is greatly appreciated.
** Update **
The problem I am trying to solve is to correctly handle an incoming stripe webhook event. In the official stripe documentation they suggested I do the following:
// Match the raw body to content type application/json
app.post('/webhook', bodyParser.raw({type: 'application/json'}),
(request, response) => {
const sig = request.headers['stripe-signature'];
let event;
try {
event = stripe.webhooks.constructEvent(request.body, sig,
endpointSecret);
} catch (err) {
return response.status(400).send(Webhook Error:
${err.message});
}
Both methods, the original at the top of this post and the official stripe recommended way, construct the stripe event correctly but only if I remove the middleware in webserver. So my understanding now is that you cannot have multiple middleware to handle the same incoming data. I don't have much wiggle room when it comes to the first middleware except for being able to modify the argument (jsonOpts) that is being passed to it and comes from a .json file. I tried adding a verify field but I couldn't figure out how to add a function as its value. I hope this makes sense and sorry for not stating what problem I am trying to solve initially.
The only solution I can find without modifying the NodeBB code is to insert your middleware in a convenient hook (that will be later than you want) and then hack into the layer list in the app router to move that middleware earlier in the app layer list to get it in front of the things you want to be in front of.
This is a hack so if Express changes their internal implementation at some future time, then this could break. But, if they ever changed this part of the implementation, it would likely only be in a major revision (as in Express 4 ==> Express 5) and you could just adapt the code to fit the new scheme or perhaps NodeBB will have given you an appropriate hook by then.
The basic concept is as follows:
Get the router you need to modify. It appears it's the app router you want for NodeBB.
Insert your middleware/route as you normally would to allow Express to do all the normal setup for your middleware/route and insert it in the internal Layer list in the app router.
Then, reach into the list, take it off the end of the list (where it was just added) and insert it earlier in the list.
Figure out where to put it earlier in the list. You probably don't want it at the very start of the list because that would put it after some helpful system middleware that makes things like query parameter parsing work. So, the code looks for the first middleware that has a name we don't recognize from the built-in names we know and insert it right after that.
Here's the code for a function to insert your middleware.
function getAppRouter(app) {
// History:
// Express 4.x throws when accessing app.router and the router is on app._router
// But, the router is lazy initialized with app.lazyrouter()
// Express 5.x again supports app.router
// And, it handles the lazy construction of the router for you
let router;
try {
router = app.router; // Works for Express 5.x, Express 4.x will throw when accessing
} catch(e) {}
if (!router) {
// Express 4.x
if (typeof app.lazyrouter === "function") {
// make sure router has been created
app.lazyrouter();
}
router = app._router;
}
if (!router) {
throw new Error("Couldn't find app router");
}
return router;
}
// insert a method on the app router near the front of the list
function insertAppMethod(app, method, path, fn) {
let router = getAppRouter(app);
let stack = router.stack;
// allow function to be called with no path
// as insertAppMethod(app, metod, fn);
if (typeof path === "function") {
fn = path;
path = null;
}
// add the handler to the end of the list
if (path) {
app[method](path, fn);
} else {
app[method](fn);
}
// now remove it from the stack
let layerObj = stack.pop();
// now insert it near the front of the stack,
// but after a couple pre-built middleware's installed by Express itself
let skips = new Set(["query", "expressInit"]);
for (let i = 0; i < stack.length; i++) {
if (!skips.has(stack[i].name)) {
// insert it here before this item
stack.splice(i, 0, layerObj);
break;
}
}
}
You would then use this to insert your method like this from any NodeBB hook that provides you the app object sometime during startup. It will create your /webhook route handler and then insert it earlier in the layer list (before the other body-parser middleware).
let rawMiddleware = bodyParser.raw({type: 'application/json'});
insertAppMethod(app, 'post', '/webhook', (request, response, next) => {
rawMiddleware(request, response, (err) => {
if (err) {
next(err);
return;
}
const sig = request.headers['stripe-signature'];
let event;
try {
event = stripe.webhooks.constructEvent(request.body, sig, endpointSecret);
// you need to either call next() or send a response here
} catch (err) {
return response.status(400).send(`Webhook Error: ${err.message}`);
}
});
});
The bodyParser.json() middleware does the following:
Check the response type of an incoming request to see if it is application/json.
If it is that type, then read the body from the incoming stream to get all the data from the stream.
When it has all the data from the stream, parse it as JSON and put the result into req.body so follow-on request handlers can access the already-read and already-parsed data there.
Because it reads the data from the stream, there is no longer any more data in the stream. Unless it saves the raw data somewhere (I haven't looked to see if it does), then the original RAW data is gone - it's been read from the stream already. This is why you can't have multiple different middleware all trying to process the same request body. Whichever one goes first reads the data from the incoming stream and then the original data is no longer there in the stream.
To help you find a solution, we need to know what end-problem you're really trying to solve? You will not be able to have two middlewares both looking for the same content-type and both reading the request body. You could replace bodyParser.json() that does both what it does now and does something else for your purpose in the same middleware, but not in separate middleware.
I have a Demo Server replying JSON objects only to the client request.
I am planning to use QNetworkAccessManager as the client, this is what I did.
I defined a lambda function handling Server reply
std::function<void(QNetworkReply*)> processReplyLB = [&](QNetworkReply *reply){
static int cnt = 0;
std::cout<<"--------------------"<<(++cnt)<<"---------------------"<<std::endl;
QList<QByteArray> headerList = reply->rawHeaderList();
foreach (QByteArray header, headerList) {
std::cout<<header.constData()<<" - "<<reply->rawHeader(header).constData()<<std::endl;
}
processResult = false;
if(reply->error()){
std::cout<<"REPLY ERROR"<<std::endl;
std::cout<<reply->errorString().toUtf8().constData()<<std::endl;
} else {
QString value = reply->readAll();
std::cout<<"value = "<<value.toUtf8().constData()<<std::endl;
QJsonDocument doc = QJsonDocument::fromJson(value.toUtf8());
if(doc.isNull()){
std::cout<<"JSON document is null"<<std::endl;
}else if(doc.isEmpty()){
std::cout<<"JSON document is empty"<<std::endl;
} else if(!doc.isObject()){
std::cout<<"JSON document is not an object"<<std::endl;
} else {
QJsonObject obj = doc.object();
QString responseStr = obj.value("result").toString();
processResult = (responseStr == "ok");
if(obj.contains("message")){
QJsonValue messageValue = obj.value("message");
std::cout<<messageValue.toString().toUtf8().constData()<<std::endl;
}
}
}
reply->deleteLater();
std::cout<<"--------------------"<<(cnt)<<"---------------------"<<std::endl;
};
and I connected this lambda slot to QNetworkAccessManager in two functions used for
check if client session does exist on Server(sends a get request).
login using id and password (send a post request with parameters).
in main function, if I invoke checkSession() or login() respectively, the result is fine. but if I try to call
login();
checkSession();
in sequence, then I will get lambda invoked four times with checkSession() result came as the first, following by a null json, login json result and finally another null json.
I know QNetworkAccessManager works asynchronously, EventLoop can solve this problem, but it is not applicable in real development mode due to I am writing a client background service component.
So how can we design this client so that I can make sure login result is processed before checkSession?
BTW, I used Java Servlet for Server without asynchronous. they are just trivial doGet and doPost processes.
I got a Web API that performs a function and posts a JSON response back to a calling page.
This is standard Web API behaviour and works beautifully.
Now I want to modify the controller so that in addition to the post back the user is redirected back to the page on the calling web site where the result of the Web API call can be displayed (in JSON).
So basically I want to:
(1) Server side post back the results in JSON to a page and redirect to the same page from the Web API
(2) On the caller's site, I want to display the JSON that was posted back.
How do I do this?
I already tried for many hours ...
e.g.:
using (WebClient client = new WebClient())
{
client.Headers.Add("Content-Type", "text/json");
client.Headers.Add("Accept", "text/json");
try
{
ErrorText = client.UploadString(redirectURL, "POST", JsonConvert.SerializeObject(orderresponse));
Response.Redirect(redirectURL);
}
catch (WebException err)
{
ErrorText = err.Message; //Todo - write to logfile
}
}
Instead of doing the redirect on the server, instruct the client to do it by using the appropriate HTTP status code. For example:
public HttpResponseMessage Post(MyModel model)
{
// handle the post
MyResult result = ...;
// redirect
var response = Request.CreateResponse<MyResult>(HttpStatusCode.Moved, result);
response.Headers.Location = new Uri("http://www.yourdomain.com/redirectURI");
return response;
}