Public transactions in quorum stuck in pending in transaction pool - ethereum

I followed Quorum's docs and have created a 2 node network using the raft-consensus. In the genesis block i had pre-allocated funds to one of the accounts. Now I am trying to do a public transaction of some ethers to the other node.
However the transaction is getting stuck in the transaction pool and the balances of both nodes remain unchanged.
I have used the same genesis.json file that was provided in the documentation. Is there something I am missing?
Once the two nodes were brought up, I tried running -
eth.sendTransaction({from:current-node-address, to: second-node's-address, value:0x200,gas:21000})
On checking the transactionReceipt with the transaction hash that was generated, it displays null.

It sounds like your network is not minting blocks, so you may have some Raft misconfiguration.
Check the log files for any error messages.
You can also check that both nodes are in the network and that one of them is minting (is the leader) by using the command raft in the geth console.

Related

Get CALL_EXCEPTION details

I am running private geth node and I am wondering if there is any way to find the root cause of transaction exception. When I send the transaction, all I can see is:
transaction failed [ See:
https://links.ethers.org/v5-errors-CALL_EXCEPTION ]
And when I run the same transaction in hardhat network, I get more details:
VM Exception while processing transaction: reverted with panic code
0x11 (Arithmetic operation underflowed or overflowed outside of an
unchecked block)
Is it possible to get the same info from my geth node?
The revert reason is extracted using transaction replay, see the example implementation. This sets requirements for what data your node must store in order to be able to replay the transaction. See your node configuration and detailed use case for further diagnosis.
Your node must support EIP-140 and EIP-838. This has been case for many years now so it is unlikely your node does not support this.
Unless a smart contract explicitly reverts, the default reverts (payable function called with value, math errors) JSON-RPC error messages depend on the node type and may vary across different nodes
Hardhat is internally using Ganache simulated node, not GoEtheruem

Why can't transactions be sent directly to validators/mining pools?

Now 99% of all interactions with the blockchain occur through Infuria or Alchemy (MetaMask - API Infuria).
Nobody raises their geth nodes.
Because of this, most people are skeptical about the word "decentralization", since the application still has a point of failure.
Why is it impossible to send transactions directly from the browser to the validator? What prevents this?
After all, this is the last obstacle before decentralization. If the browser/extensions stored hundreds of addresses of mining pools to which you can send a transaction, then such an application is almost fail-safe.
Generally, a signed transaction is sent from a wallet software to a node (in the peer-to-peer Ethereum network) that broadcasts it to the rest of the network as a "transaction waiting to be mined" (i.e. placed in a mempool).
Miners usually take transactions from the mempool and place them in blocks.
It is technically possible for a miner to accept a transaction from another source (or create and sign it themselves), and place it in a block.
But it comes with an inconvenience for the transaction sender - they need to wait until this specific miner mines a block containing their transaction. If they sent the transaction to the mempool instead, any miner could have picked it up and include in their block. And there is currently no standardized way of sending a transaction to the miner directly - so each might have a different channel and different rules.
So to answer your question:
Why can't transactions be sent directly to validators/mining pools?
They can. But it's just faster (for the transaction sender) to use the mempool and let the transaction be mined by anyone, instead of waiting for one specific mining pool to mine a block.

Fiware: Data loss prevention

I’m working with the 0.27.0 version of context broker. I'm using the Cygnus generic enabler and I have established a MQTT agent that connects external devices to the context broker.
My major concern right now is how to prevent from data loss. I established the context broker and the Cygnus mongodb databases as replica sets, but that won't ensure that all data will be persisted into the databases. I have seen that Cygnus uses Apache flume. Looking at its configuration, the re-injection retries can be configured:
# Number of channel re-injection retries before a Flume event is definitely discarded (-1 means infinite retries)
cygnusagent.sources.http-source.handler.events_ttl = -1
¿It is a good idea to establish the retries value to -1? I have read about events re-injected in the channel forever.
¿What can be done to ensure that all the data will be persisted?
¿Is there any functionality into fiware ecosystem oriented to that purpose?
Regarding Cygnus, the TTL is for sure the way of controlling the persistence retries after an error. A retry means the data is reinjected in the internal channel communicating the source (which receives Orion notifications) and the sink (which persists the data in the final storage) for future persistence attempts.
Possible values for this TTL are:
TTL = 0: there are no retries, i.e. if the first time a notified data cannot be persisted in the final storage (because of a network fail, a storage error, whatever) then the data is dropped.
TTL > 0: there are as much retries as configured TTL. Once exhausted the TTL the data is dropped.
TTL = -1: infinite retries, i.e. the data is reinjected in the channel forever until it is persisted or the channel gets full.
As commented, a -1 TTL may consume the channel capacity if the final storage never gets OK, avoiding new received data is put into the channel. Nevertheless, if the final storage never gets OK, such a drawback does not matter, right? :)
Thus, we could say the rules for choosing a TTL are:
If you don't want retries, simply configure 0.
If you want retries but you don't mind to loose data afeter certain number of retries, then configure a positive value.
If you want retries but you don't want to loose data, then configure -1 and a large channel capacity since the final storage may be down for an unknown time.
In any case, the TTL feature is changing during this sprint. The behaviour will be the same, but instead of being applied to single events, it will applied to batches of events (batches may be about 1 single event, of course). You'll see this change in the next release of Cygnus (0.13.0), and it will be available at the end of February 2016 (at the moment of writing this, the next week :)). My recommendation is to wait for such a release if you want to instensively use the TTL feature.

Azure service bus queue design

Each user in my application has gmail account. the application needs to be in sync with incoming emails. for each user every 1 minute the application should ask gmail servers if there is something new. 99% of the time nothing is new.
From what I know gmail dosen't provide web-hooks
In order to reduce the load from my servers and specially from the DB I want to use the service bus queue in the following manner.
queue properties:
query method: PEEK_AND_LOCK
lock time : 1 minute
max delivery count: X
flow:
queue listener receiving message A from the queue and process it.
if nothing is new the listener will not delete the message from the queue
the message will be delivered again after lock time (1 minute)
basically instead of sending new message to the queue again and again to be processed we just leave them in the queue and relying on the re-delivery mechanism.
we are expecting many users in the near future (100,000-500,000) which means many messages in the queue in a given moment which needs to be processed each minute.
lets assume the messages size is very small and less the 5GB all together
I am assuming that the re delivery mechanism is used mainly for error handling and I wonder whether our design is reasonable and the queue is apt for that task? or if there are any other suggestions to achieve our goal.
Thanks
You are trying to use the Service Bus Queue as a scheduler which it not really is. As a result SB Queue will be main bottleneck. With your architecture and expected number of users you will find yourself quickly blocked by limitations of the Service Bus queue. For example you have only max 100 concurrent connections, which means only 100 listeners (assuming long-pooling method).
Another issue might be max delivery count property of SB Queue. Even if you set it to int.MaxValue now, there is no guarantee that Azure Team will not limit it in the future.
Alternative solution might be that you implement your own scheduler worker role (using already existing popular tools, like Quartz.NET for example). Then you can experiment - you can host N jobs (which actually do Gmail api requests) in one worker role (each job runs every X minute(s)), and each job may handle M number of users concurently. Worker role could be easily scaled if number of users increases. Numbers N and M can depend on the worker role configuration and can be determined on practice. If applicable, just to save some resources, X can be made variable, for example, based on the time of the day (maybe you don't need to check emails so often at night). Hope it helps.

Different Retry policies for different messages in MSMQ

We have a requirement for an API, which allows asynchronous updates via a MSMQ message queue, that I'm putting together which will allow the developer consuming the API to specify different retry policies per message. So a high priority client system, e.g. for sales will submit all messages with 5 delivery attempts (retries) and 15 minutes between each attempt, whereas a low priority client system, e.g. back-end mail shot system will allow users to update their marketing preferences, submitting messages with 3 retries and an hour between each attempt.
Is there a way in the System.Messaging MSMQ (version 3 or 4) implementation to specify number of retries, retry delay and things like whether messages are sent to a dead letter queue or just deleted? (and if so, how?)
I would be open to using other messaging frameworks if they fulfilled this requirement.
Is there a way in the System.Messaging MSMQ (version 3 or 4) implementation to specify number of retries
Depending on which operating system/msmq version you're using, specifying retry semantics is highly sophisticated in WCF. The following is for Windows 2008 and MSMQ4 using a transactional queue.
The main setting on the binding is called MaxRetryCycles. One retry cycle is an attempt to successfully read a message from a queue and process it inside the handling method. This "attempt" can actually be made up of multiple attempts, as defined by the msmq binding property ReceiveRetryCount. ReceiveRetryCount is the number of times an application will try to read the message and process it before rolling back the de-queue transaction. This marks the end of one retry cycle.
You can also introduce a delay in between cycles with the RetryCycleDelay property.
A more complicated consideration is what to do with the messages which fail even after multiple retry cycles.
allow the developer consuming the API to specify different retry policies per message
I am not sure how you could do this with MSMQ - as far as I'm aware it's only possible to set retry semantics on a per-endpoint basis. If you're using transactions then you can't even allow API users to set the priority of the messages being sent (transactional queues guarantee delivery in order).
The only thing you could do is host a another instance of your API as high-priority and one for low priority. These could be hosted on different environments, and this has the added benefit that low priority messages won't be competing for system resources with high priority messages.
Hope this helps.