Is there anyway that i can limit the calling of apply method to just 1 in the Transaction Processor? By default it is called twice.
I guess your question is based on the log traces you see.
Short answer: apply method also the core business logic in your transaction family is executed once for a input transaction. There is a different reason for you to see the logs appear twice. Well, in reality transaction execution happens and state transitions are defined with respect to the context. Read the long answer for detailed understanding.
Long answer: If you're observing logs, then you may need to go a little deeper into the way Hyperledger Sawtooth works. Let's get started.
Flow of events (at very high level):
Client sends the transaction embed in a batches.
Validator adds all the transactions in the pending queue.
Based on the consensus engine's request, the validator will start creating the block.
For the block creation, a current state context information is passed along with the transaction request to the Transaction Processor. Eventually send to the right Transaction Family's apply method.
The apply method's result either success or failure is recorded. The transaction is removed from the pending queue if is invalid or it is added to the block.
If the response of the apply method is internal error then that is resubmitted.
If a transaction is added to the block. Depending on the consensus algorithm, the created block is broadcasted to all the nodes.
Every node executes the transactions in the arriving block. The node that created the block will also execute. This probably is what you're talking about.
Related
Does anyone know the use cases for the trace_block method by Openethereum?
Here's the link to it:
https://docs.alchemy.com/reference/trace-block
I cannot find any use cases listed for this method anywhere. Even in the official docs, it's not written.
The trace_block method is used to trace the execution of a block. It returns a list of traces, one for each transaction in the block. Each trace contains information about the execution of the transaction, including the execution of each message call or contract creation operation.
curl -X POST --data '{"jsonrpc":"2.0","method":"trace_block","params":["0x1"],"id":1}' https://eth-mainnet.alchemyapi.io/v2/your-api-key
Okay, so after my research, I came to know that:
The trace_block method is used to analyze the behavior of a blockchain.
For example, you can find the average gasPrice for a block by getting the traces of all the transactions in the block (traces contain the gasPrice for transactions). Then you can get the average gasPrice block by block and form a graph to visualize how the gasPrice has changed over time.
I found it in this great article by Alchemy
You can do the following:
Get contract execution revert reason (including historical txs).
Gas usage per individual call action.
Byte-code ops, steps taken and gas used per OP code.
Maybe more, but there are the ones I know of.
I have implemented kafka stream application. Let's say one of the object's field which the stream is currently processing contains a number instead of a string value. Currently when there is an exception thrown in the processing logic eg. .transform() method, whole stream is killed and my application stops to process data.
I would like to skip such invalid record and keep processing next records available on a input topic. Additionally I don't want to implement any try-catch statements in my stream processing code.
To achieve this, I implemented StreamsUncaughtExceptionHandler so it returns StreamThreadExceptionResponse.REPLACE_THREAD enum in order to spawn new thread and keep on processing next records waiting on the input topic. However, it turned out that the stream consumer offset is not committed and when new a thread is started, it takes old record which just have killed the previous stream thread... Since the logic is the same, new thread will also fail to process the error record and again fail. Some kind of a loop spawning new thread and failing on a same record every time.
Is there any clean way of skipping failing record and keep the stream processing next records?
Please note, I am not asking about DeserializationExceptionHandler or ProductionExceptionHandler.
When it comes to the application-level code, it is mostly up to the application how the exception is handled. This use case has come up before. See these previous Stack Overflow threads.
Example on handling processing exception in Spring Cloud Streams with Kafka Streams Binder and the functional style processor
How to stop sending to kafka topic when control goes to catch block Functional kafka spring
Try to see if those answers can be applied to your scenario.
You can filter the event that dont match a pattern or validate the events before you transform them
Now 99% of all interactions with the blockchain occur through Infuria or Alchemy (MetaMask - API Infuria).
Nobody raises their geth nodes.
Because of this, most people are skeptical about the word "decentralization", since the application still has a point of failure.
Why is it impossible to send transactions directly from the browser to the validator? What prevents this?
After all, this is the last obstacle before decentralization. If the browser/extensions stored hundreds of addresses of mining pools to which you can send a transaction, then such an application is almost fail-safe.
Generally, a signed transaction is sent from a wallet software to a node (in the peer-to-peer Ethereum network) that broadcasts it to the rest of the network as a "transaction waiting to be mined" (i.e. placed in a mempool).
Miners usually take transactions from the mempool and place them in blocks.
It is technically possible for a miner to accept a transaction from another source (or create and sign it themselves), and place it in a block.
But it comes with an inconvenience for the transaction sender - they need to wait until this specific miner mines a block containing their transaction. If they sent the transaction to the mempool instead, any miner could have picked it up and include in their block. And there is currently no standardized way of sending a transaction to the miner directly - so each might have a different channel and different rules.
So to answer your question:
Why can't transactions be sent directly to validators/mining pools?
They can. But it's just faster (for the transaction sender) to use the mempool and let the transaction be mined by anyone, instead of waiting for one specific mining pool to mine a block.
While working on a BPMN model, we encountered an issue concerning exception flows.
Whenever an exception flow was needed, we gave it its own end event. Although BPMN recommend using gateways to merge normal flow with exception flow. I can see no advantage in doing so, only some additional complications.
However, consider the case in which we have a subprocess with an activity X and an attached-to-boundary non-interrupting event. When triggered, a parallel token will be created and put on the branch of the exception flow. After running through exception flow, an end event consumes this token.
I assume this consumption doesn't trigger the upper-level flow(the one containing this particular subprocess) to continue, as there still is a token left in activity X. As this activity is ended and normal flow is executed , this token is consumed as well and the subprocess doesn't contain any token. This will trigger the upper-level flow to continue.
As this is the case, I can't think of any case where the merger of exception flow and normal flow should be necessary. (Except for the one where activities after activity x have to be run in exception flow as well, causing them to be executed multiple times).
I assume this consumption doesn't trigger the upper-level flow(the one containing this particular subprocess) to continue, as there still is a token left in activity X.
This is an accurate statement; the following models have identical semantics (the non-interrupting multiple event trigger is used as a placeholder):
The governing section in the BPMN specification is Section 10.5.3, which requires that "all the tokens that were generated within the process MUST be consumed by an End Event before the Process has been completed."
As this is the case, I can't think of any case where the merger of exception flow and normal flow should be necessary.
One case where the merger may be necessary occurs when the two flows must be merged before a task later in the process can commence. As a simple example, take the following models:
In the model at top, Activity Y may commence as soon as Activity X completes, regardless of whether there are instances of Exceptional activity running in parallel. In the model at bottom, Activity Y cannot start until all instances (if any) of Exceptional activity have completed. If the semantics of the second example are wanted, then a merger of the normal and exception flows are needed.
The last diagram is not inaccurate and, effectively, invalid.
By definition, the exceptional flow cannot form part of the smooth flow. By definition, the exceptional flow cannot be evaluated using IF.
The final diagram (erroneously) introduces a gateway which will, eventually receive two tokens and (erroneously) trigger activity Y, twice.
I would suggest that the language is wrong - IF the exceptional flow must be complete before Y triggers, the author has described an INTERRUPTING BOUNDARY EVENT. Depicting that event solves the modelling problem introduced by poor grammar. Only one token is produced and activity Y is triggered, only once.
I have studied Message Queues System in my class but I still don't get it how these Message Queues System work in real time scenarios? Is there any tutorial which can help me to get the complete picture?
Can someone explain me how these systems work?
An example: My thread or process can send a message to your message queue, and having sent it, my code goes on to do something else. Your code, when it gets around to it, reads the next message from the message queue, and then decides what to do about that message. Message queues avoid needing to have a critical section or mutex shared between the two threads, or processes. The underlying message queue layer itself takes care of making sure that messages get into the queue without race conditions affecting the integrity of the queue.
Message queues can be used for both one-way and two-way, asynchronous messaging. For one-way use, my thread can use it to keep your thread appraised of key events in my thread, without acknowledgement back from your thread. For two-way use, after my thread sends a message to your thread, your thread may need to send data back to my thread via my message queue.
The message queue layer uses lower level synchronization schemes to insure that no two writers to the queue can write at the same time. It insures that all writes to the queue are atomic. It also insures that a reader of the queue cannot read a partially written message from the queue.
Most message queue APIs also offer support for reading messages from the queue based on a filter that you designate. Say for instance that you consider messages from a time critical thread to be more important that other messages. You can each time you check your queue for messages, first check for messages from the critical thread, and service those messages first. Your thread would then go onto to process the rest of the messages as normal, provided no more messages from the critical thread are found.
A C tutorial of the UNIX message queues
That's a complex topic but to put it simply:
Message Queues are one of the best ways, if not the best, to
implement distributed systems.
Now you might ask, what is a distributed system? It is an integrated system that spans multiple machines, clients or nodes which execute their tasks in parallel in a non-disruptive way. A distributed system should be robust enough to continue to operate when one or more nodes fail, stop working, lag or are taken down for maintenance.
Then you might ask, what is a message queue? It is a message-oriented middleware that enables the development of a distributed system by using asynchronous messages for inter-node communication through the network.
And finally you might ask, what is all that good for? This is good for implementing applications with a lot of moving parts called nodes which needs real-time monitoring and real-time reaction capabilities. To summarize they provide: parallelism (nodes can truly run in parallel), tight integration (all nodes see the same messages in the same order), decoupling (nodes can evolve independently), failover/redundancy (when a node fails, another one can be running and building state to take over immediately), scalability/load balancing (just add more nodes), elasticity (nodes can lag during activity peaks without affecting the system as a whole) and resiliency (nodes can fail / stop working without taking the whole system down).
Check this article which discusses a message queue infrastructure in detail.