I want to broadcast a raw ETH transaction and pay the fees for it in web3j, i used the following code to get the hex value of the transaction
EthGetTransactionCount ethGetTransactionCount = web3j.ethGetTransactionCount(credentials.getAddress(), DefaultBlockParameterName.PENDING).send();
BigInteger nonce = ethGetTransactionCount.getTransactionCount();
RawTransaction rawTransaction = RawTransaction.createEtherTransaction(nonce, gasPrice, gasLimit, to, value);
// Sign the Transaction
byte[] signedMessage = TransactionEncoder.signMessage(rawTransaction, credentials);
String hexValue = Numeric.toHexString(signedMessage);
System.out.println(hexValue);
How can i broadcast and pay the transaction fees of the hex value using my wallet ?
simply doing something like this:
EthSendTransaction result = web3j.ethSendRawTransaction(hexValue).send();
String hash = result.getTransactionHash();
Related
I am trying to figure out how to get this yubihsm2 to work with signing eth transactions. I have been using the python lib and so far i have had some basic setup. Below is an abbreviation of what i have
web3_endpoint = ''
web3 = Web3(HTTPProvider(web3_endpoint))
hsm = YubiHsm.connect("http://localhost:12345")
session = hsm.create_session_derived(1, "password")
key = session.get_object(1,OBJECT.ASYMMETRIC_KEY)
#key = AsymmetricKey.generate(session, 1, "EC Key", 1, CAPABILITY.SIGN_ECDSA, ALGORITHM.EC_K256)
pub_key = key.get_public_key()
#raw_pub = pub_key.public_bytes(
# encoding=serialization.Encoding.DER,
# format=serialization.PublicFormat.SubjectPublicKeyInfo
# )
raw_pub = pub_key.public_bytes(
encoding=serialization.Encoding.X962,
format=serialization.PublicFormat.UncompressedPoint
)
print ("Public key (Uncompressed):\n",binascii.b2a_hex(raw_pub))
unindexPub = raw_pub[1:]
public_key_hash = Web3.keccak(unindexPub)
address_bytes = public_key_hash[-20:]
address = address_bytes.hex()
print(address)
Now so far i can consistently get the same public key and it looks correct. I then get the same public key each time. When i say correct, the formatting looks correct and is the correct number of bytes.
1). should i be using the commented out public key formatting or the uncompressed X962 encoding that i have above.
From there, this is where things get a bit weird
transaction = {
'to': Web3.toChecksumAddress('0x785AB1daE1b0Ee3f2412aCF55e4153A9517b07e1'),
'gas': 21000,
'gasPrice': Web3.toWei(5, 'gwei'),
'value': 1,
'nonce': 1,
'chainId': 4,
}
serializable_transaction = serializable_unsigned_transaction_from_dict(transaction)
transaction_hash = serializable_transaction.hash()
print(transaction_hash.hex())
# sign the transaction hash and calculate v value
signature = key.sign_ecdsa(transaction_hash,hashes.SHA3_256())
r, s = ecdsa.util.sigdecode_der(signature, ecdsa.SECP256k1.generator.order())
print("r: "+str(r)+"\ns: "+str(s))
v = 28
# encode the transaction along with the full signature and send it
encoded_transaction = encode_transaction(serializable_transaction, vrs=(v, r, s))
web3.eth.sendRawTransaction(encoded_transaction)
I am settings v to 28.. i also test it with 27.. I could use the correct amount with the chainid.. but it's not necessary right from the perspective of trying to get a valid signature (recoverable to get the same public key each time). Sometimes i am getting the error "invalid sender" and other times i am getting the error "insufficient gas." If i take the signature output and use a javascript lib to try to find the public key, each time i am getting a different public key. But i keep consistently generating the same public key from the yubihsm2 in this python app.
I have also commented out in sign_ecdsa the hashing function as i am passing in the data already hashed (in order to use keccak256).
Is there something i am missing? Why are these transactions not signing correctly for eth?
i am getting some of those serialization helpers from enter link description here
helper serialization functions
Thanks
Using EF Core 2.2.6 and Pomelo.EntityFrameworkCore.MySql 2.2.6 (with MySqlConnector 0.59.2)). I have a model for UserData:
public class UserData
{
[DatabaseGenerated(DatabaseGeneratedOption.None)]
public ulong ID { get; private set; }
[Required]
public Dictionary<string, InventoryItem> Inventory { get; set; }
public UserData()
{
Data = new Dictionary<string, string>();
}
}
I have a REST method that can be called that will add items to the user inventory:
using (var transaction = context.Database.BeginTransaction())
{
UserData data = await context.UserData.FindAsync(userId);
// there is code here to detect duplicate entries/etc, but I've removed it for brevity
foreach (var item in items) data.Inventory.Add(item.ItemId, item);
context.UserData.Update(data);
await context.SaveChangesAsync();
transaction.Commit();
}
If two or more calls to this method are made with the same user id then I get concurrent accesses (despite the transaction). This causes the data to sometimes be incorrect. For example, if the inventory is empty and then two calls are made to add items simultaneously (item A and item B), sometimes the database will only contain either A or B, and not both. From logging it appears that it is possible for EF to read from the database while the other read/write is still occurring, causing the code to have the incorrect state of the inventory for when it tries to write back to the db. So I tried marking the isolation level as serializable.
using (var transaction = context.Database.BeginTransaction(System.Data.IsolationLevel.Serializable))
Now I sometimes see an exception:
MySql.Data.MySqlClient.MySqlException (0x80004005): Deadlock found when trying to get lock; try restarting transaction
I don't understand how this code could deadlock... Anyways, I tried to proceed by wrapping this whole thing in a try/catch, and retry:
public static async Task<ResponseError> AddUserItem(Controller controller, MyContext context, ulong userId, List<InventoryItem> items, int retry = 5)
{
ResponseError result = null;
try
{
using (var transaction = context.Database.BeginTransaction(System.Data.IsolationLevel.Serializable))
{
UserData data = await context.UserData.FindAsync(userId);
// there is code here to detect duplicate entries/etc, but I've removed it for brevity
foreach (var item in items) data.Inventory.Add(item.ItemId, item);
context.UserData.Update(data);
await context.SaveChangesAsync();
transaction.Commit();
}
}
catch (Exception e)
{
if (retry > 0)
{
await Task.Delay(SafeRandomGenerator(10, 500));
return await AddUserItem(controller, context, userId, items, retry--);
}
else
{
// store exception and return error
}
}
return result;
}
And now I am back to the data being sometimes correct, sometimes not. So I think the deadlock is another problem, but this is the only method accessing this data. So, I'm at a loss. Is there a simple way to read from the database (locking the row in the process) and then writing back (releasing the lock on write) using EF Core? I've looked at using concurrency tokens, but this seems overkill for what appears (on the surface to me) to be a trivial task.
I added logging for mysql connector as well as asp.net server and can see the following failure:
fail: Microsoft.EntityFrameworkCore.Database.Command[20102]
=> RequestId:0HLUD39EILP3R:00000001 RequestPath:/client/AddUserItem => Server.Controllers.ClientController.AddUserItem (ServerSoftware)
Failed executing DbCommand (78ms) [Parameters=[#p1='?' (DbType = UInt64), #p0='?' (Size = 4000)], CommandType='Text', CommandTimeout='30']
UPDATE `UserData` SET `Inventory` = #p0
WHERE `ID` = #p1;
SELECT ROW_COUNT();
A total hack is to just delay the arrival of the queries by a bit. This works because the client is most likely to generate these calls on load. Normally back-to-back calls aren't expected, so spreading them out in time by delaying on arrival works. However, I'd rather find a correct approach, since this just makes it less likely to be an issue:
ResponseError result = null;
await Task.Delay(SafeRandomGenerator(100, 500));
using (var transaction = context.Database.BeginTransaction(System.Data.IsolationLevel.Serializable))
// etc
This isn't a good answer, because it isn't what I wanted to do, but I'll post it here as it did solve my problem. My problem was that I was trying to read the database row, modify it in asp.net, and then write it back, all within a single transaction and while avoiding deadlocks. The backing field is JSON type, and MySQL provides some JSON functions to help modify that JSON directly in the database. This required me to write SQL statements directly instead of using EF, but it did work.
The first trick was to ensure I could create the row if it didn't exist, without requiring a transaction and lock.
INSERT INTO UserData VALUES ({0},'{{}}','{{}}') ON DUPLICATE KEY UPDATE ID = {0};
I used JSON_REMOVE to delete keys from the JSON field:
UPDATE UserData as S set S.Inventory = JSON_REMOVE(S.Inventory,{1}) WHERE S.ID = {0};
and JSON_SET to add/modify entries:
UPDATE UserData as S set S.Inventory = JSON_SET(S.Inventory,{1},CAST({2} as JSON)) WHERE S.ID = {0};
Note, if you're using EF Core and want to call this using FromSql then you need to return the entity as part of your SQL statement. So you'll need to add something like this to each SQL statement:
SELECT * from UserData where ID = {0} LIMIT 1;
Here is a full working example as an extension method:
public static async Task<UserData> FindOrCreateAsync(this IQueryable<UserData> table, ulong userId)
{
string sql = "INSERT INTO UserData VALUES ({0},'{{}}','{{}}') ON DUPLICATE KEY UPDATE ID = {0}; SELECT * FROM UserData WHERE ID={0} LIMIT 1;";
return await table.FromSql(sql, userId).SingleOrDefaultAsync();
}
public static async Task<UserData> JsonRemoveInventory(this DbSet<UserData> table, ulong userId, string key)
{
if (!key.StartsWith("$.")) key = $"$.\"{key}\"";
string sql = "UPDATE UserData as S set S.Inventory = JSON_REMOVE(S.Inventory,{1}) WHERE S.ID = {0}; SELECT * from UserData where ID = {0} LIMIT 1;";
return await table.AsNoTracking().FromSql(sql, userId, key).SingleOrDefaultAsync();
}
Usage:
var data = await context.UserData.FindOrCreateAsync(userId);
await context.UserData.JsonRemoveInventory(userId, itemId);
Aloha.
Web3 version is 0.20, and, according to documentation:
web3.eth.sendTransaction
web3.eth.sendTransaction(transactionObject [, callback])
Sends a transaction to the network.
Parameters
Object - The transaction object to send:
from: String - The address for the sending account. Uses the web3.eth.defaultAccount property, if not specified.
to: String - (optional) The destination address of the message, left undefined for a contract-creation transaction.
value: Number|String|BigNumber - (optional) The value transferred for the transaction in Wei, also the endowment if it's a contract-creation transaction.
gas: Number|String|BigNumber - (optional, default: To-Be-Determined) The amount of gas to use for the transaction (unused gas is refunded).
gasPrice: Number|String|BigNumber - (optional, default: To-Be-Determined) The price of gas for this transaction in wei, defaults to the mean network gas price.
data: String - (optional) Either a byte string containing the associated data of the message, or in the case of a contract-creation transaction, the initialisation code.
nonce: Number - (optional) Integer of a nonce. This allows to overwrite your own pending transactions that use the same nonce.
Number|String - (optional) If you pass this parameter it will not use the default block set with web3.eth.defaultBlock.
Function - (optional) If you pass a callback the HTTP request is made asynchronous. See this note for details.
I have function placeBet() which accepts multiple parameters:
function placeBet(uint8 _outcome, uint desiredMatchIndex, uint _amount) public payable{
// find a way to store a bid in order to be easily searchable, in order to easily send money to winners;
// require(!roundEnd, "Interactions with contract are locked, be careful next time!");
// require(state == State.Active, "Betting is over, game have already started!");
require(msg.value > 0, "It isn't possible to place a bet without a money ");
if(!isDuplicate(msg.sender)) addressIndices.push(msg.sender);
testina(msg.sender, _outcome, desiredMatchIndex);
existingBets[msg.sender].push(Bet({
bettor: msg.sender,
// name: name,
amount: _amount,
bet: desiredMatchIndex,
outcome: _outcome
}));
//emit event, finally;
}
, so my question is how should I include needed additional parameters (outcome, desiredMatchIndex, amount ) [maybe last one is reduntant ] alongside
transactionObject using web3.js?
Thanks : )
Oh, i am sorry, there was an example in documentation, just not adequately referenced.
/ Explicitly sending a transaction to this method
myContractInstance.myMethod.sendTransaction(param1 [, param2, ...] [, transactionObject] [, callback])
HOW IT SHOULD WORK: User clicks on a listbox with multitude of IDs. Then, the edit controls are filed with the data from the specified ID (row).
HOW IT WORKS: It crashes at res->getString
case LBN_SELCHANGE:
{
string str;
int index = SendMessage(GetDlgItem(hwnd, IDC_lbList), LB_GETCURSEL, 0, 0);
string sint = boost::lexical_cast<std::string>(index);
string smt="SELECT client, telnr, atnrem, papatn, gedapr
FROM tremreg WHERE id = ";
string query = smt + sint;
res = stmt->executeQuery(query.c_str());
//starts crashing here
str = res->getString("client");
SetDlgItemText(hwnd, IDC_eClient, str.c_str());
str = res->getString("telnr");
SetDlgItemText(hwnd, IDC_eTelNr, str.c_str());
str = res->getString("antrem");
SetDlgItemText(hwnd, IDC_eRemAtn, str.c_str());
str = res->getString("paprem");
SetDlgItemText(hwnd, IDC_ePapAtn, str.c_str());
str = res->getString("gedapr");
SetDlgItemText(hwnd, IDC_eGedApr, str.c_str());
delete res;
delete stmt;
}
break;
CRASH INFO:
Unhandled exception at 0x753C812F in Registracija.exe:
Microsoft C++ exception:sql::InvalidArgumentException at memory location 0x0018F7AC.
Digging around google I found somebody mentioning that VARCHAR type values should be fetched with getBlob. That did not help at all though, and crashed just the same. getInt works however, and is used in other parts of the program.
QUESTION: What steps should I take to make it work?
does the same query run properly if you do it in the database directly? Also, are all the returned fields really Strings? And is it valid to call them by name, rather than by the sequence number? I.e. getString(1), etc.
The weird behavior is that a java.sql.Timestamp that I create using the System.currentTimeMillis() method, is stored in my MySQL database as 1970-01-01 01:00:00.
The two timestamps I am creating are to mark the beginning and end of a monitoring task I am trying to perform, what follows are excepts from the code where the behavior occurs
final long startTime = System.currentTimeMillis();
while(numberOfTimeStepsPassed < numTimeStep) {
/*
* Code in here
*/
}
final long endTime = System.currentTimeMillis();
return mysqlConnection.insertDataInformation(matrixOfRawData, name,Long.toString(startTime),
Long.toString(endTime), Integer.toString(numTimeStep),
Integer.toString(matrixOfRawData[0].length), owner,
type);
And here is the code used for inserting the time stamps and other data into the MySQL database
public String insertDataInformation(final double [][] matrix,
final String ... params) {
getConnection(lookUpName);
String id = "";
PreparedStatement dataInformationInsert = null;
try {
dataInformationInsert =
databaseConnection.prepareStatement(DATA_INFORMATION_PREPARED_STATEMENT);
id = DatabaseUtils.createUniqueId();
int stepsMonitored = Integer.parseInt(params[STEPS_MONITORED]);
int numberOfMarkets = Integer.parseInt(params[NUMBER_OF_MARKETS]);
dataInformationInsert.setNString(ID_INDEX, id);
dataInformationInsert.setNString(NAME_INDEX, params[0]);
dataInformationInsert.setTimestamp(START_INDEX, new Timestamp(Long.parseLong(params[START_INDEX])));
dataInformationInsert.setTimestamp(END_INDEX, new Timestamp(Long.parseLong(params[END_INDEX])));
dataInformationInsert.setInt(STEPS_INDEX, stepsMonitored);
dataInformationInsert.setInt(MARKETS_INDEX, numberOfMarkets);
dataInformationInsert.setNString(OWNER_INDEX, params[OWNER]);
dataInformationInsert.setNString(TYPE_INDEX, params[TYPE]);
dataInformationInsert.executeUpdate();
insertRawMatrix(matrix, id, Integer.toString(stepsMonitored), Integer.toString(numberOfMarkets));
} catch (SQLException sqple) {
// TODO Auto-generated catch block
sqple.printStackTrace();
System.out.println(sqple.getSQLState());
} finally {
close(dataInformationInsert);
dataInformationInsert = null;
close(databaseConnection);
}
return id;
}
The important lines of code are :
dataInformationInsert.setTimestamp(START_INDEX, new Timestamp(Long.parseLong(params[START_INDEX])));
dataInformationInsert.setTimestamp(END_INDEX, new Timestamp(Long.parseLong(params[END_INDEX])));
The JavaDocs on the TimeStamp ( http://docs.oracle.com/javase/1.5.0/docs/api/java/sql/Timestamp.html ) says that it takes in time in milliseconds since 1st January 1970 and a simple print test confirms this.
What I am looking for is:
A reason for this behavior when trying to store timestamps in a MySQL database through java.sql.Timestamp?
Any solutions to this behavior?
Any possible alternatives?
Any possible improvements?
EDIT:
Been asked to include what START_INDEX and END_INDEX are:
private static final int END_INDEX = 4;
private static final int START_INDEX = 3;
Apologises for not putting them in the original post.
Okay, look at your call:
insertDataInformation(matrixOfRawData, name, Long.toString(startTime),
Long.toString(endTime), Integer.toString(numTimeStep),
Integer.toString(matrixOfRawData[0].length), owner,
type);
So params will have values:
0: name
1: start time
2: end time
3: numTimeStep
4: matrixOfRowData[0].length
5: owner
6: type
Then you're doing:
dataInformationInsert.setTimestamp(START_INDEX,
new Timestamp(Long.parseLong(params[START_INDEX])));
... where START_INDEX is 3.
So you're using the value corresponding to numTimeStep as the value for the timestamp... I suspect you don't want to do that.
I would strongly advise you to create a simple object type (possibly a nested type in the same class) to let you pass these parameters in a strongly typed, simple to get right fashion. The string conversion and the access by index are both unwarranted, and can easily give rise to errors.