Chainlink: API array response to bytes - ethereum

I am using the fulfillOracleRequest2 with bytes data
fulfillOracleRequest2(bytes32 requestId, uint256 payment, address callbackAddress, bytes4 callbackFunctionId, uint256 expiration, bytes data)
my API and external adapter parse is returning this:
parse jsonparse
["629343835796877311","629343835797458943","629343835797471231","629343835797479423","629343835797508095","629343835797516287","629343835796869119","629343835796881407","629343835796893695","629343835797344255","629343835797348351","629343835797467135","629343835797475327","629343835797483519","629343835797491711","629343835797499903","629343835797512191","629343835798421503","629343835796873215","629343835796885503","629343835796889599","629343835796910079","629343835796926463","629343835797327871","629343835797331967","629343835797352447","629343835797377023","629343835797495807","629343835797503999","629343835797528575","629343835797544959","629343835798409215","629343835798413311","629343835798417407","629343835798478847","629343835798487039","629343835796811775","629343835796828159","629343835796901887","629343835796914175","629343835796918271","629343835797008383","629343835797012479","629343835797336063","629343835797340159","629343835797360639","629343835797381119","629343835797385215","629343835797413887","629343835797524479","629343835797536767","629343835797540863","629343835797803007","629343835797811199","629343835798425599","629343835798429695","629343835798433791","629343835798474751","629343835798482943","629343835798495231","629343835796803583","629343835796815871","629343835796819967","629343835796905983","629343835796922367","629343835797000191","629343835797004287","629343835797024767","629343835797364735","629343835797368831","629343835797372927","629343835797393407","629343835797397503","629343835797409791","629343835797442559","629343835797446655","629343835797532671","629343835797549055","629343835797745663","629343835797786623","629343835797794815","629343835797807103","629343835798380543","629343835798388735","629343835798491135","629343835798499327","629343835798548479","629343835798552575","629343838201618431","629343838201634815"]
each of those uint64 actually is a string because javascript has issues dealing with big numbers.
I was trying to first encode those strings into uint64 and then in bytes with
parse [type="jsonparse" path="data" data="$(fetch)"]
encode_data0 [type="ethabiencode" abi="(uint64[] value)" data="{\\"value\\": $(parse)}"]
encode_data [type="ethabiencode" abi="(bytes32 requestId, bytes value)" data="{\\"requestId\\": $(decode_log.requestId), \\"value\\": $(encode_data0.value)}"]
encode_tx [type="ethabiencode"
abi="fulfillOracleRequest2(bytes32 requestId, uint256 payment, address callbackAddress, bytes4 callbackFunctionId, uint256 expiration, bytes data)"
data="{\\"requestId\\": $(decode_log.requestId), \\"payment\\": $(decode_log.payment), \\"callbackAddress\\": $(decode_log.callbackAddr), \\"callbackFunctionId\\": $(decode_log.callbackFunctionId), \\"expiration\\": $(decode_log.cancelExpiration), \\"data\\": $(encode_data)}"
]
the problem is that encode_data0 is producing a weird string of weird values:
OK
encode_data0 ethabiencode
"0x0000000000000000000000000000000000000000000000000000000000000020000000000000000000000000000000000000000000000000000000000000005a00000000000000000000000000000000000000000000000008bbe0e25e412fff00000000000000000000000000000000000000000000000008bbe0e25e4a0fff00000000000000000000000000000000000000000..."
abi: (uint64[] value)
data: {"value": $(parse)}
FAIL
encode_data ethabiencode
data: value at key 'encode_data0' is a string, not a map or slice: keypath not found
abi: (bytes32 requestId, bytes value)
data: {"requestId": $(decode_log.requestId), "value": $(encode_data0.value)}
How do I go from this array of strings (returned by my API) into bytes that I can pass to fulfillOracleRequest2?

Related

Solidity: decode byte data into two structs

I have a function call which only accepts bytes data (dydx _getCallActions)
_getCallAction(bytes memory data)
During contract execution the data is passed to a user defined function named: "callFunction"
When decoding into a single struct, it works, however I want to to decode the data into two separate structs.
function callFunction(bytes calldata _data){
// This works, when passed in encoded data matching Struct1Type
Struct1Type memory data1 = abi.decode(_data, (Struct1Type));
}
function callFunction(bytes calldata _data){
// Doesnt work
Struct1Type memory data1, Struct2Type memory data2 = abi.decode(_data, (Struct1Type,Struct2Type));
}
I could decode the data into a single struct and then selectively cast it into the two desired structs, but this seems gas inefficient

You can split the array by the total byte length of the first struct rounded up to a multiplier of 32 - the slot length - and then decode each chunk separately.
In the example below, the length of Struct1Type is just 8 bytes, but the memory and storage slots take up the whole 32byte word. That's why we're splitting at the 32nd index.
Code:
pragma solidity ^0.8;
contract MyContract {
struct Struct1Type {
uint8 number;
}
struct Struct2Type {
uint16 number;
}
function callFunction(bytes calldata _data) external pure returns (Struct1Type memory, Struct2Type memory) {
// `:32` returns a chunk "from the beginning to the 32nd index"
Struct1Type memory data1 = abi.decode(_data[:32], (Struct1Type));
// `32:` returns a chunk "from the 32nd index to the end"
Struct2Type memory data2 = abi.decode(_data[32:], (Struct2Type));
return (data1, data2);
}
}
Input:
# two values: `1` and `2`
0x00000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000002
Output:
0: tuple(uint8): 1
1: tuple(uint16): 2

Go JSON Marshaller errors when converting int64 bytes

I am writing a time alias to JSON Marshal some time into a Unix format (left some of my experimenting test code in there
type NotifyTime time.Time
// MarshalJSON implements marshaller interface. Marshals time.Time into unix time in bytes
func (t NotifyTime) MarshalJSON() ([]byte, error) {
// unixTime := time.Time(t).Unix()
unixTime := 1626132059 // unixTime := time.Now().Unix()
buffer := make([]byte, 8)
binary.PutVarint(buffer, int64(unixTime))
// binary.LittleEndian.PutUint64(buffer, uint64(unixTime))
// try to convert back
fmt.Println(string(buffer))
unixIntValue := int64(binary.LittleEndian.Uint64(buffer))
fmt.Println(unixIntValue)
return buffer, nil
}
When I run json.Marshal on an object with NotifyTime struct, it errs, with the following,
json: error calling MarshalJSON for type notify.NotifyTime: invalid character '¶' looking for beginning of value
type TestMe struct {
Time NotifyTime `json:"time"`
}
testJSON := TestMe{
Time: NotifyTime(time.Now()),
}
marshalled, err := json.Marshal(testJSON)
I have switched to marshalling it as a string, but still curious as to why this happens. When stepping through the code it seems to be because
on function compact on go/1.16.2/libexec/src/encoding/json/indent.go:17
it is looping over the marshalled bytes of the JSON
and the first (0th) byte fails the checks in
go/1.16.2/libexec/src/encoding/json/scanner.go:247

Let's put aside the aspect of encoding a time.Time and lets focus on how the int64 is being turned into JSON.
binary.PutVarint uses an encoding that is appropriate for low level wire or file formats. For the constant 1626132059, this writes into buffer [182 185 230 142 12 0 0 0]. The first character is 182 PILCROW SIGN in UTF-8. This is where '¶' comes from. You are getting an error like:
json: error calling MarshalJSON for type main.NotifyTime: invalid character '¶' looking for beginning of value
This is not the beginning of a valid JSON value. You will need to find an encoding of int64 that is a JSON value, such as a decimal number 1626132059 or a string of hexadecimal digits "60ecce5b".
In general you need to be careful putting binary string values into JSON as these can contain special characters that need to be escaped.

Unmarshal JSON object published over MQTT in Go

I am receiving sensor data over MQTT. I want to check if the temperature is over 20 degrees, and if it is, send a message.
var f MQTT.MessageHandler = func(client MQTT.Client, msg MQTT.Message) {
type Data struct {
Sensor string `json:"sensor"`
Temp []int `json: "temperature"`
Hum []int `json: "humidity"`
}
var sensorData []Data
message := ""
err := json.Unmarshal(msg.Payload(), &sensorData)
if err != nil {
panic(err)
}else {
//access certain element in the Data struct
for _, i := range sensorData {
if i.Temp[2] > 20 { //where the temperature is stored
fmt.Println("Temperature too high")
message = "Temperature too high"
}else{
fmt.Println("Perfect temperature")
message = "Perfect temperature"
}
}
}
// Publish further instructions to "sensor/instruction"
token := client.Publish("sensor/instruction", 0, false, message)
token.Wait()
}
Currently I am publishing two JSON objects to be received. I think part of the problem is distinguishing between the two objects. This is the error I am getting panic: json: cannot unmarshal object into Go value of type []main.Data. These are the JSON objects I am publishing to a topic:
{'sensor': 'DHT22', 'temperature': [22.7, 22.7, 22.8], 'humidity': [51.9, 52.0, 52.0]}
{'actuator': 'arduinoLED', 'blink': ['false', 'false', 'false']}
The msg.Payload() gives
{"sensor": "DHT22", "temperature": [22.9, 22.9, 22.9], "humidity": [50.9, 50.9, 50.9]}
{"actuator": "arduinoLED", "blink": ["false", "false", "false"]
These objects are published constantly in a loop. I want to unmarshal the first object and access a specific element.

Slice vs single object
You are declaring a slice:
var sensorData []Data
But then your payload is not an array but rather only one object:
{'sensor': 'DHT22', 'temperature': [22.7, 22.7, 22.8], 'humidity': [51.9, 52.0, 52.0]}
So that error message is telling you it cannot unmarshal a single object as a slice.
Try changing that declaration to:
var sensorData Data
Then, instead of iterating over it, you need to just check that one instance of the data.
Additional mismatches between the payload and the struct type
After that, you'll probably get another error since the temperature and humidity array seem to contain decimal numbers:
`{'sensor': 'DHT22', 'temperature': [22.7, 22.7, 22.8], 'humidity': [51.9, 52.0, 52.0]`
But you are declaring those as int slices in your code:
Temp []int `json: "temperature"`
Hum []int `json: "humidity"`
So you'll need to change those to be []float64
Differentiating the two different payloads
About differentiating the two object types, it'd be better if you try to do that, but even if you don't, Go's json library will ignore problems if field names do not match, so what will happen is that when de-serializing your actuator payloads into Data, all fields will have their default values, but no error will be received.
This check will probably throw a panic, cause the array will be empty in that case:
if i.Temp[2] > 20
One "hacky" way of solving this issue would be to only process the data if the sensor field is not a blank string.
Assuming you always receive a sensor name in the sensor messages, the only case when that will result in an empty string is if you just processed one of the other messages.

There are two main reasons for your error One is you have float value for temperature and humidity but you are passing slice of int
type Data struct {
Sensor string `json:"sensor"`
Temp []int `json: "temperature"` // this should be []float64
Hum []int `json: "humidity"` // this should be []float64
}
Other is there are two objects in msg.Payload which is not a slice of Data struct. Working code.
package main
import (
"encoding/json"
"fmt"
)
type Data struct {
Sensor string `json:"sensor"`
Temp []float64 `json:"temperature"`
Hum []float64 `json:"humidity"`
}
func main() {
bytes := []byte(`{
"sensor": "DHT22",
"temperature": [22.7, 22.7, 22.8],
"humidity": [51.9, 52.0, 52.0]
}`)
var sensorData Data
if err := json.Unmarshal(bytes, &sensorData); err != nil {
fmt.Println(err)
}
fmt.Println(sensorData)
}
Check working code on Go playground

How to convert json to struct using goreq?

Using Go I'm trying to get some json from a server for which I'm using the goreq library. When I print out the resulting string as follows:
s, _ := res.Body.ToString()
fmt.Println(s)
I get a correct json string:
{"success":true,"testnet":false,"message":"","result":{"btc":4014.16,"edp":4014.16},"msIn":1505820331492,"msOut":1505820331492}
So using this json-to-go webservice I converted this json message to a struct:
type Index struct {
Success bool `json:"success"`
Testnet bool `json:"testnet"`
Message string `json:"message"`
Result struct {
Btc float64 `json:"btc"`
Edp float64 `json:"edp"`
} `json:"result"`
MsIn int64 `json:"msIn"`
MsOut int64 `json:"msOut"`
}
and I use that as follows (implementation of FromJsonTo() here):
var item Index
res.Body.FromJsonTo(&item)
fmt.Println(item)
This just prints out the nulled Index struct though (while the json str is still the same):
{false false {0 0} 0 0}
Any idea what I might be doing wrong here?

By calling res.Body.ToString() you read the whole body of the response. Next, when you call res.Body.FromJsonTo(), body is empty and therefore EOF error is returned. Removing ToString() from your code should help.

send c-like struct as arrayBuffer via udp chrome socket

i'm using chrome.sockets to send udp messages.
i need to send this c-like struct:
UInt16 ID=0,
UInt Size=20,
UInt16 CRC=0,
UInt16 MsgCount=0,
UInt32 App=0,
UInt32 Port=55555,
UInt32 Token=0
the chrome.udp.send needs to get an arrayBuffer.
what i did in my code is this:
var arrayBuffer = new ArrayBuffer(20);
var dv = new DataView(arrayBuffer,0);
dv.setUint16(0,0);
dv.setUint16(2,20);
dv.setUint16(4,0);
dv.setUint16(6,0);
dv.setUint32(8,0);
dv.setUint32(12,55555);
dv.setUint32(16,0);
but when i
console.log(arrayBuffer);
i get
ArrayBuffer {}
it is empty.
How can i send a c-like struct like this correctly?
Thanks!

ArrayBuffer stores bytes - any byte values at all.
console.log() expects human-readable ASCII strings, so that it can display them.
Unfortunately, logging an ArrayBuffer doesn't help - as soon as there's an ASCII NUL character (and your data has one in the very first byte) the string stops. You will need to write a different function to accept an ArrayBuffer and produce a Hex string that you can pass to console.log()