I am working on a project to write to and read from a TP Link / Kaza power strip or smart plug.
The data that is sent is encrypted json that has been "autokey encrypted".
So far I have been able to convert a typescript encrypt function and it works well. I get the expected result. However, I need to add a "header" to my encrypted data. That data is 3 null bytes followed by a byte that is a measure of the length of the encrypted bytes.
The typescript example has this bit of code to "encrypt with headers", however, I've hit a bit of a wall trying to convert it to something usable. Can someone nudge me along the path ?
First are the two typescript functions: (borrowed from https://github.com/plasticrake/tplink-smarthome-crypto/blob/master/src/index.ts)
/**
* Encrypts input where each byte is XOR'd with the previous encrypted byte.
*
* #param input - Data to encrypt
* #param firstKey - Value to XOR first byte of input
* #returns encrypted buffer
*/
export function encrypt(input: Buffer | string, firstKey = 0xab): Buffer {
const buf = Buffer.from(input);
let key = firstKey;
for (let i = 0; i < buf.length; i += 1) {
// eslint-disable-next-line no-bitwise
buf[i] ^= key;
key = buf[i];
}
return buf;
}
/**
* Encrypts input that has a 4 byte big-endian length header;
* each byte is XOR'd with the previous encrypted byte.
*
* #param input - Data to encrypt
* #param firstKey - Value to XOR first byte of input
* #returns encrypted buffer with header
*/
export function encryptWithHeader(
input: Buffer | string,
firstKey = 0xab
): Buffer {
const msgBuf = encrypt(input, firstKey);
const outBuf = Buffer.alloc(msgBuf.length + 4);
outBuf.writeUInt32BE(msgBuf.length, 0);
msgBuf.copy(outBuf, 4);
return outBuf;
}
Second is what I have so far.
// This part works well and produces the expected results
String encrypt(String input)
{
int16_t firstKey = 0xab;
String buf;
int key;
int i;
buf = input;
key = firstKey;
i = 0;
for (;i < buf.length();(i = i + 1))
{
buf[i] ^= key;
key = buf[i];
}
return buf;
}
// This does not function yet, as I'm pretty lost..
// This was orginally converted from typescript with https://andrei-markeev.github.io/ts2c/
// I started work on converting this, but ran into errors I don't know how to solve.
String encryptWithHeader(String input){
String msgBuf;
String outBuf;
int16_t firstKey = 0xab;
char * null = NULL;
msgBuf = encrypt(input);
outBuf = msgBuf.length() +1;
//this is where I got lost...
assert(null != NULL);
null[0] = '\0';
strncat(null, outBuf, msgBuf.length());
str_int16_t_cat(null, 4);
outBuf = msgBuf + 4
return outBuf;
}
Finally, the data:
//this is the unencrypted json
String offMsg = "{\"system\":{\"set_relay_state\":{\"state\":0}}}";
//current encrypt function produces:
d0f281f88bff9af7d5ef94b6c5a0d48bf99cf091e8b7c4b0d1a5c0e2d8a381f286e793f6d4eedea3dea3
//the working "withheaders" should produce:
00002ad0f281f88bff9af7d5ef94b6c5a0d48bf99cf091e8b7c4b0d1a5c0e2d8a381f286e793f6d4eedea3dea3
Admittedly my C/C++ ability is very limited and I can spell typescript, that's about all. I have a very extensive history with PHP. As useful as that is. So, I understand the basics of data structures and whatnot, but I'm venturing off into areas I've never been in. Any help would be greatly appreciated.
It looks like the encryption is fairly simple: write the current character XORed with the key to the buffer and make that newly written character the new key. It also looks like the "withHeaders" version adds the length of the encrypted string as a 4 byte integer to the start of the buffer. I think it might be easier to allocate a character array and pass that array to a function that writes the result to that buffer. For example:
void encryptWithHeader(byte buffer[], int bufferLength, byte key, String message) {
int i;
uint32_t messageLength = message.length();
Serial.println(message);
Serial.println(message.length());
// check that we won't overrun the buffer
if ( messageLength + 5 < bufferLength) {
buffer[0] = messageLength >> 24 & 0xFF;
buffer[1] = messageLength >> 16 & 0xFF;
buffer[2] = messageLength >> 8 & 0xFF;
buffer[3] = messageLength & 0xFF;
for (i = 0; i < messageLength; i++) {
buffer[i + 4] = message[i] ^ key;
key = buffer[i + 4];
}
}
else { // we would have overrun the buffer
Serial.println("not enough room in buffer for message");
}
}
void setup() {
// put your setup code here, to run once:
Serial.begin(9600);
}
void loop() {
byte theBuffer[64];
int i;
String offMsg = "{\"system\":{\"set_relay_state\":{\"state\":0}}}";
encryptWithHeader(theBuffer, 64, 0xab, offMsg);
// now print it out to check
for (i = 0; i < offMsg.length() + 4; i++) {
if (theBuffer[i] < 0x10) // adds an extra zero if a byte prints as on1y 1 char
Serial.print("0");
Serial.print(theBuffer[i], HEX);
}
while (true)
;
}
If you want to send the character buffer to a remote device you can send it out one byte at a time:
for (i = 0; i < offMsg.length() + 4; i++)
Serial.write(theBuffer[i]);
Related
I'm trying to create a full HTML result with HTTP headers with the correct size for content-length as simple as possible.
I looked at setbuf() function but it only works with streams, but I want to directly buffer the output to memory without having a disk involved. Is there a simpler function that does what I'm trying to achieve?
I know in php, one can use ob_start() to start buffering data and ob_get_contents() to load the data in memory and determine the size so is there anything like this in C that I can use?
int main(){
char *mybuf=malloc(1048576); //Allocate 1MB memory for buffer
char *mytitle="Webpage title";
char *mydesc="This is a page version";
int myver=2;
//print HTTP header of known data
printf("HTTP/1.1 200 OK\r\n");
printf("Server: Something\r\n");
printf("Content-type: text/html; charset=UTF-8\r\n");
printf("Cache-control: no-cache\r\n");
//Buffer HTML output to get size (we don't print these 4 lines yet)
sprintf(mybuf,"<html><head><title>%s</title></head><body>\r\n",mytitle);
sprintf(mybuf+strlen(mybuf),"<h1>%s</h1>\r\n",mytitle);
sprintf(mybuf+strlen(mybuf),"<p>%s %d</p>\r\n",mydesc,myver);
sprintf(mybuf+strlen(mybuf),"</body></html>\r\n");
//Print content length header with size of buffered data
printf("Content-length: %d\r\n",strlen(mybuf));
//then print the rest of the data
printf("%s",mybuf);
free(mybuf);
return 0;
}
The C language specification does not define any facility for streams backed only by memory, without any underlying file. Your sprintf() based approach is not bad, but it could be improved by capturing and using the return value, which tells you how many characters were printed. You could use this to avoid all the strlen() calls, which could be a considerable savings if the content is large, especially if it is printed in many small pieces:
size_t total_bytes_written = 0;
long rval;
rval = sprintf(mybuf, "<html><head><title>%s</title></head><body>\r\n", mytitle);
if (rval < 0) {
// handle error ...
}
total_bytes_written += rval;
rval = sprintf(mybuf + total_bytes_written, "<h1>%s</h1>\r\n", mytitle);
// ...
Alternatively, if you are on a system that conforms to POSIX.1-2008 or later, and you are willing to limit your program to such systems, then you might find open_memstream() of interest. This gives you a stream backed by a dynamically managed block of memory. You would write to it with fprintf() or other stream IO functions, and when done, read back a pointer to the data and the total size. Something like this:
char *buf_ptr = NULL;
size_t buf_size = 0;
FILE *mem = open_memstream(&buf_ptr, &buf_size);
fprintf(mem, "<html><head><title>%s</title></head><body>\r\n", mytitle);
fprintf(mem, "<h1>%s</h1>\r\n", mytitle);
// ...
fclose(mem); // mandatory
printf("Content-length: %d\r\n", buf_size);
printf("%s", buf_ptr); // the data are guaranteed null-terminated
free(buf); // you are responsible for this
(All error checks omitted for brevity.)
This relieves you of the manual size tracking, and it gains you a dynamically-growing backing buffer so that you can handle both large and small outputs with reasonable memory efficiency.
There are multiple solutions to this problem:
your approach is OK, but you should use snprintf() to avoid potential buffer overflows:
int main() {
char *mybuf = malloc(1048576); //Allocate 1MB memory for buffer
const char *mytitle = "Webpage title";
const char *mydesc = "This is a page version";
int myver = 2;
size_t pos = 0;
//print HTTP header of known data
printf("HTTP/1.1 200 OK\r\n");
printf("Server: Something\r\n");
printf("Content-type: text/html; charset=UTF-8\r\n");
printf("Cache-control: no-cache\r\n");
//Buffer HTML output to get size (we don't print these 4 lines yet)
pos += snprintf(mybuf + pos, pos < sizeof mybuf ? sizeof mybuf - pos : 0,
"<html><head><title>%s</title></head><body>\r\n", mytitle);
pos += snprintf(mybuf + pos, pos < sizeof mybuf ? sizeof mybuf - pos : 0,
"<h1>%s</h1>\r\n", mytitle);
pos += snprintf(mybuf + pos, pos < sizeof mybuf ? sizeof mybuf - pos : 0,
"<p>%s %d</p>\r\n", mydesc, myver);
pos += snprintf(mybuf + pos, pos < sizeof mybuf ? sizeof mybuf - pos : 0,
"</body></html>\r\n");
//Print content length header with size of buffered data
printf("Content-length: %zu\r\n", pos);
printf("\r\n");
//then print the rest of the data
printf("%s", mybuf);
free(mybuf);
return 0;
}
Here is a simpler one, combining the calls and using snprintf to compute the length:
int main() {
const char *mytitle = "Webpage title";
const char *mydesc = "This is a page version";
int myver = 2;
size_t length = 0;
//print HTTP header of known data
printf("HTTP/1.1 200 OK\r\n");
printf("Server: Something\r\n");
printf("Content-type: text/html; charset=UTF-8\r\n");
printf("Cache-control: no-cache\r\n");
//Buffer HTML output to get size (we don't print these 4 lines yet)
length = snprintf(NULL, 0,
"<html><head><title>%s</title></head><body>\r\n"
"<h1>%s</h1>\r\n"
"<p>%s %d</p>\r\n"
"</body></html>\r\n",
mytitle, mytitle, mydesc, myver);
//Print content length header with size of buffered data
printf("Content-length: %zu\r\n", length);
printf("\r\n");
//then print the rest of the data
printf("<html><head><title>%s</title></head><body>\r\n"
"<h1>%s</h1>\r\n"
"<p>%s %d</p>\r\n"
"</body></html>\r\n",
mytitle, mytitle, mydesc, myver);
return 0;
}
you could use open_memstream() as explained by John Bollinger:
int main() {
const char *mytitle = "Webpage title";
const char *mydesc = "This is a page version";
int myver = 2;
size_t length = 0;
char *buf_ptr = NULL;
size_t buf_size = 0; // the buffer size
int length = 0; // the length of the output
FILE *mem = open_memstream(&buf_ptr, &buf_size);
//print HTTP header of known data
printf("HTTP/1.1 200 OK\r\n");
printf("Server: Something\r\n");
printf("Content-type: text/html; charset=UTF-8\r\n");
printf("Cache-control: no-cache\r\n");
//Buffer HTML output to get size (we don't print these 4 lines yet)
if (mem == NULL) {
// Deal with the error
printf("Internal error\r\n");
return 1;
}
length = fprintf(mem,
"<html><head><title>%s</title></head><body>\r\n"
"<h1>%s</h1>\r\n"
"<p>%s %d</p>\r\n"
"</body></html>\r\n",
mytitle, mytitle, mydesc, myver);
//Print content length header with size of buffered data
printf("Content-length: %zu\r\n", length);
printf("\r\n");
//then print the rest of the data
//beware that buf_ptr is not necessarily null terminated
printf("%.*s", length, buf_ptr);
//could also use fwrite(f, buf_ptr, 1, length, stdout);
free(buf_ptr);
return 0;
}
you can implement your own mem_printf() that allocates or reallocates a memory buffer:
#include <stdarg.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
int mem_printf(char **bufp, size_t *lenp, const char *fmt, ...) {
char buf[1024];
va_list ap;
char *newbuf;
int len;
va_start(ap, fmt);
len = vsnprintf(buf, sizeof buf, fmt, ap);
va_end(ap);
if (len < 0)
return -1;
newbuf = realloc(*bufp, *lenp + len + 1);
if (newbuf == NULL)
return -1;
*bufp = newbuf;
if ((size_t)len < sizeof buf) {
memcpy(newbuf + *lenp, buf, len + 1);
} else {
va_start(ap, fmt);
vsnprintf(newbuf + *lenp, len + 1, fmt, ap);
va_end(ap);
}
*lenp += len;
return len;
}
int main() {
const char *mytitle = "Webpage title";
const char *mydesc = "This is a page version";
int myver = 2;
char *buf_ptr = NULL;
size_t buf_len = 0;
//print HTTP header of known data
printf("HTTP/1.1 200 OK\r\n");
printf("Server: Something\r\n");
printf("Content-type: text/html; charset=UTF-8\r\n");
printf("Cache-control: no-cache\r\n");
//Buffer HTML output to get size (we don't print these 4 lines yet)
mem_printf(&buf_ptr, &buf_len,
"<html><head><title>%s</title></head><body>\r\n"
"<h1>%s</h1>\r\n"
"<p>%s %d</p>\r\n"
"</body></html>\r\n",
mytitle, mytitle, mydesc, myver);
//Print content length header with size of buffered data
printf("Content-length: %zu\r\n", buf_len);
printf("\r\n");
//then print the rest of the data
fputs(buf_ptr, stdout);
free(buf_ptr);
return 0;
}
Content-Length, as you probably know, is the size of your body. This means you have to know at least the length of the body before you can generate the header (or use chunked transfer). sprintf returns the number of bytes written, either accumulate those, or squash it into a single call:
int n = snprintf(mybuf,
mybuf_len,
"<html><head><title>%s</title></head><body>\r\n"
...
"</body></html>\r\n",
mytitle, ..., myver
);
Then generate the header(n) with the n you figured out. If you pass in NULL for instead of mybuf, you get a size without writing anything, then you could do another write a pass in header with the count you got from the first call. If you want, wrap this in a function or two:
int response(const char *header, int body_len, const char body[body_len]) {
...
}
Consider picking a template library (I like the mustache format). It's kinda messy and error prune doing this by hand with raw strings. You have a 1 MB buffer allocated there but use like a .5k, and you will shoot yourself in the foot one day anyhow.
When reading a JSON string from the serial port on an ESP8266 it cuts off the beginning of the data.
I have tried reading data from the Serial port and printing each character, however it is cutting off part of the begging of the data.
void setup() {
Serial.begin(115200);
while (!Serial) {
;
}
}
void loop() {
int curSize = 30;
char* buffer = new char[curSize];
std::fill_n(buffer, curSize, 0);
int pos = 0;
Serial.print("Sending: ");
while(Serial.available() == false) delay(500);
while (Serial.available()) {
char c = Serial.read();
Serial.print(c);
if(pos == curSize-1){
char* newBuffer = increaseBuffer(buffer, curSize, curSize + 30);
curSize += 30;
delete[] buffer;
buffer = newBuffer;
}
if(c == '\n'){
buffer[pos] = 0;
pos = 0;
break;
}
buffer[pos++] = c;
}
if(buffer[0] != 0) {
sendBuffer(buffer);
}
delete[] buffer;
}
char* increaseBuffer(char* orig, int oldSize, int newSize){
char* data = new char[newSize];
std::fill_n(data, newSize, 0);
for(int i = 0; i < newSize; i++){
if(i < oldSize) data[i] = orig[i];
else data[i] = '\0';
}
return data;
}
JSON data used (and expected output)
{"type":0,"ver":"0.0.1","T":[28,29,29,29,29,29,29,29,29,29],"H":[59.1608,59.1608,60,59.1608,60,60,60,59.1608,59.1608,59.1608],"DP":[20.36254,20.36254,20.59363,20.36254,20.59363,20.59363,20.59363,20.36254,20.36254],"HI":[30.90588,30.90588,31.0335,30.90588,31.0335,31.0335,31.0335,30.90588,30.90588]}
examples of what is actually output
Example 1: 9,29,29,29,29,29,29,29,29],"H":[59.1608,59.1608,60,59.1608,60,60,60,59.1608,59.1608,59.1608],"DP":[20.36254,20.36254,20.59363,20.36254,20.59363,20.59363,20.59363,20.36254,20.36254],"HI":[30.90588,30.90588,31.0335,30.90588,31.0335,31.0335,31.0335,30.90588,30.90588]}
Example 2: 29,29,29,29,29,29,29,29,29],"H":[59.1608,59.1608,60,59.1608,60,60,60,59.1608,59.1608,59.1608],"DP":[20.36254,20.36254,20.59363,20.36254,20.59363,20.59363,20.59363,20.36254,20.36254],"HI":[30.90588,30.90588,31.0335,30.90588,31.0335,31.0335,31.0335,30.90588,30.90588]}
Try making the delay 1 instead of 500 in the blocking loop that's waiting for data to start coming in. I'm going to guess what happens is that on one iteration of that loop Serial.available() is false and during the delay you start to get data coming in that ends up getting written over by the time your delay ends to check again.
What I'm picturing is the following. If you were to expand out that delay(500) to be delay(1) called 500 times.
while(Serial.available() == false){
delay(1);
delay(1);
// ...
delay(1); // first character comes in
delay(1);
delay(1); // second character comes in
// ...
delay(1); // n character comes in
}
Then after the delay is over you start actually collecting the characters that are coming in.
I want to save String array to binary file.
I searched and tried, didn't come up with normal binary file contents.
(not like 00 0A 10) came up with (8죅? <죅? A죅? E죅?) - written in korean.
the writing part of the code was
FILE *fp_course = fopen("KJH_course.dat", "wb");
while ((row = mysql_fetch_row(res)) != NULL) { //JUST FOR FETCHING DATA FROM MYSQL, WORKED WELL
printf("%s %s %s %s\n", row[0], row[1], row[2], row[3]);
fwrite(row, sizeof(row), 1, fp_course);
};
the data in row is {'32141183', 'JHK', '2'}
(the last row[3] was used for larger array later)
did i wrote something wrong?
The MySQL documentation has an example:
MYSQL_ROW row;
unsigned int num_fields;
unsigned int i;
num_fields = mysql_num_fields(result);
while ((row = mysql_fetch_row(result)))
{
unsigned long *lengths;
lengths = mysql_fetch_lengths(result);
for(i = 0; i < num_fields; i++)
{
printf("[%.*s] ", (int) lengths[i],
row[i] ? row[i] : "NULL");
}
printf("\n");
}
Note that MYSQL_ROW returned does not necessarily have nul terminated strings (as per a comment, quoted below), so you can't safely call strlen() on the columns - instead you have to call mysql_fetch_lengths().
MYSQL_ROW is an array of null-terminated strings. (However, you cannot treat these as null-terminated strings if field values may contain binary data, because such values may contain null bytes internally.)
The sizeof operator does not return the length of a string, I suppose that row is a pointer, sizeof it is simply the size of a pointer, you can't do what you want the way you tried.
You need the length of each item, so
for (int i = 0; i < 4 ; ++i) {
fwrite(row[i], 1, strlen(row[i]), fp_course);
}
VERY IMPORTANT NOTE
The contents written to the file this way, are useless, because you don't know the length of each string so there is no way to read them back.
So perhaps you are required to do something like
for (int i = 0; i < 4 ; ++i) {
char *string;
int length;
string = row[i];
length = strlen(string);
fwrite(&length, sizeof(length), 1, fp_course);
fwrite(string, 1, length, fp_course);
}
this way you can read the data back.
Can anyone help me figure out how to piece these two pieces of code together so I get the result I need? My eyes are crossing from looking at this. I know this is a breeze for probably everyone other than myself, but I am not a programmer and this is just for one small personal project.
So far, after hours and hours of reading and watching any videos I could find relating to Arduino, Pubnub and sensors, I have sensor reading publishing to Pubnub. I created a Freeboard account for visualization and that's all working. The problem is, the data being published is wrong.
Basically, I'm wanting to read a battery voltage and publish it to PubNub. I can get the Arduino (Uno R3) to read the voltage and I can adjust the values in the code to match the actual voltage. The problem I run into is taking that bit of code that works and stuffing it into the JSON array that gets published to PubNub.
If anyone would be willing to help me and maybe explain a little (or not - I'm okay if I just get it working), I would SO appreciate the time, help and effort.
Thanks!
//Each sketch works indepently. I need to merge them to get the correct reading published.
//VoltagePubNub.ino
(This is the one that publishes, which is what I want. I just want the published value to be the value of the second sketch.)
#include <SPI.h>
#include <Ethernet.h>
#include <PubNub.h>
#include <aJSON.h>
// Some Ethernet shields have a MAC address printed on a sticker on the shield;
// fill in that address here, or choose your own at random:
const static byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
// Memory saving tip: remove myI and dnsI from your sketch if you
// are content to rely on DHCP autoconfiguration.
IPAddress myI(192, 168, 2, 114);
IPAddress dnsI(8, 8, 8, 8);
const static char pubkey[] = "publish_key";
const static char subkey[] = "subscribe_key";
const static char channel[] = "channel_name";
char uuid[] = "UUID";
#define NUM_CHANNELS 1 // How many analog channels do you want to read?
const static uint8_t analog_pins[] = {A0}; // which pins are you reading?
void setup()
{
Serial.begin(9600);
Serial.println("Serial set up");
Ethernet.begin((byte*) mac, myI, dnsI);
Serial.println("Ethernet set up");
delay(1000);
Serial.println("Ethernet set up");
PubNub.begin(pubkey, subkey);
Serial.println("PubNub set up");
delay(5000);
}
void loop()
{
Ethernet.maintain();
EthernetClient *client;
// create JSON objects
aJsonObject *msg, *analogReadings;
msg = aJson.createObject();
aJson.addItemToObject(msg, "analogReadings", analogReadings = aJson.createObject());
// get latest sensor values then add to JSON message
for (int i = 0; i < NUM_CHANNELS; i++) {
String analogChannel = String(analog_pins[i]);
char charBuf[analogChannel.length()+1];
analogChannel.toCharArray(charBuf, analogChannel.length()+1);
int analogValues = analogRead(analog_pins[i]);
aJson.addNumberToObject(analogReadings, charBuf, analogValues);
}
// convert JSON object into char array, then delete JSON object
char *json_String = aJson.print(msg);
aJson.deleteItem(msg);
// publish JSON formatted char array to PubNub
Serial.print("publishing a message: ");
Serial.println(json_String);
client = PubNub.publish(channel, json_String);
if (!client) {
Serial.println("publishing error");
} else
free(json_String);
client->stop();
delay(5000);
}
//VoltageSensor.ino
(This is the one with the correct value, but no publish feature.)
int analogInput = A0;
float vout = 0.0;
float vin = 0.0;
float R1 = 31000.0; //
float R2 = 8700.0; //
int value = 0;
int volt = 0;
void setup(){
pinMode(analogInput, INPUT);
Serial.begin(9600);
Serial.print("DC VOLTMETER");
Serial.println("");
}
void loop(){
// read the value at analog input
value = analogRead(analogInput);
vout = (value * 4.092) / 1024.0;
vin = vout / (R2/(R1+R2));
Serial.print("INPUT V= ");
Serial.println(vin,2);
delay(2000);
}
It may not be the most glamorous or the proper way of doing it, but I got this to do what I need. I edited the first sketch with the following code:
// create JSON objects
aJsonObject *msg, *analogReadings;
msg = aJson.createObject();
aJson.addItemToObject(msg, "analogReadings", analogReadings = aJson.createObject());
// get latest sensor values then add to JSON message
for (int i = 0; i < NUM_CHANNELS; i++) {
float vout = 0.0;
float vin = 0.0;
float R1 = 33060.0; //
float R2 = 7600.0; //
int value = 0;
int volt = 0;
//Serial.print("INPUT V= ");
//Serial.println(vin,2);
String analogChannel = String(analog_pins[i]);
value = analogRead(analog_pins[i]);
vout = (value * 4.092) / 1024.0;
vin = vout / (R2/(R1+R2));
char charBuf[analogChannel.length()+1];
analogChannel.toCharArray(charBuf, analogChannel.length()+1);
float theVoltage = (vin);
int analogValues = analogRead(analog_pins[i]);
aJson.addNumberToObject(analogReadings, charBuf, theVoltage);
}
// convert JSON object into char array, then delete JSON object
char *json_String = aJson.print(msg);
aJson.deleteItem(msg);
Now the value is published to PubNub and is graphed on Freeboard.io at this link .
I have a simple application that (for now) simulates error correction in a large array.
This bit generates the data and adds 16 bytes of Reed-Solomon parity to each block of 255 bytes.
ReedSolomonEncoder encoder = new ReedSolomonEncoder(QR_CODE_FIELD_256);
int[][] data = new int[params.getNumBlocks()][255];
int[][] original = new int[params.getNumBlocks()][];
int value = 0;
for (int i = 0; i < params.getNumBlocks(); i++) {
int[] block = data[i];
for (int j = 0; j < 239; j++) {
value = (value + 1) % 256;
block[j] = value;
}
encoder.encode(block, 16);
original[i] = Arrays.copyOf(block, block.length);
// Corrupt a byte
block[50] += 1;
}
This is my kernel:
public class RsKernel implements Kernel {
private final int[] block;
public RsKernel(int[] block) {
this.block = block;
}
#Override
public void gpuMethod() {
block[50] -= 1;
}
}
it merely manually reverts the corrupted byte in each block (it doesn't do actual Reed-Solomon error-correction).
I run the kernels with the following code:
ArrayList<Kernel> kernels = new ArrayList<>(params.getNumBlocks());
for (int[] block : data) {
kernels.add(new RsKernel(block));
}
new Rootbeer().run(kernels);
And I verify decoding with JUnit's assertArrayEquals:
Assert.assertArrayEquals(original, data);
The curious bit is that if I run this code with up to 8192 (what a suspiciously convenient number) blocks (kernels), the data is reported to have been decoded correctly; for 8193 blocks and above, it is not decoded correctly:
Exception in thread "main" arrays first differed at element [8192][50]; expected:<51> but was:<52>
at org.junit.Assert.internalArrayEquals(Assert.java:437)
at org.junit.Assert.internalArrayEquals(Assert.java:428)
at org.junit.Assert.assertArrayEquals(Assert.java:167)
at org.junit.Assert.assertArrayEquals(Assert.java:184)
at com.amphinicy.blink.rootbeer.RootBeerDemo.main(Jasmin)
What could cause this behaviour?
Here is the output of java -jar rootbeer-1.1.14.jar -printdeviceinfo:
device count: 1
device: GeForce GT 525M
compute_capability: 2.1
total_global_memory: 1073414144 bytes
num_multiprocessors: 2
max_threads_per_multiprocessor: 1536
clock_rate: 1200000 Hz
Looking at the code, I'm thinking it may be because the following:
// Corrupt a byte
block[50] += 1;
Could be adding one to 255, giving 256 which would not be a valid byte. Corrupting the byte might work better with something like this:
block[50] ^= 0x40;
Which would flip the bit in position 7 instead of adding to corrupt the byte.