Booksleeve - Setting multiple Hash Values and retrieve them at once - booksleeve

I'm trying to set multiple fields of a hash independently and retrieve them all at once later on. Is the code below supposed to work or am doing something wrong here? When I run it locally against redis-2.0.0-x64 on Windows downloaded from the Service Stack site, the result of task3 will always just contains field2.
var con = RedisConnectionManager.Instance.GetConnection();
var key = "somekey1";
// set the field value and expiration
var task = con.Hashes.Set(1, key, "field1", token.ToByteArray())
.ContinueWith((x)=> con.Keys.Expire(1, key, 7200));
task.Wait();
var task2 = con.Hashes.Set(1, key, "field2", "fooobar");
task2.Wait();
var task3 = con.Hashes.GetAll(1, key);
task3.Wait();

Cannot reproduce this on a reasonably current version of redis - looks fine here:
using (var con = Config.GetUnsecuredConnection())
{
var key = "somekey1";
// set the field value and expiration
var task = con.Hashes.Set(1, key, "field1", Encoding.UTF8.GetBytes("hello world"))
.ContinueWith((x) => con.Keys.Expire(1, key, 7200));
task.Wait();
var task2 = con.Hashes.Set(1, key, "field2", "fooobar");
task2.Wait();
var task3 = con.Hashes.GetAll(1, key);
task3.Wait();
Assert.AreEqual(2, task3.Result.Count);
Assert.AreEqual("hello world", Encoding.UTF8.GetString(task3.Result["field1"]));
Assert.AreEqual("fooobar", Encoding.UTF8.GetString(task3.Result["field2"]));
}
or, more efficiently (uses pipelining, by not waiting):
using (var con = Config.GetUnsecuredConnection())
{
var key = "somekey1";
// set the field value and expiration
con.Hashes.Set(1, key, "field1", Encoding.UTF8.GetBytes("hello world"));
con.Keys.Expire(1, key, 7200);
con.Hashes.Set(1, key, "field2", "fooobar");
var task = con.Hashes.GetAll(1, key);
con.Wait(task);
Assert.AreEqual(2, task.Result.Count);
Assert.AreEqual("hello world", Encoding.UTF8.GetString(task.Result["field1"]));
Assert.AreEqual("fooobar", Encoding.UTF8.GetString(task.Result["field2"]));
}
So yes: should work, and seems to work. Note I'm using 2.4.5 locally; 2.0 is very old now, and has a number of important bugs. It is entirely possible that there was a bug relating to HSET plus EXPIRE at the server - in which case there isn't much the client library can do to compensate.
I strongly suggest using a more "current" build of redis-server than 2.0. If one isn't available, you might want to try the dmajkic binaries (goes to 2.4.5), or the [MSOpenTech]*https://github.com/MSOpenTech/redis) source. However!!! This is just for local debugging. For production work at the current time (i.e. until the MSOpenTech code is happy-happy), I would recommend using a linux server to host redis. Ubuntu Server works well.

Related

AES encryption on mysql and node.js

I have struggle below question with days, and posted same question earlier and didn't get any positive feedback.
Im using mysql in build aes_encrypt method to encrypt new and existing data.
https://dev.mysql.com/doc/refman/8.0/en/encryption-functions.html
SET ##SESSION.block_encryption_mode = 'aes-256-ecb';
INSERT INTO test_aes_ecb ( column_one, column_two )
values ( aes_encrypt('text','key'), aes_encrypt('text', 'key'));
I used ecb ciper, so It no need to use iv for that. Issue is I can't decrypt it from node.js side.
Im using sequelize and tried to call data through model --> decrypt from node side.
I tried with below libraries,
"aes-ecb": "^1.3.15",
"aes256": "^1.1.0",
"crypto-js": "^4.1.1",
"mysql-aes": "0.0.1",
Below are code snippets from sequelize call
async function testmysqlAESModel () {
const users = await test.findAll();
console.log('users', users[0].column_one);
var decrypt = AES.decrypt( users[0].column_one, 'key' );
}
Its returning buffer data and couldn't decrypt from node side, Can someone provide proper example for that? Im struggling for days.
EDIT
Inserted record to mysql as below query.
SET ##SESSION.block_encryption_mode = 'aes-256-ecb';
INSERT INTO test_aes_ecb ( id, column_one, column_two )
VALUES (1, 2,AES_ENCRYPT('test',UNHEX('gVkYp3s6v9y$B&E)H#McQeThWmZq4t7w')));
In nodejs called like this,
testmysqlAESModel();
async function testmysqlAESModel () {
const users = await test.findAll();
console.log('users', users[0].column_one);
var decipher = crypto.createDecipheriv(algorithm, Buffer.from("gVkYp3s6v9y$B&E)H#McQeThWmZq4t7w", "hex"), "");
var encrypted = Buffer.from(users[0].column_one); // Note that this is what is stored inside your database, so that corresponds to users[0].column_one
var decrypted = decipher.update(encrypted, 'binary', 'utf8');
decrypted += decipher.final('utf8');
console.log(decrypted);
}
Im getting below error,
I used below link to create 256bit key.
https://www.allkeysgenerator.com/Random/Security-Encryption-Key-Generator.aspx
Still couldn't fix, can you provide sample project or any kind of supporting code snippet for that ?
There are multiple issues here:
Ensure that your key has the correct length. AES is specified for certain key length (i.e. 128, 196 and 256 bit). if you use any other key length, then your key will be padded (zero extended) or truncated by the crypto library. This is a non-standard process, and different implementations will do this differently. To avoid this, use a key in the correct length and store it has hex instead of ascii (to avoid charset issues)
Potential issues regarding password to key inference. Some AES implementations use methods to infer keys from passwords/passphrases. Since you are using raw keys in MySQL, you do not want to infer anything but want to use raw keys in NodeJS as well. This means that if you are using the native crypto module, that you want to use createDecipheriv instead of createDecipher.
Caution: The AES mode you are using (ECB) is inherently insecure, because equal input leads to equal output. There are ways around that using other AES modes, such as CBC or GCM. You have been warned.
Example:
MySQL SELECT AES_ENCRYPT('text',UNHEX('F3229A0B371ED2D9441B830D21A390C3')) as test; returns the buffer [145,108,16,83,247,49,165,147,71,115,72,63,152,29,218,246];
Decoding this in Node could look like this:
var crypto = require('crypto');
var algorithm = 'aes-128-ecb';
var decipher = crypto.createDecipheriv(algorithm, Buffer.from("F3229A0B371ED2D9441B830D21A390C3", "hex"), "");
var encrypted = Buffer.from([145,108,16,83,247,49,165,147,71,115,72,63,152,29,218,246]); // Note that this is what is stored inside your database, so that corresponds to users[0].column_one
var decrypted = decipher.update(encrypted, 'binary', 'utf8');
decrypted += decipher.final('utf8');
console.log(decrypted);
This prints text again.
Note that F3229A0B371ED2D9441B830D21A390C3 is the key in this example, you would obviously have to create your own. Just ensure that your key has the same length as the example, and is a valid hex string.

Couchbase: MetaData.Metrics always contain default values?

I am testing Couchbase, and I am making a very simply query:
public async Task SelectRandomJobs(int nbr)
{
IBucket bucket = await cluster.BucketAsync("myBucket");
IScope scope = bucket.Scope("myScope");
IQueryResult<JObject> result = await scope.QueryAsync<JObject>("SELECT * FROM myCollection WHERE Id = {id}");
// The Metrics.* has default values
Console.WriteLine(result.MetaData.Metrics.ElaspedTime);
}
Here are the values:
I was expecting ElaspedTime (misspelled!) and ExecutionTime to be not null. There is a AnalyticsQueryAsync method, but that did work for me (error 24045).
Why are those values null?
-- UPDATE --
I followed the advice of Eric, but I got the same results:
So you will need to enable Metrics for this query, I have provided a code sample below with two possible ways of doing this, it is covered in our docs but maybe could be easier to find or have better examples, this is something I will investigate further and see if we can make it clearer in future editions of the docs.
I have used the travel-sample dataset and tried to set the code up similar to your example so that it will be easy to implement for you.
As for why the times are null by default and the other fields are zero, that seems to just be a design decision for this class.
About the misspelling, we have filed a ticket to get the spelling corrected. Thank you for pointing that out.
using System;
using System.Threading.Tasks;
using Couchbase;
using Couchbase.Query;
namespace _3x_simple
{
class Program
{
static async Task Main(string[] args)
{
var cluster = await Cluster.ConnectAsync("couchbase://localhost", "Administrator", "password");
var bucket = await cluster.BucketAsync("travel-sample");
var myScope = bucket.Scope("inventory");
//scope path
var options = new QueryOptions().Metrics(true);
var queryResult = await myScope.QueryAsync<dynamic>("SELECT * FROM airline LIMIT 10;", options);
//cluster path
//var queryResult = await cluster.QueryAsync<dynamic>("SELECT * FROM `travel-sample`.inventory.airline LIMIT 10;", options => options.Metrics(true));
Console.WriteLine($"Execution time before read: {queryResult.MetaData.Metrics.ExecutionTime}");
await foreach(var row in queryResult){
Console.WriteLine(row);
}
Console.WriteLine($"Execution time after read: {queryResult.MetaData.Metrics.ExecutionTime}");
Console.WriteLine("Press any key to exit...");
Console.Read();
}
}
}
You won't see the execution time until after the results are read. The reason you are seeing default values for those fields is because you are trying to read that information at the wrong time/place considering your async operation.

Username in WebTokenRequestResult is empty

in a Windows 10 UWP I try use WebAuthenticationCoreManager.RequestTokenAsync to get the result from a login with a Microsoft account.
I get a WebTokenRequestResult with Success. ResponseData[0] contains a WebAccount with an ID - but the UserName is empty.
The scope of the call is wl.basic - so I should get a lot of information...
I'm not sure how to retrieve extra information - and for the current test the Username would be OK.
I checked out the universal samples - and there I found a snippet which tries to do what I'm trying - an output of webTokenRequestResult.ResponseData[0].WebAccount.UserName.
By the way - the example output is also empty.
Is this a bug - or what do I (and the MS in the samples) have to do to get the users profile data (or at least the Username)?
According to the documentation (https://learn.microsoft.com/en-us/windows/uwp/security/web-account-manager), you have to make a specific REST API call to retrieve it:
var restApi = new Uri(#"https://apis.live.net/v5.0/me?access_token=" + result.ResponseData[0].Token);
using (var client = new HttpClient())
{
var infoResult = await client.GetAsync(restApi);
string content = await infoResult.Content.ReadAsStringAsync();
var jsonObject = JsonObject.Parse(content);
string id = jsonObject["id"].GetString();
string name = jsonObject["name"].GetString();
}
As to why the WebAccount property doesn't get set... shrugs
And FYI, the "id" returned here is entirely different from the WebAccount.Id property returned with the authentication request.

Getting identity column value using entity framework 4.1

I'm using entity framework 4.1 (VS 2010, SQL Server 2012) for inserting data into a database.
First I create an instance of an object, fill the properties with values and call AddObject(), like this:
VideoData videodata = new VideoData();
videodata.StartCaptureTime = startCaptureTime;
videodata.EndCaptureTime = endCaptureTime;
videodata.CameraID = CameraID;
using (var context = new PercEntities())
{
if (context.VideoDatas.Where(c => c.VideoID == videoID).Count() == 0)
{
var videoData = new VideoData
{
StartCaptureTime = startCaptureTime,
EndCaptureTime = endCaptureTime,
CameraID = CameraID,
};
context.VideoDatas.AddObject(videoData);
context.SaveChanges();
}
}
The thing is, that the table in the database has an identity column:
VideoID int IDENTITY(1,1)
and I need to get the value inserted by the identity function in order to fill additional objects, that have the VideoID as a foreign key. for example:
FrameData frameData = new FrameData();
frameData.VideoID = videodata.VideoID;
frameData.Path = path;
The only thing I could think of was to query for the max identity right after AddObject(videoData), but I'm afraid of race conditions.
I'm new to Entity Framework, so I'd be happy for any guidance on this.
If you have other objects which require VideoID as FK you just need to correctly configure your navigation properties between VideoData and those other types and EF will handle it for you.
Call to AddObject does not insert your data to database and because of that you cannot get the identity value after this call. Only call to SaveChanges will push all your changes to database and during this call EF will handle referential integrity internally (but only if you have your model correctly configured with relations).
After calling SaveChanges your VideoID should be populated automatically if you have everything correctly configured.

Do MERGE using Linq to SQL

SQL Server 2008 Ent
ASP.NET MVC 2.0
Linq-to-SQL
I am building a gaming site, that tracks when a particular player (toon) had downed a particular monster (boss). Table looks something like:
int ToonId
int BossId
datetime LastKillTime
I use a 3d party service that gives me back latest information (toon,boss,time).
Now I want to update my database with that new information.
Brute force approach is to do line-by-line upsert. But It looks ugly (code-wise), and probably slow too.
I think better solution would be to insert new data (using temp table?) and then run MERGE statement.
Is it good idea? I know temp tables are "better-to-avoid". Should I create a permanent "temp" table just for this operation?
Or should I just read entire current set (100 rows at most), do merge and put it back from within application?
Any pointers/suggestions are always appreciated.
An ORM is the wrong tool for performing batch operations, and Linq-to-SQL is no exception. In this case I think you have picked the right solution: Store all entries in a temporary table quickly, then do the UPSERT using merge.
The fastest way to store the data to the temporary table is to use SqlBulkCopy to store all data to a table of your choice.
If you're using Linq-to-SQL, upserts aren't that ugly..
foreach (var line in linesFromService) {
var kill = db.Kills.FirstOrDefault(t=>t.ToonId==line.ToonId && t.BossId==line.BossId);
if (kill == null) {
kill = new Kills() { ToonId = line.ToonId, BossId = line.BossId };
db.Kills.InsertOnSubmit(kill);
}
kill.LastKillTime = line.LastKillTime;
}
db.SubmitChanges();
Not a work of art, but nicer than in SQL. Also, with only 100 rows, I wouldn't be too concerned about performance.
Looks like a straight-forward insert.
private ToonModel _db = new ToonModel();
Toon t = new Toon();
t.ToonId = 1;
t.BossId = 2;
t.LastKillTime = DateTime.Now();
_db.Toons.InsertOnSubmit(t);
_db.SubmitChanges();
To update without querying the records first, you can do the following. It will still hit the db once to check if record exists but will not pull the record:
var blob = new Blob { Id = "some id", Value = "some value" }; // Id is primary key (PK)
if (dbContext.Blobs.Contains(blob)) // if blob exists by PK then update
{
// This will update all columns that are not set in 'original' object. For
// this to work, Blob has to have UpdateCheck=Never for all properties except
// for primary keys. This will update the record without querying it first.
dbContext.Blobs.Attach(blob, original: new Blob { Id = blob.Id });
}
else // insert
{
dbContext.Blobs.InsertOnSubmit(blob);
}
dbContext.Blobs.SubmitChanges();
See here for an extension method for this.