I wrote a program to use chrome's login cookies to do something automatically, but since Chrome encrypt all the cookies at January, my program can't work anymore.
I'm trying to decrypt cookies, and success in java on mac os by This Topic, but my usual running environment is win7 os, so I have to decrypt that on windows.
I found os_crypt_win.cc in Chromium's source code, it has a encrypt part:
bool OSCrypt::EncryptString(const std::string& plaintext, std::string* ciphertext) {
DATA_BLOB input;
input.pbData = const_cast<BYTE*>(reinterpret_cast<const BYTE*>(plaintext.data()));
input.cbData = static_cast<DWORD>(plaintext.length());
DATA_BLOB output;
BOOL result = CryptProtectData(&input, L"", NULL, NULL, NULL, 0, &output);
if (!result)
return false;
// this does a copy
ciphertext->assign(reinterpret_cast<std::string::value_type*>(output.pbData), output.cbData);
LocalFree(output.pbData);
return true;
}
I imitate this part in java with JNA:
String encrypted = bytesToHex(Crypt32Util.cryptProtectData(Native.toByteArray(plaintext), 0));
or
String encrypted = bytesToHex(Crypt32Util.cryptProtectData(plaintext.getBytes());
or
String encrypted = bytesToHex(Crypt32Util.cryptProtectData(plaintext.getBytes("UTF-8"));
or
String encrypted = bytesToHex(Crypt32Util.cryptProtectData(plaintext.getBytes("UTF-16"));
But I got a wrong encrypted values different with the value store in Chrome.
Did I used a wrong method to encrypt this, or did I miss something important?
Can you help me figure this out?
You used the correct method to encrypt the values.
How are the values "wrong"? if they are just different from the one's stored in chrome that is not a problem.
The reason for that is very simple:
from msdn:
"The function creates a session key to perform the encryption. The
session key is derived again when the data is to be decrypted."
from msdn blog:
"A random session key is created for each call to CryptProtectData.
This key is derived from the master key, some random data, and some
optional entropy passed in by the user. The session key is then used
to do the actual encryption."
The important thing you should check is whether you are able to decrypt the values using DecryptUnprotectData.
Related
Moving from PTK to Luna and now in code, I need to pass cbc_param, before the inside object was the only mechanism.
Can somebody explain what is this about?
How PTK manage cbc_param?
What is the difference with and without cbc_param?
var cbc_param = pkcs11.C_GenerateRandom(new Buffer(16));
pkcs11.C_EncryptInit(
session,
{
mechanism: pkcs11js.CKM_AES_CBC,
parameter: cbc_param
},
secretKey
);
According to the PKCS11 documentation CBC mode has a 16-byte initialization vector (IV) parameter. This parameter is mandatory, basically it is 16 random bytes that you need use to encrypt/decrypt in CBC mode. It's ok to save it with cipher text, IV don't have to be a secret, but it must be random.
(Possibly a duplicate of Can't send a keyedMessage to brokers with partitioner.class=kafka.producer.DefaultPartitioner, although the OP of that question didn't mention kafka-python. And anyway, it never got an answer.)
I have a Python program that has been successfully (for many months) sending messages to the Kafka broker, using essentially the following logic:
producer = kafka.KafkaProducer(bootstrap_servers=[some_addr],
retries=3)
...
msg = json.dumps(some_message)
res = producer.send(some_topic, value=msg)
Recently, I tried to upgrade it to send messages to different partitions based on a definite key value extracted from the message:
producer = kafka.KafkaProducer(bootstrap_servers=[some_addr],
key_serializer=str.encode,
retries=3)
...
try:
key = some_message[0]
except:
key = None
msg = json.dumps(some_message)
res = producer.send(some_topic, value=msg, key=key)
However, with this code, no messages ever make it out of the program to the broker. I've verified that the key value extracted from some_message is always a valid string. Presumably I don't need to define my own partitioner, since, according to the documentation:
The default partitioner implementation hashes each non-None key using the same murmur2 algorithm as the java client so that messages with the same key are assigned to the same partition.
Furthermore, with the new code, when I try to determine what happened to my send by calling res.get (to obtain a kafka.FutureRecordMetadata), that call throws a TypeError exception with the message descriptor 'encode' requires a 'str' object but received a 'unicode'.
(As a side question, I'm not exactly sure what I'd do with the FutureRecordMetadata if I were actually able to get it. Based on the kafka-python source code, I assume I'd want to call either its succeeded or its failed method, but the documentation is silent on the point. The documentation does say that the return value of send "resolves to" RecordMetadata, but I haven't been able to figure out, from either the documentation or the code, what "resolves to" means in this context.)
Anyway: I can't be the only person using kafka-python 1.3.3 who's ever tried to send messages with a partitioning key, and I have not seen anything on teh Intertubes describing a similar problem (except for the SO question I referenced at the top of this post).
I'm certainly willing to believe that I'm doing something wrong, but I have no idea what that might be. Is there some additional parameter I need to supply to the KafkaProducer constructor?
The fundamental problem turned out to be that my key value was a unicode, even though I was quite convinced that it was a str. Hence the selection of str.encode for my key_serializer was inappropriate, and was what led to the exception from res.get. Omitting the key_serializer and calling key.encode('utf-8') was enough to get my messages published, and partitioned as expected.
A large contributor to the obscurity of this problem (for me) was that the kafka-python 1.3.3 documentation does not go into any detail on what a FutureRecordMetadata really is, nor what one should expect in the way of exceptions its get method can raise. The sole usage example in the documentation:
# Asynchronous by default
future = producer.send('my-topic', b'raw_bytes')
# Block for 'synchronous' sends
try:
record_metadata = future.get(timeout=10)
except KafkaError:
# Decide what to do if produce request failed...
log.exception()
pass
suggests that the only kind of exception it will raise is KafkaError, which is not true. In fact, get can and will (re-)raise any exception that the asynchronous publishing mechanism encountered in trying to get the message out the door.
I also faced the same error. Once I added json.dumps while sending the key, it worked.
producer.send(topic="first_topic", key=json.dumps(key)
.encode('utf-8'), value=json.dumps(msg)
.encode('utf-8'))
.add_callback(on_send_success).add_errback(on_send_error)
I have a very specific requirement where some columns need to be encrypted using aes_encrypt / aes_decrypt. We need to encrypt the information at SQL level using a eas so it can be read using another app or directly from MySQL using a query and aes_encrypt / aes_decrypt.
Our app was developed using CakePHP 3 and database is MySQL 5.6.25.
I found and carefully follow the instruction on this selected answer: Encyption/Decryption of Form Fields in CakePHP 3
Now the data is being saved encrypted on the database... the problem is that we still need to be able to use aes_decrypt on MySQL to decrypt the information and it's returning NULL.
On CakePHP 3, config/app.php:
'Security' => ['salt' => '1234567890']
Then encrypted using:
Security::encrypt($value, Security::salt());
Data is saved on MySQL but aes_decrypt() returns NULL
SELECT AES_DECRIPT(address_enc, '1234567890') FROM address;
How can I setup CakePHP 3 to correctly encrypt information so I can later decrypt it on MySQL using aes_decrypt() ?
[EDIT]
My MYSQL table:
CREATE TABLE IF NOT EXISTS `address` (
`id` int(11) NOT NULL,
`address` varchar(255) DEFAULT NULL,
`address_enc` blob,
`comment` varchar(255) DEFAULT NULL,
`comment_enc` blob
) ENGINE=MyISAM AUTO_INCREMENT=2 DEFAULT CHARSET=utf8;
Note: address and comment are just for testings.
Then, on CakePHP, I created a custom database type:
src/Database/Type/CryptedType.php
<?php
namespace App\Database\Type;
use Cake\Database\Driver;
use Cake\Database\Type;
use Cake\Utility\Security;
class CryptedType extends Type
{
public function toDatabase($value, Driver $driver)
{
return Security::encrypt($value, Security::salt());
}
public function toPHP($value, Driver $driver)
{
if ($value === null) {
return null;
}
return Security::decrypt($value, Security::salt());
}
}
src/config/bootstrap.php
Register the custom type.
use Cake\Database\Type;
Type::map('crypted', 'App\Database\Type\CryptedType');
src/Model/Table/AddressTable.php
Finally map the cryptable columns to the registered type, and that's it, from now on everything's being handled automatically.
use Cake\Database\Schema\Table as Schema;
class AddressTable extends Table
{
// ...
protected function _initializeSchema(Schema $table)
{
$table->columnType('address_enc', 'crypted');
$table->columnType('comment_enc', 'crypted');
return $table;
}
// ...
}
Do you really need to do that?
I'm not going to argue about the pros and cons of storing encrypted data in databases, but whether trying to decrypt on SQL level is a good idea, is a question that should be asked.
So ask yourself whether you really need to do that, maybe it would be better to implement the decryption at application level instead, it would probably make things easier with regards to replicating exactly what Security::decrypt() does, which is not only decrypting, but also integrity checking.
Just take a look at what Security::decrypt() does internally.
https://github.com/cakephp/cakephp/blob/3.1.7/src/Utility/Security.php#L201
https://github.com/cakephp/cakephp/blob/3.1.7/src/Utility/Crypto/OpenSsl.php#L77
https://github.com/cakephp/cakephp/blob/3.1.7/src/Utility/Crypto/Mcrypt.php#L89
It should be pretty easy to re-implement that in your other application.
Watch out, you may be about to burn your fingers!
I am by no means an encryption expert, so consider the following as just a basic example to get things started, and inform yourself about possible conceptual, and security related problems in particular!
Handling encryption/decryption of data without knowing exactly what you are doing, is a very bad idea - I can't stress that enough!
Decrypting data at SQL level
That being said, using the example code from my awful (sic) answer that you've linked to, ie using Security::encrypt(), and Security::salt() as the encryption key, will by default leave you with a value that has been encrypted in AES-256-CBC mode, using an encryption key derived from the salt concatenated with itself (first 32 bytes of its SHA256 representation).
But that's not all, additionally the encrypted value gets an HMAC hash, and the initialization vector pepended, so that you do not end up with "plain" encrypted data that you could directly pass to AES_DECRYPT().
So if you'd wanted to decrypt this on MySQL level (for whatever reason), then you'd first of all have to set the proper block encryption mode
SET block_encryption_mode = 'aes-256-cbc';
sparse out the HMAC hash (first 64 bytes) and the initialization vector (following 16 bytes)
SUBSTRING(`column` FROM 81)
and use the first 32 bytes of hash('sha256', Security::salt() . Security::salt()) as the encryption key, and the initialization vector from the encrypted value for decryption
SUBSTRING(`column`, 65, 16)
So in the end you'd be left with something like
SET block_encryption_mode = 'aes-256-cbc';
SELECT
AES_DECRYPT(
SUBSTRING(`column` FROM 81), -- the actual encryted data
'the-encryption-key-goes-here',
SUBSTRING(`column`, 65, 16) -- the intialization vector
)
FROM table;
Finally you maybe also want to cast the value (CAST(AES_DECRYPT(...) AS CHAR)), and remove possible zero padding (not sure whether AES_DECRYPT() does that automatically).
Data integrity checks
It should be noted that the HMAC hash that is prepended to the encrypted value, has a specific purpose, it is used to ensure integrity, so by just dropping it, you'll lose that. In order to keep it, you'd have to implement a (timing attack safe) HMAC256 generation/comparison on SQL level too. This leads us back to the intial question, do you really need to decrypt on SQL level?
[Solution] The solution for this particular requirement (we need to encrypt the information at SQL level using a eas so it can be read using another app or directly from MySQL using a query and aes_encrypt / aes_decryp) was to create a custom database type in CakePHP them, instead of using CakePHP encryption method, we implemented PHP Mcrypt.
Now the information is saved to the database from our CakePHP 3 app and the data be read at MySQL/phpMyAdmin level using eas_decrypt and aes_encrypt.
FOR ANYONE STRUGGLING TO DECRYPT WITH MYSQL: This generally applies to anyone using symmetric AES encryption/decryption - specifically when trying to decrypt with AES_DECRYPT.
For instance, if you are using aes-128-ecb, and your encrypted data is 16 bytes long with no padding, you need to add padding bytes to your encrypted data before trying to decrypt (because mySQL is expecting PKCS7 padding). Because MySQL uses PKCS7, you need to add 16 more bytes, in this case those pad bytes are 0x10101010101010101010101010101010. We take the left 16 bytes because when we encrypt the 0x10101010101010101010101010101010, we get 32 bytes, and we only need the first 16.
aes_decrypt(concat(<ENCRYPTED_BYTES>, left(aes_encrypt(<PAD BYTES>, <KEY>), 16)), <KEY>)
I am implementing Windows MDM. In this, Device sends CSR which is PKCS#10 Certificate request.
When I go to http://certlogik.com/decoder, and decode this CSR,I get Subject as
"CN=B1C43CD0-1624-5FBB-8E54-34CF17DFD3A1\00"
This "\00", we want to remove from subject property.Because of this we are not able to install company hub app during enrolment.
I want to change this subject to any value such as "CN=myMDM".How can I change Subject property of CSR?
It is permissible to have a certificate subject differ from the subject of the PKCS#10 (i.e. CSR). See the -subj option to OpenSSL's 'req' command.
For reasons behind this ability, consider this; Your CSR is signed by your private key. The signature is used to verify that the contents of the CSR have not been modified (this includes the subject). Your public key is included in the CSR. A CA creates your cert and uses whatever parts of the CSR subject it sees fit. The cert, along with the CA-specified subject and the public key from your CSR is signed by the private key of the CA. This signature is used to verify that the contents of your cert have not been modified (this includes the CA-specified subject).
You cannot change it once you have the certificate request, as CSR's should be signed with your private key, and the signing is definitely including the subject line.
So if you change anything you need to resign, which requires you to parse the CSR. So basically you should rebuild your CSR and simply strip off the character with value 00 (null terminator character) when you supply your common name (CN).
You are left with the following options:
correctly generate the CSR, as explained above,
change the CSR, removing the zero valued byte and recalculating all the lengths, and skip verification,
create a special certificate creator that puts the correct subject like out of the CSR after verifying or skipping the signature verification of the CSR;
I am using Linq to SQL as my DAL layer and during unit test I found out that my objects are not being returned from database but from the DataContext cache.
The strange thing is that when the objects are returned from the cache why does it require a separate call to the database to fetch all the fields.
Anyway, I implemented a ClearCache method that will clear out the cache. But I am only clearing the cache in the unit test and not in the API code.
The reason is that once the object is inserted it is good to load from the cache then to fetch it again from the database.
What do you think?
UPDATE:
public static void ClearCache(this EStudyModelDataContext context)
{
const BindingFlags Flags = BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic;
var method = context.GetType().GetMethod("ClearCache", Flags);
method.Invoke(context, null);
}
in my case work only with 4 bindings:
DATACONTEXT.GetType().InvokeMember(
"ClearCache",
BindingFlags.Instance |
BindingFlags.Public |
BindingFlags.NonPublic |
BindingFlags.InvokeMethod,
null, DATACONTEXT, null);
You are correct that L2S will return items from the cache when it can. I learned about this the hard way. A clean way to handle this is to do a context.RefreshRow after each Insert and Update operation. This refreshes the cache and guarantees the cache is current.
There is one big caveat to manually calling 'ClearCache' - it is that you're doing exactly that! So - if you had any prior database operations (pending), you just flushed them. You have to be very careful that you haven't walked on other "On Submit" changes. I learned this the hard way - so I had to add a few more 'SubmitChanges' statements into my code - as a direct result of implementing ClearCache after some stored procedure executions (bulk deletes).