How can I verify if a firmware (.bin file) is signed or not , if there is a tool or script that can be used as much I understand sigcheck does not work with firmwares, any help on this ?
A bin file is generally just a binary file. There is no such thing as a (single) standard for firmware files. Undoubtedly the file internally has structure but for us it is impossible to tell what it is. Generally a tool only works for a specific protocol.
RSA signatures themselves are the result of modular exponentiation with the public exponent. The result is then converted to a byte array that is almost indistinguishable from random. The size has the size of the modulus, which is the same size as the key size, in bytes. RSA may be used in various formats such as RSA with PKCS#1 v1.5 padding or RSA with PSS padding; the output however looks the same. Other signature types exist, e.g. ECDSA signatures that are completely different from RSA. They have a different (smaller) size and can be both dynamically and statically sized. ECDSA makes a lot of sense if the signature needs to be checked on resource constrained hardware.
Generally the signature is calculated as the very last step, so often the last bytes of the .bin file will be the signature. If you're lucky the signature is performed using a standard, e.g. PGP compliant signatures or CMS compliant signatures. For CMS signatures you may check if the file contains so called ASN.1 structures. These are encoded using something called BER, which can be parsed (e.g. using openssl asn1parse -inform BER).
So it all depends on the protocol. There is no single tool that can check if something is signed or not, the protocols are too different and signatures themselves just look like random data.
Related
I understand that RSA keys can be generated using different sha algorithms. Using openssl, I don't seem to have the option of specifying what algorithm the key generator should use. I suspect it's using sha256.
How can I generate RSA keys using different sha algorithms (such as sha512) in either a bash shell or in Ruby? Does the openssl library support generating RSA keys using different algorithms? If not, does anyone know of another library I can use? (In ruby, OpenSSL::PKey::RSA doesn't seem to allow for choosing an algorithm, but the documentation is hard for me to follow soo...?)
Apologies if this question has already been answered, but I haven't been able to find an answer.
Maybe I should also note (in case I am wrong): it is my understanding that choosing a size for the generated RSA key (i.e. RSA 2048) is separate from choosing the hashing algorithm (i.e. sha512).
UPDATE - Some background
I want to sign Java Web Tokens with an RSA key. The JWT library I'm using gives me the impression that RSA keys can be generated using different hashing algorithms (RS256, RS384, RS512). Generating a key using openssl doesn't seem to let me choose what hashing algorithm is used though.
Thanks!!
RSA keys, and "the RSA algorithm" don't have any notion of a hash algorithm.
An RSA key is just two prime numbers and one other number (from the (p, q, e) triplet all the other values can be derived). e is usually chosen as 0x010001 (though other reasonable values exist) and p and q are generated randomly (while almost any CSPRNG is going to have a backing hash algorithm the CSPRNG itself is usually considered a black box that just emits randomness).
Where a hash algorithm comes into play is in RSA Signatures.
For an RSA Signature the original data is hashed under an algorithm and then the hash value, algorithm identifier, and private key are used to produce a signature (for PKCS v1.5 signatures... for PSS there's also a second (effectively fixed) identifier and some more random bytes).
RS256 is the JWA (JSON Web Algorithms) identifier for "RSASSA-PKCS1-v1_5 using SHA(-2)-256".
JWA section 3.3 says
This section defines the use of the RSASSA-PKCS1-v1_5 digital
signature algorithm as defined in Section 8.2 of RFC 3447 [RFC3447]
(commonly known as PKCS #1), using SHA-2 [SHS] hash functions.
A key of size 2048 bits or larger MUST be used with these algorithms.
The RSASSA-PKCS1-v1_5 SHA-256 digital signature is generated as
follows: generate a digital signature of the JWS Signing Input using
RSASSA-PKCS1-v1_5-SIGN and the SHA-256 hash function with the desired
private key. This is the JWS Signature value.
(emphasis mine)
So no requirement is made on the RSA key, other than that the spec was written in 2015 so they mandated a 2015-compatible minimum keysize.
I was tasked to write image upload to remote server and save those images locally. It was quite easy to do it with Base64 transfer through JSON and storing with Node.js. However, is there a reason to not use this type of file upload, to use AJAX or other ways? (Other than the 30% bandwidth increase, which I know of. You can still include that in your answer in order for it to be full).
The idea of base64 encoding is to avoid binary data for protocols based on text. Outside this situation, it's I think always a bad idea.
Pros
Avoidance of binary data for protocols based on text, and independance from external files.
Avoidance of delimiter collision.
Cons
Time and space increased complexity; for space it's 33–36% (33% by the encoding itself, up to 3% more by the inserted line breaks).
API response payloads are larger/too large.
User Experience is negatively impacted, unless one invoke some lazy loading.
By including all image data together in one API response, the app
must receive all data before drawing anything on screen. This means
users will see on-screen loading states for longer and the app will
appear sluggish as users wait.
This is however mitigated with Axios and some lazy loader such as react-lazyload or lazyload or so.
CDN caching is harder. Contrary to image files, the Base64 strings inside an API response cannot be delivered via a CDN cache. The whole API response must be delivered by CDN. (cf., Don’t use Base64 encoded images on mobile and Why "optimizing" your images with Base64 is almost always a bad idea)
Image caching on the device is no longer possible.
Content management becomes harder on server side. Most content management tools handle images as binary files. But then when managing in binary, there is the time overhead of encoding/decoding.
No security gain and overhead in engineering to mitigate (Sanitizing, Input Validation, Escaping). Example of XSS attack: Preventing XSS with Base64 encoding: The False sense of web application security
The developers of that site might have opted to make the website appear more secure by having cryptic URLs and whatnot. However, that
doesn't mean this is security by obscurity.
If their website is vulnerable to SQL injection and they try to hide that by encoding the URLs, then it's security by obscurity. If their website is well secured against SQL injection; XSS; CSRF; etc., and they deiced to encode the URLs like that, then it's just plain stupidity.
It does not help with text encoded images such as svg (Probably Don’t Base64 SVG)
Data URIs aren't supported on IE6 or IE7, nor on Opera before 7.2 (Which browsers support data URIs and since which version?)
References
https://en.wikipedia.org/wiki/Base64
https://en.wikipedia.org/wiki/Delimiter#Delimiter_collision
SO: What is base 64 encoding used for?
https://medium.com/snapp-mobile/dont-use-base64-encoded-images-on-mobile-13ddeac89d7c
https://css-tricks.com/probably-dont-base64-svg/
https://security.stackexchange.com/questions/46362/purpose-of-using-base64-encoded-urls
https://bunnycdn.com/blog/why-optimizing-your-images-with-base64-is-almost-always-a-bad-idea/
https://www.davidbcalhoun.com/2011/when-to-base64-encode-images-and-when-not-to/
Data Encoding
Every data Encoding and Decoding can be used duo various reasons, which came up with benefits and downsides.
like:
Error-detection encodings : which can detect errors but increase data usage.
Encryption encodings: turns data to cipher which intruder wont decipher.
There are a lot of Encoding Algorithms which Alter Data in
Which has some usefullness to do that.
but with
Base64 Encoding, its encode every 6-bit data into one character (8-bit) .
3 Byte to 4 Byte but it only includes alphanumeric(62 distinc) and 2 signs.
its benefits is it Dose not have special chars and signs
Base64 Purpose
it make possible to transfer Any Data with Channels Which Prohibits us to have:
special chars like ' " / \ ...
non-printable Ascii like \0 \n \r \t \a
8-bit Ascii codes (ascii with 1 MSB )
binary files usually includes any data which if turns in ascii can be any 8-bit character.
in some protocols and application there are I/O Interfaces Which Does only accepts a handful of chars (alphanumeric with a few of signs).
duo to:
prevent to code injection (ex: SQL injection or any prgramming-language-syntax-like characters ; )
or just some character has already has a meaning in their protocol (ex: in URI QueryString character & has a meaning and cannot be in any QueryString Value)
or maybe the input is not intended to accept non-alphanumerical values. (ex: it should accept only Human Names)
but with base64 encoding you can encode anything and transfer it with
any channel you want.
Example:
you can encode an image or application and save it in DBMS with SQL
you can include some binary data in URI
you can send binary files in a protocol which has been designed to accepts only human chats as alphanumerical like IRC Channel
Base64 is a just a converting format that HTTP server cannot accept binary data in the contents except the HTTP Header type is binary or acceptable format defined by web-server.
As you might know, JSON can contain various formats and information; thus, you can contain such as
{
IMG_FILENAME="HELLO",
IMG_TYPE="IMG/JPEG",
DATA="~~~BASE64 ENCODED IMAGE~~~~"
}
You can send JSON file through AJAX or other method. But, as I told you, HTTP server have various limitation because it should keep RFC2616 (https://www.rfc-editor.org/rfc/rfc2616).
In short, Sending Through JSON can contain various data.
AJAX is just a type of sending as other ways does.
I used same solution in one of my project.
The only concern is the request body size. If all your images are small, like a few M, then you should be fine.
My server is asp.net core, its maxAllowedContentLength value is 30000000, which is approximately 28.6MB. When the image size is over this, the request failed with error "request body too large".
I think node.js should have similar setting, make sure to adjust it to meet your need.
Please note that when the request size is too big, the possibility of request timeout increases accordingly due to the network traffic. This will be an issue especially for the requests from phones.
I think the use of base64 is valid.
The only doubt is the size of the request, but this can be circumvented if you divide this base64 in the frontend, if a 30mb file you could divide each request into 5mb and in the backend put the parts together, this is useful even to do the "keep downloading" "when you have a problem with the network and corrupt some part.
Hugs
Base64 converts your data to an ASCII representation of the binary data. It allows you to embed your data in text streams such as JSON for example. Base64 increases the size of the data transferred by 33%.
multipart/form-data is the standard way of transferring binary data in HTTP requests. It allows you to use specific encodings / content types for each part you'd like to transfer. In my opinion, you should stick to multipart uploads unless you have specific requirements or device/SDK capabilities.
Checkout these links
What is difference Between Base64 and Multipart?
Base64 image upload VS Binary image upload?
The Mainline DHT, used in BitTorrent to distribute lists of peers, implements a custom RPC protocol called KRPC. KRPC consists of BEncoded dictionaries, which are essentially a more compact form of JSON.
Is there any benefit of using BEncode over something like BSON (or even just compressing the data)?
I suspect that bencode is being used for historical reasons and to lighten the burden on developers wanting to implement the DHT extension. Since all BitTorrent clients must have a working bencode implementation to work with torrent files (which are simply bencoded dictionaries of metadata), implementing the DHT with bencode would require no new project dependencies.
Also, consider that the DHT is using bencode along with binary encoding. Among others, the 26 byte node identifier string (20 bytes for the node-id, 6 bytes for the ipv4 address / port) is being stored as a binary string (see: http://bittorrent.org/beps/bep_0005.html#contact-encoding), so in-effect, there is already some minimal data-compression that is happening.
There are many libraries and utilities for trying to guess a file's MIME type from content, rather than simply using file name extensions. While some libraries simply take an open file handle, others expect to be passed actual data.
When passing data to a library, how many bytes are required to obtain an accurate MIME type?
Alternatively, at what point are you passing in extra information for no gain in reliability?
You could refer this File Signature Table
It describes a lot of file types.
It is difficult to say how many bytes needed to be read to detect file type because of some files have static signatures at 512 bytes offset (like PDB), and other in trailer (like TGA).
The most of files listed in the table have signature no more than 24 bytes.
It can vary. Wikipedia suggests 2-4 bytes
http://en.wikipedia.org/wiki/File_signature
Be then later on
http://en.wikipedia.org/wiki/List_of_file_signatures
points that file signatures can be very can be very long. Then you have to deal with issue of some files like text don't have signature.
I have a encryption scheme implemented, the constituent components: The symetric cypher and its chaining mode, and the HMAC algorithm are hard-coded into the binary. Additionally, the parameters of the algorithms (HMAC key, symetric-key symetric IVEC) are specified in binary files, one for each parameter.
I would like to specify the choice of algorithms, and modes, and their parameters in a single file. Do I need my own format, or is this possible using existing OpenSSL infrastructure ? If there is infrastructure, could someone please provide some references.
p.s., I know of the config file parsing code, and the PEM/x.509 code in OpenSSL. However anything built from this won't be cohesive.
I would like to specify the choice of algorithms, and modes, and their
parameters in a single file
That type of agility sounds like you will allow the user to make a choice. Don't do it, since they might pick a bad cipher (or cipher combination). Make good choices for them.
The symetric cypher and its chaining mode, and the HMAC algorithm are
hard-coded into the binary.
I would first look into an authenticated encryption mode - EAX, CCM, or GCM. I believe OpenSSL only has CCM at the moment (or is it GCM?). If you can't use an authenticated encrpytion mode, move on to Encrypt-Then-Authenticate (ie, encrypt then authnticate the cipher text with a HMAC or CMAC), which it sounds like you are doing.
Additionally, the parameters of the algorithms (HMAC key, symetric-key
symetric IVEC)
HMAC (and CMAC) are good. Don't use a CBC-MAC since it suffers from weaknesses on variable length messages.
symetric-key [in binary file]
Hmmm...
symmetric IVEC
IVs are considered public parameters. Pick a random IV for the message, and send it along with the cipher text. Make sure to MAC both the cipher text and IV to detect tampering.
Do I need my own format, or is this possible using existing OpenSSL infrastructure
Look at BIOs for I/O. The encoding is up to you. You can write out raw bytes, you could Base{16|32|64} encode it, you can encode and store it in XML, or store it as name/value pairs.