Some site describe config & make for OpenSSL with zlib while I can do it without zlib.
It means zlib is not necessary for openSSL in some case.
Does anyone tell me what case OpenSSL does compression or decompression?
The answer from #Giacomo1968 is useful.
I want to know how to choose if I use –z or not?
The answer from #Giacomo1968 is useful.
I want to know how to choose if I use –z or not?
That's easy. Compression leaks information in protocols like HTTPS and SPDY, so you should not use it. Since you should not use it, there's no reason to configure with it. See Rizzo and Duong's CRIME attack.
There's another option to configure you might be interested in: no-comp. It disables compression independent of zlib.
Does anyone tell me what case OpenSSL does compression or decompression?
By default, compression is enabled unless you disable it at compile time or runtime. If compression is available, then you have to disable it at runtime with the SSL_OP_NO_COMPRESSION context options:
const SSL_METHOD* method = SSLv23_method();
if(method == NULL) handleFailure();
ctx = SSL_CTX_new(method);
if(ctx == NULL) handleFailure();
const long flags = SSL_OP_NO_SSLv2 | SSL_OP_NO_SSLv3 | SSL_OP_NO_COMPRESSION;
SSL_CTX_set_options(ctx, flags);
For completeness, Firefox does not support compression. Firefox's configure used to be broken out of the box, so the browser was not vulnerable to the compression attacks. See the bug report, Build NSS with the TLS zlib compression code and add the security.ssl.enable_compression preference to enable it.
The answer is right in the manual. It relates to the -z option:
“Compress or decompress clear text using zlib before encryption or after decryption. This option exists only if OpenSSL with compiled with zlib or zlib-dynamic option.”
I'd like to add some information on "zlib ssl compression, CRIME attack, BREACH attack", that I found not clear enough when read the answers.
A quick summary:
TLS used zlib for compression, that is vulnerable to CRIME attack. To fix that, it requires either clients(browsers) or servers to refuse compression in TLS. Apache httpd and nginx changed tls compression to false as default. Browser vendors(Firefox did) disable tls compression. So disabling compression (build without zlib), everything still works.
CRIME attack doesn't only affect TLS protocols. HTTP also uses compression so it's also affected. BREACH is a variant attack of CRIME targeting HTTP. But it can be mitigated via CSRF tokens.
Good references and check:
https://www.wikiwand.com/en/CRIME
https://bugzilla.mozilla.org/show_bug.cgi?id=580679
https://bz.apache.org/bugzilla/show_bug.cgi?id=53219#c10
https://www.wikiwand.com/en/BREACH_(security_exploit)
Related
We have a number of loosely coupled apps, some in PHP and some in Python.
It would be beneficial to have some centralized place where they could get both global and app-specific configuration information.
Something like, for Python:
conf=config_server.get_params(url='http://config_server/get/My_app/all', auth=my_auth_data)
and then ideally use parameters as potentially nested attributes, eg. conf.APP.URL, conf.GLOBAL.MAX_SALES
I was considering making my own config server app, but wasn't sure, what would be the pros and cons of such approach vs. eg. storing config in centralized database or any other multiple-site accessible mode.
Also, if I perhaps was missing some readily available tool with good support, which could do this (I had a look at Puppet and Ansible, but they seemed to be very evolved tools for doing so much more than this. I also looked at software recommnedation SE for this, but they have a number of such question unanswered already).
I think it would be a good idea for your configuration mechanism not to be hard-coded to obtain configuration data via a particular technology (such as file, web server or database), but rather be able to obtain configuration data from any of several different technologies. I illustrate this with the following pseudo-code examples:
cfg = getConfig("file.cfg"); # from a file
cfg = getConfig("file#file.cfg"); # also from a file
cfg = getConfig("url#http://config_server/file.cfg"); # from the specified URL
cfg = getConfig("exec#getConfigFromDB.py"); # from stdout of command
The parameter passed to getConfig() might be obtained from, say, a command-line option. The "exec#..." format is a flexible mechanism, but carries the potential danger of somebody specifying a malicious command to execute, for example, "exec#rm -rf /".
This approach means you can experiment with whatever you consider to be an ideal source-of-configuration-data technology and later, if you discover that technology to be inappropriate, it will be trivial to discard it and use a different source-of-configuration-data technology instead. Indeed, the decision for which source-of-configuration-data technology to use might vary from one use case/user to another.
I developed a C++ and Java configuration file parser (sorry, no Python or PHP implementations) called Config4*. If you look at chapters 2 (overview of syntax) and 3 (overview of API) of the Config4* Getting Started Guide, you will notice that it supports the kind of flexible approach I discuss in this answer (the "url#... format is not supported, but "exec#curl -sS ..." provides the same functionality). 99 percent of the time, I end up using configuration files, but I find it comforting to know that my applications can trivially switch to using a different-source-of-configuration-data technology whenever the need might arise.
Thrift's primary goal is to enable efficient and reliable communication across programming languages. but I think HTTP-RPC can also do that, web developer almost everyone knows how to work on http and it is easier to implement HTTP-RPC(json) than Thrift,
Maybe Thrift-RPC is faster, then who can tell me the difference in perfermance between them?
A few reasons other than speed:
Thrift generates the client and server code completely, including the data structures you are passing, so you don't have to deal with anything other than writing the handlers and invoking the client. and everything, including parameters and returns are automatically validated and parsed. so you are getting sanity checks on your data for free.
Thrift is more compact than HTTP, and can easily be extended to support things like encryption, compression, non blocking IO, etc.
Thrift can be set up to use HTTP and JSON pretty easily if you want it (say if your client is somewhere on the internet and needs to pass firewalls)
Thrift supports persistent connections and avoids the continuous TCP and HTTP handshakes that HTTP incurs.
Personally, I use thrift for internal LAN RPC and HTTP when I need connections from outside.
I hope all this makes sense to you. You can read a presentation I gave about thrift here:
http://www.slideshare.net/dvirsky/introduction-to-thrift
It has links to a few other alternatives to thrift.
Here is good resource on performance comparison of different serializers: https://github.com/eishay/jvm-serializers/wiki/
Speaking specifically of Thrift vs JSON: Thrift performance is comparable to the best JSON libraries (jackson, protostuff), and serialized size is somewhat lower.
IMO, strongest thrift advantages are convenient interoperable RPC invocations and convenient handling of binary data.
I have a encryption scheme implemented, the constituent components: The symetric cypher and its chaining mode, and the HMAC algorithm are hard-coded into the binary. Additionally, the parameters of the algorithms (HMAC key, symetric-key symetric IVEC) are specified in binary files, one for each parameter.
I would like to specify the choice of algorithms, and modes, and their parameters in a single file. Do I need my own format, or is this possible using existing OpenSSL infrastructure ? If there is infrastructure, could someone please provide some references.
p.s., I know of the config file parsing code, and the PEM/x.509 code in OpenSSL. However anything built from this won't be cohesive.
I would like to specify the choice of algorithms, and modes, and their
parameters in a single file
That type of agility sounds like you will allow the user to make a choice. Don't do it, since they might pick a bad cipher (or cipher combination). Make good choices for them.
The symetric cypher and its chaining mode, and the HMAC algorithm are
hard-coded into the binary.
I would first look into an authenticated encryption mode - EAX, CCM, or GCM. I believe OpenSSL only has CCM at the moment (or is it GCM?). If you can't use an authenticated encrpytion mode, move on to Encrypt-Then-Authenticate (ie, encrypt then authnticate the cipher text with a HMAC or CMAC), which it sounds like you are doing.
Additionally, the parameters of the algorithms (HMAC key, symetric-key
symetric IVEC)
HMAC (and CMAC) are good. Don't use a CBC-MAC since it suffers from weaknesses on variable length messages.
symetric-key [in binary file]
Hmmm...
symmetric IVEC
IVs are considered public parameters. Pick a random IV for the message, and send it along with the cipher text. Make sure to MAC both the cipher text and IV to detect tampering.
Do I need my own format, or is this possible using existing OpenSSL infrastructure
Look at BIOs for I/O. The encoding is up to you. You can write out raw bytes, you could Base{16|32|64} encode it, you can encode and store it in XML, or store it as name/value pairs.
According to mysql document
"Encoding with a 128-bit key length is used, but you can extend it up to 256 bits by modifying the source."
But they didn't seem to provide instruction where to change. Anyone experience with this situation? which source file should change?
Note: I use these steps to compile.
I found little help from mysql mailing list
file include/my_aes.h
#define AES_KEY_LENGTH 128 /* must be 128 192 or 256 */
as I'm using OpenSuSe 11.1 need to have following tools
sudo zypper install gcc gcc-c++ ncurses-devel
then just compile it by this instruction - here
Credit to LenZ and tripanel.net
It's probably going to be a more maintainable solution to carry out the encryption in the client application.
Moreover, you'll also then get the benefit of then having the data carried over the network encrypted and not sending the key over the network (Of course you can use SSL to connect to mysql to mitigate this anyway).
If this does not seem like a good approach, please post your requirements.
You probably do not want to compile your own mysql binaries; there are more useful things for developers to do than building their own mysql binaries. MySQL / Sun's ones are extensively tested and won't contain performance regressions (we hope).
The mysql AES_ENCRYPT() functions are also potentially not secure because they haven't documented
How they hash the password into the key
What cipher mode they use
If they're done in a vulnerable way, the encryption could be very weak. It depends on your use-case whether this matters.
Many times I saw logging of errors like these:
System.out.println("Method aMethod with parameters a:"+a+" b: "+b);
print("Error in line 88");
so.. What are the best practices to log an error?
EDIT:
This is java but could be C/C++, basic, etc.
Logging directly to the console is horrendous and frankly, the mark of an inexperienced developer. The only reason to do this sort of thing is 1) he or she is unaware of other approaches, and/or 2) the developer has not thought one bit about what will happen when his/her code is deployed to a production site, and how the application will be maintained at that point. Dealing with an application that is logging 1GB/day or more of completely unneeded debug logging is maddening.
The generally accepted best practice is to use a Logging framework that has concepts of:
Different log objects - Different classes/modules/etc can log to different loggers, so you can choose to apply different log configurations to different portions of the application.
Different log levels - so you can tweak the logging configuration to only log errors in production, to log all sorts of debug and trace info in a development environment, etc.
Different log outputs - the framework should allow you to configure where the log output is sent to without requiring any changes in the codebase. Some examples of different places you might want to send log output to are files, files that roll over based on date/size, databases, email, remoting sinks, etc.
The log framework should never never never throw any Exceptions or errors from the logging code. Your application should not fail to load or fail to start because the log framework cannot create it's log file or obtain a lock on the file (unless this is a critical requirement, maybe for legal reasons, for your app).
The eventual log framework you will use will of course depend on your platform. Some common options:
Java:
Apache Commons Logging
log4j
logback
Built-in java.util.logging
.NET:
log4net
C++:
log4cxx
Apache Commons Logging is not intended for applications general logging. It's intended to be used by libraries or APIs that don't want to force a logging implementation on the API's user.
There are also classloading issues with Commons Logging.
Pick one of the [many] logging api's, the most widely used probably being log4j or the Java Logging API.
If you want implementation independence, you might want to consider SLF4J, by the original author of log4j.
Having picked an implementation, then use the logging levels/severity within that implementation consistently, so that searching/filtering logs is easier.
The easiest way to log errors in a consistent format is to use a logging framework such as Log4j (assuming you're using Java). It is useful to include a logging section in your code standards to make sure all developers know what needs to be logged. The nice thing about most logging frameworks is they have different logging levels so you can control how verbose the logging is between development, test, and production.
A best practice is to use the java.util.logging framework
Then you can log messages in either of these formats
log.warning("..");
log.fine("..");
log.finer("..");
log.finest("..");
Or
log.log(Level.WARNING, "blah blah blah", e);
Then you can use a logging.properties (example below) to switch between levels of logging, and do all sorts of clever stuff like logging to files, with rotation etc.
handlers = java.util.logging.ConsoleHandler
.level = WARNING
java.util.logging.ConsoleHandler.level = ALL
com.example.blah = FINE
com.example.testcomponents = FINEST
Frameworks like log4j and others should be avoided in my opinion, Java has everything you need already.
EDIT
This can apply as a general practice for any programming language. Being able to control all levels of logging from a single property file is often very important in enterprise applications.
Some suggested best-practices
Use a logging framework. This will allow you to:
Easily change the destination of your log messages
Filter log messages based on severity
Support internationalised log messages
If you are using java, then slf4j is now preferred to Jakarta commons logging as the logging facade.
As stated slf4j is a facade, and you have to then pick an underlying implementation. Either log4j, java.util.logging, or 'simple'.
Follow your framework's advice to ensuring expensive logging operations are not needlessly carried out
The apache common logging API as mentioned above is a great resource. Referring back to java, there is also a standard error output stream (System.err).
Directly from the Java API:
This stream is already open and ready
to accept output data.
Typically this stream corresponds to
display output or another output
destination specified by the host
environment or user. By convention,
this output stream is used to display
error messages or other information
that should come to the immediate
attention of a user even if the
principal output stream, the value of
the variable out, has been redirected
to a file or other destination that is
typically not continuously monitored.
Aside from technical considerations from other answers it is advisable to log a meaningful message and perhaps some steps to avoid the error in the future. Depending on the errors, of course.
You could get more out of a I/O-Error when the message states something like "Could not read from file X, you don't have the appropriate permission."
See more examples on SO or search the web.
There really is no best practice for logging an error. It basically just needs to follow a consistent pattern (within the software/company/etc) that provides enough information to track the problem down. For Example, you might want to keep track of the time, the method, parameters, calling method, etc.
So long as you dont just print "Error in "