I'm storing some files within my mysql database. The corresponding domain class looks as follows:
import org.springframework.web.multipart.commons.CommonsMultipartFile
import grails.persistence.Entity
#Entity
class Document extends BaseClass {
String documentReference
CommonsMultipartFile cmfFile
static constraints = {
documentReference nullable: true, maxSize: 500
}
static mapping = {
cmfFile sqlType: "mediumblob"
}
}
I managed to successfully store different files within the table in the database. Now I want to enable the user to download any of these files using the following action:
def download(Document documentInstance) {
if (documentInstance == null) {
notFound()
return
}
response.setContentType(documentInstance?.cmfFile?.contentType)
response.setHeader("Content-disposition", "attachment;filename=${documentInstance?.cmfFile?.originalFilename}")
response.outputStream << documentInstance?.cmfFile?.getBytes()
response.outputStream.flush()
return true
}
My problem now is that downloading works perfectly fine with .docx, textfiles or images. However, when I'm trying to download e.g. .pdf or .zip all files are empty. I don't know what the difference is as I'm also passing on the content type.
I would be greatful for any help! Thank you!
I also use Grails (2.3.11) and I store files in MySQL and it works like a charm.
The differences I can see are:
Domain
I use Blob type instead of CommonsMultipartFile and type: 'blob' instead of sqlType: 'mediumblob'. To set value to such a field you can use new javax.sql.rowset.serial.SerialBlob(byte[]).
Controller
I use (adjusted to your naming):
response.setContentType(servletContext.getMimeType(documentInstance?.cmfFile?.originalFilename))
response.setHeader("Content-disposition", "inline;filename=${documentInstance?.cmfFile?.originalFilename}")
response.outputStream << documentInstance?.cmfFile?.getBytes()
No flush(), no return true.
Notes
I also noticed you use quite small maxSize constraint. 500 B is not too much. Are you sure it's enough for your PDFs or ZIPs?
Besides, are you sure documentInstance?.cmfFile is not null?
Related
This question has two parts:
By default, what URL protocols are considered valid for specifying resources to Cypher's LOAD CSV command?
So far, I've successfully loaded CSV files into Neo4j using http and file protocols. A comment on this unrelated question indicates that ftp works as well, but I haven't had tried this because I have no use case.
What practical options do I have to configure non-standard URI protocols? I'm running up against a Neo.TransientError.Statement.ExternalResourceFailure: with "Invalid URL specified (unknown protocol)". Other than digging into the Neo4j source, is there anyway to modify this validation/setting, provided that the host machine is capable of resolving the resource with the specified protocol?
Neo4j relies on the capabilities of the JVM. According to https://docs.oracle.com/javase/7/docs/api/java/net/URL.html the default protocols are:
http, https, ftp, file, jar
Please note that file URLs are interpreted from the server's point of view and not from the client side (a common source of confusion).
To use custom URLs you need to understand how the JVM deals with those. The javadocs for URL class explain an approach by using a system property to provide custom URL handlers. It should be good enough to provide this system property in neo4j-wrapper.conf and drop the jar file containing your handler classes into the plugins folder. (Note: I did not validate that approach myself, but I'm pretty confident that it will work).
Here is a complete example, using the technique of implementing your own URLStreamHandler to handle the resource protocol. You must name your class 'Handler', and the last segment of the package name must be the protocol name (in this case, resource)
src/main/java/com/example/protocols/resource/Handler.java:
package com.example.protocols.resource;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.net.URL;
import java.net.URLConnection;
import java.net.URLStreamHandler;
public class Handler extends URLStreamHandler {
private final ClassLoader classLoader;
public Handler() {
this.classLoader = getClass().getClassLoader();
}
#Override
protected URLConnection openConnection(URL url) throws IOException {
URL resource = classLoader.getResource(url.getPath());
if (resource == null) {
throw new FileNotFoundException("Resource file not found: " + url.getPath());
}
return resource.openConnection();
}
}
From here, we need to set the system property java.protocol.handler.pkgs to include the base package com.example.protocols so that the protocol is registered. This can be done statically in a Neo4j ExtensionFactory. Since the class gets loaded by Neo4j, we know that the static block will be executed. We also need to provide our own URLAccessRule, since Neo4j by default only allows use of a few select protocols. This can also happen in the ExtensionFactory.
src/main/java/com/example/protocols/ProtocolInitializerFactory.java:
package com.example.protocols;
import org.neo4j.annotations.service.ServiceProvider;
import org.neo4j.graphdb.security.URLAccessRule;
import org.neo4j.kernel.extension.ExtensionFactory;
import org.neo4j.kernel.extension.ExtensionType;
import org.neo4j.kernel.extension.context.ExtensionContext;
import org.neo4j.kernel.lifecycle.Lifecycle;
import org.neo4j.kernel.lifecycle.LifecycleAdapter;
#ServiceProvider
public class ProtocolInitializerFactory extends ExtensionFactory<ProtocolInitializerFactory.Dependencies> {
private static final String PROTOCOL_HANDLER_PACKAGES = "java.protocol.handler.pkgs";
private static final String PROTOCOL_PACKAGE = ProtocolInitializerFactory.class.getPackageName();
static {
String currentValue = System.getProperty(PROTOCOL_HANDLER_PACKAGES, "");
if (currentValue.isEmpty()) {
System.setProperty(PROTOCOL_HANDLER_PACKAGES, PROTOCOL_PACKAGE);
} else if (!currentValue.contains(PROTOCOL_PACKAGE)) {
System.setProperty(PROTOCOL_HANDLER_PACKAGES, currentValue + "|" + PROTOCOL_PACKAGE);
}
}
public interface Dependencies {
URLAccessRule urlAccessRule();
}
public ProtocolInitializerFactory() {
super(ExtensionType.DATABASE, "ProtocolInitializer");
}
#Override
public Lifecycle newInstance(ExtensionContext context, Dependencies dependencies) {
URLAccessRule urlAccessRule = dependencies.urlAccessRule();
return LifecycleAdapter.onInit(() -> {
URLAccessRule customRule = (config, url) -> {
if ("resource".equals(url.getProtocol())) { // Check the protocol name
return url; // Optionally, you can validate the URL here and throw an exception if it is not valid or should not be allowed access
}
return urlAccessRule.validate(config, url);
};
context.dependencySatisfier().satisfyDependency(customRule);
});
}
}
After setting this up, follow the guide to packaging these classes as a Neo4j plugin and drop it into your database's plugins directory.
Admittedly, needing to override the default URLAccessRule feels a little bit shady. It may be better to simply implement the URLStreamHandler, and use another CSV loading method like APOC's apoc.load.csv. This will not require overriding the URLAccessRule, but it will require setting the Java system property java.protocol.handler.pkgs.
I have a JSON file that I'd like to load using Laravel. I'm learning Laravel and would like to know the right way to do this. I have the JSON files in a folder called json in the public folder.
In my routes.php I have the following:
Route::get('/json/{jsonfile}', array(
'as' => 'load-json',
'uses' => 'JSONController#loadJSON'
));
In JSONController I have:
public function loadJSON($jsonfile) {
// not sure what to do here
return View::make('json.display')
->with('jsonfile', $jsonfile);
}
Also is my naming convention ok or do you have better suggestions?
Always be careful when allowing a user inputed data to decide what files to read and write. Here is simple code that will take in the filename and look in the apps/storage/json folder. I'm less familiar with what Illuminate does to protect against system injections but you might want at the very least to make sure that 'filename' doesn't contain anything but alphanumeric characters with a validator.
Unless the JSON (or any file) needs to be public you shouldn't keep it in the public folder. This way they must go through your app (and permissions) to view it. Also you can have more restrictive permissions outside the public folder.
public function loadJSON($filename) {
$path = storage_path() . "/json/${filename}.json"; // ie: /var/www/laravel/app/storage/json/filename.json
if (!File::exists($path)) {
throw new Exception("Invalid File");
}
$file = File::get($path); // string
// Verify Validate JSON?
// Your other Stuff
}
I have a huge message file which i need to split into multiples files for different languages.
For example :
I created one folder for English locale i.e. en and another for French locale , fr inside conf folder.
en contains messages1_en.properties and messages2_en.properties
fr contains messages1_fr.properties and messages2_fr.properties
How to access these properties files inside my view.
Thanks
The only way to do that without introducing your own alternative implementation and use that instead of the built in Messages is to use hacked locales, so you would do fr_type1, fr_type2 or something like that to select the right alternative.
This is probably a bad idea since it's always risky to use an API in a different way from how it was intended to be used, there is a high risk of unexpected behaviour and it might be brittle since there is no guarantee that you will be able to use made up locales in future versions etc.
If you look at the Messages implementation you could probably get some ideas of how to implement your own without much fuss.
Good luck!
That's an old question, but i had a close issue, and i didn't find a solution anywhere.
This example use a configuration key to load messages from a file with a custom name. But you can easily modify it to load messages file from a subdirectory and/or multiple messages files.
Override play.api.i18n.DefaultMessagesApiProvider
#Singleton
class CustomMessagesApiProvider #Inject() (
environment: Environment,
config: Configuration,
langs: Langs,
httpConfiguration: HttpConfiguration)
extends DefaultMessagesApiProvider(environment, config, langs, httpConfiguration) {
def filename =
config.get[String]("play.i18n.filename")
override protected def loadAllMessages: Map[String, Map[String, String]] = {
langs.availables.map(_.code).map { lang =>
(lang, loadMessages(filename +"." + lang))
}.toMap
.+("default" -> loadMessages(filename))
.+("default.play" -> loadMessages(filename+".default"))
}
}
Add Guice binding in Module.java
#Override
public void configure() {
bind(DefaultMessagesApiProvider.class).to(CustomMessagesApiProvider.class);
}
It's my first Scala class, so maybe it can be improved. But it works.
To load multiple files (it compiles but not tested)
override protected def loadAllMessages: Map[String, Map[String, String]] = {
langs.availables.map(_.code).map { lang =>
(lang,
loadMessageFiles("." + lang))
}.toMap
.+("default" -> loadMessageFiles(""))
.+("default.play" -> loadMessageFiles(".default"))
}
private def loadMessageFiles(suffix: String) = {
loadMessages("messages-1" + suffix) ++ loadMessages("messages-2" + suffix)
}
I want to create a JSON file for use as part of a simple web prototyping exercise. LinqPAD is perfect for accessing the data from my DB in just the shape I need, however I cannot get it out as JSON very easily.
I don't really care what the schema is, because I can adapt my JavaScript to work with whatever is returned.
Is this possible?
A more fluent solution is to add the following methods to the "My Extensions" File in Linqpad:
public static String DumpJson<T>(this T obj)
{
return
obj
.ToJson()
.Dump();
}
public static String ToJson<T>(this T obj)
{
return
new System.Web.Script.Serialization.JavaScriptSerializer()
.Serialize(obj);
}
Then you can use them like this in any query you like:
Enumerable.Range(1, 10)
.Select(i =>
new
{
Index = i,
IndexTimesTen = i * 10,
})
.DumpJson();
I added "ToJson" separately so it can be used in with "Expessions".
This is not directly supported, and I have opened a feature request here. Vote for it if you would also find this useful.
A workaround for now is to do the following:
Set the language to C# Statement(s)
Add an assembly reference (press F4) to System.Web.Extensions.dll
In the same dialog, add a namespace import to System.Web.Script.Serialization
Use code like the following to dump out your query as JSON
new JavaScriptSerializer().Serialize(query).Dump();
There's a solution with Json.NET since it does indented formatting, and renders Json dates properly. Add Json.NET from NuGet, and refer to Newtonsoft.Json.dll to your “My Extensions” query and as well the following code :
public static object DumpJson(this object value, string description = null)
{
return GetJson(value).Dump(description);
}
private static object GetJson(object value)
{
object dump = value;
var strValue = value as string;
if (strValue != null)
{
var obj = JsonConvert.DeserializeObject(strValue);
dump = JsonConvert.SerializeObject(obj, Newtonsoft.Json.Formatting.Indented);
}
else
{
dump = JsonConvert.SerializeObject(value, Newtonsoft.Json.Formatting.Indented);
}
return dump;
}
Use .DumpJson() as .Dump() to render the result. It's possible to override more .DumpJson() with different signatures if necessary.
As of version 4.47, LINQPad has the ability to export JSON built in. Combined with the new lprun.exe utility, it can also satisfy your needs.
http://www.linqpad.net/lprun.aspx
I'm working on a Grails 1.0.4 project that has to be released in less than 2 weeks, and the customer just came up with a requirement that all data in the database should be encrypted.
Since encryption of every database access in the application itself could take a lot of time and will be error prone, the solution I seek is some kind of encryption transparent to the application.
Is there a way to setup Hibernate to encrypt all data in all tables (except maybie the id and version columns) or should I seek a MySQL solution (we're using MySQL 5.0) ?
EDIT:
Thanks for all of your posts for alternative solutions, if the customer changes mind it would be great. As for now, the requirement is "No plain text in the Database".
Second thing I'd like to point out is that I'm using Grails, for those not fammiliar with it, It's a convention over configuration, so even small changes to the application that are not by convention should be avoided.
If you end doing the work in the application, you can use Hibernate custom types and it wouldn't add that many changes to your code.
Here's an encrypted string custom type that I've used:
import org.hibernate.usertype.UserType
import org.apache.log4j.Logger
import java.sql.PreparedStatement
import java.sql.ResultSet
import java.sql.SQLException
import java.sql.Types
class EncryptedString implements UserType {
// prefix category name with 'org.hibernate.type' to make logging of all types easier
private final Logger _log = Logger.getLogger('org.hibernate.type.com.yourcompany.EncryptedString')
Object nullSafeGet(ResultSet rs, String[] names, Object owner) throws SQLException {
String value = rs.getString(names[0])
if (!value) {
_log.trace "returning null as column: $names[0]"
return null
}
_log.trace "returning '$value' as column: $names[0]"
return CryptoUtils.decrypt(value)
}
void nullSafeSet(PreparedStatement st, Object value, int index) throws SQLException {
if (value) {
String encrypted = CryptoUtils.encrypt(value.toString())
_log.trace "binding '$encrypted' to parameter: $index"
st.setString index, encrypted
}
else {
_log.trace "binding null to parameter: $index"
st.setNull(index, Types.VARCHAR)
}
}
Class<String> returnedClass() { String }
int[] sqlTypes() { [Types.VARCHAR] as int[] }
Object assemble(Serializable cached, Object owner) { cached.toString() }
Object deepCopy(Object value) { value.toString() }
Serializable disassemble(Object value) { value.toString() }
boolean equals(Object x, Object y) { x == y }
int hashCode(Object x) { x.hashCode() }
boolean isMutable() { true }
Object replace(Object original, Object target, Object owner) { original }
}
and based on this it should be simple to create similar classes for int, long, etc. To use it, add the type to the mapping closure:
class MyDomainClass {
String name
String otherField
static mapping = {
name type: EncryptedString
otherField type: EncryptedString
}
}
I omitted the CryptoUtils.encrypt() and CryptoUtils.decrypt() methods since that's not Grails-specific. We're using AES, e.g. "Cipher cipher = Cipher.getInstance('AES/CBC/PKCS5Padding')". Whatever you end up using, make sure it's a 2-way crypto, i.e. don't use SHA-256.
If the customer is worried about someone physically walking away with the hard drive then using a full disk solution like Truecrypt should work. If there worried about traffic being sniffed then take a look at this part of the mysql documentation on ssl over JDBC. Remember if someone compromises your server all bets are off.
the customer could easily do this without changing a thing in your application.
first, encrypt the communications between the server by turning on SSL in the mysql layer, or use an SSH tunnel.
second, store the mysql database on an encrypted volume.
any attack that can expose the filesystem of the mysql database or the credentials needed to log in to the mysql server is not mitigated by encrypting the data since that same attack can be used to retrieve the encryption key from the application itself.
Well it has been a long time since I've asked the question. In the meantime, thanks for all the answers. They were great when dealing with the original idea of encrypting the entire database, but the requirement changed to only encrypting sensitive user info, like name and address. So the solution was something like the code down below.
We've implemented an Encrypter which reads the encryption method from the record ( so there can be different encryption per record) and use it to connect transient duplicate fields to the ones encrypted in the database. The added bonus/drawbacks are:
The data is also encrypted in memory, so every access to the method getFirstName descrypts the data (I guess there is a way to cache decrypted data, but I dont need it in this case)
Encrypted fields cannot be used with default grails/hibernate methods for search through database, we've made custom methods in services that get data, encrypt it and then use the encrypted data in the where clause of a query. It's easy when using User.withCriteria
class User {
byte[] encryptedFirstName
byte[] encryptedLastName
byte[] encryptedAddress
Date dateCreated // automatically set date/time when created
Date lastUpdated // automatically set date/time when last updated
EncryptionMethod encryptionMethod = ConfigurationHolder.config.encryption.method
def encrypter = Util.encrypter
static transients = [
'firstName',
'lastName',
'address',
'encrypter'
]
static final Integer BLOB_SIZE = 1024
static constraints = {
encryptedFirstName maxSize: BLOB_SIZE, nullable: false
encryptedLastName maxSize: BLOB_SIZE, nullable: false
encryptedAddress maxSize: BLOB_SIZE, nullable: true
encryptionMethod nullable: false
} // constraints
String getFirstName(){
decrypt('encryptedFirstName')
}
void setFirstName(String item){
encrypt('encryptedFirstName',item)
}
String getLastName(){
decrypt('encryptedLastName')
}
void setLastName(String item){
encrypt('encryptedLastName',item)
}
String getAddress(){
decrypt('encryptedAddress')
}
void setAddress(String item){
encrypt('encryptedAddress',item)
}
byte[] encrypt(String name, String value) {
if( null == value ) {
log.debug "null string to encrypt for '$name', returning null"
this.#"$name" = null
return
}
def bytes = value.getBytes(encrypter.ENCODING_CHARSET)
def method = getEncryptionMethod()
byte[] res
try {
res = encrypter.encrypt( bytes, method )
} catch(e) {
log.warn "Problem encrypting '$name' data: '$string'", e
}
log.trace "Encrypting '$name' with '$method' -> '${res?.size()}' bytes"
this.#"$name" = res
}
String decrypt(String name) {
if(null == this.#"$name") {
log.debug "null bytes to decrypt for '$name', returning null"
return null
}
def res
def method = getEncryptionMethod()
try {
res = new String(encrypter.decrypt(this.#"$name", method), encrypter.ENCODING_CHARSET )
} catch(e) {
log.error "Problem decrypting '$name'", e
}
log.trace "Decrypting '$name' with '$method' -> '${res?.size()}' bytes"
return res
}
}
Another option is to use a JDBC driver that encrypts/decrypts data on the fly, two way. Bear in mind that any solution will probably invalidate searches by encrypted fields.
IMHO the best solution is the one proposed by longneck, it will make everything much easier, from administration to development. Besides, bear in mind that any solution with client-side encryption will render all your db data unusable outside of the client, ie, you will not be able to use nice tools like a jdbc client or MySQL query browser, etc.
Jasypt integrates with Hibernate: http://jasypt.org/hibernate3.html. However, queries which use WHERE clauses cannot be used
Generated ids, version, mapped foreign keys - basically everything maintained by Hibernate - are out unless you intend to declare custom CRUD for all of your classes and manually encrypt them in queries.
For everything else you've got a couple of choices:
#PostLoad and #PrePersist entity listeners will take care of all non-query operations.
Implementing custom String / Long / Integer / etc... types to handle encryption will take care of both query and CRUD operations; however the mapping will become rather messy.
You can write a thin wrapper around a JDBC driver (as well as Connection / Statement / PreparedStatement / ResultSet / etc...) to do the encryption for you.
As far as queries go you'll have to handle encryption manually (unless you're going with #2 above) but you should be able to do so via a single entry point. I'm not sure how (or if) Grails deals with this, but using Spring, for example, it would be as easy as extending HibernateTemplate.