How to store duration in groovy with mysql - mysql

Setting up an application which does some reporting on call logs using groovy. I want to use mysql is the background and upload the data to the database externally (i.e. a script that runs every midnight which loads todays calls)
I have gotten the connection between mysql and groovy (somewhat) working, it clearly validates the database even if I cant extract or store any data in it.
However, one of the values in the external log is a duration, normally in the format "00:05:42" (%H:%m:%s), how ever, I am having some issues how to store it in mysql in a way that groovy is happy with. Tried date first, but that gives me the issue that it will expect a calendar date as part of it.
Then I started trying to use TimeDuration from grovy.time.*, but this seems to store things as a tinyblob. So, is there a Time version that works in mysql and groovy likes (tried a number of different ones, and while mysql seems happy enough, groovy don't quite seem to know what to do about it.
The domain at the moment looks like this:
import groovy.time.*
import java.sql.Time
import java.sql.Timestamp
class Calls {
Time CallTime
TimeDuration Length
int Line
String SourceNr
String Direction
String DialNr
String Phone
String User
static constraints = {
}
static mapping = {
version false
}
// id generator: 'hilo',
// params: [table: 'hi_value', column: 'next_value', max_lo: 100]
//id composite: ['DateTime', 'SourceNr','DialNr']
}

If you cannot find a duration type that can parse the format you describe and that is stored correctly in MySQL, I suggest using a simple long value in seconds and doing the conversion to/from string in the getter and setter.
(This is just some quick code, do some proper tests on boundary cases before using it, plus maybe some input validation.)
class Calls {
long length
/** Parse a string with format hhh:mm:ss into a value in seconds. */
public void setLength(String s) {
length = s.split(':').collect(Integer.&parseInt).inject {
a, b -> a * 60 + b
}
}
/** Format a number in seconds into a hhh:mm:ss string. */
public String getLength() {
def a = [length]
for (i in [1, 2]) {
a << (long)(a[-1] / 60)
a[-2] %= 60
}
String.format("%02d:%02d:%02d", *a.reverse())
}
}

Related

Typescript - auto conversion from a Json string with a date string to an object with a Date property

When calling a REST service returning JSON, how can I format a date so that it is converted automatically into a typescript Date object?
The result of calling a REST call is this message:
{"id":3796,...,"startTempValue":"2019-05-26T19:39:01Z"}
I also tried this ISO format:
{"id":3796,...,"startTempValue": "2019-05-26T19:39:01.000Z"}
The model object is:
export class Settings {
public id: number;
public shortName;
public description: string;
public value: string;
public possibleValues: string;
public startTempValue: Date;
}
The result of startTempValue is a string ... ?! I want the startTempValue to be a Date object.
Of course, I could convert the Date string to a Date object manually. So, doing something like the code below at reception of the REST service result. But there should be a better way. Another alternative would be to convert it at the server to an epoch (milliconds). That is possible, but still, this 'string' variant is more readable.
if ( this.settings[i].startTempValue !== undefined &&
this.settings[i].startTempValue !== null) {
this.settings[i].startTempValue = new Date(this.settings[i].startTempValue);
} else {
this.settings[i].startTempValue = null;
}
JSON only deals with primitives like numbers, strings, arrays and objects.
There is no information within JSON that describes what its contents mean. So even if something looks like a date to you, it's still just a string.
So how could you convert it manually? Well, if you know in advanced what property names are always dates, you could loop through the result and convert these preemptively. Or you could loop through all the values and take anything that looks like a date and automatically convert.
A third option would be to have some kind of schema that marks certain things as dates and use that for your automatic conversion.
The point is that there is no built-in way to infer this, and it's kind of up to you do to something else with your data

Find string length without using builtin string.length method in web methods?

I want to find length of given string with out using pub.string.length (built in function) in web methods .
Ex:If we give name "abcdef" i want result like 6 .
You could write your own java service, but that sounds redundant to me.
Here's an (as yet untested) code sample:
public static final void checkStringSize(IData pipeline)
throws ServiceException {
// pipeline
IDataCursor pipelineCursor = pipeline.getCursor();
String inputString = IDataUtil.getString( pipelineCursor, "inputString" );
pipelineCursor.destroy();
long length = -1;
length = inputString.length();
// pipeline
IDataCursor pipelineCursor_1 = pipeline.getCursor();
IDataUtil.put( pipelineCursor_1, "length", ""+length );
pipelineCursor_1.destroy();
}
You could manually count the length of the string by using pub.string:substring until it gets an error.
I wouldn't recommend this - it's inelegant, possibly quite slow and it still uses a function from pub.string, which you appear to be avoiding.
Anyway, here's a way to do it, click the link to see the image.
webMethods code sample - https://i.stack.imgur.com/FE9oU.jpg
Just make sure the input string cannot be null - otherwise the substring won't throw an error and it will have an infinite loop.

Grails JSON Marshaller Shows a Different Date value than Original Date

In my Grails app, the original date read from the database is equal to:
{ endDate=2015-10-19 19:00:00.0}
While the JSON result is:
{"endDate": "2015-10-19T16:00:00Z"}
I think this is maybe related to time zone conversion. How could I show the original date without any timezone conversions in JSON?
Depending on which time zone you're in, 2015-10-19 19:00:00.0 and 2015-10-19T16:00:00Z may not be different times, they may be just different representations of the same time (instant).
In my case, I use a custom marshaller to ensure that times in my API's JSON response always use the UTC time zone. My custom marshaller looks like this:
import org.springframework.stereotype.Component
#Component
class DateMarshaller implements CustomMarshaller {
#Override
def getSupportedTypes() {
Date
}
#Override
Closure getMarshaller() {
{ Date date ->
TimeZone tz = TimeZone.getTimeZone('UTC')
date?.format("yyyy-MM-dd'T'HH:mm:ss'Z'", tz)
}
}
}
Remember to register the package this marshaller is in for Spring bean scanning in Config.groovy. The interface it implements is:
interface CustomMarshaller {
/**
* Indicates the type(s) of object that this marshaller supports
* #return a {#link Class} or collection of {#link Class}
* if the marshaller supports multiple types
*/
def getSupportedTypes()
Closure getMarshaller()
}
Then I have a service that registers all my instances of CustomMarshaller for the relevant type(s):
import grails.converters.JSON
import org.springframework.context.ApplicationContext
import org.springframework.context.ApplicationContextAware
import javax.annotation.PostConstruct
class MarshallerRegistrarService implements ApplicationContextAware {
static transactional = false
ApplicationContext applicationContext
// a combination of eager bean initialization and #PostConstruct ensures that the marshallers are registered when
// the app (or a test thereof) starts
boolean lazyInit = false
#PostConstruct
void registerMarshallers() {
Map<String, CustomMarshaller> marshallerBeans = applicationContext.getBeansOfType(CustomMarshaller)
marshallerBeans.values().each { CustomMarshaller customMarshaller ->
customMarshaller.supportedTypes.each { Class supportedType ->
JSON.registerObjectMarshaller supportedType, customMarshaller.marshaller
}
}
}
}
This is a fairly involved solution, but In my case I'm using Grails 2.5.X. If I was using Grails 3.X I'd try to use JSON views instead.
If my memory is correct, JSON spec does not actually define a format for date format. But everybody uses ISO 8601, so it sort of de-facto standard. And most just always use Zulu time zone.
I searched myself some time ago, on ways to force Grails JSON to render dates in certain time zone, but failed. In my Grails web app, I just declare date fields as text and format them to proper time zone and format in my own code. On the good side, it also has an added benefit that you can guarantee it to stay that way in future. (I'm, using Grails since about 1.1 and did see breaking changes on several occasions.)
Working on an old application that is built with grails 2.3.11 in which for some unbeknownst reason #Dónal's solution wasn't working. Fortunately, I found an easier/more straightforward approach to register a custom marshaller on mrhaki's blog.
So, I made a copy of the default DateMarshaller that resides in org.codehaus.groovy.grails.web.converters.marshaller.json and modified the date formatter in my custom DateMarshaller and then registered the marshaller using the snippet below, taken from mrhaki's post.
dateMarshaller(ObjectMarshallerRegisterer) {
// Assign custom marshaller instance.
marshaller = new DateMarshaller()
// Set converter class type.
converterClass = JSON
// Optional set priority. Default value is 0.
priority = 1
}
Just make sure that your Custom Marshaller has a higher priority than the default one. Works like a charm!

Doctrine 2 and MongoDB: How can I declare a field that will store a json document?

I would like to use MongoDB to store the logs of my json api. I am using the Doctrine 2 ODM.
Reading through the manual I understand that to store my json api response, I should use the #Field annotation with some well chosen data type. However, I do not find the one that would fit the BSON data type.
Is there any data type that will allow me to get things working with something like
$myRecord->setResponse($json_response);
and later on perform MongoDB queries like
db.Logs.find({"response.user": "foo"});
? What is the correct way to deal with this ?
After a few tries, here I am: the correct type to do this is the Hash type. Our document has a response field that looks like this:
/**
* #MongoDB\Hash
*/
protected $response;
And the call to set the value is as follow:
$myRecord->setResponse(json_decode($result, true));
This is it!
Edit: As I had some issues to retrieve the stored values, I came to use mongodb callbacks :/
/**
* (...)
* #MongoDB\HasLifecycleCallbacks
*/
class Log
{
(...)
/** #MongoDB\PreLoad */
public function preLoad(array &$data)
{
$data["response"] = json_encode($data["response"]);
}
/** #MongoDB\PrePersist */
public function prePersist()
{
$a_json = json_decode($this->getResponse(), true);
$this->setResponse($a_json);
}
Which makes the set call straight, by the way:
$myRecord->setResponse($result);

Hibernate Encryption of Database Completely Transparent to Application

I'm working on a Grails 1.0.4 project that has to be released in less than 2 weeks, and the customer just came up with a requirement that all data in the database should be encrypted.
Since encryption of every database access in the application itself could take a lot of time and will be error prone, the solution I seek is some kind of encryption transparent to the application.
Is there a way to setup Hibernate to encrypt all data in all tables (except maybie the id and version columns) or should I seek a MySQL solution (we're using MySQL 5.0) ?
EDIT:
Thanks for all of your posts for alternative solutions, if the customer changes mind it would be great. As for now, the requirement is "No plain text in the Database".
Second thing I'd like to point out is that I'm using Grails, for those not fammiliar with it, It's a convention over configuration, so even small changes to the application that are not by convention should be avoided.
If you end doing the work in the application, you can use Hibernate custom types and it wouldn't add that many changes to your code.
Here's an encrypted string custom type that I've used:
import org.hibernate.usertype.UserType
import org.apache.log4j.Logger
import java.sql.PreparedStatement
import java.sql.ResultSet
import java.sql.SQLException
import java.sql.Types
class EncryptedString implements UserType {
// prefix category name with 'org.hibernate.type' to make logging of all types easier
private final Logger _log = Logger.getLogger('org.hibernate.type.com.yourcompany.EncryptedString')
Object nullSafeGet(ResultSet rs, String[] names, Object owner) throws SQLException {
String value = rs.getString(names[0])
if (!value) {
_log.trace "returning null as column: $names[0]"
return null
}
_log.trace "returning '$value' as column: $names[0]"
return CryptoUtils.decrypt(value)
}
void nullSafeSet(PreparedStatement st, Object value, int index) throws SQLException {
if (value) {
String encrypted = CryptoUtils.encrypt(value.toString())
_log.trace "binding '$encrypted' to parameter: $index"
st.setString index, encrypted
}
else {
_log.trace "binding null to parameter: $index"
st.setNull(index, Types.VARCHAR)
}
}
Class<String> returnedClass() { String }
int[] sqlTypes() { [Types.VARCHAR] as int[] }
Object assemble(Serializable cached, Object owner) { cached.toString() }
Object deepCopy(Object value) { value.toString() }
Serializable disassemble(Object value) { value.toString() }
boolean equals(Object x, Object y) { x == y }
int hashCode(Object x) { x.hashCode() }
boolean isMutable() { true }
Object replace(Object original, Object target, Object owner) { original }
}
and based on this it should be simple to create similar classes for int, long, etc. To use it, add the type to the mapping closure:
class MyDomainClass {
String name
String otherField
static mapping = {
name type: EncryptedString
otherField type: EncryptedString
}
}
I omitted the CryptoUtils.encrypt() and CryptoUtils.decrypt() methods since that's not Grails-specific. We're using AES, e.g. "Cipher cipher = Cipher.getInstance('AES/CBC/PKCS5Padding')". Whatever you end up using, make sure it's a 2-way crypto, i.e. don't use SHA-256.
If the customer is worried about someone physically walking away with the hard drive then using a full disk solution like Truecrypt should work. If there worried about traffic being sniffed then take a look at this part of the mysql documentation on ssl over JDBC. Remember if someone compromises your server all bets are off.
the customer could easily do this without changing a thing in your application.
first, encrypt the communications between the server by turning on SSL in the mysql layer, or use an SSH tunnel.
second, store the mysql database on an encrypted volume.
any attack that can expose the filesystem of the mysql database or the credentials needed to log in to the mysql server is not mitigated by encrypting the data since that same attack can be used to retrieve the encryption key from the application itself.
Well it has been a long time since I've asked the question. In the meantime, thanks for all the answers. They were great when dealing with the original idea of encrypting the entire database, but the requirement changed to only encrypting sensitive user info, like name and address. So the solution was something like the code down below.
We've implemented an Encrypter which reads the encryption method from the record ( so there can be different encryption per record) and use it to connect transient duplicate fields to the ones encrypted in the database. The added bonus/drawbacks are:
The data is also encrypted in memory, so every access to the method getFirstName descrypts the data (I guess there is a way to cache decrypted data, but I dont need it in this case)
Encrypted fields cannot be used with default grails/hibernate methods for search through database, we've made custom methods in services that get data, encrypt it and then use the encrypted data in the where clause of a query. It's easy when using User.withCriteria
class User {
byte[] encryptedFirstName
byte[] encryptedLastName
byte[] encryptedAddress
Date dateCreated // automatically set date/time when created
Date lastUpdated // automatically set date/time when last updated
EncryptionMethod encryptionMethod = ConfigurationHolder.config.encryption.method
def encrypter = Util.encrypter
static transients = [
'firstName',
'lastName',
'address',
'encrypter'
]
static final Integer BLOB_SIZE = 1024
static constraints = {
encryptedFirstName maxSize: BLOB_SIZE, nullable: false
encryptedLastName maxSize: BLOB_SIZE, nullable: false
encryptedAddress maxSize: BLOB_SIZE, nullable: true
encryptionMethod nullable: false
} // constraints
String getFirstName(){
decrypt('encryptedFirstName')
}
void setFirstName(String item){
encrypt('encryptedFirstName',item)
}
String getLastName(){
decrypt('encryptedLastName')
}
void setLastName(String item){
encrypt('encryptedLastName',item)
}
String getAddress(){
decrypt('encryptedAddress')
}
void setAddress(String item){
encrypt('encryptedAddress',item)
}
byte[] encrypt(String name, String value) {
if( null == value ) {
log.debug "null string to encrypt for '$name', returning null"
this.#"$name" = null
return
}
def bytes = value.getBytes(encrypter.ENCODING_CHARSET)
def method = getEncryptionMethod()
byte[] res
try {
res = encrypter.encrypt( bytes, method )
} catch(e) {
log.warn "Problem encrypting '$name' data: '$string'", e
}
log.trace "Encrypting '$name' with '$method' -> '${res?.size()}' bytes"
this.#"$name" = res
}
String decrypt(String name) {
if(null == this.#"$name") {
log.debug "null bytes to decrypt for '$name', returning null"
return null
}
def res
def method = getEncryptionMethod()
try {
res = new String(encrypter.decrypt(this.#"$name", method), encrypter.ENCODING_CHARSET )
} catch(e) {
log.error "Problem decrypting '$name'", e
}
log.trace "Decrypting '$name' with '$method' -> '${res?.size()}' bytes"
return res
}
}
Another option is to use a JDBC driver that encrypts/decrypts data on the fly, two way. Bear in mind that any solution will probably invalidate searches by encrypted fields.
IMHO the best solution is the one proposed by longneck, it will make everything much easier, from administration to development. Besides, bear in mind that any solution with client-side encryption will render all your db data unusable outside of the client, ie, you will not be able to use nice tools like a jdbc client or MySQL query browser, etc.
Jasypt integrates with Hibernate: http://jasypt.org/hibernate3.html. However, queries which use WHERE clauses cannot be used
Generated ids, version, mapped foreign keys - basically everything maintained by Hibernate - are out unless you intend to declare custom CRUD for all of your classes and manually encrypt them in queries.
For everything else you've got a couple of choices:
#PostLoad and #PrePersist entity listeners will take care of all non-query operations.
Implementing custom String / Long / Integer / etc... types to handle encryption will take care of both query and CRUD operations; however the mapping will become rather messy.
You can write a thin wrapper around a JDBC driver (as well as Connection / Statement / PreparedStatement / ResultSet / etc...) to do the encryption for you.
As far as queries go you'll have to handle encryption manually (unless you're going with #2 above) but you should be able to do so via a single entry point. I'm not sure how (or if) Grails deals with this, but using Spring, for example, it would be as easy as extending HibernateTemplate.