i have extracted the fix message as below from Unix server and now need to convert this message into JSON. how can we do this?
8=FIXT.1.1|9=449|11=ABCD1|35=AE|34=1734|49=REPOFIXUAT|52=20140402-11:38:34|56=TR_UAT_VENDOR|1128=8|15=GBP|31=1.7666|32=50000000.00|55=GBP/USD|60=20140402-11:07:33|63=B|64=20140415|65=OR|75=20140402|150=F|167=FOR|194=1.7654|195=0.0012|460=4|571=7852455|1003=2 USD|1056=88330000.00|1057=N|552=1|54=2|37=20140402-12:36:48|11=NOREF|453=4|448=ZERO|447=D|452=3|448=MBY2|447=D|452=1|448=LMEB|447=D|452=16|448=DOR|447=D|452=11|826=0|78=1|79=default|80=50000000.00|5967=88330000.00|10=111
Note: I tried to make this a comment on the answer provided by #selbie, but the text was too long for a comment, so I am making it an answer.
#selbie's answer will work most of the time, but there are two edge cases in which it could fail.
First, in a tag=value field where the value is of type STRING, it is legal for value to contain the = character. To correctly cope with this possibility, the Java statement:
pair = item.split("=");
should be changed to:
pair = item.split("=", 2);
The second edge case is when there are a pair of fields, the first of which is of type LENGTH and the second is of type DATA. In this case, the value of the LENGTH fields specifies the length of the DATA field (without the delimiter), and it is legal for the value of the DATA field to contain the delimiter character (ASCII character 1, but denoted as | in both the question and Selbie's answer). Selbie's code cannot be modified in a trivial manner to deal with this edge case. Instead, you will need a more complex algorithm that consults a FIX data dictionary to determine the type of each field.
Since you didn't tag your question for any particular programming language, I'll give you a few sample solutions:
In javascript:
let s = "8=FIXT.1.1|9=449|11=ABCD1|35=AE|34=1734|49=REPOFIXUAT|52=20140402-11:38:34|56=TR_UAT_VENDOR|1128=8|15=GBP|31=1.7666|32=50000000.00|55=GBP/USD|60=20140402-11:07:33|63=B|64=20140415|65=OR|75=20140402|150=F|167=FOR|194=1.7654|195=0.0012|460=4|571=7852455|1003=2 USD|1056=88330000.00|1057=N|552=1|54=2|37=20140402-12:36:48|11=NOREF|453=4|448=ZERO|447=D|452=3|448=MBY2|447=D|452=1|448=LMEB|447=D|452=16|448=DOR|447=D|452=11|826=0|78=1|79=default|80=50000000.00|5967=88330000.00|10=111"
let obj = {};
items = s.split("|")
items.forEach(item=>{
let pair = item.split("=");
obj[pair[0]] = pair[1];
});
let jsonString = JSON.stringify(obj);
Python:
import json
s = "8=FIXT.1.1|9=449|11=ABCD1|35=AE|34=1734|49=REPOFIXUAT|52=20140402-11:38:34|56=TR_UAT_VENDOR|1128=8|15=GBP|31=1.7666|32=50000000.00|55=GBP/USD|60=20140402-11:07:33|63=B|64=20140415|65=OR|75=20140402|150=F|167=FOR|194=1.7654|195=0.0012|460=4|571=7852455|1003=2 USD|1056=88330000.00|1057=N|552=1|54=2|37=20140402-12:36:48|11=NOREF|453=4|448=ZERO|447=D|452=3|448=MBY2|447=D|452=1|448=LMEB|447=D|452=16|448=DOR|447=D|452=11|826=0|78=1|79=default|80=50000000.00|5967=88330000.00|10=111"
obj = {}
for item in s.split("|"):
pair = item.split("=")
obj[pair[0]] = pair[1]
jsonString = json.dumps(obj)
Porting the above solutions to other languages is an exercise for yourself. There's comments below about semantic ordering and handling cases where the the = or | chars are part of the content. That's on you to explore if you need to support those scenarios.
First week having to deal with a MYSQL database and JSON field types and I cannot seem to figure out why values are encoded automatically and then returned in encoded format.
Given the following SQL
-- create a multiline string with a tab example
SET #str ="Line One
Line 2 Tabbed out
Line 3";
-- encode it
SET #j = JSON_OBJECT("str", #str);
-- extract the value by name
SET #strOut = JSON_EXTRACT(#J, "$.str");
-- show the object and attribute value.
SELECT #j, #strOut;
You end up with what appears to be a full formed JSON object with a single attribute encoded.
#j = {"str": "Line One\n\tLine 2\tTabbed out\n\tLine 3"}
but using JSON_EXTRACT to get the attribute value I get the encoded version including outer quotes.
#strOut = "Line One\n\tLine 2\tTabbed out\n\tLine 3"
I would expect to get my original string with the \n \t all unescaped to the original values and no outer quotes. as such
Line One
Line 2 Tabbed out
Line 3
I can't seem to find any JSON_DECODE or JSON_UNESCAPE or similar functions.
I did find a JSON_ESCAPE() function but that appears to be used to manually build a JSON object structure in a string.
What am I missing to extract the values to the original format?
I like to use handy operator ->> for this.
It was introduced in MySQL 5.7.13, and basically combines JSON_EXTRACT() and JSON_UNQUOTE():
SET #strOut = #J ->> '$.str';
You are looking for the JSON_UNQUOTE function
SET #strOut = JSON_UNQUOTE( JSON_EXTRACT(#J, "$.str") );
The result of JSON_EXTRACT() is intentionally a JSON document, not a string.
A JSON document may be:
An object enclosed in { }
An array enclosed in [ ]
A scalar string value enclosed in " "
A scalar number or boolean value
A null — but this is not an SQL NULL, it's a JSON null. This leads to confusing cases because you can extract a JSON field whose JSON value is null, and yet in an SQL expression, this fails IS NULL tests, and it also fails to be equal to an SQL string 'null'. Because it's a JSON type, not a scalar type.
Is there any way to initialize a Sequential value not in one fellow swoop?
Like, can I declare it, then use a for loop to populate it, step by step?
As this could all happen inside a class body, the true immutability of the Sequential value could then kick in once the class instance construction phase has been completed.
Example:
Sequential<String> strSeq;
for (i in span(0,10)) {
strSeq[i] = "hello";
}
This code doesn't work, as I get this error:
Error:(12, 9) ceylon: illegal receiving type for index expression:
'Sequential' is not a subtype of 'KeyedCorrespondenceMutator' or
'IndexedCorrespondenceMutator'
So what I can conclude is that sequences must be assigned in one statement, right?
Yes, several language guarantees hinge on the immutability of sequential objects, so that immutability must be guaranteed by the language – it can’t just trust you that you won’t mutate it after the initialization is done :)
Typically, what you do in this situation is construct some sort of collection (e. g. an ArrayList from ceylon.collection), mutate it however you want, and then take its .sequence() when you’re done.
Your specific case can also be written as a comprehension in a sequential literal:
String[] strSeq = [for (i in 0..10) "hello"];
The square brackets used to create a sequence literal accept not only a comma-separated list of values, but also a for-comprehension:
String[] strSeq = [for (i in 0..10) "hello"];
You can also do both at the same time, as long as the for-comprehension comes last:
String[] strSeq = ["hello", "hello", for (i in 0..8) "hello"];
In this specific case, you could also do this:
String[] strSeq = ["hello"].repeat(11);
You can also get a sequence of sequences via nesting:
String[][] strSeqSeq = [for (i in 0..2) [for (j in 0..2) "hello"]];
And you can do the cartesian product (notice that the nested for-comprehension here isn't in square brackets):
[Integer, Character][] pairs = [for (i in 0..2) for (j in "abc") [i, j]];
Foo[] is an abbreviation for Sequential<Foo>, and x..y translates to span(x, y).
If you know upfront the size of the sequence you want to create, then a very efficient way is to use an Array:
value array = Array.ofSize(11, "");
for (i in 0:11) {
array[i] = "hello";
}
String[] strSeq = array.sequence();
On the other hand, if you don't know the size upfront, then, as described by Lucas, you need to use either:
a comprehension, or
some sort of growable array, like ArrayList.
This question's answers are a community effort. Edit existing answers to improve this post. It is not currently accepting new answers or interactions.
I've been using the == operator in my program to compare all my strings so far.
However, I ran into a bug, changed one of them into .equals() instead, and it fixed the bug.
Is == bad? When should it and should it not be used? What's the difference?
== tests for reference equality (whether they are the same object).
.equals() tests for value equality (whether they contain the same data).
Objects.equals() checks for null before calling .equals() so you don't have to (available as of JDK7, also available in Guava).
Consequently, if you want to test whether two strings have the same value you will probably want to use Objects.equals().
// These two have the same value
new String("test").equals("test") // --> true
// ... but they are not the same object
new String("test") == "test" // --> false
// ... neither are these
new String("test") == new String("test") // --> false
// ... but these are because literals are interned by
// the compiler and thus refer to the same object
"test" == "test" // --> true
// ... string literals are concatenated by the compiler
// and the results are interned.
"test" == "te" + "st" // --> true
// ... but you should really just call Objects.equals()
Objects.equals("test", new String("test")) // --> true
Objects.equals(null, "test") // --> false
Objects.equals(null, null) // --> true
You almost always want to use Objects.equals(). In the rare situation where you know you're dealing with interned strings, you can use ==.
From JLS 3.10.5. String Literals:
Moreover, a string literal always refers to the same instance of class String. This is because string literals - or, more generally, strings that are the values of constant expressions (§15.28) - are "interned" so as to share unique instances, using the method String.intern.
Similar examples can also be found in JLS 3.10.5-1.
Other Methods To Consider
String.equalsIgnoreCase() value equality that ignores case. Beware, however, that this method can have unexpected results in various locale-related cases, see this question.
String.contentEquals() compares the content of the String with the content of any CharSequence (available since Java 1.5). Saves you from having to turn your StringBuffer, etc into a String before doing the equality comparison, but leaves the null checking to you.
== tests object references, .equals() tests the string values.
Sometimes it looks as if == compares values, because Java does some behind-the-scenes stuff to make sure identical in-line strings are actually the same object.
For example:
String fooString1 = new String("foo");
String fooString2 = new String("foo");
// Evaluates to false
fooString1 == fooString2;
// Evaluates to true
fooString1.equals(fooString2);
// Evaluates to true, because Java uses the same object
"bar" == "bar";
But beware of nulls!
== handles null strings fine, but calling .equals() from a null string will cause an exception:
String nullString1 = null;
String nullString2 = null;
// Evaluates to true
System.out.print(nullString1 == nullString2);
// Throws a NullPointerException
System.out.print(nullString1.equals(nullString2));
So if you know that fooString1 may be null, tell the reader that by writing
System.out.print(fooString1 != null && fooString1.equals("bar"));
The following are shorter, but it’s less obvious that it checks for null:
System.out.print("bar".equals(fooString1)); // "bar" is never null
System.out.print(Objects.equals(fooString1, "bar")); // Java 7 required
== compares Object references.
.equals() compares String values.
Sometimes == gives illusions of comparing String values, as in following cases:
String a="Test";
String b="Test";
if(a==b) ===> true
This is because when you create any String literal, the JVM first searches for that literal in the String pool, and if it finds a match, that same reference will be given to the new String. Because of this, we get:
(a==b) ===> true
String Pool
b -----------------> "test" <-----------------a
However, == fails in the following case:
String a="test";
String b=new String("test");
if (a==b) ===> false
In this case for new String("test") the statement new String will be created on the heap, and that reference will be given to b, so b will be given a reference on the heap, not in String pool.
Now a is pointing to a String in the String pool while b is pointing to a String on the heap. Because of that we get:
if(a==b) ===> false.
String Pool
"test" <-------------------- a
Heap
"test" <-------------------- b
While .equals() always compares a value of String so it gives true in both cases:
String a="Test";
String b="Test";
if(a.equals(b)) ===> true
String a="test";
String b=new String("test");
if(a.equals(b)) ===> true
So using .equals() is always better.
The == operator checks to see if the two strings are exactly the same object.
The .equals() method will check if the two strings have the same value.
Strings in Java are immutable. That means whenever you try to change/modify the string you get a new instance. You cannot change the original string. This has been done so that these string instances can be cached. A typical program contains a lot of string references and caching these instances can decrease the memory footprint and increase the performance of the program.
When using == operator for string comparison you are not comparing the contents of the string, but are actually comparing the memory address. If they are both equal it will return true and false otherwise. Whereas equals in string compares the string contents.
So the question is if all the strings are cached in the system, how come == returns false whereas equals return true? Well, this is possible. If you make a new string like String str = new String("Testing") you end up creating a new string in the cache even if the cache already contains a string having the same content. In short "MyString" == new String("MyString") will always return false.
Java also talks about the function intern() that can be used on a string to make it part of the cache so "MyString" == new String("MyString").intern() will return true.
Note: == operator is much faster than equals just because you are comparing two memory addresses, but you need to be sure that the code isn't creating new String instances in the code. Otherwise you will encounter bugs.
String a = new String("foo");
String b = new String("foo");
System.out.println(a == b); // prints false
System.out.println(a.equals(b)); // prints true
Make sure you understand why. It's because the == comparison only compares references; the equals() method does a character-by-character comparison of the contents.
When you call new for a and b, each one gets a new reference that points to the "foo" in the string table. The references are different, but the content is the same.
Yea, it's bad...
== means that your two string references are exactly the same object. You may have heard that this is the case because Java keeps sort of a literal table (which it does), but that is not always the case. Some strings are loaded in different ways, constructed from other strings, etc., so you must never assume that two identical strings are stored in the same location.
Equals does the real comparison for you.
Yes, == is bad for comparing Strings (any objects really, unless you know they're canonical). == just compares object references. .equals() tests for equality. For Strings, often they'll be the same but as you've discovered, that's not guaranteed always.
Java have a String pool under which Java manages the memory allocation for the String objects. See String Pools in Java
When you check (compare) two objects using the == operator it compares the address equality into the string-pool. If the two String objects have the same address references then it returns true, otherwise false. But if you want to compare the contents of two String objects then you must override the equals method.
equals is actually the method of the Object class, but it is Overridden into the String class and a new definition is given which compares the contents of object.
Example:
stringObjectOne.equals(stringObjectTwo);
But mind it respects the case of String. If you want case insensitive compare then you must go for the equalsIgnoreCase method of the String class.
Let's See:
String one = "HELLO";
String two = "HELLO";
String three = new String("HELLO");
String four = "hello";
one == two; // TRUE
one == three; // FALSE
one == four; // FALSE
one.equals(two); // TRUE
one.equals(three); // TRUE
one.equals(four); // FALSE
one.equalsIgnoreCase(four); // TRUE
I agree with the answer from zacherates.
But what you can do is to call intern() on your non-literal strings.
From zacherates example:
// ... but they are not the same object
new String("test") == "test" ==> false
If you intern the non-literal String equality is true:
new String("test").intern() == "test" ==> true
== compares object references in Java, and that is no exception for String objects.
For comparing the actual contents of objects (including String), one must use the equals method.
If a comparison of two String objects using == turns out to be true, that is because the String objects were interned, and the Java Virtual Machine is having multiple references point to the same instance of String. One should not expect that comparing one String object containing the same contents as another String object using == to evaluate as true.
.equals() compares the data in a class (assuming the function is implemented).
== compares pointer locations (location of the object in memory).
== returns true if both objects (NOT TALKING ABOUT PRIMITIVES) point to the SAME object instance.
.equals() returns true if the two objects contain the same data equals() Versus == in Java
That may help you.
== performs a reference equality check, whether the 2 objects (strings in this case) refer to the same object in the memory.
The equals() method will check whether the contents or the states of 2 objects are the same.
Obviously == is faster, but will (might) give false results in many cases if you just want to tell if 2 Strings hold the same text.
Definitely the use of the equals() method is recommended.
Don't worry about the performance. Some things to encourage using String.equals():
Implementation of String.equals() first checks for reference equality (using ==), and if the 2 strings are the same by reference, no further calculation is performed!
If the 2 string references are not the same, String.equals() will next check the lengths of the strings. This is also a fast operation because the String class stores the length of the string, no need to count the characters or code points. If the lengths differ, no further check is performed, we know they cannot be equal.
Only if we got this far will the contents of the 2 strings be actually compared, and this will be a short-hand comparison: not all the characters will be compared, if we find a mismatching character (at the same position in the 2 strings), no further characters will be checked.
When all is said and done, even if we have a guarantee that the strings are interns, using the equals() method is still not that overhead that one might think, definitely the recommended way. If you want an efficient reference check, then use enums where it is guaranteed by the language specification and implementation that the same enum value will be the same object (by reference).
If you're like me, when I first started using Java, I wanted to use the "==" operator to test whether two String instances were equal, but for better or worse, that's not the correct way to do it in Java.
In this tutorial I'll demonstrate several different ways to correctly compare Java strings, starting with the approach I use most of the time. At the end of this Java String comparison tutorial I'll also discuss why the "==" operator doesn't work when comparing Java strings.
Option 1: Java String comparison with the equals method
Most of the time (maybe 95% of the time) I compare strings with the equals method of the Java String class, like this:
if (string1.equals(string2))
This String equals method looks at the two Java strings, and if they contain the exact same string of characters, they are considered equal.
Taking a look at a quick String comparison example with the equals method, if the following test were run, the two strings would not be considered equal because the characters are not the exactly the same (the case of the characters is different):
String string1 = "foo";
String string2 = "FOO";
if (string1.equals(string2))
{
// this line will not print because the
// java string equals method returns false:
System.out.println("The two strings are the same.")
}
But, when the two strings contain the exact same string of characters, the equals method will return true, as in this example:
String string1 = "foo";
String string2 = "foo";
// test for equality with the java string equals method
if (string1.equals(string2))
{
// this line WILL print
System.out.println("The two strings are the same.")
}
Option 2: String comparison with the equalsIgnoreCase method
In some string comparison tests you'll want to ignore whether the strings are uppercase or lowercase. When you want to test your strings for equality in this case-insensitive manner, use the equalsIgnoreCase method of the String class, like this:
String string1 = "foo";
String string2 = "FOO";
// java string compare while ignoring case
if (string1.equalsIgnoreCase(string2))
{
// this line WILL print
System.out.println("Ignoring case, the two strings are the same.")
}
Option 3: Java String comparison with the compareTo method
There is also a third, less common way to compare Java strings, and that's with the String class compareTo method. If the two strings are exactly the same, the compareTo method will return a value of 0 (zero). Here's a quick example of what this String comparison approach looks like:
String string1 = "foo bar";
String string2 = "foo bar";
// java string compare example
if (string1.compareTo(string2) == 0)
{
// this line WILL print
System.out.println("The two strings are the same.")
}
While I'm writing about this concept of equality in Java, it's important to note that the Java language includes an equals method in the base Java Object class. Whenever you're creating your own objects and you want to provide a means to see if two instances of your object are "equal", you should override (and implement) this equals method in your class (in the same way the Java language provides this equality/comparison behavior in the String equals method).
You may want to have a look at this ==, .equals(), compareTo(), and compare()
Function:
public float simpleSimilarity(String u, String v) {
String[] a = u.split(" ");
String[] b = v.split(" ");
long correct = 0;
int minLen = Math.min(a.length, b.length);
for (int i = 0; i < minLen; i++) {
String aa = a[i];
String bb = b[i];
int minWordLength = Math.min(aa.length(), bb.length());
for (int j = 0; j < minWordLength; j++) {
if (aa.charAt(j) == bb.charAt(j)) {
correct++;
}
}
}
return (float) (((double) correct) / Math.max(u.length(), v.length()));
}
Test:
String a = "This is the first string.";
String b = "this is not 1st string!";
// for exact string comparison, use .equals
boolean exact = a.equals(b);
// For similarity check, there are libraries for this
// Here I'll try a simple example I wrote
float similarity = simple_similarity(a,b);
The == operator check if the two references point to the same object or not. .equals() check for the actual string content (value).
Note that the .equals() method belongs to class Object (super class of all classes). You need to override it as per you class requirement, but for String it is already implemented, and it checks whether two strings have the same value or not.
Case 1
String s1 = "Stack Overflow";
String s2 = "Stack Overflow";
s1 == s2; //true
s1.equals(s2); //true
Reason: String literals created without null are stored in the String pool in the permgen area of heap. So both s1 and s2 point to same object in the pool.
Case 2
String s1 = new String("Stack Overflow");
String s2 = new String("Stack Overflow");
s1 == s2; //false
s1.equals(s2); //true
Reason: If you create a String object using the new keyword a separate space is allocated to it on the heap.
== compares the reference value of objects whereas the equals() method present in the java.lang.String class compares the contents of the String object (to another object).
I think that when you define a String you define an object. So you need to use .equals(). When you use primitive data types you use == but with String (and any object) you must use .equals().
If the equals() method is present in the java.lang.Object class, and it is expected to check for the equivalence of the state of objects! That means, the contents of the objects. Whereas the == operator is expected to check the actual object instances are same or not.
Example
Consider two different reference variables, str1 and str2:
str1 = new String("abc");
str2 = new String("abc");
If you use the equals()
System.out.println((str1.equals(str2))?"TRUE":"FALSE");
You will get the output as TRUE if you use ==.
System.out.println((str1==str2) ? "TRUE" : "FALSE");
Now you will get the FALSE as output, because both str1 and str2 are pointing to two different objects even though both of them share the same string content. It is because of new String() a new object is created every time.
Operator == is always meant for object reference comparison, whereas the String class .equals() method is overridden for content comparison:
String s1 = new String("abc");
String s2 = new String("abc");
System.out.println(s1 == s2); // It prints false (reference comparison)
System.out.println(s1.equals(s2)); // It prints true (content comparison)
All objects are guaranteed to have a .equals() method since Object contains a method, .equals(), that returns a boolean. It is the subclass' job to override this method if a further defining definition is required. Without it (i.e. using ==) only memory addresses are checked between two objects for equality. String overrides this .equals() method and instead of using the memory address it returns the comparison of strings at the character level for equality.
A key note is that strings are stored in one lump pool so once a string is created it is forever stored in a program at the same address. Strings do not change, they are immutable. This is why it is a bad idea to use regular string concatenation if you have a serious of amount of string processing to do. Instead you would use the StringBuilder classes provided. Remember the pointers to this string can change and if you were interested to see if two pointers were the same == would be a fine way to go. Strings themselves do not.
You can also use the compareTo() method to compare two Strings. If the compareTo result is 0, then the two strings are equal, otherwise the strings being compared are not equal.
The == compares the references and does not compare the actual strings. If you did create every string using new String(somestring).intern() then you can use the == operator to compare two strings, otherwise equals() or compareTo methods can only be used.
In Java, when the == operator is used to compare 2 objects, it checks to see if the objects refer to the same place in memory. In other words, it checks to see if the 2 object names are basically references to the same memory location.
The Java String class actually overrides the default equals() implementation in the Object class – and it overrides the method so that it checks only the values of the strings, not their locations in memory.
This means that if you call the equals() method to compare 2 String objects, then as long as the actual sequence of characters is equal, both objects are considered equal.
The == operator checks if the two strings are exactly the same object.
The .equals() method check if the two strings have the same value.