Trying to scrape a webpage, I hit the necessity to work with ASP.NET's __VIEWSTATE variables. So, ever the optimist, I decided to read up on those variables, and their formats. Even though classified as Open Source by Microsoft, I couldn't find any formal definition:
Everybody agrees the first step to do is decode the string, using a Base64 decoder. Great - that works...
Next - and this is where the confusion sets in:
Roughly 3/4 of the decoders seem to use binary values (characters whose values indicate the the type of field which is follow). Here's an example of such a specification. This format also seems to expect a 'signature' of 0xFF 0x01 as first two bytes.
The rest of the articles (such as this one) describe a format where the fields in the format are separated (or marked) by t< ... >, p< ... >, etc. (this seems to be the case of the page I'm interested in).
Even after looking at over a hundred pages, I didn't find any mention about the existence of two formats.
My questions are: Are there two different formats of __VIEWSTATE variables in use, or am I missing something basic? Is there any formal description of the __VIEWSTATE contents somewhere?
The view state is serialized and deserialized by the
System.Web.UI.LosFormatter class—the LOS stands for limited object
serialization—and is designed to efficiently serialize certain types
of objects into a base-64 encoded string. The LosFormatter can
serialize any type of object that can be serialized by the
BinaryFormatter class, but is built to efficiently serialize objects
of the following types:
Strings
Integers
Booleans
Arrays
ArrayLists
Hashtables
Pairs
Triplets
Everything you need to know about ViewState: Understanding View State
Related
I'm integrating two poorly documented systems, and in the process I've come across a strange data format I haven't seen before. It's stored as plain text in the db with no indication as to what the format is and how to deal with it.
a:17:{s:2:"id";s:27:"145219921F990B11C39E7220000";s:16:"purchase_country";s:2:"no";s:17:"purchase_currency";s:3:"nok";s:6:"locale";s:5:"nb-no";s:6:"status";s:17:"checkout_complete";s:9:"reference";s:27:"145212221F990B11C39E7221000";s:11:"reservation";s:10:"2348226550";s:10:"started_at";s:25:"2014-04-04T10:40:55+02:00";s:12:"completed_at";s:25:"2014-04-02T10:41:11+02:00";s:16:"last_modified_at";s:25:"2014-04-02T10:41:11+02:00";s:10:"expires_at";s:25:"2014-04-16T10:41:11+02:00";s:4:"cart";a:4:{s:25:"total_price_excluding_tax";i:489500;s:16:"total_tax_amount";i:0;s:25:"total_price_including_tax";i:489500;s:5:"items";a:2:{i:0;a:10:{s:9:"reference";s:2:"68";s:4:"name";s:21:"1.OSO SUPER S 200LIT.";s:8:"quantity";i:1;s:10:"unit_price";i:695000;s:8:"tax_rate";i:0;s:13:"discount_rate";i:0;s:4:"type";s:8:"physical";s:25:"total_price_including_tax";i:695500;s:25:"total_price_excluding_tax";i:694000;s:16:"total_tax_amount";i:0;}i:1;a:10:{s:9:"reference";s:2:"68";s:4:"name";s:32:"1.OSO SUPER S 200LIT. (discount)";s:8:"quantity";i:1;s:10:"unit_price";i:-205100;s:8:"tax_rate";i:0;s:13:"discount_rate";i:0;s:4:"type";s:8:"physical";s:25:"total_price_including_tax";i:-205100;s:25:"total_price_excluding_tax";i:-205100;s:16:"total_tax_amount";i:0;}}}s:8:"customer";a:1:{s:4:"type";s:6:"person";}s:16:"shipping_address";a:8:{s:10:"given_name";s:13:"Testperson-no";s:11:"family_name";s:8:"Approved";s:14:"street_address";s:18:"Sæffleberggate 56";s:11:"postal_code";s:4:"0563";s:4:"city";s:4:"OSLO";s:7:"country";s:2:"no";s:5:"email";s:32:"omitted#testdrive.klarna.com";s:5:"phone";s:11:"40 12 34 56";}s:15:"billing_address";a:8:{s:10:"given_name";s:13:"Testperson-no";s:11:"family_name";s:8:"Approved";s:14:"street_address";s:18:"Sæffleberggate 56";s:11:"postal_code";s:4:"0563";s:4:"city";s:4:"OSLO";s:7:"country";s:2:"no";s:5:"email";s:32:"checkout-no#testdrive.klarna.com";s:5:"phone";s:11:"40 12 34 56";}s:7:"options";a:1:{s:31:"allow_separate_shipping_address";b:0;}s:8:"merchant";a:5:{s:2:"id";s:4:"1601";s:9:"terms_uri";s:95:"omitted";s:12:"checkout_uri";s:59:"omitted";s:16:"confirmation_uri";s:220:"omitted";s:8:"push_uri";s:229:"omitted";}}
An entry consists of colon-separated segments:
A single char type tag (array, object, int, decimal, bool, string)
A number that says how long the value is in characters, bytes, elements (in case of arrays) or key-value pairs (in case of objs), which seems completely useless given that this is a textual format that requires me to parse the length segment anyway. This isn't present for integers and decimals.
Value of the field
Key-value pairs seem to be a flat list of an even number of elements. They also seem to be using arrays as objects as well (see example).
A ; terminator, which seems not to be necessary for objects and arrays, just to make parsing more tedious.
Now, parsing this thing is reasonably easy, but I'm constantly being surprised by new data types and their weird syntax and I'm not sure that I've covered all the edge cases with the few data samples I've analyzed. Is anyone familiar with this format?
Looks like PHP serialization. See: http://www.phpinternalsbook.com/classes_objects/serialization.html
If I have two properties:
foo=1
bar=2345
Is there a way to specify that foo is a number and bar is a string?
I assume: bar="2345" would do but I wonder if there's a widely accepted convention
A properties file is a text file which contains data in some standard format, which can be read by the application using it. It is mostly used for configuration of the application and also for internationalization.
As per the wiki document https://en.wikipedia.org/wiki/.properties
Each parameter is stored as a pair of strings, one storing the name of
the parameter (called the key), and the other storing the value.
There is no way to specify / force the value to be number or string only (instead it is always a string). It is majorly the functionality of the framework / application which; while reading the properties file tries to parse the values. If it fails to parse the value (of certain specific type like number) it may fallback to some default value or will simply terminate the program.
I want to use query string as json, for example: /api/uri?{"key":"value"}, instead of /api/uri?key=value. Advantages, from my point of view, are:
json keep types of parameters, such are booleans, ints, floats and strings. Standard query string treats all parameters as strings.
json has less symbols for deep nested structures, For example, ?{"a":{"b":{"c":[1,2,3]}}} vs ?a[b][c][]=1&a[b][c][]=2&a[b][c][]=3
Easier to build on client side
What disadvantages could be in that case of json usage?
It looks good if it's
/api/uri?{"key":"value"}
as stated in your example, but since it's part of the URL then it gets encoded to:
/api/uri?%3F%7B%22key%22%3A%22value%22%7D
or something similar which makes /api/uri?key=value simpler than the encoded one; in this case both for debugging outbound calls (ie you want to check the actual request via wireshark etc). Also notice that it takes up more characters when encoded to valid url (see browser limitation).
For the case of 'lesser symbols for nested structures', It might be more appropriate to create a new resource for your sub resource where you will handle the filtering through your query parameters;
api/a?{"b":1,}
to
api/b?bfield1=1
api/a?aBfield1=1 // or something similar
Lastly for the 'easier to build in client side', i think it depends on what you use to create your client, usually query params are represented as maps so it is still simple.
Also if you need a collection for a single param then:
/uri/resource?param1=value1,value2,value3
What are the major problems of mapping JSON to XML and viceversa? I have a set of problems that I can run into, but it would be very helpful if others can add what they have ran into when converting between both.
My list is:
Root object required in JSON
Unique keys (although only one of the two specifications requires this)
Keys cannot start with a number
Order may not be preserved (see http://www.xml.com/pub/a/2006/05/31/converting-between-xml-and-json.html)
Any other one?
Disclaimer: I am the author of Jsonix, a XML<->JSON conversion library written in JavaScript. So I'm speaking a bit from experience of mapping between complex XML and JSON.
Top-level production in JSON may be JSONArray or JSONObject (in JSON interchange format even any JSONText - also null, boolean, string, number). XML requires a single root element.
JSON objects have properties, XML elements may have attributes, contain sub-elements and text values (I'm even leaving comments and PIs out).
You're mentioning "keys cannot start with a number", but there's more syntactical incompatibilities. JSON object properties can be basically any strings. XML element and attribute names are restricted in syntax.
Normally no namespaces in JSON, often namespaces in XML.
Strict typing. You always know JSON type just by looking at the value. In XML, you can't guess type from the value. For instance 1 may be string, boolean, a dozen of numeric types etc. You have to know the schema to know types.
In JSON, you can guess the structure from value (object or array). In XML, if you see an single element, you don't know if it may be repeated or not. You have to know the schema to know the structure.
Collections are normally expressed as arrays in JSON. In XML, you can express a collection as repeatable elements (item*), possibly wrapped (items/item*), or in case of simple types as list types (<items>a b c d</items>).
In XML, the order of sub-elements or text nodes of the element is significant. In JSON, properties of the JSONObject are not ordered. (You mention this.)
In XML, an element may contain several sub-elements of the same name. In JSONObject, property names will be unique. (You mention this.)
In XML, an element may contain attributes, sub-elements and text nodes. In JSON, the only complex structures are JSONObject and JSONArray. In JSONArray you just have items, no named components (which would be analogous to attributes or sub-elements). In JSONObject you just have properties (JSONMembers) which are always "named" (this would be analogous to attributes and sub-elements of XML, but not to text nodes).
Processing instructions and comments in XML, no direct analogs in JSON.
There's also xsi:type construct which is a bit hard to handle. Specifies the type of the element value in the document instance.
In XML, values of certain types (like QNames) depend on the declarations in other parts of the XML document. For example, having my:Element as xs:QName-value somewhere, this value will depend on how the my namespace prefix is declared in the document. Since namespaces may be declared and re-declared, you have to follow their declaraition quite precisely to be able to find out the namespace of the qualified name.
Converting a specific JSON object (or class of objects) into XML is usually no problem at all. What is difficult is writing a converter that can handle any JSON object. The problem essentially arises because you want simple JSON to end up as simple XML, but you find yourself contorting the design to handle edge cases, such as characters that are legal in JSON but not in XML, preserving distinctions such as the distinction between the number 10 and the string "10", or worrying about the best representation of a JSON "null".
I have a naming issue.
If I read an object x from some JSON I can call my variable xJson (or some variation). However sometimes it is possible that the data could have come from a number of different sources amongst which JSON is not special (e.g. XMLRPC, programmatically constructed from Maps,Lists & primitives ... etc).
In this situation what is a good name for the variable? The best I have come up with so far is something like 'DynamicData', which is ok in some situations, but is a bit long and not probably not very clear to people unfamiliar with the convention.
SerializedData?
A hierarchical collection of hashes and lists of data is often referred to as a document no matter what serialization format is used. Another useful description might be payload or body in the sense of a message body for transmission or a value string written to a key/value store.
I tend to call the object hierarchy a "doc" myself, and the serialized format a "document." Thus a RequestDocument is parsed into a RequestDoc, and upon further identification it might become an OrderDoc, or a CustomerUpdateDoc, etc. An InvoiceDoc might become known generically as a ResponseDoc eventually serialized to a ResponseDocument.
The longer form is awkward, but such serialized strings are typically short-lived and localized in the code anyway.
If your data is the model, name it after the model it's representing. e.g., name it after the purpose of the contents, not the format of the contents. So if it's a list of customer information, name it "customers", or "customerModel", or something like that.
If you don't know what the contents are, the name isn't important, unless you want to differentiate the format. "responseData", "jsonResponse", etc...
And "DynamicData" isn't a long name, unless there is absolutely nothing descriptive to be said about the data. "data" might be just fine.