XSLT has (amongst other things) what seems to be a pretty unique programming model that supports this 'pattern'
(this example take from the answer to XSLT Identity template overriding)
<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:axsl="http://www.w3.org/1999/XSL/TransformAlias">
<xsl:output method="xml" version="1.0" encoding="UTF-8" indent="yes" />
<xsl:strip-space elements="*"/>
<xsl:namespace-alias stylesheet-prefix="axsl" result-prefix="xsl" />
<!-- identity transform -->
<xsl:template match="#*|node()">
<xsl:copy>
<xsl:apply-templates select="#*|node()"/>
</xsl:copy>
</xsl:template>
<xsl:template match="xsl:object[Time ='00000000']"/>
</xsl:stylesheet>
i.e. the ability to clone (map) an existing data structure based on an recursive identity template, but map certain subtrees based on subsequent overriding templates
so here the the mapping will remove anything matching "xsl:object[Time ='00000000']"
If we think of similar constructs in (statically typed) languages these seems similar to a Functor, where certain predefined parts of a data structure are parametised (up front), and then a 'map' is defined to map this parametised part.
The interesting thing about apply-templates though is this isnt done up front, the 'program' will map any data structure, defaulting to the identity function if no overrides match, but no up front, definition of the structure happens.
Is there any other languages that have this sort of thing?
F# has a library called 'typeshape' that can typically be used to recursively process and 'map' data structures (though i need to actually try it to see where the limits are),
any more?
static typing would seem to be an obsticle? in terms of types, the algorithm can transform the type of the structure in seemingly complex ways that a static type system would find hard to describe (without resorting to dependent types).
Related
I am on a red hat system and I have multiple XML files generated from various SOAP requests that are in a format that is not compatible with MySQL's LoadXML function. I need to load the data into MySQL tables. One table will be setup for each type of XML file, depending on the data received via the Soap XML API.
Sample format of one of the files is as this, but each file will have a different number of columns and different column names. I am trying to find a way to convert them to a compatible format in the most generic way possible since I will have to create any customized solution for each API request/response.
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Body>
<dbd:DataRetrievalRequestResponse xmlns:dbd="dbd.v1">
<DataObjects>
<ObjectSelect>
<mdNm>controller-ac</mdNm>
<meNm>WALL-EQPT-A</meNm>
</ObjectSelect>
<DataInstances>
<DataInstance>
<instanceId>DSS1</instanceId>
<Attribute>
<name>Name</name>
<value>DSS1</value>
</Attribute>
<Attribute>
<name>Operational Mode</name>
<value>mode-fast</value>
</Attribute>
<Attribute>
<name>Rate - Down</name>
<value>1099289</value>
</Attribute>
<Attribute>
<name>Rate - Up</name>
<value>1479899</value>
</Attribute>
</DataInstance>
<DataInstance>
<instanceId>DSS2</instanceId>
<Attribute>
<name>Name</name>
<value>DSS2</value>
</Attribute>
<Attribute>
<name>Operational Mode</name>
<value>mode-fast</value>
</Attribute>
<Attribute>
<name>Rate - Down</name>
<value>1299433</value>
</Attribute>
<Attribute>
<name>Rate - Up</name>
<value>1379823</value>
</Attribute>
</DataInstance>
</DataInstances>
</DataObjects>
</dbd:DataRetrievalRequestResponse>
</soap:Body>
</soap:Envelope>
Of course I want the data to be entered into a mysql table with column names 'id, Name, Group' rows for each unique instance
Name
Operational Mode
Rate - Down
Rate - Up
DSS1
mode-fast
1099289
1479899
DSS2
mode-fast
1299433
1379823
Do I need to create an XSLT and preprocess this XML data from command line prior to running it to LoadXML to get it into a format that MySQL LoadXML function will accept? This would not be a problem, but I am not familiar with XSLT transformations.
Is there a way to reformat the above XML to straight CSV (preferred), or to another XML format that is compatible, such as the examples given in mysql documentation for loadxml?
<row>
<field name='column1'>value1</field>
<field name='column2'>value2</field>
</row>
I tried doing LOAD DATA INFILE and using ExtractValue function, but some of the values have spaces in them, and the delimiter for ExtractValue is hard coded to single-space. This makes it unusable as a workaround.
Your question is very general (which is fine!) so my answer is also quite general.
Firstly, it's certainly true that XSLT is an ideal generic tool for problems of this sort. I have absolutely no doubt that every one of your SOAP messages could be coerced into a suitable form, using an XSLT that's customised for each type of message, while still remaining structurally very similar, which is what you'd want if you're new to XSLT.
I'm not sure how familiar you are with XPath, XML, XML namespaces, etc, but I think the task here is simple enough to tackle, and if you do have any tricky XPath expressions to write you can always come back to StackOverflow and ask for help.
From what you've said it sounds like you're confident that each SOAP message can be mapped to a single table. I'm going to suggest an XSLT pattern that would be customisable for each type of SOAP message, where you have an xsl:for-each statement that iterates over each row, and within that you create a row element and populate it with fields.
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
<!-- indent the output, for ease of reading -->
<xsl:output indent="yes"/>
<!-- process the document -->
<xsl:template match="/">
<!-- create the root element of the output -->
<resultset>
<!-- create each row of the output, by iterating over the
repeating elements in the SOAP message -->
<xsl:for-each
select="//DataInstance">
<row>
<!-- create each field -->
<!-- This field is defined individually, and the value
is produced by evaluating the 'instanceId' xpath
relative to the current DataInstance -->
<field name="id"><xsl:value-of select="instanceId"/></field>
<!-- these field can be generated with a loop -->
<xsl:for-each select="Attribute">
<field name="{name}"><xsl:value-of select="value"/></field>
</xsl:for-each>
</row>
</xsl:for-each>
</resultset>
</xsl:template>
</xsl:stylesheet>
Result of this, run over your sample SOAP message:
<resultset>
<row>
<field name="id">DSS1</field>
<field name="Name">DSS1</field>
<field name="Operational Mode">mode-fast</field>
<field name="Rate - Down">1099289</field>
<field name="Rate - Up">1479899</field>
</row>
<row>
<field name="id">DSS2</field>
<field name="Name">DSS2</field>
<field name="Operational Mode">mode-fast</field>
<field name="Rate - Down">1299433</field>
<field name="Rate - Up">1379823</field>
</row>
</resultset>
If you can follow this general pattern, you should be able to write a custom XSLT for every kind of SOAP message in your collection. You will just need to modify the various XPath expressions in the stylesheet:
//DataInstance means "every DataInstance"
instanceId means "the instanceId that's a child of the current ("context") element.
name means "the name element that's a child of the current element.
value means "the value element that's a child of the current element.
In the example SOAP message you gave, the Attribute element maps to a field, so all those elements could be copied generically, with another xsl:for-each, but for your other documents you may have to just define each field element individually, as I did for the id element in my answer.
I'm working with a server.xml file...
Case 1:
<?xml version="1.0" encoding="UTF-8"?>
<Resource name="${app.name}" />
In catalina.properties i have declared the app.name
app.name=or
Case 2:
<?xml version="1.0" encoding="UTF-8"?>
<Resource name="or" />
The problem is why case 2 is working and case 1 not?
Why in case 1 XML entities not parsing?
I.e the output is :
<Resource name= "or" /> //in case 1
<Resource name= "or" /> //in case 2
Key point: Entity expansion happens during XML parsing.
Case 1
In case 1, during parsing, there are no entities in Resources/#name – just ${app.name}, which the program calling the XML parser would presumably go on to substitute the literal text, or, for the variable:
<Resource name="or" />
Downstream processing likely doesn't know how to deal with or, and you have your "not working" case.
Case 2
In case 2, or exists in the XML file prior to parsing. After parsing, effectively, the program calling the XML parser sees the entities expanded:
<Resource name="or" />
and is able to "work" because it knows what to do when #name is "or".
Note that had catalina.properties been an XML file, the expansion would have occurred then that file was parsed, and you'd be back to your "working" case.
Solution
Options include one of the following:
Hardwire the entities in server.xml rather than in catalina.properties.
Force the property substitution to happen prior to XML parsing of server.xml.
Use Unicode characters directly (not encoded as XML entities) in your catalina.properties file.
I have a JSON file to convert into XML format with below fields. For "Formats" and "MediaFormats", they are list of integers.
"Stars": 4.5000000000,
"Reviews": 11,
"Formats": [5,6],
"MediaFormats": [1, 2]
My expected result is
<Price>29</Price>
<Stars>4.5</Stars>
<Reviews>11</Reviews>
<Formats>5,6</Formats>
<MediaFormats>1,2</MediaFormats>
I tried XmlDocument xmlDoc = JsonConvert.DeserializeXmlNode but the actual result is
<Price>29</Price>
<Stars>4.5</Stars>
<Reviews>11</Reviews>
<Formats>5</Formats>
<Formats>6</Formats>
<MediaFormats>1</MediaFormats>
<MediaFormats>2</MediaFormats>
Any idea how to solve this issue?
Post-process the result with XSLT. No off-the-shelf JSON-to-XML converter (or XML-to-JSON converter) is going to give you the result you want every time; you must be prepared to customise it, and XSLT is the best tool for this.
You can achieve the required format using
<xsl:for-each-group select="*" group-adjacent="node-name(.)">
<xsl:copy>
<xsl:value-of select="current-group()" separator=","/>
</xsl:copy>
</xsl:for-each-group>
Of course, if you're using XSLT anyway, then you could consider using XSLT 3.0's xml-to-json() and json-to-xml() functions so it's all done with one tool.
Is there a standard way to transform an input XML document with structure (scheme) of my choice to an output JSON object with structure (scheme) of my choice?
If it were transformation from input XML to output XML, I would use XSLT.
I can image the following three approaches:
Direct transformation from XML to JSON, i.e. a way to describe transformation XML -> JSON just like XSLT describes transformation XML -> XML.
I am aware of JSONML. It is a lossless JSON representation of arbitrary XML document. However, the resulting JSON object does not have the structure of my choice. If there were some standard way to describe transformation JSON -> JSON, I would chain XML -> JSONML and JSONML -> JSON.
If there were the opposite to JSONML (let's call it "XMLSON", i.e. a lossless XML notation of arbitrary JSON object), I would chain XML -> XMLSON (via XSLT) and XMLSON -> JSON.
All the three options have some "if there were". I wonder if there really is some technology to achieve the goal.
Thanks.
XSLT 3 has support to transform any XML to an XML representation of JSON defined in https://www.w3.org/TR/xslt-30/#schema-for-json and then allows you to use https://www.w3.org/TR/xslt-30/#func-xml-to-json to convert that particular XML to JSON.
The output of XSLT does not need to be XML, so if you are comfortable using that, you can go ahead and use it to output JSON.
A quick search showed up this, which might be a good example for you to start from: https://github.com/bramstein/xsltjson
It defines an XSLT function which takes an XML tree as input, and generates a string as output. Looking into the source, the basic approach is to generate an XML tree with nodes for each JSON object, array, and value, and then apply templates to that which output the JSON syntax itself.
For instance, to output a JSON array, it first generates an XML node of <json:array>...</json:array>, and then applies this template:
<xsl:template match="json:array" mode="json">
<xsl:variable name="values">
<xsl:apply-templates mode="json"/>
</xsl:variable>
<xsl:text/>[<xsl:text/>
<xsl:value-of select="string-join($values/value,',')"/>
<xsl:text/>]<xsl:text/>
</xsl:template>
i need to know how to link my xsl transformation to my database i currently have it set up doing a html conversion but need to insert the data into a database the xslt is a single file only used for conversion and is run in a php script, i saw a thread on it a few days ago but forgot to save it and now cant for the life of me find anything on this. the xml is a feed not a file this is what the xsl looks like:
<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:template match="/">
<xsl:variable name="ref" select="lr_rates/hoel_ref"/>
<xsl:for-each select="//room">
<xsl:variable name="ref" select="ref"/>
<xsl:variable name="type" select="type_description"/>
<xsl:variable name="descr" select="description"/>
<xsl:variable name="avail" select="rooms_available"/>
<xsl:variable name="rate" select="rack_rate"/>
<xsl:variable name="cur" select="rate/requested_currency"/>
</xsl:for-each>
</xsl:template>
</xsl:stylesheet>
then this is what the database request might look like
"DELETE FROM `pt_rooms` WHERE hotel_ref = '$id'";
"INSERT INTO `pt_rooms`(hotel_ref,room_ref,room_type,description,availability,price,currency) VALUES ('$id','$ref','$type','$descr','$avail','$rate','$cur')";
i think i would probably need a mysql_connect statement as-well as its not an application just a singl xsl in a php script if anyone could link to a good explanation
XSLT is a XML Transformation, meaning you convert any XML file from XML to any other text-based format that you can come up with. It doesn't magically interact with your database without having some kind of processing code that takes the result and executes the queries. Since you need this and mentioned PHP, you would be better of processing the XML in PHP in the first place and populate the database that way.