In SQL Server a table can have a XML column
CREATE TABLE [dbo].[Temp](
[ID] [int] IDENTITY(1,1) NOT NULL,
[C1] [xml] NULL
)
XML column allows multiple root content, it's legal to put below into xml column
<a>1</a><a>2</a>
When doing query, I got this error,
There are multiple root elements
I think the cause of the error is CLR data type LINQPad/Linq-to-SQL auto-generated.
public XElement C1;
XElement does not allow multiple root content.
Is it possible to intervene the code generation process, so change the data type as string?
public String C1;
btw, the assembly LinqPad generated is at
Yes, use the advanced properties on the SQL Connection dialog to map XML columns to string.
Related
I apologize in advance if this is very simple and I am just missing it.
Would any of you know how to put custom attributes as column headers? I currently have a simple opt in survey on connect and I would like to have each of the 4 items as column headers and the score in the table results. I pull the data using an ODBC connection to excel so ideally I would like to just add this on the end of my current table if I can figure out how to do it.
This is how it currently looks in the output
{"effortscore":"5","promoterscore":"5","satisfactionscore":"5","survey_opt_in":"True"}
If you have any links or something that I can follow to try improve my knowledge.
Thanks in advance
There are multiple options to query data in JSON format in Athena, and based on your use case (data source, query frequency, query destination, etc.) you can choose what makes more sense.
String Column + JSON functions
This is usually the most straightforward option and a good starting point. You define the survey_output as a string column, and when you need to extract the specific attributes from the JSON string, you can apply the JSON functions in Trino/Athena: https://trino.io/docs/current/functions/json.html. For example:
SELECT
id,
json_query(
survey_output,
'lax $.satisfactionscore'
) AS satisfactionscore
FROM customers
String Column + JSON functions + View
The following way to simplify access to data without json_query functions is to define a VIEW on that table using the json_query syntax in the VIEW creation. You define the view once by a DBA, and when the users query the data, they see the columns they care about. For example:
CREATE VIEW survey_results AS
SELECT
id,
json_query(
survey_output,
'lax $.satisfactionscore'
) AS satisfactionscore
FROM customers;
With such dynamic view creation, you have more flexibility in what data will be easily exposed to the users.
Create a Table with STRUCT
Another option is to create the external table from the data source (files in S3, for example) with the STRUCT definition.
CREATE EXTERNAL TABLE survey (
id string,
survey_results struct<
effortscore:string,
promoterscore:string,
satisfactionscore:string,
survey_opt_in:string
>
)
ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'
LOCATION 's3://<YOUR BUCKET HERE>/<FILES>'
I have this stored procedure:
Dbo.SprocName (#Id UNIQUEIDENTIFIER,
#ResponseCode INT OUTPUT,
#ResponseDescription VARCHAR(500) OUTPUT)
And it returns a dataset called say Result as a nvarchar(MAX) (always a single row).
I've tried OLE and ADO connections and as well as result sets. I've tried creating a table variable and storing the value there.
Nothing works.
I can see in the database that it's running successfully then it fails when returning the result data set.
I’ve done some debugging and I can assure the result string is returned as should be. The problem is that I don’t know how to handle this on SSIS.
The error that I get is:
Input string was not in a correct format
I appreciate any ideas.
Thanks.
EDIT: I have tried using a table variable again and it works. I guess I didn't do it well first time. sorry about that. Thanks!
One potential cause for your problem could be a mismatch in data types between SSIS and SQL Server.
An SSIS GUID data type does not match a SQL Server uniqueidentifier - the SSIS GUID has curly braces (e.g., {00000000-0000-0000-0000-000000000000}), while the SQL value does not. SQL cannot recognize the value as a unique identifier, and fails to convert.
To pass down a GUID, you will need to remove those curly braces, either in SSIS or in SQL. One approach I've used it to send it across as a VARCHAR and then strip out the curly braces, e.g.,
DECLARE #GUID VARCHAR(40) = '{00000000-0000-0000-0000-000000000000}'
DECLARE #CnvtGUID UNIQUEIDENTIFIER = REPLACE(REPLACE(#GUID, '}', ''), '{', '')
SELECT #GUID, #CnvtGUID
I'm working with mongoDB, and I used a wrapper mongo/Postegres.
Now, I can find my tables and data.
I want to do some statistics but I can't reach objects that got json type in Postgres.
My problem is that I got all the object in json but I need to separate the fields.
I used this :
CREATE FOREIGN TABLE rents( _id NAME, status text, "from" json )
SERVER mongo_server
OPTIONS (database 'tr', collection 'rents');
The field "from" is an object.
I found something like this :
enter code here
but nothing happened
The error (why a screenshot??) means that the data are not in valid json format.
As a first step, you could define the column as type text instead of json. Then querying the foreign table will probably work, and you can see what is actually returned and why PostgreSQL thinks that this is not valid JSON.
Maybe you can create a view on top of the foreign table that converts the value to valid JSON for further processing.
I'm trying to use Talend to get JSON data that is stored in MySQL as a VARCHAR datatype and export it into PostgreSQL 9.4 table of the following type:
CREATE TABLE myTable( myJSON as JSONB)
When I try running the job I get the following error:
ERROR: column "json_string" is of type json but expression is of type
character varying
Hint: You will need to rewrite or cast the expression. Position:
54
If I use python or just plain SQL with PostgreSQL insert I can insert a string such as '{"Name":"blah"}' and it understands it.
INSERT INTO myTable(myJSON) VALUES ('{"Name":"blah"}');
Any Idea's how this can be done in Talend?
You can add a type-cast by opening the "Advanced Settings" tab on you "tPostgresqlOutput" component. Consider the following example:
In this case, the input row to "tPostgresqlOutput_1" has one column data. This column is of type String and is mapped to the database column data of type VARCHAR (as by the default suggested by Talend):
Next, open the component settings for tPostgresqlOutput_1 and locate the "Advanced settings" tab:
On this tab, you can replace the existing data column by a new expression:
In the name column, specify the target column name.
In the SQL Expression column, do your type casting. In this case: "?::json"`. Note the usage of the placeholder character?`` which will be replaced with the original value.
In Position, specify Replace. This will replace the value proposed by Talend with your SQL expression (including the type cast).
As Reference Column use the source value.
This should do the trick.
Here is a sample schema for where in i have the input row 'r' which has question_json and choice_json columns which are json strings. From which i know the key what i wanted to extract and here is how i do
you should look at the columns question_value and choice_value. Hope this helps you
Given the following HTML "<b>demo</b>"
I want to save it to an SQL 2005 table
Mandatory, no encoded, no escaped characters in the saved field from database
Saved html must be as small as possible
On my efforts, my stored html is always saved as encoded > html
EDIT:
Debugging my code I found that my HTML string is sended encoded to my StoredProcedure. Server.HtmlDecode won't entirely decode my HTML !
It's TinyMCE fault.
A direct insert of an arbitrary varchar string should always work:
create table MarkupTable(
id int identity(1,1) /*please don't berate me for using identity!*/
, markup varchar(max) /* this will use only the space taken by your markup string */
)
insert MarkupTable select '<html><b>demo</b></html>'
select * from MarkupTable
Now - the question is: what piece of code is encoding your markup???
Some starting places:
Are you using C# objects properly: SqlParameter(SqlDbType.VarChar).Value = yourMarkupString;
Are you using FOR XML anywhere?
Are you using anything like Security.Escape(markup)?
Is your string being returned as part of an XmlDocument or SOAP envelope? (it WILL be escaped by the serializer)
Are you getting a doubly encoded string?
Try:
Server.HtmlDecode(Server.HtmlDecode(markupString));
For space requirements:
You could run the string through a whitespace stripping algorithm (anything becomes a single space or tab.
You could compress the text and not even store varchar(max), but varbinary(max)
You can do the Server.HtmlEncode and save it in a varchar field in the database (SQL 2005). When you extarct it you can do a Server.HtmlDecode and load it into a label/literal and it will rendered normally.
Don't get caught out using the "text" data type for this data. It's going to be removed from SQL server and you don't want to cause yourself a headache later on.
Store the data in a nvarchar(max) data type.
As far as making it as small as possible, stripping out unnecessary white space may help to reduce the size of the data.