Related
I have the following tables (pseudo names below) in SQL which I need to use to produce some dynamic JSON out from :
HeaderTable
RequestID int
RequestType varchar(50)
Sample Data
1 : User Name Change
2 : User Name Change
ValuesTable
RequestID int
Alias varchar(50)
FieldValue varchar(50)
Sample Data
1 : MobileNo : 07777777777
1 : Name : Fred Bloggs
2 : MobileNo : 07888888888
2 : Name : John Smith
The JSON I need to end up with is as follows :
[
{
"request_type":"User Name Change",
"request_details":[
{
"MobileNo":"07777777777",
"Name":"Fred Bloggs"
},
{
"MobileNo":"07888888888",
"Name":"John Smith"
}
]
}
]
So I effectively need to pass my alias value as the key name in the JSON.
My code so far is as follows but I assume I might need some sort of dynamic SQL?
SELECT hdr.RequestType AS request_type
, (
SELECT vals.FieldValue AS [request_details.value]
FROM ValuesTable vals
WHERE vals.RequestID = hdr.[RequestID]
FOR JSON PATH
) request_details
FROM HeaderTable hdr
FOR JSON PATH
I'm not sure if any of the other SQL JSON function might be useful here or if I need to somehow churn out some dynamic SQL as my only hope?
I don't think you can build the required JSON directly (usinf FOR JSON), but you may try to build one part of the JSON output using basic string concatenation and aggregation. Note, that starting for SQL Server 2016, you need to use FOR XML PATH for aggregation:
Data:
SELECT *
INTO HeaderTable
FROM (VALUES
(1, 'User Name Change')
) v (RequestID, RequestType)
SELECT *
INTO ValuesTable
FROM (VALUES
(1, 'MobileNo', '07777777777'),
(1, 'Name', 'Fred Bloggs'),
(1, 'Address', 'Full address'),
(2, 'MobileNo', '07888888888'),
(2, 'Name', 'John Smith')
) v (RequestID, Alias, FieldValue)
Statement for SQL Server 2017:
SELECT
hdr.RequestType AS request_type,
JSON_QUERY((
SELECT CONCAT(
'[{',
STRING_AGG(
CONCAT(
'"',
STRING_ESCAPE(vals.Alias, 'json'),
'":"',
STRING_ESCAPE(vals.FieldValue, 'json'), '"'
),
','
),
'}]'
)
FROM ValuesTable vals
WHERE vals.RequestID = hdr.[RequestID]
)) AS request_details
FROM HeaderTable hdr
FOR JSON PATH
Statement for SQL Server 2016:
SELECT
hdr.RequestType AS request_type,
JSON_QUERY(CONCAT(
'[{',
STUFF(
(
SELECT CONCAT(',"', vals.Alias, '":"', vals.FieldValue, '"')
FROM ValuesTable vals
WHERE vals.RequestID = hdr.[RequestID]
FOR XML PATH(''), TYPE
).value('.', 'varchar(max)'),
1, 1, ''
),
'}]'
)) AS request_details
FROM HeaderTable hdr
FOR JSON PATH
Result:
[
{
"request_type":"User Name Change",
"request_details":[
{
"MobileNo":"07777777777",
"Name":"Fred Bloggs",
"Address":"Full address"
}
]
}
]
Say I have two JSON strings as follows:
[{"RowId":102787,"UserId":1,"Activity":"This is another test","Timestamp":"2017-11-25T14:37:30.3700000"}]
[{"RowId":102787,"UserId":2,"Activity":"Testing the Update function","Timestamp":"2017-11-25T14:37:30.3700000"}]
Both have the same properties but two of the properties in the second string have different values than the first (UserId and Activity). Is it possible, in Azure SQL Database T-SQL, to generate a third JSON string that contains the values in the second string that are different from the first? In other words, I'd like a string returned that looks like this:
[{"UserId":2,"Activity":"Testing the Update function"}]
Also, the solution should assume that the properties in the JSON strings are not known. I need this to be a generic solution for any two JSON strings.
Have not tried this on Azure, but it seems to work on SQL Server 2017
There is probably a more elegant way to get to the final JSON string other than through string manipulation, perhaps we can update the answer as better ways are found.
-- Expected : [{"UserId":2,"Activity":"Testing the Update function"}]
DECLARE #jsonA NVARCHAR(MAX) = '[{"RowId":102787,"UserId":1,"Activity":"This is another test","Timestamp":"2017-11-25T14:37:30.3700000"}]'
,#jsonB NVARCHAR(MAX) = '[{"RowId":102787,"UserId":2,"Activity":"Testing the Update function","Timestamp":"2017-11-25T14:37:30.3700000"}]'
,#result NVARCHAR(MAX) = ''
SELECT #jsonA = REPLACE(REPLACE(#jsonA, ']', ''), '[', '')
,#jsonB = REPLACE(REPLACE(#jsonB, ']', ''), '[', '')
;WITH DSA AS
(
SELECT *
FROM OPENJSON(#jsonA)
)
,DSB AS
(
SELECT *
FROM OPENJSON(#jsonB)
)
SELECT #result += CONCAT (
'"', B.[key], '":'
,IIF(B.[type] = 2, B.[value], CONCAT('"', B.[value], '"')) -- havent checked types other than 1 and 2; think there's a bool type?
,','
)
FROM DSA A
JOIN DSB B ON A.[key] = B.[key]
WHERE A.[value] != B.[value]
SELECT CONCAT('[{', LEFT(#result, LEN(#result) - 1), '}]')
I'm trying to migrate a MySQL-based app over to Microsoft SQL Server 2005 (not by choice, but that's life).
In the original app, we used almost entirely ANSI-SQL compliant statements, with one significant exception -- we used MySQL's group_concat function fairly frequently.
group_concat, by the way, does this: given a table of, say, employee names and projects...
SELECT empName, projID FROM project_members;
returns:
ANDY | A100
ANDY | B391
ANDY | X010
TOM | A100
TOM | A510
... and here's what you get with group_concat:
SELECT
empName, group_concat(projID SEPARATOR ' / ')
FROM
project_members
GROUP BY
empName;
returns:
ANDY | A100 / B391 / X010
TOM | A100 / A510
So what I'd like to know is: Is it possible to write, say, a user-defined function in SQL Server which emulates the functionality of group_concat?
I have almost no experience using UDFs, stored procedures, or anything like that, just straight-up SQL, so please err on the side of too much explanation :)
No REAL easy way to do this. Lots of ideas out there, though.
Best one I've found:
SELECT table_name, LEFT(column_names , LEN(column_names )-1) AS column_names
FROM information_schema.columns AS extern
CROSS APPLY
(
SELECT column_name + ','
FROM information_schema.columns AS intern
WHERE extern.table_name = intern.table_name
FOR XML PATH('')
) pre_trimmed (column_names)
GROUP BY table_name, column_names;
Or a version that works correctly if the data might contain characters such as <
WITH extern
AS (SELECT DISTINCT table_name
FROM INFORMATION_SCHEMA.COLUMNS)
SELECT table_name,
LEFT(y.column_names, LEN(y.column_names) - 1) AS column_names
FROM extern
CROSS APPLY (SELECT column_name + ','
FROM INFORMATION_SCHEMA.COLUMNS AS intern
WHERE extern.table_name = intern.table_name
FOR XML PATH(''), TYPE) x (column_names)
CROSS APPLY (SELECT x.column_names.value('.', 'NVARCHAR(MAX)')) y(column_names)
I may be a bit late to the party but this method works for me and is easier than the COALESCE method.
SELECT STUFF(
(SELECT ',' + Column_Name
FROM Table_Name
FOR XML PATH (''))
, 1, 1, '')
SQL Server 2017 does introduce a new aggregate function
STRING_AGG ( expression, separator).
Concatenates the values of string expressions and places separator
values between them. The separator is not added at the end of string.
The concatenated elements can be ordered by appending WITHIN GROUP (ORDER BY some_expression)
For versions 2005-2016 I typically use the XML method in the accepted answer.
This can fail in some circumstances however. e.g. if the data to be concatenated contains CHAR(29) you see
FOR XML could not serialize the data ... because it
contains a character (0x001D) which is not allowed in XML.
A more robust method that can deal with all characters would be to use a CLR aggregate. However applying an ordering to the concatenated elements is more difficult with this approach.
The method of assigning to a variable is not guaranteed and should be avoided in production code.
Possibly too late to be of benefit now, but is this not the easiest way to do things?
SELECT empName, projIDs = replace
((SELECT Surname AS [data()]
FROM project_members
WHERE empName = a.empName
ORDER BY empName FOR xml path('')), ' ', REQUIRED SEPERATOR)
FROM project_members a
WHERE empName IS NOT NULL
GROUP BY empName
Have a look at the GROUP_CONCAT project on Github, I think I does exactly what you are searching for:
This project contains a set of SQLCLR User-defined Aggregate functions (SQLCLR UDAs) that collectively offer similar functionality to the MySQL GROUP_CONCAT function. There are multiple functions to ensure the best performance based on the functionality required...
To concatenate all the project manager names from projects that have multiple project managers write:
SELECT a.project_id,a.project_name,Stuff((SELECT N'/ ' + first_name + ', '+last_name FROM projects_v
where a.project_id=project_id
FOR
XML PATH(''),TYPE).value('text()[1]','nvarchar(max)'),1,2,N''
) mgr_names
from projects_v a
group by a.project_id,a.project_name
With the below code you have to set PermissionLevel=External on your project properties before you deploy, and change the database to trust external code (be sure to read elsewhere about security risks and alternatives [like certificates]) by running ALTER DATABASE database_name SET TRUSTWORTHY ON.
using System;
using System.Collections.Generic;
using System.Data.SqlTypes;
using System.IO;
using System.Runtime.Serialization;
using System.Runtime.Serialization.Formatters.Binary;
using Microsoft.SqlServer.Server;
[Serializable]
[SqlUserDefinedAggregate(Format.UserDefined,
MaxByteSize=8000,
IsInvariantToDuplicates=true,
IsInvariantToNulls=true,
IsInvariantToOrder=true,
IsNullIfEmpty=true)]
public struct CommaDelimit : IBinarySerialize
{
[Serializable]
private class StringList : List<string>
{ }
private StringList List;
public void Init()
{
this.List = new StringList();
}
public void Accumulate(SqlString value)
{
if (!value.IsNull)
this.Add(value.Value);
}
private void Add(string value)
{
if (!this.List.Contains(value))
this.List.Add(value);
}
public void Merge(CommaDelimit group)
{
foreach (string s in group.List)
{
this.Add(s);
}
}
void IBinarySerialize.Read(BinaryReader reader)
{
IFormatter formatter = new BinaryFormatter();
this.List = (StringList)formatter.Deserialize(reader.BaseStream);
}
public SqlString Terminate()
{
if (this.List.Count == 0)
return SqlString.Null;
const string Separator = ", ";
this.List.Sort();
return new SqlString(String.Join(Separator, this.List.ToArray()));
}
void IBinarySerialize.Write(BinaryWriter writer)
{
IFormatter formatter = new BinaryFormatter();
formatter.Serialize(writer.BaseStream, this.List);
}
}
I've tested this using a query that looks like:
SELECT
dbo.CommaDelimit(X.value) [delimited]
FROM
(
SELECT 'D' [value]
UNION ALL SELECT 'B' [value]
UNION ALL SELECT 'B' [value] -- intentional duplicate
UNION ALL SELECT 'A' [value]
UNION ALL SELECT 'C' [value]
) X
And yields: A, B, C, D
Tried these but for my purposes in MS SQL Server 2005 the following was most useful, which I found at xaprb
declare #result varchar(8000);
set #result = '';
select #result = #result + name + ' '
from master.dbo.systypes;
select rtrim(#result);
#Mark as you mentioned it was the space character that caused issues for me.
About J Hardiman's answer, how about:
SELECT empName, projIDs=
REPLACE(
REPLACE(
(SELECT REPLACE(projID, ' ', '-somebody-puts-microsoft-out-of-his-misery-please-') AS [data()] FROM project_members WHERE empName=a.empName FOR XML PATH('')),
' ',
' / '),
'-somebody-puts-microsoft-out-of-his-misery-please-',
' ')
FROM project_members a WHERE empName IS NOT NULL GROUP BY empName
By the way, is the use of "Surname" a typo or am i not understanding a concept here?
Anyway, thanks a lot guys cuz it saved me quite some time :)
2021
#AbdusSalamAzad's answer is the correct one.
SELECT STRING_AGG(my_col, ',') AS my_result FROM my_tbl;
If the result is too big, you may get error "STRING_AGG aggregation result exceeded the limit of 8000 bytes. Use LOB types to avoid result truncation." , which can be fixed by changing the query to this:
SELECT STRING_AGG(convert(varchar(max), my_col), ',') AS my_result FROM my_tbl;
For my fellow Googlers out there, here's a very simple plug-and-play solution that worked for me after struggling with the more complex solutions for a while:
SELECT
distinct empName,
NewColumnName=STUFF((SELECT ','+ CONVERT(VARCHAR(10), projID )
FROM returns
WHERE empName=t.empName FOR XML PATH('')) , 1 , 1 , '' )
FROM
returns t
Notice that I had to convert the ID into a VARCHAR in order to concatenate it as a string. If you don't have to do that, here's an even simpler version:
SELECT
distinct empName,
NewColumnName=STUFF((SELECT ','+ projID
FROM returns
WHERE empName=t.empName FOR XML PATH('')) , 1 , 1 , '' )
FROM
returns t
All credit for this goes to here:
https://social.msdn.microsoft.com/Forums/sqlserver/en-US/9508abc2-46e7-4186-b57f-7f368374e084/replicating-groupconcat-function-of-mysql-in-sql-server?forum=transactsql
For SQL Server 2017+, use STRING_AGG() function
SELECT STRING_AGG(Genre, ',') AS Result
FROM Genres;
Sample result:
Result
Rock,Jazz,Country,Pop,Blues,Hip Hop,Rap,Punk
SQL statement:
(select top 1 [egrp_name] from [Enotify Group] where [egrp_id] in (a.grp_id) )
e value of a.grp_id is '0,1145' and i am getting error
Conversion failed when converting the varchar value '0,1145' to data type int.
Can anybody tell me how can i change '0,1145' to 0,1145 in above case, so my query does work and also if their is any other way to do this
You can use a split-string function to change your comma delimited string into a table.
select top(1) [egrp_name]
from [Enotify Group]
where [egrp_id] in (
select Value
from dbo.SplitInts(a.grp_id)
);
One version of a split-string function that you can use if you like:
create function dbo.SplitInts(#Values nvarchar(max)) returns table with schemabinding
as
return
(
select T2.X.value(N'.', N'int') as Value
from (select cast(N'<?X '+replace(#Values, N',', N'?><?X ') + N'?>' as xml).query(N'.')) as T1(X)
cross apply T1.X.nodes(N'/processing-instruction("X")') as T2(X)
);
Or you can use like.
select top(1) [egrp_name]
from [Enotify Group]
where ','+a.grp_id +',' like '%,'+cast(egrp_id as varchar(11))+',%' ;
I'm trying to migrate a MySQL-based app over to Microsoft SQL Server 2005 (not by choice, but that's life).
In the original app, we used almost entirely ANSI-SQL compliant statements, with one significant exception -- we used MySQL's group_concat function fairly frequently.
group_concat, by the way, does this: given a table of, say, employee names and projects...
SELECT empName, projID FROM project_members;
returns:
ANDY | A100
ANDY | B391
ANDY | X010
TOM | A100
TOM | A510
... and here's what you get with group_concat:
SELECT
empName, group_concat(projID SEPARATOR ' / ')
FROM
project_members
GROUP BY
empName;
returns:
ANDY | A100 / B391 / X010
TOM | A100 / A510
So what I'd like to know is: Is it possible to write, say, a user-defined function in SQL Server which emulates the functionality of group_concat?
I have almost no experience using UDFs, stored procedures, or anything like that, just straight-up SQL, so please err on the side of too much explanation :)
No REAL easy way to do this. Lots of ideas out there, though.
Best one I've found:
SELECT table_name, LEFT(column_names , LEN(column_names )-1) AS column_names
FROM information_schema.columns AS extern
CROSS APPLY
(
SELECT column_name + ','
FROM information_schema.columns AS intern
WHERE extern.table_name = intern.table_name
FOR XML PATH('')
) pre_trimmed (column_names)
GROUP BY table_name, column_names;
Or a version that works correctly if the data might contain characters such as <
WITH extern
AS (SELECT DISTINCT table_name
FROM INFORMATION_SCHEMA.COLUMNS)
SELECT table_name,
LEFT(y.column_names, LEN(y.column_names) - 1) AS column_names
FROM extern
CROSS APPLY (SELECT column_name + ','
FROM INFORMATION_SCHEMA.COLUMNS AS intern
WHERE extern.table_name = intern.table_name
FOR XML PATH(''), TYPE) x (column_names)
CROSS APPLY (SELECT x.column_names.value('.', 'NVARCHAR(MAX)')) y(column_names)
I may be a bit late to the party but this method works for me and is easier than the COALESCE method.
SELECT STUFF(
(SELECT ',' + Column_Name
FROM Table_Name
FOR XML PATH (''))
, 1, 1, '')
SQL Server 2017 does introduce a new aggregate function
STRING_AGG ( expression, separator).
Concatenates the values of string expressions and places separator
values between them. The separator is not added at the end of string.
The concatenated elements can be ordered by appending WITHIN GROUP (ORDER BY some_expression)
For versions 2005-2016 I typically use the XML method in the accepted answer.
This can fail in some circumstances however. e.g. if the data to be concatenated contains CHAR(29) you see
FOR XML could not serialize the data ... because it
contains a character (0x001D) which is not allowed in XML.
A more robust method that can deal with all characters would be to use a CLR aggregate. However applying an ordering to the concatenated elements is more difficult with this approach.
The method of assigning to a variable is not guaranteed and should be avoided in production code.
Possibly too late to be of benefit now, but is this not the easiest way to do things?
SELECT empName, projIDs = replace
((SELECT Surname AS [data()]
FROM project_members
WHERE empName = a.empName
ORDER BY empName FOR xml path('')), ' ', REQUIRED SEPERATOR)
FROM project_members a
WHERE empName IS NOT NULL
GROUP BY empName
Have a look at the GROUP_CONCAT project on Github, I think I does exactly what you are searching for:
This project contains a set of SQLCLR User-defined Aggregate functions (SQLCLR UDAs) that collectively offer similar functionality to the MySQL GROUP_CONCAT function. There are multiple functions to ensure the best performance based on the functionality required...
To concatenate all the project manager names from projects that have multiple project managers write:
SELECT a.project_id,a.project_name,Stuff((SELECT N'/ ' + first_name + ', '+last_name FROM projects_v
where a.project_id=project_id
FOR
XML PATH(''),TYPE).value('text()[1]','nvarchar(max)'),1,2,N''
) mgr_names
from projects_v a
group by a.project_id,a.project_name
With the below code you have to set PermissionLevel=External on your project properties before you deploy, and change the database to trust external code (be sure to read elsewhere about security risks and alternatives [like certificates]) by running ALTER DATABASE database_name SET TRUSTWORTHY ON.
using System;
using System.Collections.Generic;
using System.Data.SqlTypes;
using System.IO;
using System.Runtime.Serialization;
using System.Runtime.Serialization.Formatters.Binary;
using Microsoft.SqlServer.Server;
[Serializable]
[SqlUserDefinedAggregate(Format.UserDefined,
MaxByteSize=8000,
IsInvariantToDuplicates=true,
IsInvariantToNulls=true,
IsInvariantToOrder=true,
IsNullIfEmpty=true)]
public struct CommaDelimit : IBinarySerialize
{
[Serializable]
private class StringList : List<string>
{ }
private StringList List;
public void Init()
{
this.List = new StringList();
}
public void Accumulate(SqlString value)
{
if (!value.IsNull)
this.Add(value.Value);
}
private void Add(string value)
{
if (!this.List.Contains(value))
this.List.Add(value);
}
public void Merge(CommaDelimit group)
{
foreach (string s in group.List)
{
this.Add(s);
}
}
void IBinarySerialize.Read(BinaryReader reader)
{
IFormatter formatter = new BinaryFormatter();
this.List = (StringList)formatter.Deserialize(reader.BaseStream);
}
public SqlString Terminate()
{
if (this.List.Count == 0)
return SqlString.Null;
const string Separator = ", ";
this.List.Sort();
return new SqlString(String.Join(Separator, this.List.ToArray()));
}
void IBinarySerialize.Write(BinaryWriter writer)
{
IFormatter formatter = new BinaryFormatter();
formatter.Serialize(writer.BaseStream, this.List);
}
}
I've tested this using a query that looks like:
SELECT
dbo.CommaDelimit(X.value) [delimited]
FROM
(
SELECT 'D' [value]
UNION ALL SELECT 'B' [value]
UNION ALL SELECT 'B' [value] -- intentional duplicate
UNION ALL SELECT 'A' [value]
UNION ALL SELECT 'C' [value]
) X
And yields: A, B, C, D
Tried these but for my purposes in MS SQL Server 2005 the following was most useful, which I found at xaprb
declare #result varchar(8000);
set #result = '';
select #result = #result + name + ' '
from master.dbo.systypes;
select rtrim(#result);
#Mark as you mentioned it was the space character that caused issues for me.
About J Hardiman's answer, how about:
SELECT empName, projIDs=
REPLACE(
REPLACE(
(SELECT REPLACE(projID, ' ', '-somebody-puts-microsoft-out-of-his-misery-please-') AS [data()] FROM project_members WHERE empName=a.empName FOR XML PATH('')),
' ',
' / '),
'-somebody-puts-microsoft-out-of-his-misery-please-',
' ')
FROM project_members a WHERE empName IS NOT NULL GROUP BY empName
By the way, is the use of "Surname" a typo or am i not understanding a concept here?
Anyway, thanks a lot guys cuz it saved me quite some time :)
2021
#AbdusSalamAzad's answer is the correct one.
SELECT STRING_AGG(my_col, ',') AS my_result FROM my_tbl;
If the result is too big, you may get error "STRING_AGG aggregation result exceeded the limit of 8000 bytes. Use LOB types to avoid result truncation." , which can be fixed by changing the query to this:
SELECT STRING_AGG(convert(varchar(max), my_col), ',') AS my_result FROM my_tbl;
For my fellow Googlers out there, here's a very simple plug-and-play solution that worked for me after struggling with the more complex solutions for a while:
SELECT
distinct empName,
NewColumnName=STUFF((SELECT ','+ CONVERT(VARCHAR(10), projID )
FROM returns
WHERE empName=t.empName FOR XML PATH('')) , 1 , 1 , '' )
FROM
returns t
Notice that I had to convert the ID into a VARCHAR in order to concatenate it as a string. If you don't have to do that, here's an even simpler version:
SELECT
distinct empName,
NewColumnName=STUFF((SELECT ','+ projID
FROM returns
WHERE empName=t.empName FOR XML PATH('')) , 1 , 1 , '' )
FROM
returns t
All credit for this goes to here:
https://social.msdn.microsoft.com/Forums/sqlserver/en-US/9508abc2-46e7-4186-b57f-7f368374e084/replicating-groupconcat-function-of-mysql-in-sql-server?forum=transactsql
For SQL Server 2017+, use STRING_AGG() function
SELECT STRING_AGG(Genre, ',') AS Result
FROM Genres;
Sample result:
Result
Rock,Jazz,Country,Pop,Blues,Hip Hop,Rap,Punk