I am trying to get data from the US Census on SQL Server Studio for analysis (trade data at port level). I have downloaded a JSON file for now from their API (ideally, I will do a call from SQL studio later). I then read the file with OPEN ROW SET and OPEN JSON, I can read the file but when I add the with clause to get the column, I only get NULL values.
Declare #JSON varchar(max)
SELECT #JSON=BulkColumn
FROM OPENROWSET (BULK 'C:\Users\amartinez\US.json', SINGLE_CLOB) j
SELECT * FROM OPENJSON (#JSON)
WITH (
[CTY_CODE] varchar(max) '$.CTY_CODE',
[CTY_NAME] varchar(max) '$.CTY_NAME',
[I_ENDUSE] varchar(max) '$.I_ENDUSE',
[I_ENDUSE_LDESC] varchar(max) '$.I_ENDUSE_LDESC',
[GEN_VAL_MO] int '$.GEN_VAL_MO',
[CON_VAL_MO] int '$.CON_VAL_MO',
[time] varchar(max) '$.time'
) as tradeF;
Input file
Please try the following solution.
Your JSON is a JSON array, so it needs a slightly different syntax.
SQL
DECLARE #JSON VARCHAR(MAX);
SELECT #JSON=BulkColumn
FROM OPENROWSET (BULK 'C:\Users\amartinez\US.json', SINGLE_CLOB) j;
SELECT tradeF.*
FROM OPENJSON (#JSON)
WITH (
[CTY_CODE] varchar(max) '$[0]',
[CTY_NAME] varchar(max) '$[1]',
[I_ENDUSE] varchar(max) '$[2]',
[I_ENDUSE_LDESC] varchar(max) '$[3]',
[GEN_VAL_MO] varchar(max) '$[4]',
[CON_VAL_MO] varchar(max) '$[5]',
[time] varchar(max) '$[6]'
) as tradeF;
Related
I have a ole automation query that sends json data to an rest api. This works great when you hardcode the json data in the query.
I am trying to use the output from a for json query instead.
So my basic goal is to store the json from the query output in a usable variable
I have tried this
but null always gets assigned to the variable
```
declare #JsonOutput xml
EXEC SP_EXECUTESQL #SqlText, N'#JsonOutput xml OUTPUT', #JsonOutput = #JsonOutput OUTPUT
```
the above output does return the json but the #JsonOutput variable never gets set with the JSON data and is unusable..
..................
Update:
I found a work around but was still wondering if there was a better way. I'm loading the json into a temp table then selecting it and stores it in a usable variable and it posts successfully. It does however seem to restrict the amount of json I can put in the temp table field.
'''
set nocount on;
drop table #b
;WITH x(a) as
(
select JSON_QUERY(rtrim(JSON_MODIFY('[]', 'append $', authorIds))) AS authorIds,[bases], [isCircular], [name], 'ts_SOThWL5L' [schemaId], [registryId], [namingStrategy]
FROM [dbo].[vw_bulkDNAPostCreate]
where myid between 600 and 630
FOR JSON PATH, ROOT('dnaSequences')
)
SELECT a INTO #b FROM x
select a from #b
DECLARE #IntVariable INT;
DECLARE #SQLString NVARCHAR(500);
DECLARE #ParmDefinition NVARCHAR(500);
DECLARE #JSON varchar(max);
set #SQLString = N'select #JSONOUT = a from #b'
SET #ParmDefinition = N'#JSONOUT varchar(max) OUTPUT';
EXECUTE sp_executesql #SQLString, #ParmDefinition,#JSONOUT=#JSON OUTPUT;
--SELECT #JSON;
declare #url nvarchar(222) = N'https://TestApiEndpoint..net'
declare #authHeader nvarchar(222) = N'Basic tokenadskjfldasjfs'
Declare #Body as varchar(max) = #JSON
EXEC [dbo].[MyAPIPostProc]
#authHeader = #authHeader,
#Body = #JSON,
#url = #url
'''
I read the docs of handling a JSON file here. So far I am able to read the file and get a result:
QRY: SELECT * FROM OPENROWSET (BULK 'c:\ne.db', SINGLE_CLOB) as import
Result: {"res":{"number":"123", "info":"c-PM6900"},"_id":"aHMIeu6ZwB9lIBZk"} {"res":{"number":"456", "info":"a-PMs900"},"_id":"aHaIeu6ZwB9sIBZ1"}....
if I qry this, I only get the first row with the res nested:
Declare #JSON varchar(max)
SELECT #JSON=BulkColumn
FROM OPENROWSET (BULK 'C:\ne.db', SINGLE_CLOB) import
SELECT *
FROM OPENJSON (#JSON)
What I want to achieve, is to read every entry of the JSON file and insert "res" from the json query into a row of a table in the database containing columns "number","info","id". If anyone could help me to finish this, I would appreciate.
The JSON file contains about 400000 lines and comes from a NodeJS script which uses nedb.
Here is the example file: LINK
The JSON in the file is not a valid JSON, it contains multiple root elements or a single row for each JSON object. It's strange, but OPENJSON() reads only the first element in this JSON input without generating an error.
But you may try to transform the input JSON into a valid JSON array ({...} {...} into [{}, {...}]) and parse this JSON array with OPENJSON() and explicit schema. If the input file has a single row for each JSON object, you need to know the new line separator (it's usually CHAR(10)):
DECLARE #json nvarchar(MAX)
-- Read the file's content
-- SELECT #json = BulkColumn
-- FROM OPENROWSET (BULK 'C:\ne.db', SINGLE_CLOB) AS [Insert]
-- Only for test
SELECT #json =
N'{"res":{"number":"123", "info":"c-PM6900"},"_id":"aHMIeu6ZwB9lIBZk"}' +
CHAR(10) +
N'{"res":{"number":"456", "info":"a-PMs900"},"_id":"aHaIeu6ZwB9sIBZ1"}'
SELECT [number], [info], [_id]
FROM OPENJSON(CONCAT('[', REPLACE(#json, CONCAT('}', CHAR(10), '{'), '},{'), ']')) WITH (
[number] varchar(3) '$.res.number',
[info] varchar(10) '$.res.info',
_id varchar(50) '$._id'
)
Result:
number info _id
123 c-PM6900 aHMIeu6ZwB9lIBZk
456 a-PMs900 aHaIeu6ZwB9sIBZ1
You need to use a couple of calls to OPENJSON to achieve this, with a WITH:
DECLARE #JSON nvarchar(MAX) = N'{"res":{"number":"123", "info":"c-PM6900"},"_id":"aHMIeu6ZwB9lIBZk"} {"res":{"number":"456", "info":"a-PMs900"},"_id":"aHaIeu6ZwB9sIBZ1"}'
SELECT J._id,
r.number,
r.info
FROM OPENJSON(#JSON)
WITH (_id varchar(30),
res nvarchar(MAX) AS JSON) J
CROSS APPLY OPENJSON(J.res)
WITH(number int,
info varchar(10)) r;
Because the OP appears to think I am telling them to change their DECLARE and assignment statement... to confirm how you get the value into the #JSON, from the OP's own question:
DECLARE #JSON varchar(max);
SELECT #JSON=BulkColumn
FROM OPENROWSET (BULK 'C:\ne.db', SINGLE_CLOB);
Final edit, is also appears that the OP's JSON is malformed, as I would expect a comma, or something, before the second res definition. Guessing we need to split it into rows as well, which means some string splitting:
SELECT J._id,
r.number,
r.info
FROM STRING_SPLIT(REPLACE(#JSON,N'}} {"res"',N'}}|{"res"'),'|') SS --I assume a pipe (|`) won't appear in the data
CROSS APPLY OPENJSON(SS.[value])
WITH (_id varchar(30),
res nvarchar(MAX) AS JSON) J
CROSS APPLY OPENJSON(J.res)
WITH(number int,
info varchar(10)) r;
db<>fiddle
I have the below query
DEClare #Plan_ID nvarchar(100)
set #Plan_ID=REPLACE('2151,1886',',',''',''')
and
SELECT distinct Plan_Dict_Id from REF_Plan_Dictionary WHERE
CAST(Plan_Dict_Id as int) in (#Plan_ID),
Pls help , Plan_Dict_Id datatype is INT, I want to pass the values to where , but getting error "Conversion failed when converting the varchar value '2151','1886' to data type int."
If this is one of the later versions of SQL Server then you could do something like this...
DECLARE #Plan_ID nvarchar(100)
SET #Plan_ID= '2151,1886'
SELECT DISTINCT Plan_Dict_Id
FROM REF_Plan_Dictionary d
JOIN string_split(#Plan_ID, ',') s ON d.Plan_Dict_Id = s.[value]
If you are using an older version you could put the whole statement in a string and then execute it, however you have a problem in your first line as you are not adding quotes around the string... I've accounted for that below.
DEClare #Plan_ID nvarchar(100) set #Plan_ID=REPLACE('2151,1886',',',''',''')
DECLARE #sql nvarchar(max)
SET #SQL = 'select distinct Plan_Dict_Id from REF_Plan_Dictionary where CAST(Plan_Dict_Id as int) in (''' + #Plan_ID + ''')'
EXEC (#sql)
I'm creating a stored procedure which gets some parameters and in turn these parameters are sent to another stored procedure which I'm calling from openrowset but I'm getting some syntax errors.
CREATE PROCEDURE UpdatePrevFYConfigData
-- Add the parameters for the stored procedure here
#startDate datetime,
#endDate datetime,
#productGroup varchar(8000) = 'All',
#projectType varchar(500) = 'All',
#businessUnit nvarchar(50) = 'All',
#developmentLocation nvarchar(100) = 'All'
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
declare #start varchar(50)
declare #end varchar(50)
set #start = cast(#startDate as varchar(40))
set #end = cast(#endDate as varchar(40))
-- Insert statements for procedure here
select round(avg(a.DeviationDeadline),2) as DeviationDeadline,
round(avg(a.DeviationDefinition),2) as DeviationDefinition,
round(avg(a.DeviationRDCosts),2) as DeviationRDCosts,
round(avg(a.FunctionsAdded) + avg(a.FunctionsDeleted),2) as NotRealizedFuncs,
round(avg(a.DeviationPM2000Aufwand),2) as DeviationPM200Aufwand,
round(avg(b.Defect),2) as Defect
into #tempTable
from openrowset('SQLNCLI',
'Server=.\sqlexpress;Trusted_Connection=yes;',
'SET NOCOUNT ON;SET FMTONLY OFF;EXEC [BSC_DB].dbo.SelectScorecardGraphData
'''+#start+''',
'''+#end+''',
'''+#productGroup+''',
'''+#projectType+''',
''1'',
''0'',
''All'',
''Current'',
'''+#businessUnit+''',
'''+#developmentLocation+'''
') as a,
openrowset('SQLNCLI', 'Server=.\sqlexpress;Trusted_Connection=yes;', 'SET NOCOUNT ON;SET FMTONLY OFF;EXEC [BSC_DB].dbo.GetSPCDefectDistributionData
'''+cast(#startDate as varchar(40))+''',
'''+cast(#endDate as varchar(40))+''',
''Defect'',
'''+#projectType+''',
'''+#productGroup+''',
'''+#businessUnit+''',
'''+#developmentLocation+'''') as b
update dbo.EA_ProcessScorecard_Config_Tbl
set EPC_Deviation = case EPC_Metric
when 'PM200' then (select DeviationDefinition from #tempTable)
when 'PM300' then (select DeviationDeadline from #tempTable)
when 'Cost' then (select DeviationRDCosts from #tempTable)
when 'PM150' then (select DeviationPM200Aufwand from #tempTable)
when 'Defect' then (select Defect from #tempTable)
when 'Funcs' then (select NotRealizedFuncs from #tempTable)
END
where EPC_Description = 'PrevFY' and EPC_FYYear = '0'
drop table #tempTable
END
GO
I'm not able to create it and I get the error message:
Msg 102, Level 15, State 1, Procedure UpdatePrevFYConfigData,
Line 38 Incorrect syntax near '+'.
... but if I use hard coded values for the parameters it works!!
Please help!
Both OPENROWSET and OPENDATASOURCE should be used only for accessing external data for, let's say, quick and dirty solutions, or when it is not possible to configure a permanent linked server. These functions do not provide all of the functionality available from a linked server.
The arguments of OPENROWSET and OPENDATASOURCE do not support variables. They have to be specified as string-literal. If variables need to be passed in as arguments to these functions, a query string containing these variables can be constructed dynamically and executed using the EXEC statement.
Similar to (not syntax checked)
DECLARE #sqlCommand varchar(1000)
SET #sqlCommand = 'SELECT *
FROM OPENROWSET(''SQLNCLI'',''server=.\sqlexpress;Trusted_Connection=yes'',''SET NOCOUNT ON;SET FMTONLY OFF;EXEC [BSC_DB].dbo.SelectScorecardGraphData ''''' + cast(#param1 as varchar(10)) + ''''',''' + cast(#param2 as varchar(n)) ''')'
EXEC #sqlCommand
And so on...
Hope that helps. Kind regards,
Stefan
-- FOR USING OPENROWSETS
EXEC sp_configure 'Ad Hoc Distributed Queries'
,1
RECONFIGURE
DECLARE #SQL NVARCHAR(MAX)
SET #SQL = 'INSERT INTO #TABLESIZESYEAR SELECT NULL AS [TABLE NAME], * FROM OPENROWSET
(''SQLOLEDB'',''Server=(local);TRUSTED_CONNECTION=YES;'',''set fmtonly off EXEC one.[dbo].[InvestigateDataGrowthByYearAndClient] #pDATECOLUMN =' + #YEARCOLUMN + ' ,
#pTABLENAME = ' + #TABLENAME + ' WITH RESULT SETS(
([YEAR NAME] NVARCHAR(5) NULL
, [NUMBER OF ROWS] CHAR(11)
, [RESERVED SPACE] VARCHAR(18)
, [DATA SPACE] VARCHAR(18)
, [INDEX SIZE] VARCHAR(18)
, [UNUSED SPACE] VARCHAR(18) )
)
;'') '
DECLARE #ParmDefinition NVARCHAR(500) = '#pDATECOLUMN NVARCHAR(20)
,#YEARCOLUMN NVARCHAR(20)
,#pTABLENAME NVARCHAR(60)';
EXECUTE sp_executesql #sql
,#ParmDefinition
,#YEARCOLUMN = #YEARCOLUMN
,#pDATECOLUMN = #YEARCOLUMN
,#pTABLENAME = #TABLENAME
I have A stored procedure written
DECLARE #AreaID AS INT
DECLARE #DayPrior AS INT
DECLARE #TableName AS VARCHAR(50)
DECLARE #StoreQuery AS NVARCHAR(MAX)
DECLARE #SQL_ExtractDlSql AS NVARCHAR(MAX)
DECLARE #ParameterDefinition AS NVARCHAR(2000)
SET #AreaID = 1
SET #DayPrior = 1
SET #TableName = 'Tbl_Lube'
SET #SQL_ExtractDlSql = 'SELECT Download_SQL From
HDDDataPackage.dbo.tbl_HDD_DataDownloadSetting
Where AreaId=#AreaID AND TableName=#TableName'
SET #ParameterDefinition = '#AreaID INT,#DayPrior INT,#TableName VARCHAR(50)'
EXECUTE sp_executesql #SQL_ExtractDlSql,
#ParameterDefinition,
#AreaID,
#DayPrior,
#TableName
PRINT #SQL_ExtractDlSql
I above Stored procedure, I get What I wanted on print, but instead of doing that, I wanted the selected data stores as a variable.
Then what I done is I declared a variable #StoreSql and made some modified into the SQL statement show below:
Variable Declaration:
DECLARE #StoreSql AS NVARCHAR(MAX)
Modified the Code:
SET #SQL_ExtractDlSql = 'SELECT #StoreSql = Download_SQL From
HDDDataPackage.dbo.tbl_HDD_DataDownloadSetting
Where AreaId=#AreaID AND TableName=#TableName'
I executed the code, I get error Must declare the scalar variable "#StoreSql".
I have totally no idea how to deal with this.Any one can help.Beside,I am new to sqlserver,new to stored procedures.
Please don't hesitate to point out my any mistake,any..such as logic or etc. I learn from mistake.
Regards
LiangCk
The dynamic SQL string cannot reference variables from the outer scope. You need to declare it as an output parameter
SET #ParameterDefinition = '#AreaID INT,
#DayPrior INT,
#TableName VARCHAR(50),
#StoreSql NVARCHAR(MAX) OUTPUT'
EXECUTE sp_executesql #SQL_ExtractDlSql,
#ParameterDefinition,
#AreaID,
#DayPrior,
#TableName,
#StoreSql OUTPUT