Can some one please advise how to extract text from the field below? It is a JSON file which is stored as a CLOB datatype field in an Oracle table.
The field name is Rules and its value is as below:
{
"condition":[
{
"name":"",
"property":"ipaddress",
"type":"range",
"operator":"range",
"start":"2.117.11.1",
"end":"2.117.11.254"
}
],
"operator":{
"property":"or"
},
"outcome":{
"name":"BRSBRS",
"reason":"Site was created on Fri Apr 20 2018 09:45:46 GMT+0100 (GMT Daylighttime)"
}
}
I want to extract the two IP address from above in two different fields as shown below:
Assuming that you had a typo in your string, and it is, in fact, a valid JSON document (meaning: you are missing a colon after "condition", you can use the json_table() function, available since Oracle 12.1.
create table tbl (id number primary key, rules clob);
insert into tbl (id, rules)
select 1001, '{"condition":[{"name":"","property":"ipaddress","type":"range","operator":"range","start":"2.117.11.1","end":"2.117.11.254"}],"operator":{"property":"or"},"outcome": {"name":"BRSBRS","reason":"Site was created on Fri Apr 20 2018 09:45:46 GMT+0100 (GMT Daylighttime)"}}'
from dual;
select id, start_ip_address, end_ip_address
from tbl,
json_table(rules format json, '$.condition[*]'
columns start_ip_address varchar2(19) path '$.start',
end_ip_address varchar2(19) path '$.end'
)
;
ID START_IP_ADDRESS END_IP_ADDRESS
---- ---------------- ----------------
1001 2.117.11.1 2.117.11.254
Related
Want to execute a query to view single date-month-year time column to separate date column, month column and year column.
eg
joining_date
01-JAN-22 12.00.00AM
to
joining_date|joining_month|joining_year
01 | JAN | 22
You have some ways of doing this:
If your data is always in this 01-JAN-22 12.00.00AM format , no matter what comes after 22, you can use substring.
select substring('01-JAN-22 12.00.00AM',1,2) as joining_date,
substring('01-JAN-22 12.00.00AM',4,3) as joining_month,
substring('01-JAN-22 12.00.00AM',8,2) as joining_year;
Result:
joining_date joining_month joining_year
01 JAN 22
Another option is converting the string to proper date datatype an use MySQL functions, like :
select DAY(str_to_date('01-JAN-22 12.00.00AM', "%d-%b-%y")) as joining_date,
MONTH(str_to_date('01-JAN-22 12.00.00AM', "%d-%b-%y")) as joining_month,
YEAR(str_to_date('01-JAN-22 12.00.00AM', "%d-%b-%y")) as joining_year ;
Result:
joining_date joining_month joining_year
1 1 2022
Fiddle
Use YEAR, MONTH and DAY syntax:
SELECT
YEAR(`joining_date`) as joiningYear,
MONTH(`joining_date`) as joiningMonth,
DAY(`joining_date`) as joiningDay
FROM tableName
If you want your month name, then use MONTHNAME:
SELECT
YEAR(`joining_date`) as joiningYear,
MONTHNAME(`joining_date`) as joiningMonth,
DAY(`joining_date`) as joiningDay
FROM tableName
I need to load several similar csv files into work tables with the same format for onward processing but for some of the data I get 'ORA-00984: column not allowed here' errors.
I can't change the layout of the csv but the ordering of the columns in the work table and the format of the sqlldr control file are in my control.
What do I need to change to get sqlldr to load this data?
EDIT: Solution: The following change to the .ctl file:
col6_fixedchar constant "abc", fixes the issue, interestingly sqlldr is quite happy with interpreting "3600" as a number.
Below is a sample:
table:
create table test_sqlldr
(
col1_date date,
col2_char varchar2(15),
col3_int number(5),
col4_int number(5),
col5_int number(5),
-- fixed and dummy fields
col6_fixedchar varchar2(15),
col7_nullchar varchar2(20),
col8_fixedint number(5)
);
csv:
cat /tmp/test_sqlldr.csv
2019-08-27 09:00:00,abcdefghi,3600,0,0
2019-08-27 09:00:00,jklmnopqr,3600,0,0
2019-08-27 09:00:00,stuvwxyza,3600,3598,3598
2019-08-27 09:00:00,bcdefghij,3600,0,0
ctl:
cat /tmp/test_sqlldr.ctl
load data infile '/tmp/test_sqlldr.csv'
insert into table test_sqlldr
fields terminated by ',' optionally enclosed by '"' TRAILING NULLCOLS
(
col1_date timestamp 'yyyy-mm-dd hh24:mi:ss',
col2_char,
col3_int,
col4_int,
col5_int,
col6_fixedchar "abc",
col8_fixedint "3600"
)
This generates the following output:
/opt/oracle/product/112020_cl_64/cl/bin/sqlldr <db credentials> control='/tmp/test_sqlldr.ctl' ; cat test_sqlldr.log
SQL*Loader: Release 12.2.0.1.0 - Production on Wed Aug 28 10:26:00 2019
Copyright (c) 1982, 2017, Oracle and/or its affiliates. All rights reserved.
Path used: Conventional
Commit point reached - logical record count 4
Table TEST_SQLLDR:
0 Rows successfully loaded.
Check the log file:
test_sqlldr.log
for more information about the load.
SQL*Loader: Release 12.2.0.1.0 - Production on Wed Aug 28 10:26:00 2019
Copyright (c) 1982, 2017, Oracle and/or its affiliates. All rights reserved.
Control File: /tmp/test_sqlldr.ctl
Data File: /tmp/test_sqlldr.csv
Bad File: /tmp/test_sqlldr.bad
Discard File: none specified
(Allow all discards)
Number to load: ALL
Number to skip: 0
Errors allowed: 50
Bind array: 64 rows, maximum of 256000 bytes
Continuation: none specified
Path used: Conventional
Table TEST_SQLLDR, loaded from every logical record.
Insert option in effect for this table: INSERT
TRAILING NULLCOLS option in effect
Column Name Position Len Term Encl Datatype
------------------------------ ---------- ----- ---- ---- ---------------------
COL1_DATE FIRST * , O(") DATETIME yyyy-mm-dd hh24:mi:ss
COL2_CHAR NEXT * , O(") CHARACTER
COL3_INT NEXT * , O(") CHARACTER
COL4_INT NEXT * , O(") CHARACTER
COL5_INT NEXT * , O(") CHARACTER
COL6_FIXEDCHAR NEXT * , O(") CHARACTER
SQL string for column : "abc"
COL8_FIXEDINT NEXT * , O(") CHARACTER
SQL string for column : "3600"
Record 1: Rejected - Error on table TEST_SQLLDR, column COL4_INT.
ORA-00984: column not allowed here
Record 2: Rejected - Error on table TEST_SQLLDR, column COL4_INT.
ORA-00984: column not allowed here
Record 3: Rejected - Error on table TEST_SQLLDR, column COL4_INT.
ORA-00984: column not allowed here
Record 4: Rejected - Error on table TEST_SQLLDR, column COL4_INT.
ORA-00984: column not allowed here
Table TEST_SQLLDR:
0 Rows successfully loaded.
4 Rows not loaded due to data errors.
0 Rows not loaded because all WHEN clauses were failed.
0 Rows not loaded because all fields were null.
Space allocated for bind array: 115584 bytes(64 rows)
Read buffer bytes: 1048576
Total logical records skipped: 0
Total logical records read: 4
Total logical records rejected: 4
Total logical records discarded: 0
Run began on Wed Aug 28 10:26:00 2019
Run ended on Wed Aug 28 10:26:00 2019
Elapsed time was: 00:00:00.14
CPU time was: 00:00:00.03
Try: col6_fixedchar CONSTANT "abc"
I have a table with the following fields:
id, type, date, changelog.
The changelog field has 10 useful pieces of information I would like to split out into their own fields. both new and old: name, month, year, zipcode, status
So I would like to create a table with the following fields:
id, type, date, old_name, new_name, old_month, new_month, old_year, new_year, old_zipcode, new_zipcode, old_status, new_status.
When all 5 pieces of information exist it is easy but when some are missing I can’t get it to work. Any help is appreciated.
a typical changelog field doesn't have all of these pieces of information, just what is being updated.
for example:
id type date changelog
101 upd 1/1/2019 ---!hash:ActiveSupport
name:
- Adam
- Chris
month:
- 7
- 12
status:
- 1
- 3
Which would translate to:
id type date old_name new_name old_month new_month old_year new_year old_zipcode new_zipcode old_status new_status
101 upd 1/1/19 Adam Chris 7 12 1 3
This is not a complete solution (it assumes you can already parse out the values when you know they are present), but it addresses how to handle when those values are missing:
INSERT INTO tableV2 (id, type, date, old_name, new_name, and so on....)
SELECT id, type, date
, CASE WHEN INSTR(changelog, 'name:') = 0 THEN NULL
ELSE (parse the value out here)
END AS old_name
, CASE WHEN INSTR(changelog, 'name:') = 0 THEN NULL
ELSE (parse the value out here)
END AS new_name
, and so on....
FROM tableV1
;
The parsing, while not trivial, probably won't be too difficult other than the tediousness of it. You'll need to take the found "tag" location, find the 3 newlines following it (first for the tag, latter for each value), and then use those along with other string functions such as SUBSTR, LEFT... and maybe some CHAR_LENGTH(tag string) like CHAR_LENGTH('name:') to make the parsing repeatable for each tag with minor modification.
I am going to convert my current PostgreSQL database into a MongoDB version. For example, I have a table to record tweets, and another table to record multiple hashtags used by a specific tweet. What I wanna do is to use SQL to get a table like below and then export it as a .csv file so that I could import it to MongoDB.
Example:
2018-04-02 18:12:32 This plane has no outlet for me to charge my p... [{'tag': 'GucciGarden', 'airline': 'American A...
The problem that I met is that I can get a .csv file contains json array like "[{'tag': 'GucciGarden', 'airline': 'American A...", but it is a String type! And when I import it into MongoDB. The quote will be kept, which makes sth wrong.
And here is my SQL code:
SELECT tweets.tweet_id,tweets.text,
(SELECT array_to_json(array_agg(row_to_json(d)))
from (
SELECT tags.tag
FROM
tags
WHERE tags.tweet_id=tweets.tweet_id
) d
) as Tags
from tweets
Here is the result that I import into MongoDB:
{
"_id" : ObjectId("5ac59c272221ade1185ec241"),
"tweet_id" : 9.80869021435351e+17.0,
"created_at" : "2018-04-02 18:06:13",
"text" : "RT #MiraSorvino: Brad Myles shares #Delta that awareness is working- 9,000 #humantrafficking cases identified by #polarisproject National H��",
"screen_name" : "MMexville",
"favorite_count" : 0.0,
"retweet_count" : 40.0,
"source" : "the public",
"tags" : "[{'tag': 'humantrafficking', 'airline': 'Delta Air Lines'}]"}
this is because [{'tag': is not a valid json - you should have used double quotes and cast to json, eg:
let's say smth like your sample:
t=# create table c (i int, t text, j text);
CREATE TABLE
t=# insert into c values(1,'text',$$[{'tag': 'GucciGarden'}]$$);
INSERT 0 1
t=# select * from c;
i | t | j
---+------+--------------------------
1 | text | [{'tag': 'GucciGarden'}]
(1 row)
so then smth like your qry:
t=# select to_json(c) from (select i,t,replace(j,$$'$$,'"')::json j from c) c;
to_json
-------------------------------------------------
{"i":1,"t":"text","j":[{"tag": "GucciGarden"}]}
(1 row)
of course you will have positive false replacements of single quotes, eg 'tag': 'Gucci's Garden' will break the query logic, so you will have to make a more sophisticated replacement. probably with regular expressions to be neater.
I have table with column 'ID', 'File_Name'
Table
ID File_Name
123 ROSE1234_LLDAtIInstance_03012014_04292014_190038.zip
456 ROSE1234_LLDAtIInstance_08012014_04292014_190038.zip
All I need is to pickup the first date given in file name.
Required:
ID Date
123 03012014
456 08012014
Here's one method assuming 8 characters after 2nd _ is always true.
It finds the position of the first _ then looks for the position of the 2nd _ using the position of the first _+1 then it looks for the 8 characters after the 2nd _
SELECT Id
, substr(File_name, instr(File_name,'_',instr(File_name,'_')+1)+1,8) as Date
FROM Table
or
a more elegant way would be to use a RegExp_Instr Function which eliminates the need for nesting instr.
SELECT Id, substr(File_name,REGEXP_INSTR(FileName,'_',1,2)+1,8) as date
FROM dual;
Why don't you simply put the date in separate column? E.g. you can than query the (indexed) date. The theory says the date is a property of the file. It's about avoiding errors, maintainability and so on. What in the zip files? Excel sheets I suppose :-)
Use a much simplified call to REGEXP_SUBSTR( ):
SQL> with tbl(ID, File_name) as (
2 select 123, 'ROSE1234_LLDAtIInstance_03012014_04292014_190038.zip' from dual
3 union
4 select 456, 'ROSE1234_LLDAtIInstance_08012014_04292014_190038.zip' from dual
5 )
6 select ID,
7 REGEXP_SUBSTR(File_name, '_(\d{8})_', 1, 1, NULL, 1) "Date"
8 from tbl;
ID Date
---------- ----------------------------------------------------
123 03012014
456 08012014
SQL>
For 11g, click here for the parameters to REGEXP_SUBSTR( ).
EDIT: Making this a virtual column would be another way to handle it. Thanks to Epicurist's post for the idea. The virtual column will contain a date value holding the filename date once the ID and filename are committed. Add it like this:
alter table X_TEST add (filedate date generated always as (TO_DATE(REGEXP_SUBSTR(Filename, '_(\d{8})_', 1, 1, NULL, 1), 'MMDDYYYY')) virtual);
So now just insert the ID and Filename, commit and there's your filedate. Note that its read-only.