Converting tomcat logs to CSV using sed [closed] - csv

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed last year.
Improve this question
I have some application log data on a Linux server, which looks like this:
Jan 11 14:24:42 AttackSimulator.abcd [1587566256,49294,"ryan.wright#abcd.com",3237159933,1,0,0,3,"2314","https",443,2899903330,"https://googleads.g.doubleclick.net/pagead/ads",0,"","","","","1 - Default Policy","","googleads.g.doubleclick.net","GET",4,0,5]
Jan 11 14:24:42 AttackSimulator.abcd [1587566256,49294,"melisa.zeunert#abcd.com",3237159933,1,0,0,3,"2339,37788","http",80,387803624,"http://ping.citrix.com",0,"","","","","3 - Extended Policy High","","ping.citrix.com","HEAD",3,0,4]
I want to output this in a CSV file. Everything between the [ ] should be part of the CSV with column names such as DateandTime, AccountID, UserID, ClientIP etc. Something that looks like this:
DateandTime, AccountID, UserID, ClientIP
1587566256,49294,"ryan.wright#abcd.com",3237159933
1587566256,49294,"melisa.zeunert#abcd.com",3237159933
(This is just an example with only 4 columns. Actual output would have 25)
Any input would be helpful

Using sed
$ sed 's/.*\[\(.*\)]/\1/;1iDateandTime, AccountID, UserID, ClientIP ' file
DateandTime, AccountID, UserID, ClientIP
1587566256,49294,"ryan.wright#scnx.com",3237159933,1,0,0,3,"2314","https",443,2899903330,"https://googleads.g.doubleclick.net/pagead/ads",0,"","","","","1 - Default Policy","","googleads.g.doubleclick.net","GET",4,0,5
1587566256,49294,"melisa.zeunert#scnx.com",3237159933,1,0,0,3,"2339,37788","http",80,387803624,"http://ping.citrix.com",0,"","","","","3 - Extended Policy High","","ping.citrix.com","HEAD",3,0,4
.*\[ - Exclude everything up to the last [ square bracket.
\(.*\)] - Include everything within the parenthesis up to the last ] square bracket
\1 - Return with back reference the contents stored inside the parenthesis
1i - Insert on line 1

Related

How can I add brackets to each word of a string which contains multiple words in SQL? [closed]

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 5 months ago.
The community reviewed whether to reopen this question 5 months ago and left it closed:
Original close reason(s) were not resolved
Improve this question
How can I add brackets to each word of a string which contains multiple words in SQL? For example, I have:
string example = 'word1,word2,word3,word4,word5'
How can I convert it to something like this:
string example = '[word1],[word2],[word3],[word4],[word5]'
We can use a combination of REPLACE() and string concatenation:
SELECT val, CONCAT('[', REPLACE(val, ',', '],[') , ']') AS output
FROM yourTable;
On MySQL 8+, we can use a regex replacement:
SELECT val, REGEXP_REPLACE(val, '([^,]+)', '[$1]') AS output
FROM yourTable;

How to extract the domain in a string by using regex? [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 1 year ago.
Improve this question
I got a string representing users data.
What is the proper regex to extract domain in this string?
I know that I have to find all strings with 2 characters matching the condition that it comes after the last "." after a "#".
However I still failed to implement it.
import re
regex = r"#.+\.([a-z]{2}),"
your_string = ("001,Francisca,Dr Jhonaci,jhonadr#abc.com,32yearsold,120.238.225.0\n"
"002,Lavenda,Bocina,lavenboci#banck.ac.uk,50yearsold,121.186.221.182\n"
"003,Laura,Eglington,elinton#python.co.jp,26yearsold,36.55.173.63\n"
"004,Timo,Baum,timobaum#tennis.co.cn,22yearsold,121.121.110.10")
matches = re.finditer(regex, your_string, re.MULTILINE)
for match in matches:
result = match.group(1)
print(result)
The comma seems to be the delimiter in the string.
To not cross-matching a comma (to prevent matching too much), and also not cross-matching a second # char you can use a negated character class starting with [^
If the entry can also be at the end of the string, you can assert either a , or the end of the string.
#[^#,]*\.([A-Za-z]{2})(?=,|$)
Regex demo
import re
regex = r"#[^#,]*\.([A-Za-z]{2})(?=,|$)"
s = ("001,Francisca,Dr Jhonaci,jhonadr#abc.com,32yearsold,120.238.225.0\n"
"002,Lavenda,Bocina,lavenboci#banck.ac.uk,50yearsold,121.186.221.182\n"
"003,Laura,Eglington,elinton#python.co.jp,26yearsold,36.55.173.63\n"
"004,Timo,Baum,timobaum#tennis.co.cn,22yearsold,121.121.110.10")
print(re.findall(regex, s, re.M))
Output
['uk', 'jp', 'cn']
Use the comma after the email instead of the last point.
Using this regex
#.+\.(\w+)(?<!com),
the capturing group will contain the info that you want.

Mysql query to return regex match between two columns [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
I have two columns in a database I want to compare. One is email addresses and the other is username.
I'm looking to return results if the first part of username matches the first component of the email address (stuff before the # sign).
Sample data:
username emailaddress
badzzycshulzey9802 badzzycshulzey#gmail.com
trogleddg1919 trogleddg#gmail.com
Tried a variety of queries but I can't seem to get this one.... thanks!
You don't really need a regexp for this, strings functions can do it:
select *
from mytable
where username like concat(substr(emailaddress, 1, locate('#', emailaddress) - 1), '%')
locate('#', emailaddress) gives you the position of the arobas in the email address, and substr(emailaddress, 1, locate(...) - 1) extracts everything before that. Then, we can check if the username starts with that part of the email address using like with a wildcard on the right side.

Find all entries that contain more than one colon character [closed]

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 7 years ago.
Improve this question
I need to find all entries that contain more than one colon (:) character.
However when I do LIKE %:% it shows the entire table because of http://. How can I find more than one colon?
SELECT *
FROM `downloads`
WHERE `url` LIKE '%:%'
LIMIT 0 , 30
If you want to find a colon that occurs after the scheme of your URL, then change your LIKE clause accordingly:
SELECT *
FROM `downloads`
WHERE `url` LIKE '%:%:%'
LIMIT 0 , 30
The first colon will be in your scheme, and the second will be somewhere else in the Url after the scheme.
A word of caution, however - it is completely valid to have a colon in the Url when a port number is specified, e.g.: http://localhost:8080

Append $ character to a MySQL DB field so that length is increased by 40 % [closed]

Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 8 years ago.
Improve this question
I want to append any character, say $, to the name of the student field in database such that its length is increased by 40 %, and save the result in the database.
E.g.:
Name: VijayKumar
After update:
Name: VijayKumar$$$$
You would use concat() and repeat():
select concat(name, repeat('$', ceil(length(name) * 0.4))