I am new to Data Scraping. I am reading the data from a file having JSON objects as one row
{"name": "Soul Sweet \u2018Taters (Step-by-Step!)", "ingredients": "4 whole Medium Sweet Potatoes\n1 cup Sugar\n1 cup Milk\n2 whole Eggs\n1 teaspoon Vanilla Extract\n1 teaspoon Salt\n1 cup Brown Sugar\n1 cup Pecans\n1/2 cup Flour\n3/4 sticks Butter", "cookTime": "PT30M","prepTime": "PT45M"}
{"name": "Cranberry-Pomegranate Sauce", "ingredients": "1 bag (about 12 To 16 Oz) Fresh Cranberries\n16 ounces, fluid Pomegranate Juice\n3/4 cups Sugar, More Or Less To Taste", "cookTime": "PT15M","prepTime": "PT2M"}
{"name": "Whiskey Maple Cream Sauce", "ingredients": "1-1/2 cup Heavy Cream\n5 Tablespoons Pure Maple Syrup\n3 Tablespoons Light Corn Syrup\n1 Tablespoon Whiskey (can Add More If Desired)","cookTime": "PT15M","prepTime": "PT5M"}
I am looking for assistance on below:
Filter rows that contain text sugar in the Ingredients column.
convert ISO prep time and ISO cook time to human-readable format to add a new column total time(prep time+cook time) for filtered rows.The ISO time is in minutes and even in Hours
Expected Output
name
total_cook_time
Soul Sweet \u2018Taters (Step-by-Step!)
75M
Cranberry-Pomegranate Sauce
17M
My sample code
import json
from datetime import datetime
currentdate=datetime.today().strftime('%Y/%m/%d')
absolutepath='project/sniper/'+'/'+currentdate+'/*.json'
new_data = []
totaltime = {}
data = [json.loads(line) for line in open('absolutepath', 'r')]
for d in data:
if 'sugar' in d.get('ingredients').lower() # case senstitive
new_data.append(d.get('name'))
total_time = int(d['prepTime'].replace('PT', '').replace('M','')) + int(d['cookTime'].replace('PT', '').replace('M',''))
totaltime[d['name']] = total_time
This is one way to do it, as long as the durations are the same format. Otherwise more logic and/or regex could be friendlier. Assumes you are working with a list of recipes called 'data'.
The new_data list holds recipes with Sugar and totaltime dictionary is the recipe name total time.
new_data = []
totaltime = {}
for d in data:
# print(d.get('ingredients')
if 'Sugar' in d.get('ingredients'): # case senstitive
new_data.append(d.get('ingredients'))
total_time = int(d['prepTime'].replace('PT', '').replace('M','')) + int(d['cookTime'].replace('PT', '').replace('M',''))
totaltime[d['name']] = total_time
totaltime
{'Soul Sweet ‘Taters (Step-by-Step!)': 75, 'Cranberry-Pomegranate Sauce': 17}
I have original dataset in json format. Let's load it in R.
library("rjson")
setwd("mydir")
getwd()
json_data <- fromJSON(paste(readLines("N1.json"), collapse=""))
uu <- unlist(json_data)
uutext <- uu[names(uu) == "text"]
And I have another dataset mydata2
mydata=read.csv(path to data/words)
I need to find the words in mydata2, only which are present in messages in json file. And then write this messages into the new document, "xyz.txt" How to do it?
chalk indirect pick reaction team skip pumpkin surprise bless ignorance
1 time patient road extent decade cemetery staircase monarch bubble abbey
2 service conglomerate banish pan friendly position tight highlight rice disappear
3 write swear break tire jam neutral momentum requirement relationship matrix
4 inspire dose jump promote trace latest absolute adjust joystick habit
5 wrong behave claim dedicate threat sell particle statement teach lamb
6 eye tissue prescription problem secretion revenge barrel beard mechanism platform
7 forest kick face wisecrack uncertainty ratio complain doubt reflection realism
8 total fee debate hall soft smart sip ritual pill category
9 contain headline lump absorption superintendent digital increase key banner second
i mean
chalk -1 number1 indirect -2 number2
template
Word1-1 number1-1; Word1-2 number 1-2; …; Word 1-10 number 1-10
Word2-1 number2-1; Word2-2 number 2-2; …; Word 2-10 number 2-10
Next time pls include real data. Simplified model:
library(data.table)
word = c("test","meh","blah")
jsonF = c("let's do test", "blah is right", "test blah", "test test")
outp <- list()
for (i in 1:length(word)) {
outp[[i]] = as.data.frame(grep(word[i],jsonF,v=T,fixed=T)) # possibly, ignore.case=T
}
qq = rbindlist(outp)
qq = unique(qq)
print(qq)
1: let's do test
2: test blah
3: test test
4: blah is right
Edit: quick and dirty paste/collapse:
library(data.table)
x = LETTERS[1:10]
y = LETTERS[11:20]
df = rbind(x,y)
L = list()
for (i in 1:nrow(df)) {
L[i] = paste0(df[i,],"-",seq(1,10)," ",i,"-",seq(1,10),collapse="; ")
}
Fin = cbind(L)
View(Fin)
Gives:
> Fin
L
[1,] "A-1 1-1; B-2 1-2; C-3 1-3; D-4 1-4; E-5 1-5; F-6 1-6; G-7 1-7; H-8 1-8; I-9 1-9; J-10 1-10"
[2,] "K-1 2-1; L-2 2-2; M-3 2-3; N-4 2-4; O-5 2-5; P-6 2-6; Q-7 2-7; R-8 2-8; S-9 2-9; T-10 2-10"
I need a little help, in our class we've been playing around with GREP and SED commands in an attempt to learn how they work. More specifically we've been using sed commands to manipulate text and add tags.
So, we we're given an assignment, we've been given 500 lines of CSV fake data and it is our job to create a sed command that will automatically tag the data and tag any new data added down the road (theoretically).
Here's a few lines of our fake UN-TAGGED data, this is by default how we received it, as you can see all the data starts with a first name and ends with a web email:
FirstName,LastName,Company,Address,City,County,State,ZIP,Phone,Fax,Email,Web
"Essie","Vaill","Litronic Industries","14225 Hancock Dr","Anchorage","Anchorage","AK","99515","907-345-0962","907-345-1215","essie#vaill.com","http://www.essievaill.com"
"Cruz","Roudabush","Meridian Products","2202 S Central Ave","Phoenix","Maricopa","AZ","85004","602-252-4827","602-252-4009","cruz#roudabush.com","http://www.cruzroudabush.com"
"Billie","Tinnes","D & M Plywood Inc","28 W 27th St","New York","New York","NY","10001","212-889-5775","212-889-5764","billie#tinnes.com","http://www.billietinnes.com"
"Zackary","Mockus","Metropolitan Elevator Co","286 State St","Perth Amboy","Middlesex","NJ","08861","732-442-0638","732-442-5218","zackary#mockus.com","http://www.zackarymockus.com"
"Rosemarie","Fifield","Technology Services","3131 N Nimitz Hwy #-105","Honolulu","Honolulu","HI","96819","808-836-8966","808-836-6008","rosemarie#fifield.com","http://www.rosemariefifield.com"
"Bernard","Laboy","Century 21 Keewaydin Prop","22661 S Frontage Rd","Channahon","Will","IL","60410","815-467-0487","815-467-1244","bernard#laboy.com","http://www.bernardlaboy.com"
"Sue","Haakinson","Kim Peacock Beringhause","9617 N Metro Pky W","Phoenix","Maricopa","AZ","85051","602-953-2753","602-953-0355","sue#haakinson.com","http://www.suehaakinson.com"
"Valerie","Pou","Sea Port Record One Stop Inc","7475 Hamilton Blvd","Trexlertown","Lehigh","PA","18087","610-395-8743","610-395-6995","valerie#pou.com","http://www.valeriepou.com"
"Lashawn","Hasty","Kpff Consulting Engineers","815 S Glendora Ave","West Covina","Los Angeles","CA","91790","626-960-6738","626-960-1503","lashawn#hasty.com","http://www.lashawnhasty.com"
"Marianne","Earman","Albers Technologies Corp","6220 S Orange Blossom Trl","Orlando","Orange","FL","32809","407-857-0431","407-857-2506","marianne#earman.com","http://www.marianneearman.com"
"Justina","Dragaj","Uchner, David D Esq","2552 Poplar Ave","Memphis","Shelby","TN","38112","901-327-5336","901-327-2911","justina#dragaj.com","http://www.justinadragaj.com"
"Mandy","Mcdonnell","Southern Vermont Surveys","343 Bush St Se","Salem","Marion","OR","97302","503-371-8219","503-371-1118","mandy#mcdonnell.com","http://www.mandymcdonnell.com"
"Conrad","Lanfear","Kahler, Karen T Esq","49 Roche Way","Youngstown","Mahoning","OH","44512","330-758-0314","330-758-3536","conrad#lanfear.com","http://www.conradlanfear.com"
"Cyril","Behen","National Paper & Envelope Corp","1650 S Harbor Blvd","Anaheim","Orange","CA","92802","714-772-5050","714-772-3859","cyril#behen.com","http://www.cyrilbehen.com"
"Shelley","Groden","Norton, Robert L Esq","110 Broadway St","San Antonio","Bexar","TX","78205","210-229-3017","210-229-9757","shelley#groden.com","http://www.shelleygroden.com"
Our teacher wanted us to create sed commands that would automatically indent the data, add TR to the front and back of the data and add TD tags to each new field.
<HTML>
<HEAD><Title>Lab 4b by Andrey</Title></HEAD>
<BODY>
<table border="1">
<TR><TD>FirstName</TD><TD>LastName</TD><TD>Company</TD><TD>Address</TD><TD>City</TD><TD>County</TD><TD>State</TD><TD>ZIP</TD><TD>Phone</TD><TD>Fax</TD><TD>Email</TD><TD>Web</TD></TR>
<TR><TD>Essie</TD><TD>Vaill</TD><TD>Litronic Industries</TD><TD>14225 Hancock Dr</TD><TD>Anchorage</TD><TD>Anchorage</TD><TD>AK</TD><TD>99515</TD><TD>907-345-0962</TD><TD>907-345-1215</TD><TD>essie#vaill.com</TD><TD>http://www.essievaill.com</TD><TR>
<TR><TD>Cruz</TD><TD>Roudabush</TD><TD>Meridian Products</TD><TD>2202 S Central Ave</TD><TD>Phoenix</TD><TD>Maricopa</TD><TD>AZ</TD><TD>85004</TD><TD>602-252-4827</TD><TD>602-252-4009</TD><TD>cruz#roudabush.com</TD><TD>http://www.cruzroudabush.com</TD><TR>
<TR><TD>Billie</TD><TD>Tinnes</TD><TD>D & M Plywood Inc</TD><TD>28 W 27th St</TD><TD>New York</TD><TD>New York</TD><TD>NY</TD><TD>10001</TD><TD>212-889-5775</TD><TD>212-889-5764</TD><TD>billie#tinnes.com</TD><TD>http://www.billietinnes.com</TD><TR>
<TR><TD>Zackary</TD><TD>Mockus</TD><TD>Metropolitan Elevator Co</TD><TD>286 State St</TD><TD>Perth Amboy</TD><TD>Middlesex</TD><TD>NJ</TD><TD>08861</TD><TD>732-442-0638</TD><TD>732-442-5218</TD><TD>zackary#mockus.com</TD><TD>http://www.zackarymockus.com</TD><TR>
<TR><TD>Rosemarie</TD><TD>Fifield</TD><TD>Technology Services</TD><TD>3131 N Nimitz Hwy #-105</TD><TD>Honolulu</TD><TD>Honolulu</TD><TD>HI</TD><TD>96819</TD><TD>808-836-8966</TD><TD>808-836-6008</TD><TD>rosemarie#fifield.com</TD><TD>http://www.rosemariefifield.com<$
<TR><TD>Bernard</TD><TD>Laboy</TD><TD>Century 21 Keewaydin Prop</TD><TD>22661 S Frontage Rd</TD><TD>Channahon</TD><TD>Will</TD><TD>IL</TD><TD>60410</TD><TD>815-467-0487</TD><TD>815-467-1244</TD><TD>bernard#laboy.com</TD><TD>http://www.bernardlaboy.com</TD><TR>
<TR><TD>Sue</TD><TD>Haakinson</TD><TD>Kim Peacock Beringhause</TD><TD>9617 N Metro Pky W</TD><TD>Phoenix</TD><TD>Maricopa</TD><TD>AZ</TD><TD>85051</TD><TD>602-953-2753</TD><TD>602-953-0355</TD><TD>sue#haakinson.com</TD><TD>http://www.suehaakinson.com</TD><TR>
<TR><TD>Valerie</TD><TD>Pou</TD><TD>Sea Port Record One Stop Inc</TD><TD>7475 Hamilton Blvd</TD><TD>Trexlertown</TD><TD>Lehigh</TD><TD>PA</TD><TD>18087</TD><TD>610-395-8743</TD><TD>610-395-6995</TD><TD>valerie#pou.com</TD><TD>http://www.valeriepou.com</TD><TR>
<TR><TD>Lashawn</TD><TD>Hasty</TD><TD>Kpff Consulting Engineers</TD><TD>815 S Glendora Ave</TD><TD>West Covina</TD><TD>Los Angeles</TD><TD>CA</TD><TD>91790</TD><TD>626-960-6738</TD><TD>626-960-1503</TD><TD>lashawn#hasty.com</TD><TD>http://www.lashawnhasty.com</TD><T$
<TR><TD>Marianne</TD><TD>Earman</TD><TD>Albers Technologies Corp</TD><TD>6220 S Orange Blossom Trl</TD><TD>Orlando</TD><TD>Orange</TD><TD>FL</TD><TD>32809</TD><TD>407-857-0431</TD><TD>407-857-2506</TD><TD>marianne#earman.com</TD><TD>http://www.marianneearman.com</TD$
<TR><TD>Justina</TD><TD>Dragaj</TD><TD>Uchner David D Esq</TD><TD>2552 Poplar Ave</TD><TD>Memphis</TD><TD>Shelby</TD><TD>TN</TD><TD>38112</TD><TD>901-327-5336</TD><TD>901-327-2911</TD><TD>justina#dragaj.com</TD><TD>http://www.justinadragaj.com</TD><TR>
<TR><TD>Mandy</TD><TD>Mcdonnell</TD><TD>Southern Vermont Surveys</TD><TD>343 Bush St Se</TD><TD>Salem</TD><TD>Marion</TD><TD>OR</TD><TD>97302</TD><TD>503-371-8219</TD><TD>503-371-1118</TD><TD>mandy#mcdonnell.com</TD><TD>http://www.mandymcdonnell.com</TD><TR>
<TR><TD>Conrad</TD><TD>Lanfear</TD><TD>Kahler Karen T Esq</TD><TD>49 Roche Way</TD><TD>Youngstown</TD><TD>Mahoning</TD><TD>OH</TD><TD>44512</TD><TD>330-758-0314</TD><TD>330-758-3536</TD><TD>conrad#lanfear.com</TD><TD>http://www.conradlanfear.com</TD><TR>
<TR><TD>Cyril</TD><TD>Behen</TD><TD>National Paper & Envelope Corp</TD><TD>1650 S Harbor Blvd</TD><TD>Anaheim</TD><TD>Orange</TD><TD>CA</TD><TD>92802</TD><TD>714-772-5050</TD><TD>714-772-3859</TD><TD>cyril#behen.com</TD><TD>http://www.cyrilbehen.com</TD><TR>
<TR><TD>Shelley</TD><TD>Groden</TD><TD>Norton Robert L Esq</TD><TD>110 Broadway St</TD><TD>San Antonio</TD><TD>Bexar</TD><TD>TX</TD><TD>78205</TD><TD>210-229-3017</TD><TD>210-229-9757</TD><TD>shelley#groden.com</TD><TD>http://www.shelleygroden.com</TD><TR>
</table>
</BODY>
</HTML>
So, I was messing around and I tired to create a few sed commands that would mimic the second output.
My first attempt was:
#!/bin/sh
sed -e 's=^.*$=<TR><TD>&</TD></TR>=' input.csv
Unfortunately, this program only outputs something like this where I get TR TD at the beginning and end, but no TD tags inside:
<TR><TD>"Bryan","Rovell","All N All Shop","90 Hackensack St","East Rutherford","Bergen","NJ","07073","201-939-2788","201-939-9079","bryan#rovell.com","http://www.bryanrovell.com"</TD></TR>
<TR><TD>"Joey","Bolick","Utility Trailer Sales","7700 N Council Rd","Oklahoma City","Oklahoma","OK","73132","405-728-5972","405-728-5244","joey#bolick.com","http://www.joeybolick.com"</TD></TR>
I've also attempted to create individual seds to tag field, but instead I've only managed to tag each word, so I'm kinda stuck.
I'm partially on the right track, I think, but I need helping indenting and adding TD to the beginning & end of every field, along with TR to the beginning and end of each new column.
This is the main part of it:
$ sed -r 's:^"?: <TR><TD>:; s:"?,"?:</TD><TD>:g; s:"?$:</TD></TR>:' file
<TR><TD>FirstName</TD><TD>LastName</TD><TD>Company</TD><TD>Address</TD><TD>City</TD><TD>County</TD><TD>State</TD><TD>ZIP</TD><TD>Phone</TD><TD>Fax</TD><TD>Email</TD><TD>Web</TD></TR>
<TR><TD>Essie</TD><TD>Vaill</TD><TD>Litronic Industries</TD><TD>14225 Hancock Dr</TD><TD>Anchorage</TD><TD>Anchorage</TD><TD>AK</TD><TD>99515</TD><TD>907-345-0962</TD><TD>907-345-1215</TD><TD>essie#vaill.com</TD><TD>http://www.essievaill.com</TD></TR>
<TR><TD>Cruz</TD><TD>Roudabush</TD><TD>Meridian Products</TD><TD>2202 S Central Ave</TD><TD>Phoenix</TD><TD>Maricopa</TD><TD>AZ</TD><TD>85004</TD><TD>602-252-4827</TD><TD>602-252-4009</TD><TD>cruz#roudabush.com</TD><TD>http://www.cruzroudabush.com</TD></TR>
<TR><TD>Billie</TD><TD>Tinnes</TD><TD>D & M Plywood Inc</TD><TD>28 W 27th St</TD><TD>New York</TD><TD>New York</TD><TD>NY</TD><TD>10001</TD><TD>212-889-5775</TD><TD>212-889-5764</TD><TD>billie#tinnes.com</TD><TD>http://www.billietinnes.com</TD></TR>
<TR><TD>Zackary</TD><TD>Mockus</TD><TD>Metropolitan Elevator Co</TD><TD>286 State St</TD><TD>Perth Amboy</TD><TD>Middlesex</TD><TD>NJ</TD><TD>08861</TD><TD>732-442-0638</TD><TD>732-442-5218</TD><TD>zackary#mockus.com</TD><TD>http://www.zackarymockus.com</TD></TR>
<TR><TD>Rosemarie</TD><TD>Fifield</TD><TD>Technology Services</TD><TD>3131 N Nimitz Hwy #-105</TD><TD>Honolulu</TD><TD>Honolulu</TD><TD>HI</TD><TD>96819</TD><TD>808-836-8966</TD><TD>808-836-6008</TD><TD>rosemarie#fifield.com</TD><TD>http://www.rosemariefifield.com</TD></TR>
<TR><TD>Bernard</TD><TD>Laboy</TD><TD>Century 21 Keewaydin Prop</TD><TD>22661 S Frontage Rd</TD><TD>Channahon</TD><TD>Will</TD><TD>IL</TD><TD>60410</TD><TD>815-467-0487</TD><TD>815-467-1244</TD><TD>bernard#laboy.com</TD><TD>http://www.bernardlaboy.com</TD></TR>
<TR><TD>Sue</TD><TD>Haakinson</TD><TD>Kim Peacock Beringhause</TD><TD>9617 N Metro Pky W</TD><TD>Phoenix</TD><TD>Maricopa</TD><TD>AZ</TD><TD>85051</TD><TD>602-953-2753</TD><TD>602-953-0355</TD><TD>sue#haakinson.com</TD><TD>http://www.suehaakinson.com</TD></TR>
<TR><TD>Valerie</TD><TD>Pou</TD><TD>Sea Port Record One Stop Inc</TD><TD>7475 Hamilton Blvd</TD><TD>Trexlertown</TD><TD>Lehigh</TD><TD>PA</TD><TD>18087</TD><TD>610-395-8743</TD><TD>610-395-6995</TD><TD>valerie#pou.com</TD><TD>http://www.valeriepou.com</TD></TR>
<TR><TD>Lashawn</TD><TD>Hasty</TD><TD>Kpff Consulting Engineers</TD><TD>815 S Glendora Ave</TD><TD>West Covina</TD><TD>Los Angeles</TD><TD>CA</TD><TD>91790</TD><TD>626-960-6738</TD><TD>626-960-1503</TD><TD>lashawn#hasty.com</TD><TD>http://www.lashawnhasty.com</TD></TR>
<TR><TD>Marianne</TD><TD>Earman</TD><TD>Albers Technologies Corp</TD><TD>6220 S Orange Blossom Trl</TD><TD>Orlando</TD><TD>Orange</TD><TD>FL</TD><TD>32809</TD><TD>407-857-0431</TD><TD>407-857-2506</TD><TD>marianne#earman.com</TD><TD>http://www.marianneearman.com</TD></TR>
<TR><TD>Justina</TD><TD>Dragaj</TD><TD>Uchner</TD><TD> David D Esq</TD><TD>2552 Poplar Ave</TD><TD>Memphis</TD><TD>Shelby</TD><TD>TN</TD><TD>38112</TD><TD>901-327-5336</TD><TD>901-327-2911</TD><TD>justina#dragaj.com</TD><TD>http://www.justinadragaj.com</TD></TR>
<TR><TD>Mandy</TD><TD>Mcdonnell</TD><TD>Southern Vermont Surveys</TD><TD>343 Bush St Se</TD><TD>Salem</TD><TD>Marion</TD><TD>OR</TD><TD>97302</TD><TD>503-371-8219</TD><TD>503-371-1118</TD><TD>mandy#mcdonnell.com</TD><TD>http://www.mandymcdonnell.com</TD></TR>
<TR><TD>Conrad</TD><TD>Lanfear</TD><TD>Kahler</TD><TD> Karen T Esq</TD><TD>49 Roche Way</TD><TD>Youngstown</TD><TD>Mahoning</TD><TD>OH</TD><TD>44512</TD><TD>330-758-0314</TD><TD>330-758-3536</TD><TD>conrad#lanfear.com</TD><TD>http://www.conradlanfear.com</TD></TR>
<TR><TD>Cyril</TD><TD>Behen</TD><TD>National Paper & Envelope Corp</TD><TD>1650 S Harbor Blvd</TD><TD>Anaheim</TD><TD>Orange</TD><TD>CA</TD><TD>92802</TD><TD>714-772-5050</TD><TD>714-772-3859</TD><TD>cyril#behen.com</TD><TD>http://www.cyrilbehen.com</TD></TR>
<TR><TD>Shelley</TD><TD>Groden</TD><TD>Norton</TD><TD> Robert L Esq</TD><TD>110 Broadway St</TD><TD>San Antonio</TD><TD>Bexar</TD><TD>TX</TD><TD>78205</TD><TD>210-229-3017</TD><TD>210-229-9757</TD><TD>shelley#groden.com</TD><TD>http://www.shelleygroden.com</TD></TR>
I expect you can figure out the rest since that's just printing the head and tail lines.
I am using a bash script to generate a report for higher management and sending it through mail with the help of cronjob but my problem is that we need Grand_Total line in bold format.
I am getting results by below multiple queries:
mysql -H -u$DBUSER -p$DBPASS test -e "SELECT IFNULL(cust_name,'Grand_Total') AS cust_name,SUM(order_amt) AS amt FROM mytable1 GROUP BY cust_name WITH ROLLUP;" > REPORT_FILE.txt
mysql -H -u$DBUSER -p$DBPASS test -e "SELECT IFNULL(cust_name,'Grand_Total') AS cust_name,SUM(order_amt) AS amt FROM mytable2 GROUP BY cust_name WITH ROLLUP;" >> REPORT_FILE.txt
mysql -H -u$DBUSER -p$DBPASS test -e "SELECT IFNULL(cust_name,'Grand_Total') AS cust_name,SUM(order_amt) AS amt FROM mytable3 GROUP BY cust_name WITH ROLLUP;" >> REPORT_FILE.txt
It will generate output in html format in REPORT_FILE.txt file.
I want to Bold my "Grand_Total" line either through mysql query (if there is any way, as per me there is no way in mysql) or after exporting data in txt file with the help of any linux command.
You need to read the file in some other scripting language such as Python, Perl etc and you can parse the file and change the required columns as per required.
You may even use sed or awk to edit the sepcific columns you want to.
Finally I got the solution by below perl script, so first my script will generate report in mytext.txt file then this perl script will search "TD" word in each line but between <TD>Grand Total</TD> to end of line and after replacing it with "TH" print it to myoutput.txt file along with the lines those don't have these keyword. So finally my Grand Total line will be displayed as Bold in report.
#!/usr/local/bin/perl
use strict;
use warnings;
open (INPUTFILE, "mytext.txt") or die ("Unable to find the file");
open (OUTFILE, ">myoutput.txt") or die ("Unable to find the file");
my #lines = <INPUTFILE>;
foreach my $str (#lines) {
if (my ($matches) = $str =~ /(<TD>Grand Total<\/TD>(<TD>-?[0-9.]+<\/TD>)*)/) {
my $orig = $matches;
$matches =~ tr/TD/TH/ ;
$str =~ s/$orig/$matches/;
print OUTFILE $str;
}else{
print OUTFILE $str;
}
}
mytext.txt conent is below-
<br><br>Shoes Bulk Inventory Report <br><br>
<TABLE BORDER=1><TR><TH>Shoes</TH><TH>size 6</TH><TH>size 7</TH><TH>size 8</TH><TH>size 9</TH><TH>size 10</TH><TH>size 11</TH><TH>LastDay_Total</TH><TH>Today_Total</TH><TH>Quantity Change</TH><TH>% Change</TH></TR><TR><TD>Marco ferro Boot - Black - 198747</TD><TD>0</TD><TD>40</TD><TD>143</TD><TD>301</TD><TD>23</TD><TD>0</TD><TD>556</TD><TD>507</TD><TD>-49</TD><TD>-8.81</TD></TR><TR><TD>Lotto Marathon Shoes White Navy - 196553</TD><TD>2</TD><TD>410</TD><TD>105</TD><TD>26</TD><TD>152</TD><TD>0</TD><TD>705</TD><TD>695</TD><TD>-10</TD><TD>-1.42</TD></TR><TR><TD>LOTTO MAGIC SHOES - Blue - 196552</TD><TD>51</TD><TD>224</TD><TD>644</TD><TD>1925</TD><TD>698</TD><TD>0</TD><TD>3672</TD><TD>3542</TD><TD>-130</TD><TD>-3.54</TD></TR><TR><TD>Lancer LCR Footwear JJ Air White Navy - 221068</TD><TD>211</TD><TD>383</TD><TD>206</TD><TD>400</TD><TD>141</TD><TD>0</TD><TD>7</TD><TD>1341</TD><TD>1334</TD><TD>19057.14</TD></TR><TR><TD>Columbus Shoes - Green - 202033</TD><TD>65</TD><TD>190</TD><TD>134</TD><TD>128</TD><TD>532</TD><TD>640</TD><TD>2068</TD><TD>1689</TD><TD>-379</TD><TD>-18.33</TD></TR><TR><TD>Columbus Race2 White Blue Sports Shoe - 217145</TD><TD>94</TD><TD>87</TD><TD>32</TD><TD>80</TD><TD>108</TD><TD>132</TD><TD>0</TD><TD>533</TD><TD>533</TD><TD>100.00</TD></TR><TR><TD>Columbus Footwear Tab105 White Royal Blue - 215963</TD><TD>1</TD><TD>4</TD><TD>0</TD><TD>1</TD><TD>0</TD><TD>10</TD><TD>27</TD><TD>16</TD><TD>-11</TD><TD>-40.74</TD></TR><TR><TD>Stylish Biker Shoe - 198909</TD><TD>3</TD><TD>89</TD><TD>53</TD><TD>20</TD><TD>105</TD><TD>0</TD><TD>272</TD><TD>270</TD><TD>-2</TD><TD>-0.74</TD></TR><TR><TD>Rocio Sports shoe Blue White - 198456</TD><TD>10</TD><TD>2</TD><TD>164</TD><TD>203</TD><TD>107</TD><TD>0</TD><TD>487</TD><TD>486</TD><TD>-1</TD><TD>-0.21</TD></TR><TR><TD>Rocio Sports Shoe White Blue - 198457</TD><TD>0</TD><TD>0</TD><TD>9</TD><TD>66</TD><TD>9</TD><TD>0</TD><TD>108</TD><TD>84</TD><TD>-24</TD><TD>-22.22</TD></TR><TR><TD>Rocio Formal Shoes - 38087</TD><TD>67</TD><TD>183</TD><TD>206</TD><TD>98</TD><TD>54</TD><TD>0</TD><TD>656</TD><TD>608</TD><TD>-48</TD><TD>-7.32</TD></TR><TR><TD>MARCO FERRO TAN SHOES - 60894</TD><TD>334</TD><TD>6</TD><TD>6</TD><TD>164</TD><TD>268</TD><TD>0</TD><TD>784</TD><TD>778</TD><TD>-6</TD><TD>-0.77</TD></TR><TR><TD>Marco Ferro Olive Shoes - Olive - 195566</TD><TD>256</TD><TD>56</TD><TD>856</TD><TD>999</TD><TD>591</TD><TD>0</TD><TD>2814</TD><TD>2758</TD><TD>-56</TD><TD>-1.99</TD></TR><TR><TD>Marco ferro Boot - Tan - 197672</TD><TD>136</TD><TD>13</TD><TD>5</TD><TD>313</TD><TD>100</TD><TD>0</TD><TD>594</TD><TD>567</TD><TD>-27</TD><TD>-4.55</TD></TR><TR><TD>Columbus Shoes - 214619</TD><TD>7</TD><TD>90</TD><TD>82</TD><TD>30</TD><TD>87</TD><TD>0</TD><TD>283</TD><TD>296</TD><TD>13</TD><TD>4.59</TD></TR><TR><TD>Columbus Shoes - Blue - 202034</TD><TD>18</TD><TD>71</TD><TD>57</TD><TD>42</TD><TD>69</TD><TD>14</TD><TD>307</TD><TD>271</TD><TD>-36</TD><TD>-11.73</TD></TR><TR><TD>Columbus Shoes - Red - 202035</TD><TD>27</TD><TD>50</TD><TD>68</TD><TD>45</TD><TD>26</TD><TD>6</TD><TD>229</TD><TD>222</TD><TD>-7</TD><TD>-3.06</TD></TR><TR><TD>Columbus Shoes - Tab - White/Red/Black - 214618</TD><TD>33</TD><TD>145</TD><TD>86</TD><TD>73</TD><TD>88</TD><TD>0</TD><TD>420</TD><TD>425</TD><TD>5</TD><TD>1.19</TD></TR><TR><TD>Columbus Shoes - Tab White Blue - 214617</TD><TD>54</TD><TD>156</TD><TD>43</TD><TD>47</TD><TD>70</TD><TD>0</TD><TD>372</TD><TD>370</TD><TD>-2</TD><TD>-0.54</TD></TR><TR><TD>Lotto Magic Shoes - 129726</TD><TD>0</TD><TD>4</TD><TD>0</TD><TD>8</TD><TD>1</TD><TD>0</TD><TD>13</TD><TD>13</TD><TD>0</TD><TD>0.00</TD></TR><TR><TD>Grand Total</TD><TD>1369</TD><TD>2203</TD><TD>2899</TD><TD>4969</TD><TD>3229</TD><TD>802</TD><TD>14374</TD><TD>15471</TD><TD>1097</TD><TD>7.63</TD></TR></TABLE>
<br><br>Small Shoes Bulk Inventory Report <br><br>
<TABLE BORDER=1><TR><TH>Shoes</TH><TH>size 4</TH><TH>size 5</TH><TH>size 6</TH><TH>size 7</TH><TH>LastDay_Total</TH><TH>Today_Total</TH><TH>Quantity Change</TH><TH>% Change</TH></TR><TR><TD>Lotto Diva Women White Pink - 203979</TD><TD>581</TD><TD>1214</TD><TD>1067</TD><TD>475</TD><TD>3341</TD><TD>3337</TD><TD>-4</TD><TD>-0.12</TD></TR><TR><TD>Grand Total</TD><TD>581</TD><TD>1214</TD><TD>1067</TD><TD>475</TD><TD>3341</TD><TD>3337</TD><TD>-4</TD><TD>-0.12</TD></TR></TABLE>
<br><br>Apparel Bulk Inventory Report<br><br>
<TABLE BORDER=1><TR><TH>Apparel</TH><TH>Size - S</TH><TH>Size - M</TH><TH>Size - L</TH><TH>Size - XL</TH><TH>Size - XXL</TH><TH>LastDay_Total</TH><TH>Today_Total</TH><TH>Quantity Change</TH><TH>% Change</TH></TR><TR><TD>Lotto Tracksuit - 199346</TD><TD>895</TD><TD>2765</TD><TD>2763</TD><TD>1646</TD><TD>778</TD><TD>8849</TD><TD>8847</TD><TD>-2</TD><TD>-0.02</TD></TR><TR><TD>Rocio Black Bomber Jacket - 201734</TD><TD>0</TD><TD>77</TD><TD>268</TD><TD>216</TD><TD>0</TD><TD>562</TD><TD>561</TD><TD>-1</TD><TD>-0.18</TD></TR><TR><TD>Rocio Black Full Sleeve Jacket with Fur Lining - 202674</TD><TD>0</TD><TD>32</TD><TD>3</TD><TD>30</TD><TD>0</TD><TD>65</TD><TD>65</TD><TD>0</TD><TD>0.00</TD></TR><TR><TD>Rocio Blue Bomber Jacket - 201735</TD><TD>0</TD><TD>117</TD><TD>324</TD><TD>313</TD><TD>0</TD><TD>754</TD><TD>754</TD><TD>0</TD><TD>0.00</TD></TR><TR><TD>Rocio Mens Black Tracksuit with Blue Piping - 199529</TD><TD>426</TD><TD>1301</TD><TD>1300</TD><TD>683</TD><TD>350</TD><TD>4060</TD><TD>4060</TD><TD>0</TD><TD>0.00</TD></TR><TR><TD>Rocio Mens Black Tracksuit with Red Piping - 199530</TD><TD>458</TD><TD>1226</TD><TD>1330</TD><TD>739</TD><TD>332</TD><TD>4085</TD><TD>4085</TD><TD>0</TD><TD>0.00</TD></TR><TR><TD>Grand Total</TD><TD>1779</TD><TD>5518</TD><TD>5988</TD><TD>3627</TD><TD>1460</TD><TD>18375</TD><TD>18372</TD><TD>-3</TD><TD>-0.02</TD></TR></TABLE>
myoutput.txt content after replacing TD to TH:
<br><br>Shoes Bulk Inventory Report <br><br>
<TABLE BORDER=1><TR><TH>Shoes</TH><TH>size 6</TH><TH>size 7</TH><TH>size 8</TH><TH>size 9</TH><TH>size 10</TH><TH>size 11</TH><TH>LastDay_Total</TH><TH>Today_Total</TH><TH>Quantity Change</TH><TH>% Change</TH></TR><TR><TD>Marco ferro Boot - Black - 198747</TD><TD>0</TD><TD>40</TD><TD>143</TD><TD>301</TD><TD>23</TD><TD>0</TD><TD>556</TD><TD>507</TD><TD>-49</TD><TD>-8.81</TD></TR><TR><TD>Lotto Marathon Shoes White Navy - 196553</TD><TD>2</TD><TD>410</TD><TD>105</TD><TD>26</TD><TD>152</TD><TD>0</TD><TD>705</TD><TD>695</TD><TD>-10</TD><TD>-1.42</TD></TR><TR><TD>LOTTO MAGIC SHOES - Blue - 196552</TD><TD>51</TD><TD>224</TD><TD>644</TD><TD>1925</TD><TD>698</TD><TD>0</TD><TD>3672</TD><TD>3542</TD><TD>-130</TD><TD>-3.54</TD></TR><TR><TD>Lancer LCR Footwear JJ Air White Navy - 221068</TD><TD>211</TD><TD>383</TD><TD>206</TD><TD>400</TD><TD>141</TD><TD>0</TD><TD>7</TD><TD>1341</TD><TD>1334</TD><TD>19057.14</TD></TR><TR><TD>Columbus Shoes - Green - 202033</TD><TD>65</TD><TD>190</TD><TD>134</TD><TD>128</TD><TD>532</TD><TD>640</TD><TD>2068</TD><TD>1689</TD><TD>-379</TD><TD>-18.33</TD></TR><TR><TD>Columbus Race2 White Blue Sports Shoe - 217145</TD><TD>94</TD><TD>87</TD><TD>32</TD><TD>80</TD><TD>108</TD><TD>132</TD><TD>0</TD><TD>533</TD><TD>533</TD><TD>100.00</TD></TR><TR><TD>Columbus Footwear Tab105 White Royal Blue - 215963</TD><TD>1</TD><TD>4</TD><TD>0</TD><TD>1</TD><TD>0</TD><TD>10</TD><TD>27</TD><TD>16</TD><TD>-11</TD><TD>-40.74</TD></TR><TR><TD>Stylish Biker Shoe - 198909</TD><TD>3</TD><TD>89</TD><TD>53</TD><TD>20</TD><TD>105</TD><TD>0</TD><TD>272</TD><TD>270</TD><TD>-2</TD><TD>-0.74</TD></TR><TR><TD>Rocio Sports shoe Blue White - 198456</TD><TD>10</TD><TD>2</TD><TD>164</TD><TD>203</TD><TD>107</TD><TD>0</TD><TD>487</TD><TD>486</TD><TD>-1</TD><TD>-0.21</TD></TR><TR><TD>Rocio Sports Shoe White Blue - 198457</TD><TD>0</TD><TD>0</TD><TD>9</TD><TD>66</TD><TD>9</TD><TD>0</TD><TD>108</TD><TD>84</TD><TD>-24</TD><TD>-22.22</TD></TR><TR><TD>Rocio Formal Shoes - 38087</TD><TD>67</TD><TD>183</TD><TD>206</TD><TD>98</TD><TD>54</TD><TD>0</TD><TD>656</TD><TD>608</TD><TD>-48</TD><TD>-7.32</TD></TR><TR><TD>MARCO FERRO TAN SHOES - 60894</TD><TD>334</TD><TD>6</TD><TD>6</TD><TD>164</TD><TD>268</TD><TD>0</TD><TD>784</TD><TD>778</TD><TD>-6</TD><TD>-0.77</TD></TR><TR><TD>Marco Ferro Olive Shoes - Olive - 195566</TD><TD>256</TD><TD>56</TD><TD>856</TD><TD>999</TD><TD>591</TD><TD>0</TD><TD>2814</TD><TD>2758</TD><TD>-56</TD><TD>-1.99</TD></TR><TR><TD>Marco ferro Boot - Tan - 197672</TD><TD>136</TD><TD>13</TD><TD>5</TD><TD>313</TD><TD>100</TD><TD>0</TD><TD>594</TD><TD>567</TD><TD>-27</TD><TD>-4.55</TD></TR><TR><TD>Columbus Shoes - 214619</TD><TD>7</TD><TD>90</TD><TD>82</TD><TD>30</TD><TD>87</TD><TD>0</TD><TD>283</TD><TD>296</TD><TD>13</TD><TD>4.59</TD></TR><TR><TD>Columbus Shoes - Blue - 202034</TD><TD>18</TD><TD>71</TD><TD>57</TD><TD>42</TD><TD>69</TD><TD>14</TD><TD>307</TD><TD>271</TD><TD>-36</TD><TD>-11.73</TD></TR><TR><TD>Columbus Shoes - Red - 202035</TD><TD>27</TD><TD>50</TD><TD>68</TD><TD>45</TD><TD>26</TD><TD>6</TD><TD>229</TD><TD>222</TD><TD>-7</TD><TD>-3.06</TD></TR><TR><TD>Columbus Shoes - Tab - White/Red/Black - 214618</TD><TD>33</TD><TD>145</TD><TD>86</TD><TD>73</TD><TD>88</TD><TD>0</TD><TD>420</TD><TD>425</TD><TD>5</TD><TD>1.19</TD></TR><TR><TD>Columbus Shoes - Tab White Blue - 214617</TD><TD>54</TD><TD>156</TD><TD>43</TD><TD>47</TD><TD>70</TD><TD>0</TD><TD>372</TD><TD>370</TD><TD>-2</TD><TD>-0.54</TD></TR><TR><TD>Lotto Magic Shoes - 129726</TD><TD>0</TD><TD>4</TD><TD>0</TD><TD>8</TD><TD>1</TD><TD>0</TD><TD>13</TD><TD>13</TD><TD>0</TD><TD>0.00</TD></TR><TR><TH>Grand Total</TH><TH>1369</TH><TH>2203</TH><TH>2899</TH><TH>4969</TH><TH>3229</TH><TH>802</TH><TH>14374</TH><TH>15471</TH><TH>1097</TH><TH>7.63</TH></TR></TABLE>
<br><br>Small Shoes Bulk Inventory Report <br><br>
<TABLE BORDER=1><TR><TH>Shoes</TH><TH>size 4</TH><TH>size 5</TH><TH>size 6</TH><TH>size 7</TH><TH>LastDay_Total</TH><TH>Today_Total</TH><TH>Quantity Change</TH><TH>% Change</TH></TR><TR><TD>Lotto Diva Women White Pink - 203979</TD><TD>581</TD><TD>1214</TD><TD>1067</TD><TD>475</TD><TD>3341</TD><TD>3337</TD><TD>-4</TD><TD>-0.12</TD></TR><TR><TH>Grand Total</TH><TH>581</TH><TH>1214</TH><TH>1067</TH><TH>475</TH><TH>3341</TH><TH>3337</TH><TH>-4</TH><TH>-0.12</TH></TR></TABLE>
<br><br>Apparel Bulk Inventory Report<br><br>
<TABLE BORDER=1><TR><TH>Apparel</TH><TH>Size - S</TH><TH>Size - M</TH><TH>Size - L</TH><TH>Size - XL</TH><TH>Size - XXL</TH><TH>LastDay_Total</TH><TH>Today_Total</TH><TH>Quantity Change</TH><TH>% Change</TH></TR><TR><TD>Lotto Tracksuit - 199346</TD><TD>895</TD><TD>2765</TD><TD>2763</TD><TD>1646</TD><TD>778</TD><TD>8849</TD><TD>8847</TD><TD>-2</TD><TD>-0.02</TD></TR><TR><TD>Rocio Black Bomber Jacket - 201734</TD><TD>0</TD><TD>77</TD><TD>268</TD><TD>216</TD><TD>0</TD><TD>562</TD><TD>561</TD><TD>-1</TD><TD>-0.18</TD></TR><TR><TD>Rocio Black Full Sleeve Jacket with Fur Lining - 202674</TD><TD>0</TD><TD>32</TD><TD>3</TD><TD>30</TD><TD>0</TD><TD>65</TD><TD>65</TD><TD>0</TD><TD>0.00</TD></TR><TR><TD>Rocio Blue Bomber Jacket - 201735</TD><TD>0</TD><TD>117</TD><TD>324</TD><TD>313</TD><TD>0</TD><TD>754</TD><TD>754</TD><TD>0</TD><TD>0.00</TD></TR><TR><TD>Rocio Mens Black Tracksuit with Blue Piping - 199529</TD><TD>426</TD><TD>1301</TD><TD>1300</TD><TD>683</TD><TD>350</TD><TD>4060</TD><TD>4060</TD><TD>0</TD><TD>0.00</TD></TR><TR><TD>Rocio Mens Black Tracksuit with Red Piping - 199530</TD><TD>458</TD><TD>1226</TD><TD>1330</TD><TD>739</TD><TD>332</TD><TD>4085</TD><TD>4085</TD><TD>0</TD><TD>0.00</TD></TR><TR><TH>Grand Total</TH><TH>1779</TH><TH>5518</TH><TH>5988</TH><TH>3627</TH><TH>1460</TH><TH>18375</TH><TH>18372</TH><TH>-3</TH><TH>-0.02</TH></TR></TABLE>