I have three .csv files that are output from saving a query in MS SQLServer. I need to load these files into an Informix database, which requires that tacking on of a trailing delimiter. That's easy to do using sed
s/$/,/g
However, each of these files also contains (as displayed by vim, but not ed or sed) an at the first character position of the first line.
I need to get rid of this character. It deletes as one character using vim's x command. How can I describe this character using sed so I can delete it without removing the line.
I've tried 1s/^.//g, but that is not working.
Try this instead:
sed -e '1s/^.//' input_file > output_file
Or if you'd like to edit the files in-place:
sed -ie '1s/^.//' input_file
(Edited) Apparently s/^.// doesn't quote do it, updated.
Remove the first character on the first line inplace:
sed -i '1s/^.//' file
try:
sed -i '1s/^.\(.*\)/\1/' file
this should remove the first character from the first line. (try it without the -i argument first to make sure)
edit: i originally posted the following, which would delete the first character from every line. upon re-reading the question i realized that isn't quite what was wanted.
sed -i 's/^.\(.*\)/\1/' file
Related
I want to replace "}{" by "},{" to make a large txt file into valid json. Need help !!!
This can be achieved with sed:
sed -i 's/}{/},{/g' filename
sed is the command, -i implies that the changes have to made to a file, which name you're giving at the end (and you should change filename, of course).
The substitution part starts with the s, between the first //, you set what you want to replace, between the last //, you set what you want instead. The g at the end makes sure that this search/replace is not only executed once, but as long as sed finds matches.
If you have any newlines present after the }, you can simply remove them all, you'll still get a valid JSON afterwards:
cat filename | tr -d '\n' | sed 's/}{/},{/g' >newfilename
This would simply delete all newlines (\n) and pass it to the command. It will create a new file, though.
Quick sed question
In a text file how do I remove all characters up to and including the first '[' found in the entire file and nothing else?
I tried
sed "s/^[^\[]*\[//" example.json
but it's stripping out all text on every line.
Alternately,
I have a set of files that are sets of JSON documents. I am trying to import them into elasticsearch, but the first document in the file is an informational document with a non-standard layout that messes up the importing of the rest of the documents. I'm trying to get rid of the first document so the subsequent documents can load properly.
Here is the document:
https://earthquake.usgs.gov/fdsnws/event/1/query?format=geojson&starttime=2014-01-01&endtime=2014-01-02
For anything other than a simple s/old/new on individual strings just use awk. This will work using any awk in any shell on every UNIX box:
awk 'sub(/[^[]*\[/,""){f=1} f' file
I used two sed commands to achieve the task to "remove all characters up to and including the first '[' found"
sed -n '/\[/,$p' example.json | sed -r '1 s/[^\[]*\[(.*)/\1/'
I know how to do this in a text editor but it shuts my computer down because I have a large file. I have tab-separated data like this all on a single line:
XP_23947974 XM_23947974 HG12390 product=blahblah NP_23947975 XM_23947975 HG12391 product=blahblah2
And I want to insert a line break at every either XP or NP. So, as it is tab-separated, in the text editor I was just going to do
Find:(\D)P_
Replace:\n\1P_
Giving
XP_23947974 XM_23947974 HG12390 product=blahblah
NP_23947975 XM_23947975 HG12391 product=blahblah2
But I want to use sed (etc) to do that. help appreciated.
This should do the trick:
sed -e 's/\(XP\|NP\)/\n\1/g'
You can test this with:
echo 'XP_23947974 XM_23947974 HG12390 product=blahblah NP_23947975 XM_23947975 HG12391 product=blahblah2' | sed -e 's/\(XP\|NP\)/\n\1/g'
I'd like to use sed to remove tabs from otherwise blank lines. For example a line containing only \t\n should change to \n. What's the syntax for this?
sed does not know about escape sequences like \t. So you will have to literally type a tab on your console:
sed 's/^ *$//g' <filename>
If you are on bash, then you can't type tab on the console. You will have to do ^V and then press tab. (Ctrl-V and then tab) to print a literal tab.
The other posted solution will work when there is 1 (and only 1) tab in the line. Note that Raze2dust points out that sed requires you to type a literal tab. An alternative is:
sed '/[^ ]/!s/ //g' file-name.txt
Which substitues away tabs from lines that only have tabs. The inverted class matches lines that contain anything bug a tab - the following '!' causes it to not match those lines - meaning only lines that have only tabs. The substitution then only runs on those lines, removing all tabs.
To replace arbitrary whitespace lines with an empty line, use
sed -r 's/^\s+$//'
The -r flag says to use extended regular expressions, and the ^\s+$ pattern matches all lines with some whitespace but no other characters.
What worked for me was:
sed -r '/^\s+$/d' my_file.txt > output.txt
I've noticed \t is not recognized by UNIX. Being said, use the actual key. In the code below, TAB represents pressing the tab key.
$ sed 's/TAB//g' oldfile > newfile
Friendly tip: to ensure you have tabs in the file you are trying to remove tabs from use the following code to see if \t appears
$ od -c filename
grep -o ".*" file > a; mv a file;
I'm trying to replace the content of some HTML tags in an HTML page using sed in a bash script. For some reason I'm not getting the proper result as it's not replacing anything. It has to be something very simple/stupid im overlooking, anyone care to help me out?
HTML to search/replace in:
Unlocked <span id="unlockedCount"></span>/<span id="totalCount"></span> achievements for <span id="totalPoints"></span> points.
sed command used:
cat index.html | sed -i -e "s/\<span id\=\"unlockedCount\"\>([0-9]\{0,\})\<\/span\>/${unlockedCount}/g" index.html
The point of this is to parse the HTML page and update the figures according to some external data. For a first run, the contents of the tags will be empty, after that they will be filled.
EDIT:
I ended up using a combination of the answers which resulted in the following code:
sed -i -e 's|<span id="unlockedCount">\([0-9]\{0,\}\)</span>|<span id="unlockedCount">'"${unlockedCount}"'</span>|g' index.html
Many thanks to #Sorpigal, #tripleee, #classic for the help!
Try this:
sed -i -e "s/\(<span id=\"unlockedCount\">\)\(<\/span>\)/\1${unlockedCount}\2/g" index.html
What you say you want to do is not what you're telling sed to do.
You want to insert a number into a tag or replace it if present. What you're trying to tell sed to do is to replace a span tag and its contents, if any or a number, with the value of in a shell variable.
You're also employing a lot of complex, annoying and erorr-prone escape sequences which are just not necessary.
Here's what you want:
sed -r -i -e 's|<span id="unlockedCount">([0-9]{0,})</span>|<span id="unlockedCount">'"${unlockedCount}"'</span>|g' index.html
Note the differences:
Added -r to turn on extended expressions without which your capture pattern would not work.
Used | instead of / as the delimiter for the substitution so that escaping / would not be necessary.
Single-quoted the sed expression so that escaping things inside it from the shell would not be necessary.
Included the matched span tag in the replacement section so that it would not get deleted.
In order to expand the unlockedCount variable, closed the single-quoted expression, then later re-opened it.
Omitted cat | which was useless here.
I also used double quotes around the shell variable expansion, because this is good practice but if it contains no spaces this is not really necessary.
It was not, strictly speaking, necessary for me to add -r. Plain old sed will work if you say \([0-9]\{0,\}\), but the idea here was to simplify.
sed -i -e 's%<span id="unlockedCount">([0-9]*)</span\>/'"${unlockedCount}/g" index.html
I removed the Useless Use of Cat, took out a bunch of unnecessary backslashes, added single quotes around the regex to protect it from shell expansion, and fixed the repetition operator. You might still need to backslash the grouping parentheses; my sed, at least, wants \(...\).
Note the use of single and double quotes next to each other. Single quotes protect against shell expansion, so you can't use them around "${unlockedCount}" where you do want the shell to interpolate the variable.