Im trying to find and replace all "^A" characters with a "\t" so that I can import the data into a mysql database. The command Im using on OSX is:
sed -i '' 's/^A/\t/' item_info_20130203 > item_info_20130203_tab
This just deletes all the contents in the file. Where am I going wrong?
The command should actually be something like this:
sed 's/'`echo "\001"`'/\t/g' item_info_20130203 > item_info_20130203_tab
or the more modern way
sed 's/'$(echo "\001")'/\t/g' item_info_20130203 > item_info_20130203_tab
or to affect the original file:
sed -i 's/'`echo "\001"`'/\t/g' item_info_20130203
If you are using GNU sed, you can use this:
sed -i 's/\x01/\t/g' item_info_20130203
According to this post, sed's -i option finds/replaces text in the original file, so the syntax > item_info_20120203_tab at the end of your command is a NOP and hence the output file is empty.
Related
Just a quick question to solve an issue I've been facing for days now: how to get an wget json response in a shell variable?
I have so far a wget command like this:
wget "http://IP:PORT/webapi/auth.cgi?account=USER&passwd=PASSWD"
The server reponse is normally something like:
{"data":{"sid":"9O4leaoASc0wgB3J4N01003"},"success":true}
What I'd like to do is to grep the sid value in a variable (as it is used as login ticket), but also the success value in order to ensure that the command has been executed correctly...
I think it is a very easy command to build, but I've never practised wget/http reponse in shell command...
Thanks a lot for your help!
EDIT: Thanks for your help. I did gave a try to both answers, but I am having the same error message (whatever I do):
--2022-07-16 14:21:38-- http://xxxxxxxx:port/webapi/auth.cgi?api=SYNO.API.Auth&method=Login&version=3&account=USER&passwd=PWD&session=SurveillanceStation&format=sid
Connecting to 192.168.1.100:5000... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/plain]
auth.cgi?api=SYNO.API.Auth&method=Login&version=3&account=USER&passwd=PASSWD&session=SurveillanceStation&format=sid: Permission denied
Cannot write to `auth.cgi?api=SYNO.API.Auth&method=Login&version=3&account=USER&passwd=PASSWD&session=SurveillanceStation&format=sid' (Permission denied).
The annoying thing: execution the URL from a web browser works just fine... :/
You can first store the result of wget command in variable and then use it:
VAR=$(wget "http://IP:PORT/webapi/auth.cgi?account=USER&passwd=PASSWD")
and then using jq extract from JSON file:
sid=$(echo $VAR|jq .data.sid)
success=$(echo $VAR|jq .success)
If you have problem with execution of wget you can try something like:
wget -O output_file 'http://xxxxxxxx:port/webapi/auth.cgi?api=SYNO.API.Auth&method=Login&version=3&account=USER&passwd=PWD&session=SurveillanceStation&format=sid'
and then set variables:
sid=$(jq .data.sid output_file )
success=$(jq .success output_file )
I do not know why I am facing this Permission Denied error. Thus I gave a try to save cookie on a dedicated folder... And it works just fine :)
The final command lloks like:
VAR=$(wget -q --keep-session-cookies --save-cookies "/var/tmp/cookie_tmp" -O- "http://IP:PORT/webapi/auth.cgi?api=SYNO.API.Auth&method=login&version=1&account=USER&passwd=PWD&session=SurveillanceStation");
Thanks for your help (I learned a lot about sed ;) )
So this can be done using the stream editor or "sed". There is a lot to learn but for this post here is an idea of a code:
sid=$(wget <your url> | sed 's/.*sid":"\(.*\)"},.*/\1/')
success=$(wget <your url> | sed 's/.*success":\(.*\)}/\1/')
This will create 2 variables $sid and $success.
you can learn more about sed in depth here.
Hope this helped!
I am using sed in package.json file to substitute the text.
But it does not replace the content.
Please help me solve this issue. I am using windows 10 and already installed sed (GNU sed version 4.2.1)
"replace": "sed -i \"s/<style type=\"text\\/css\">/<style type=\"text\\/css\" data-premailer=\"ignore\">/g\" ./dist/news.html"
Suggesting to try awk gsub() function instead of sed. awk use single quote.
I hope ' will prevent expansion of all " in gsub() function.
"replace": "awk '{gsub("<style type=\"text\\/css\">","<style type=\"text\\/css\" data-premailer=\"ignore\">");print}' ./dist/news.html"
Note that:
\\ is treated differently in awk. You might need \\\\
The output/result is not in the same file (sed -i not possible) but sent to stdout. You might want to do file redirection with > .
"replace": "awk '{gsub("<style type=\"text\\/css\">","<style type=\"text\\/css\" data-premailer=\"ignore\">");print}' ./dist/news.html > ./dist/news.1.html"
Worst case:
You can always create a batch script replace.bat to pack your scripts:
replace.bat
sed -i "s/<style type=\"text\\/css\">/<style type=\"text\\/css\" data-premailer=\"ignore\">/g\" %1
package.json
...
"replace": "cmd.exe -c 'replace.sh ./dist/news.html' "
...
Windows note:
In windows you can use powershell replace cmdlets instead of sed.
Also if you are comfortable with Linux, install windows wsl subsystem and work in Linux environment.
I need to make an automatic bump of package.json in jenkins, and really stuck with it.
The following sed command works in a pipeline but it changes the second field, I need to change the third one:
sed -i "/version/s/\\([.]\\)\\(.\\)\\(.*\\)/\\1${BUILD_NUMBER}\\3/" package.json
Output:
"version": "0.222.0"
but I need:
"version": "0.0.222"
a part of json input:
{
"name": "render",
"version": "0.0.0"
"description": "",
"main": "./dist/index.js",
"bin": {
"render-ne": "./bin/re"
},
Who can assist with it?
Assuming the field format is as was presented, (i.e.: "version": "0.0.0"), then those back references aren't really necessary here. Just substitute the text after the final .:
sed -i '/version/s/[^.]*$/'"${BUILD_NUMBER}\"/" package.json
If you tried the accepted solution on MacOS X, you get the following results:
sed: 1: "package.json": extra characters at the end of p command
After searching for similar issue found that on Darwin, sed is slightly different; this worked for me:
sed -i '' '/version/s/[^.]*$/'"${BUILD_NUMBER}\",/" package.json
I hope this helps someone who came here looking for an answer but found that the solution was not for all versions of sed.
For Mac -
Create a file called version-bump.sh with the following
🆘 RUN THIS IN THE ROOT OF YOUR REPO OR UPDATE PATH TO PACKAGE.JSON
VERSION_STRING='"version": '
CURR_VERSION=$(awk -F \" '/"version": ".+"/ { print $4; exit; }' package.json)
NEXT_VERSION=$(echo ${CURR_VERSION} | awk -F. -v OFS=. '{$NF += 1 ; print}')
sed -i '' "s/\($VERSION_STRING\).*/\1\"$NEXT_VERSION\",/" package.json
😮💨
I'm working on a VHDL project in Microsemi Libero.
When I click "Simulate" in the Libero GUI, modelSim starts up and I get to see the results of my simulation.
I'd like to get the same response from a TCL command.
I can do "Execute Script...", and point Libero at a .TCL file containing the single line
run_tool -name {SIM_PRESYNTH}
...and this appears to work just fine (I get messages like "Starting Simulation...Simulation completed...The Execute Script succeeded")... except I don't get a modelSim window opening up to show me my simulation results.
How do I get modelSim to open at the end of a simulation using a TCL command?
many thanks
Just a guess, 7 months late.
In Libero ISE if I want Synplify to popup, in the IDE, I click "Project", then "Profiles" and I set the synthesis tool not to run in batch mode.
Perhaps you can do the same for the simulator, or add a profile, which in tcl would look like this:
add_profile \
-name {Synplify_b} \
-type {synthesis} \
-tool {Synplify} \
-location {somewhere} \
-args {-batch} \
-batch 1
select_profile -name {Synplify_b}
i am using SQLCMD to run a .sql file which is 270 MB. The script file (.sql) was generated using Red Gate SQL Data Compare synchronization wizard. I cannot run it from SSMS because of insufficient memory. I log into the server and go to command prompt and it opens up command prompt
C:\Users\USER1>
then i type in
> C:\Users\USER1>SQLCMD -U sa -P PWD -d DATA_FEAT -i F:\SYNC\DATA-DATA_FEAT-20140709.sql -o F:\SYNC\DATA-DATA_FEAT-20140709result.txt
but i get
Sqlcmd: Error: Scripting error.
i am able to use Red gate to synchronize it without error. Red gate runs the same .sql file
Any Help
Thanks
I ran into this with a big script doing a lot of inserts. The solution was over in this other answer: Insert a GO periodically in the file so that everything wasn't being built up in one massive transaction. That answer even got the info from...a RedGate forum thread.
Since I'm using Linux and my file was one-statement-per-line, it was quite easy for me to use sed as outlined in this answer to add GO every few lines, e.g.:
$ sed ': loop; n; n; n; n; n; n; n; n; n; a GO
n; b loop' < bigfile.sql > bigfile2.sql
That inserts a GO every 10 lines (the number of times n appears in the sed script), which is probably overkill.
I had the same problem with an SQL file with 2.2M inserts.
Adding GO every few lines as suggested above didn't solve it for me I ended up having to split the file into smaller ones of xxx lines and running them seperately using the bash split command as follows: -
split -l xxx largefile.txt newfile.txt
I also have a nicer syntax for the sed solution listed above
sed '50~50i\GO' largefile.txt > newfile.txt
This will insert GO every 50 lines in a file and output the result to a new file.
Hope it helps someone.