I'm working on Fedora x86_64. It uses /lib64, /usr/lib64 and friends. I have the following *.pc.in file:
$ cat libcryptopp.pc.in
prefix=#prefix#
exec_prefix=#exec_prefix#
libdir=#libdir#
includedir=#includedir#
...
My config.site has the following. It was copied from Fedora's config.site at /usr/share/config.site. The copy was used because of config.site for vendor libs on Fedora x86_64.
$ cat /usr/local/share/config.site
...
# Note: This file includes also RHEL/Fedora fix for installing libraries into
# "/lib/lib64" on 64bit systems.
if test -n "$host"; then
# skip when cross-compiling
return 0
fi
if test "$prefix" = /usr \
|| { test "$prefix" = NONE && test "$ac_default_prefix" = /usr ; }
then
test "$sysconfdir" = '${prefix}/etc' && sysconfdir=/etc
test "$sharedstatedir" = '${prefix}/com' && sharedstatedir=/var
test "$localstatedir" = '${prefix}/var' && localstatedir=/var
ARCH=`uname -m`
for i in x86_64 ppc64 s390x aarch64; do
if test $ARCH = $i; then
test libdir='${exec_prefix}/lib64'
break
fi
done
fi
However, after Autoconf processes my *.pc.in file:
$ autoreconf --install --force
...
$ ./configure
...
$ cat libcryptopp.pc
prefix=/usr/local
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/include
...
Notice libdir=${exec_prefix}/lib, and not libdir=${exec_prefix}/lib64.
GCC is definitely building 64-bit binaries for the package. I did not add -mx32 or -m32:
$ gcc -dumpmachine
x86_64-redhat-linux
Why is the wrong lib/ directory being used, and how do I fix it?
The /usr/local/share/config.site is wrong. Though it was copied from Fedora's config.site and placed in /usr/local/share, the prefix directories are wrong. The prefix test should use /usr/local and not /usr.
Below is the corrected one.
$ cat /usr/local/share/config.site
...
if test -n "$host"; then
# skip when cross-compiling
return 0
fi
if test "$prefix" = /usr/local \
|| { test "$prefix" = NONE && test "$ac_default_prefix" = /usr/local ; }
then
test "$sysconfdir" = '${prefix}/etc' && sysconfdir=/etc
test "$sharedstatedir" = '${prefix}/com' && sharedstatedir=/var
test "$localstatedir" = '${prefix}/var' && localstatedir=/var
ARCH=`uname -m`
for i in x86_64 ppc64 s390x aarch64; do
if test $ARCH = $i; then
test "$libdir" = '${exec_prefix}/lib' && libdir='${exec_prefix}/lib64'
break
fi
done
fi
Now, the next question is, why Fedora's /usr/share/config.site is not handling prefix=/usr/local properly. That's an open question at Issue 1510073 : Autoconf does not honor libdir in config.site for "libdir=#libdir#" in *.pc file, which has been closed as NOT A BUG.
Related
I wrote a shell script (for practice) that should compile a C++ (.cpp) file, automatically generate an executable with clang++ and execute it. My code:
#!/bin/bash
function runcpp() {
CPPFILE=$1
if [ -z $CPPFILE ]; then
echo "You need to specify a path to your .cpp file!"
else
echo -n "Checking if '$CPPFILE' is a valid file..."
if [[ $CPPFILE == "*.cpp" ]]; then
echo -e "\rChecking if '$CPPFILE' is a valid file... successful"
echo -n "Generating executable for '$CPPFILE'..."
clang++ $CPPFILE
echo -e "\rGenerating executable for '$CPPFILE'... done"
fi
fi
}
It's not done yet, however, at line 9 (if [[ $CPPFILE == "*.cpp" ]]; then) something goes wrong: the script exits, even though the file I specified is a .cpp file. My Terminal window:
kali#kali:~$ ls -lha *.cpp
-rw-r--r-- 1 kali kali 98 Feb 9 19:35 test.cpp
kali#kali:~$ runcpp test.cpp
Checking if 'test.cpp' is a valid file...kali#kali:~$
Some command I use (task-spooler) produces text in table format and with some fields as filenames
I'd like to pipe the output of that command to generate a HTML file, s.t. I can click on the filename in the browser, and it would open in a new browser tab.
I am guessing, I should have some way to detect filenames and wrap them as file://x/y/z.txt. Is there an easy way to make it from the bash command line?
You could use sed. The regexp here is very simple and will likely require tweaking depending on your particular data.
~ $ echo -e 'http://google.com/asdf says hello\nthis is another url here: https://example.com/' | \
sed -E 's#(http[^ ]+)#\1#g'
http://google.com/asdf says hello
this is another url here: https://example.com/
~ $
like:
#!/bin/bash
make_page() {
echo "<html><body><ul>"
got=
while read -r id state file etc
do
[[ -f "$file" ]] && {
printf "<li><%s></li>\n" "$file" "$file";
got=1;
}
done < <(sed 1d) #skip header
[[ -z "$got" ]] && echo '<li>No any file</li>'
echo "</ul</body></html>"
}
[[ "$1" ]] && [[ ! -f "$1" ]] && { echo "no file $1" >&2; exit 1; }
[[ "$1" ]] && exec 3<"$1" || exec 3<&0
make_page <&3
save it to tspage.sh and use as tspage.sh filename or tspage.sh < filename
Eventually I piped to a python script:
to_html.py:
#!/usr/bin/env python
"""
Convert a piped in text input to html, while replacing file/path names with URLs
"""
import sys
import os
import re
def path_to_html_link(field):
field = field.group()
if os.path.isfile(field) or os.path.isdir(field):
return """{0} """.format(field)
else:
return field
if __name__ == "__main__":
while True:
line = sys.stdin.readline()
if not line: break # EOF
line = re.sub('/[^\s]*', path_to_html_link, line)
line = '<p><tt>' + line + '</tt></p>'
sys.stdout.write(line)
And then called
chmod +x to_html.py
ts | to_html.py > tmp.html
I'm a command line newbie and I'm trying to figure out how I can add a header to multiple .csv files. The new header should have the following: 'TaxID' and 'filename'
I've tried multiple commands like sed, ed, awk, echo but if it worked it only changed the first file it found (I said *.csv in my command) and I can only manage this for TaxID.
Can anyone help me to get the filename into the header as well and do this for all my csv files?
(Note, I'm using a Mac)
Thank you!
Here's one way to do it, there are certainly others:
$ for i in *.csv;do echo $i;cp "$i" "$i.bak" && { echo "TaxID,$i"; cat "$i.bak"; } >"$i";done
Here's a sample run:
$ cat file1.csv
1,2
3,4
$ cat file2.csv
a,b
c,d
$ for i in *.csv;do echo $i;cp "$i" "$i.bak" && { echo "TaxID,$i"; cat "$i.bak"; } >"$i";done
file1.csv
file2.csv
$ cat file1.csv.bak
1,2
3,4
$ cat file1.csv
TaxID,file1.csv
1,2
3,4
$ cat file2.csv.bak
a,b
c,d
$ cat file2.csv
TaxID,file2.csv
a,b
c,d
Breaking it down:
$ for i in *.csv; do
This loops over all the files ending in .csv in the current directory. Each will be put in the shell variable i in turn.
echo $i;
This just echoes the current filename so you can see the progress. This can be safely left out.
cp "$i" "$i.bak"
Copy the current file (whose name is in i) to a backup. This is both to preserve the file if something goes awry, and gives subsequent commands something to copy from.
&&
Only run the subsequent commands if the cp succeeds. If you can't make a backup, don't continue.
{
Start a group command.
echo "TaxID,$i";
Output the desired header.
cat "$i.bak";
Output the original file.
}
End the group command.
>"$i";
Redirect the output of the group command (the new header and the contents of the original file) to the original file. This completes one file.
done
Finish the loop over all the files.
For fun, here are a couple of other ways (one JRD beat me to), including one using ed!
$ for i in *.csv;do echo $i;perl -p -i.bak -e 'print "TaxID,$ARGV\n" if $. == 1' "$i";done
$ for i in *.csv;do echo $i;echo -e "1i\nTaxID,$i\n.\nw\nq\n" | ed "$i";done
Here is on way in perl that modifies the files in place by adding a header of TaxID,{filename}, ignoring adding the header if it thinks it already exists.
ls
a.csv b.csv
cat a.csv
1,a.txt
2,b.txt
cat b.csv
3,c.txt
4,d.txt
ls *.csv | xargs -I{} -n 1 \
perl -p -i -e 'print "TaxID,{}\n" if !m#^TaxID# && !$h; $h = 1;' {}
cat a.csv
TaxID,a.csv
1,a.txt
2,b.txt
cat b.csv
TaxID,b.csv
3,c.txt
4,d.txt
You may want to create some backups of your files, or run on a few sample copies before running in earnest.
Explanatory:
List all files in directory with .csv extenstion
ls *.csv
"Pipe" the output of ls command into xargs so the perl command can run for each file. -I{} allows the filename to be subsequently referenced with {}. -n tells xargs to only pass 1 file at a time to perl.
| xargs -I{} -n 1
-p print each line of the input (file)
-i modifying the file in place
-e execute the following code
perl -p -i -e
Perl will implicitly loop over each line of the file and print it (due to -p). Print the header if we have not printed the header already and the current line doesn't already look like a header.
'print "TaxID,{}\n" if !m#^TaxID# && !$h; $h = 1;'
This is replaced with the filename.
{}
All told, in this example the commands to be run would be:
perl -p -i -e 'print "TaxID,{}\n" if !m#^TaxID# && !$h; $h = 1;' a.csv
perl -p -i -e 'print "TaxID,{}\n" if !m#^TaxID# && !$h; $h = 1;' b.csv
perl -p -i -e 'print "TaxID,{}\n" if !m#^TaxID# && !$h; $h = 1;' c.csv
perl -p -i -e 'print "TaxID,{}\n" if !m#^TaxID# && !$h; $h = 1;' d.csv
I cannot get the gnu parallel function to implement a custom function that I built.
My function is:
function run_cuffLinks() {
inputBAM="${HOME}/Analyses/P_miniata/CleanUpPipeline/TH_${1}/${1}.realigned.bam"
if [[ ! -f $inputBAM ]]; then echo -e "$inputBAM could not be found\nexit 1" ; fi
WORKING_DIR="${HOME}/data/CuffLinks/TH_$1"
if [[ ! -d $WORKING_DIR ]]; then mkdir -p $WORKING_DIR; fi
REF="${HOME}/ReferenceSequences/GATK_pmin.scaf.fa"
if [[ ! -f $REF ]]; then echo -e "$inputBAM could not be found\nexit 1" ; exit 1; fi
GTF_FILE="${HOME}/ReferenceSequences/genes.sorted.gff3"
if [[ ! -f $GTF_FILE ]]; then echo -e "$inputBAM could not be found\nexit 1" ; exit 1; fi
cufflinks \
--output-dir $WORKING_DIR \
--num-threads 2 \
--frag-len-mean 100 \
--GTF-guide $GTF_FILE \
--frag-bias-correct $REF \
-L "HH" \
$inputBAM ;
}
When I enter:
parallel --no-notice -j+2 run_cuffLinks {} ::: sample1 sample2 sample3
I get the output:
/bin/bash: run_cuffLinks: command not found
/bin/bash: run_cuffLinks: command not found
/bin/bash: run_cuffLinks: command not found
If I include a '$' symbol in front of the function name, I get:
/bin/bash: sample1: command not found
/bin/bash: sample2: command not found
/bin/bash: sample3: command not found
I have also tried using the -pipe --recend and --rrs options, but without a positive result.
Is GNU parallel not able to process user-defined functions?
You do not write whether you have walked through the tutorial (man parallel_tutorial). In that it shows that you must export -f the function, and since you do not write that, I believe you might have forgotten that:
export -f run_cuffLinks
parallel ...
Since version 20180522 you can also use env_parallel:
env_parallel --session
[define functions and variables here that you want parallel to see]
# Use env_parallel like you would parallel
env_parallel run_cuffLinks ...
PS: Use --bibtex once to avoid --no-notice in the future.
As the title suggests how do I remove spaces from all the files in the current directory ?
Example
file name.mp3 should become filename.mp3
Note:
I am open to an answer in any language.
I am a big fan of python, so here is a python script for doing the same
import os
for f in os.listdir("."):
r = f.replace(" ","")
if( r != f):
os.rename(f,r)
with sh
for file in *' '*; do [ -f "$file" ] && mv "$file" "`echo $file|tr -d '[:space:]'`"; done
with perl 5.14 (replace y/ //dr by do{($x=$_)=~y/ //d;$x} for older versions)
# Linux/Unix
perl -e 'rename$_,y/ //drfor<"* *">'
# Windows
perl -e "rename$_,y/ //drfor<'* *'>"
with Java
import java.io.File;
public class M {
public static void main(String[] args) {
String o,n;
for (File old : new File(".").listFiles()) {
o=old.getName();
if (!o.contains(" ")) continue;
n=o.replaceAll(" ", "");
old.renameTo(new File(n));
}
}
}
for i in * ; do
if [ "$i" != ${i//[[:space:]]} ] ;
then
mv "$i" "${i//[[:space:]]}"
fi
done
${i//[[:space:]]} removes all the spaces in a string.
Since you're language agnostic, here's a ruby one-liner:
ruby -e 'Dir.foreach(".") {|f| f.count(" \t") > 0 and File.rename(f, f.delete(" \t"))}'
ls -1 | awk '/ /{a=$0;gsub(/ /,"");b="mv \""a"\" "$0;system(b);}'
This renames an old file only if the old file name contained a space
and the new file doesn't already exist.
for old in *; do
new="${old//[[:space:]]}"
[[ $old = $new || -f $new ]] || mv "$old" "$new"
done