I am trying to run an octave function from a batch file. The function is well written given how it works when launched from within the Octave GUI.
The batch file, other than pointing to the octave function, defines the only argument needed by it.
A while back this was not a function but a simple Octave script and the commands used were ok.
The only issue I am encountering now is being able to pass the variable calculated by the batch file onto the octave function.
I have recently written an octave function to do some file management. It requires only one input from the user:
function replace_TMM (file_base)
where file_base is a string to specify what directory I am working on. So it has to be something like "Z:" or "I:" or so on.
I am quite sure that the function is well written since I am able to use it from Octave GUI without any issues.
The fact is that I would like to run this function from a batch file. Inside this batch file I wrote:
SET a=%cd:~0,2%
This command is able to identify the working directory so "a" will be equal to "Z:" or similars.
Now my issue is telling the batch file to evaluate the octave function using "a" as its input argument.
I tried stuff like:
"C:\Program Files\GNU Octave\Octave-7.3.0\mingw64\bin\octave-cli.exe" -q --eval _03_REPLACE_V04("'%cd:~0,2%'")
which does not seem to work. This kind of solution gives a syntax error at batch level, it is not even able to enter the octave file.
If I instead try something like:
"C:\Program Files\GNU Octave\Octave-7.3.0\mingw64\bin\octave-cli.exe" -q _03_REPLACE_V04.m Z:
It is able to enter the octave file but it does not process the function, just skips over it to get to the end of the script.
Same goes if I try the following:
"C:\Program Files\GNU Octave\Octave-7.3.0\mingw64\bin\octave-cli.exe" _03_REPLACE_V04.m -"Z:"
In brief I bvelieve that the function itself works, it is only a matter of passing a variable from the batch to the octave.
Would really appreciate some help, thanks in advance.
UPDATE 1
I have done what was suggested by #Dariush Gavari and used the following syntax:
"C:\Program Files\GNU Octave\Octave-7.3.0\mingw64\bin\octave-cli.exe" -q --eval "replace_TMM('%a%')"
This gets me the following error message:
error: 'replace_TMM' undefined near line 1, column 1
I believed that it was because it was not ablòe to find the script containing the function. This is saved in a file called _03_REPLACE_V04.m
For this reason I have tried with
"C:\Program Files\GNU Octave\Octave-7.3.0\mingw64\bin\octave-cli.exe" -q --eval _03_REPLACE_V04.m "replace_TMM('%a%')"
Leading to the following error:
error: --eval "CODE" and script file are mutually exclusive options
usage: octave [-HVWdfhiqvx] [--debug] [--doc-cache-file file] [--echo-commands]
[--eval CODE] [--exec-path path] [--experimental-terminal-widget]
[--gui] [--help] [--image-path path] [--info-file file]
[--info-program prog] [--interactive] [--line-editing] [--no-gui]
[--no-history] [--no-init-file] [--no-init-path] [--no-line-editing]
[--no-site-file] [--no-window-system] [--norc] [-p path]
[--path path] [--persist] [--server] [--silent] [--traditional]
[--verbose] [--version] [file]
I believed that the problem could also have been having the functional nd the file with two different names. To solve this I have kept the same file name but changed the function to match it:
function _03_REPLACE_V04 (file_base)
Then in the batch:
"C:\Program Files\GNU Octave\Octave-7.3.0\mingw64\bin\octave-cli.exe" -q --eval "_03_REPLACE_V04('%a%')"
Leading to:
warning: function '_03_REPLACE_V04' defined within script file '\99_TOOLS\OCTAVE_FILES\_03_REPLACE_V04.m'
error: invalid call to script \99_TOOLS\OCTAVE_FILES\_03_REPLACE_V04.m
error: called from
_03_REPLACE_V04
In other words still no way of making it work. :)
Octave provides the argv function, which returns a cellstring array of all arguments passed to the octave executable at the time of launch, or in the case where it was used to launch a script, then this is the script's arguments.
So presumably all you have to do to get your directory from within octave is argv(){1}
If you would like to convert your filename to an absolute filename, you could also do this from within octave via the make_absolute_filename function.
Incidentally, a very useful command you should know of in octave is the lookfor command. Writing lookfor arguments in the terminal returns a list with all functions which have the word "arguments" in their description; argv is at the top of that list.
You can then do help argv to see more details on that command.
I am trying to pass the Pull request title as a parameter to the lane
I run this command for example
fastlane android distribute release_notes:$PR_TITLE
And I can see from the logs that the command is executed correctly with the complete title
[16:37:52]: $ bundle exec fastlane android distribute release_notes:ref: copy the services module inside app module
but when I print the passed argument I found it trimmed
desc "distribute new build for testers, set internal to true to pass it for internal testrs"
lane :distribute do |options|
print "\n"
print "release_notes"
print options[:release_notes]
which prints release_notes ref:, trimmed after the : and it even gets trimmed after newlines in a strange way
As you can see from your release_notes:string command, fastlane parses colons in a key/value format. So it will break if you pass in a string which includes a colon.
A more common pattern is to read the release notes from the environment variable within your lane. So instead of using options at all just do something like
notes = ENV['PR_TITLE']
I have Snakefile as following:
SAMPLES, = glob_wildcards("data/{sample}_R1.fq.gz")
rule all:
input:
expand("samtools_sorted_out/{sample}.raw.snps.indels.g.vcf", sample=SAMPLES),
expand("samtools_sorted_out/combined_gvcf")
rule combine_gvcf:
input: "samtools_sorted_out/{sample}.raw.snps.indels.g.vcf"
output:directory("samtools_sorted_out/combined_gvcf")
params: gvcf_file_list="gvcf_files.list",
gatk4="/storage/anaconda3/envs/exome/share/gatk4-4.1.0.0-0/gatk-package-4.1.0.0-local.jar"
shell:"""
java -DGATK_STACKTRACE_ON_USER_EXCEPTION=true \
-jar {params.gatk4} GenomicsDBImport \
-V {params.gvcf_file_list} \
--genomicsdb-workspace-path {output}
"""
When I test it with dry run, I got error:
RuleException in line 335 of /data/yifangt/exomecapture/Snakefile:
Wildcards in input, params, log or benchmark file of rule combine_gvcf cannot be determined from output files:
'sample'
There are two places that I need some help:
The {output} is a folder that will be created by the shell part;
The {output} folder was hard-coded manually required by the command line (and the contents are unknown ahead of time).
The problem seems to be with the {output} without expansion as compared with the {input} which does.
How should I handle with this situation? Thanks a lot!
I'm writing a snakemake pipeline to take publicly available sra files, convert them to fastq files then run them through alignment, peak calling and LD score regression.
I'm having an issue in the rule called SRA2fastq below in which I use parallel-fastq-dump to convert SRA files to paired end fastq files. This rule generates two outputs for each SRA file, SRRXXXXXXX_1, and SRRXXXXXXX_2.
Here is my config file:
samples:
fullard2018_NpfcATAC_1: SRR5367824
fullard2018_NpfcATAC_2: SRR5367798
fullard2018_NpfcATAC_3: SRR5367778
fullard2018_NpfcATAC_4: SRR5367754
fullard2018_NpfcATAC_5: SRR5367729
And here are the first few rules of my Snakefile:
# read config info into this namespace
configfile: "config.yaml"
print (config['samples'])
rule all:
input:
expand("fastq_files/{SRA}_{num}.fastq.gz", SRA=[config['samples'][x] for x in config['samples']], num=[1,2]),
expand("FastQC/{SRA}_{num}_fastqc.html", SRA=[config['samples'][x] for x in config['samples']], num=[1,2]),
"FastQC/fastq_multiqc.html",
expand("peak_files/{sample}_peaks.blrm.narrowPeak", sample=config['samples']),
"peak_files/Fullard2018_peaks.mrgd.blrm.narrowPeak",
expand("LD_annotation_files/Fullard_2018.{chr}.l2.ldscore.gz", chr=range(1,23))
rule SRA_prefetch:
params:
SRA="{SRA}"
output:
"/home/c1477909/ncbi/public/sra/{SRA}.sra"
log:
"logs/prefetch/{SRA}.log"
shell:
"prefetch {params.SRA}"
rule SRA2fastq:
input:
"/home/c1477909/ncbi/public/sra/{SRA}.sra"
output:
"fastq_files/{SRA}_1.fastq.gz",
"fastq_files/{SRA}_2.fastq.gz"
log:
"logs/SRA2fastq/{SRA}.log"
shell:
"""
parallel-fastq-dump --sra-id {input} --threads 8 \
--outdir fastq_files --split-files --gzip
"""
rule fastqc:
input:
rules.SRA2fastq.output
output:
# Output needs to end in '_fastqc.html' for multiqc to work
html="FastQC/{SRA}_{num}_fastqc.html"
log:
"logs/FASTQC/{SRA}_{num}.log"
wrapper:
"0.27.1/bio/fastqc"
rule multiqc_fastq:
input:
lambda wildcards: expand("FastQC/{SRA}_{num}_fastqc.html", SRA=[config['samples'][x] for x in config['samples']], num=[1,2])
output:
"FastQC/fastq_multiqc.html"
wrapper:
"0.27.1/bio/multiqc"
rule bowtie2:
input:
sample=lambda wildcards: expand("fastq_files/{SRA}_{num}.fastq.gz", SRA=config['samples'][wildcards.sample], num=[1,2])
output:
"bam_files/{sample}.bam"
log:
"logs/bowtie2/{sample}.txt"
params:
index=config["index"], # prefix of reference genome index (built with bowtie2-build),
extra=""
threads: 8
wrapper:
"0.27.1/bio/bowtie2/align"
However, when I run the Snakefile I get the following error:
Error in job SRA2fastq while creating output files fastq_files/SRR5367754_1.fastq.gz, fastq_files/SRR5367754_2.fastq.gz
I've seen this error many times before and it's usually caused when the name of output file generated by the program does not exactly match the output file name you specify in the corresponding snakemake rule. However, this is not the case here as if I run the command snakemake generates for this particular rule separately the files are created as expected and the file names match. Here is an example of one instance of the rule taken after running snakemake -np:
rule SRA2fastq:
input: /home/c1477909/ncbi/public/sra/SRR5367779.sra
output: fastq_files/SRR5367779_1.fastq.gz, fastq_files/SRR5367779_2.fastq.gz
log: logs/SRA2fastq/SRR5367779.log
jobid: 18
wildcards: SRA=SRR5367779
parallel-fastq-dump --sra-id /home/c1477909/ncbi/public/sra/SRR5367779.sra --threads 8 --outdir fastq_files --split-files --gzip
Note the output files generated by the parallel-fastq-dump command run separately (i.e. not using snakemake) are named as specified in the SRA2fastq rule:
ls fastq_files
SRR5367729_1.fastq.gz SRR5367729_2.fastq.gz
I'm a bit stumped by this as this error is usually easily rectified but I can't work out what the issue is. I've tried changing the output section of the SRA2fastq to:
output:
file1="fastq_files/{SRA}_1.fastq.gz",
file2="fastq_files/{SRA}_2.fastq.gz"
However, this throws the same error. I've also tried just specifying one output file but this affects the bowtie2 rule later on as I get an input files missing error.
Any ideas what's going on here? Is there something I'm missing when trying to look for multiple output files in a single rule?
Many Thanks
Here is the situation - jmeter results were recorded in .csv quite a long time ago (around 6 month). The version of jmeter was not changed (3.0) but the config files did. Now I've been trying to generate a report from the old csv as usual - using
jmeter.bat -g my.csv -o reportFolder
Also, to defeat the incompatibility of configurations, I created a file named local-saveservice.properties, and passed it through -q command line option. Playing with settings in this file, I managed to defeat several errors like "column number mismatch" or "No column xxx found in sample metadata", but I still didn't generate the report succesfully, and here is the trouble:
File 'D:\ .....\load_NSI_stepping3_2017-03-24-new.csv' does not contain the field names h
eader, ensure the jmeter.save.saveservice.* properties are the same as when the CSV file was created or the file may be
read incorrectly
An error occurred: Error while processing samples:Consumer failed with message :Consumer failed with message :Consumer f
ailed with message :Consumer failed with message :Error in sample at line:1 converting field:Latency at column:11 to:lon
g, fieldValue:'UTF-8'
However,in my .csv column number 11 has the header "Latency" and contains numeric values, though 'UTF-8' is the content of next column - "Encoding"
Here are first lines of my .csv
timeStamp,elapsed,label,responseCode,responseMessage,success,bytes,grpThreads,allThreads,URL,Latency,Encoding,SampleCount,ErrorCount,Connect,threadName
1490364040950,665,searchItemsInCatalogRequest,200,OK,true,25457,1,1,http://*.*.*.*:9080/em/.....Service,654,UTF-8,1,0,9,NSI - search item in catalog
1490364041620,507,searchItemsInCatalogRequest,200,OK,true,25318,1,1,http://*.*.*.*:9080/em/.....Service,499,UTF-8,1,0,0,NSI - search item in catalog
1490364042134,495,searchItemsInCatalogRequest,200,OK,true,24266,2,2,http://*.*.*.*:9080/em/.....Service,487,UTF-8,1,0,0,NSI - search item in catalog
1490364043595,563,searchItemsInCatalogRequest,200,OK,true,24266,2,2,http://*.*.*.*:9080/em/.....Service,556,UTF-8,1,0,6,NSI - search item in catalog
PS I had to add threadName manually, 'cos it was not saved during initial data recording (my knowledge of Jmeter was even less then now :) )
First you should update to JMeter 3.3 as there are bugs fixed in report generation in the 3 versions after 3.0.
Second add to your command line:
jmeter.bat -p <path to jmeter.properties> -q <path to your custom.properties used when you generated file> -g my.csv -o reportFolder
Ensure that in "your custom.properties" you set to false all fields that are prefixed by jmeter.save.saveservice that didn't yet exist at the time you generated the file.