I am trying to design a circos plot using BioCircos R package. BioCircos allows to save the plots as .html interactive files. However, when I run the package using RScript the saved .html file is empty. To save the .html file I used saveWidget option from htmlwidgets package. Is it something wrong with saveWidget option? The code I used follows:
#!/usr/bin/Rscript
######R script for BioCircos test
library(htmlwidgets)
library(BioCircos)
genomes <- list("chra1" = 217471166, "chra2" = 181034961, "chra3" = 153873357, "chra4" = 153961319, "chra5" = 164033575,
"chra6" = 154486312, "chra7" = 133565930, "chra8" = 147241510, "chra9" = 91218944, "chra10" = 52432566, "chrb1" = 843366180, "chrb2" = 842558404, "chrb3" = 707956555, "chrb4" = 635713434, "chrb5" = 567300182,
"chrb6" = 439630435, "chrb7" = 236595445, "chrb8" = 231667822, "chrb9" = 230778867, "chrb10" = 151572763, "chrb11" = 103205957) # custom genome
links_chromosomes_01 <- c("chra1", "chra2", "chra3", "chra4", "chra4", "chra5", "chra6", "chra7", "chra7", "chra8", "chra8", "chra9", "chra10") # Chromosomes on which the links should start
links_chromosomes_02 <- c("chrb2", "chrb3", "chrb1", "chrb9", "chrb10", "chrb4", "chrb5", "chrb6", "chrb1", "chrb8", "chrb3", "chrb7", "chrb6") # Chromosomes on which the links should end
links_pos_01 <- c(115060347, 102611974, 14761160, 128700431, 128681496, 42116205, 58890582, 40356090,
146935315, 136481944, 157464876, 39323393, 84752508, 136164354,
99573657, 102580613,
111139346, 120764772, 90748238, 122164776,
44933176, 18823342,
48771409, 128288229, 150613881, 18509106, 123913217, 51237349,
34237851, 53357604, 78270031,
25306417, 25320614,
94266153,
41447919, 28810876, 2802465,
45583472,
81968637, 27858237, 17263637,
30569409) ### links chra chromosomes
links_pos_02 <- c(410543481, 463189512, 825903588, 353914638, 354135472, 717707494, 643107332, 724899652,
583713545, 558756961, 642015290, 154999098, 340216235, 557731577,
643350872, 655077847,
85356666, 157889318, 226411560, 161566470,
109857786, 25338955,
473876792, 124495704, 46258030, 572314729, 141584107, 426419779,
531245660, 220131772, 353941099,
62422773, 62387030,
116923325,
76544045, 33452274, 7942164,
642047816,
215981114, 39278129, 23302654,
418922633) ### links chrb chromosomes
links_labels <- c("aldh1a3", "amh", "cyp26b1", "dmrt1", "dmrt3", "fgf20", "hhip", "srd5a3",
"amhr2", "dhh", "fgf9", "nr0b1", "rspo1", "wnt1",
"aldh1a2", "cyp19a1",
"lhx9", "pdgfb", "ptch2", "sox10",
"cbln1", "wt1",
"esr1", "foxl2", "gata4", "lrpprc", "serpine2", "srd5a2",
"asns", "ctnnb1", "srd5a1",
"cyp26a1", "cyp26c1",
"wnt4",
"ar", "nr5a1", "ptgds",
"fgf16",
"cxcr4", "pdgfa", "sox8",
"sox9")
tracklist <- BioCircosLinkTrack('myLinkTrack', links_chromosomes_01, links_pos_01,
links_pos_01, links_chromosomes_02, links_pos_02, links_pos_02,
maxRadius = 0.55, labels = links_labels)
#plotting results
plot_chra_chrb <- BioCircos(tracklist, genome = chra_chrb_genomes, genomeFillColor = "RdBu", chrPad = 0.02, displayGenomeBorder = FALSE, genomeLabelTextSize = "10pt", genomeTicksScale = 4e+3,
elementId = "chra_chrb_comp_plot_test.html")
saveWidget(plot_chra_chrb, "chra_chrb_comp_plot_test.html", selfcontained = F, libdir = "lib")
The command line to run this script:
Rscript /path_to/Circle_plot_test.r
I tried to use this script in RStudio (without saveWidget() command), however it took too long to run in my personnel computer and the results was not displayed. However, this could be due to memory usage setup because when I took off some data, the script easily generates the plot in RStudio and I am able to save it. Is there other way to save the .hmtl interactive files in R or am I doing something wrong using htmlwidgets package in my script?
Thanks all in advance for any help and comments.
When you said it took too long to run, that was a sign that something was wrong! You weren't getting anything when you used saveWidget, because there is nothing returned from BioCiros.
I found two things that are a problem. The first one will result in a blank output—you can't use a '.' in the element ID. This ID will be used in the HTML coding.
You were getting huge delays due to the scale you set for genomeTickScale. That scaling value is for a tick mark attribute. I'm not sure why you set it to .004. However, when I comment out that line, it renders immediately. I have no issues with saving the widget, either.
--One other thing, you had chra_chrb_genomes as the object name assigned to the parameter genome in the function BioCircos. I assumed it was the object genome from your question since it was the only unused object.
The only things I changed were in the BioCircos function:
(plot_chra_chrb <- BioCircos(tracklist, genome = genomes, #chra_chrb_genomes,
genomeFillColor = "RdBu",
chrPad = 0.02,
displayGenomeBorder = FALSE,
genomeLabelTextSize = "10pt",
# genomeTicksScale = 4e+3, # problematic
elementId = "chra_chrb_comp_plot_test" # no periods
))
How can I rewrite these 2 commands, which work fine in a mapping, so that they will work in a function?
:if has_key(glos,#g)==1<cr>:let #j=eval('glos.'.#g)<cr>
The function concerned is executed by vim without comment, but #j remains unchanged, as if they had failed, but no message/error is generated.
Here is the complete code involved, the command that loads the dictionary, the function that does not work, and the mapping from that function that does.
" read the glossary into the dictionary, glos
let glos=eval(join(readfile("glossary.dict")))
" 2click item of interest and this will
" send image filepath to xv
" if item all-caps find same at start of its line
" If capitalized at eol find same at start of its line
" if all lower-case at eol find next occurrence of same
" look lower-case or capitalized word up in glossary.txt
" find _\d\+ (page no.) alone on its line in text
com! F call F()
function! F()
normal "ayiw"cyE"by$
let #c=substitute(#c,"[,.?':;!]\+","","g")
if #c=~'images\/ss\d\d\d*'
let #i='!display -geometry +0+0 '.#c.' &'
pkill display
#i
elseif #c==toupper(#c)
let #n=search('^'.#c,'sw')
elseif #c!=#b
let #f=3
let #g=tolower(#c)
while #f>0
try
let #j=eval('glos.'.#g)
catch
let #f=#f-1
let #g=strpart(#g,0,strlen(#g)-1)
continue
endtry
break
endwh
if #f>0
let #h=substitute(#j," glosimgs.*",'','')
if #h!=#j
let #i='!xv -geometry +0+380 '.substitute(#j,'^.\{-}\( glosimgs.*\)$','\1','').' &'
!pkill xv
#i
endif
echo #h
else
echo 'No matching entry for '.#c
endif
elseif #c=~'\u\l\+$'
let #n=search('^'.#c,'sw')
elseif #c=~'\l\+$'
norm *
elseif #c=~'^_\w\+$'
let #/='^'.#c.'$'
norm nzz
endif
endfunction
map <silent> <2-LeftMouse> "ayiw"cyE"by$:let #c=substitute(#c,"[,.?':;!]\+","","g")<cr>:if #c=~'images\/ss\d\d\d*'<cr>:let #i='!display -geometry +0+0 '.#c.' &'<cr>:pkill display<cr>:#i<cr>:elseif #c==toupper(#c)<cr>:let #n=search('^'.#c,'sw')<cr>:elseif #c!=#b<cr>:let #f=3<cr>:let #g=tolower(#c)<cr>:while #f>0<cr>:try<cr>:let #j=eval('glos["'.#g.'"]')<cr>:catch<cr>:let #f=#f-1<cr>:let #g=strpart(#g,0,strlen(#g)-1)<cr>:continue<cr>:endtry<cr>:break<cr>:endwh<cr>:if #f>0<cr>:let #h=substitute(#j," glosimgs.*",'','')<cr>:if #h!=#j<cr>:let #i='!xv -geometry +0+380 '.substitute(#j,'^.\{-}\( glosimgs.*\)$','\1','').' &'<cr>:!pkill xv<cr>:#i<cr>:endif<cr><cr<cr>>:echo #h<cr>:else<cr>:echo 'No matching entry for '.#c<cr>:endif<cr>:elseif #c=~'\u\l\+$'<cr>:let #n=search('^'.#c,'sw')<cr>:elseif #c=~'\l\+$'<cr>:norm *<cr>:elseif #c=~'^_\w\+$'<cr>:let #/='^'.#c.'$'<cr>:norm nzz<cr>:endif<cr>
Specifically, I should have written:
:if has_key(**g:**glos,#g)==1:let #j=eval('**g:**glos.'.#g)
:h g: goes straight to the heart of the matter; in a function all references are local to that function, so references to any variable outside the function must be global, by prepending 'g:' to the variable name. As I created the dictionary independent of the function, the function must reference it as a global item.
Of course, if you are not aware of 'g:', it is rather difficult to find that help reference, but that's a frequent problem using help.
And, of course, the ** surrounding g: aren't required, that's what this site gives you in lieu of bolded text, apparently.
Often I find myself doing repetitive file & replace operations in a file. Most often that comes down to fixed find and replace operations; deleting some lines, changing some strings that are always the same and so on.
In Vim that is a no-brainer,
function! Modify_Strength_Files()
execute':%s/?/-/'
execute':%s/Ä/-/'
"--------------------------------------------------------
execute':%s/Ä/-/'
execute':%s///g'
"--------------------------------------------------------
execute':g/Version\ of\ Light\ Ship/d'
execute':g/Version\ of\ Data\ for\ Specific\ Regulations/d'
"--------------------------------------------------------
" execute':g/LOADING\ CONDITION/d'
" execute':g/REGULATION:\ A\.562\ IMO\ Resolution/d'
" This is to reduce multiple blank lines into one.
execute ':%s/\s\+$//e'
execute ':%s/\n\{3,}/\r\r/e'
" ---------------------
endfunction
copied verbatim.
How could a function like this be defined in Sublime Text editor, if it can be done at all, and then called to act upon the currently opened file?
Here are resources to write Sublime Text 2 plugins:
Sublime Text 2 API Reference
Sublime Text 2 Plugin Examples
How to run Sublime Text 2 commands
Setting up Sublime Text 2 Custom Keyboard Shortcuts
Example: you can write a similar plugin and bind a hot key to it, that is, batch_edit command. Then you can open a file and execute the command via that hot key. By the way, in this script, I didn't consider the file encoding. You can get the file encoding via self.view.encoding().
# -*- coding: utf-8 -*-
import sublime, sublime_plugin
import re
class BatchEditCommand(sublime_plugin.TextCommand):
def run(self, edit):
self._edit = edit
self._replace_all(r"\?", "-")
self._replace_all(u"Ä", "-")
self._delete_line_with(r"Version of Light Ship")
self._delete_line_with(r"Version of Data for Specific Regulations")
self._replace_all(r"(\n\s*\n)+", "\n\n")
def _get_file_content(self):
return self.view.substr(sublime.Region(0, self.view.size()))
def _update_file(self, doc):
self.view.replace(self._edit, sublime.Region(0, self.view.size()), doc)
def _replace_all(self, regex, replacement):
doc = self._get_file_content()
p = re.compile(regex, re.UNICODE)
doc = re.sub(p, replacement, doc)
self._update_file(doc)
def _delete_line_with(self, regex):
doc = self._get_file_content()
lines = doc.splitlines()
result = []
for line in lines:
if re.search(regex, line, re.UNICODE):
continue
result.append(line)
line_ending = {
"Windows" : "\r\n",
"Unix" : "\n",
"CR" : "\r"
}[self.view.line_endings()]
doc = line_ending.join(result)
self._update_file(doc)
I have a whole lot of html files that live in one folder. I need to convert these to markdown I found a couple gems out there that does this great one by one.
my question is...
How can I loop though each file in the folder and run the command to convert these to md on a separate folder.
UPDATE
#!/usr/bin/ruby
root = 'C:/Doc'
inDir = File.join(root, '/input')
outDir = File.join(root, '/output')
extension = nil
fileName = nil
Dir.foreach(inDir) do |file|
# Dir.foreach will always show current and parent directories
if file == '.' or item == '..' then
next
end
# makes sure the current iteration is not a sub directory
if not File.directory?(file) then
extension = File.extname(file)
fileName = File.basename(file, extension)
end
# strips off the last string if it contains a period
if fileName[fileName.length - 1] == "." then
fileName = fileName[0..-1]
end
# this is where I got stuck
reverse_markdown File.join(inDir, fileName, '.html') > File.join(outDir, fileName, '.md')
Dir.glob(directory) {|f| ... } will loop through all files inside a directory. For example using the Redcarpet library you could do something like this:
require 'redcarpet'
markdown = Redcarpet::Markdown.new(Redcarpet::Render::HTML, :autolink => true)
Dir.glob('*.md') do |in_filename|
out_filename = File.join(File.dirname(in_filename), "#{File.basename(in_filename,'.*')}.html")
File.open(in_filename, 'r') do |in_file|
File.open(out_filename, 'w') do |out_file|
out_file.write markdown.render(in_file.read)
end
end
end
I have set up Jenkins, but I would like to find out what files were added/changed between the current build and the previous build. I'd like to run some long running tests depending on whether or not certain parts of the source tree were changed.
Having scoured the Internet I can find no mention of this ability within Hudson/Jenkins though suggestions were made to use SVN post-commit hooks. Maybe it's so simple that everyone (except me) knows how to do it!
Is this possible?
I have done it the following way. I am not sure if that is the right way, but it seems to be working. You need to get the Jenkins Groovy plugin installed and do the following script.
import hudson.model.*;
import hudson.util.*;
import hudson.scm.*;
import hudson.plugins.accurev.*
def thr = Thread.currentThread();
def build = thr?.executable;
def changeSet= build.getChangeSet();
changeSet.getItems();
ChangeSet.getItems() gives you the changes. Since I use accurev, I did List<AccurevTransaction> accurevTransList = changeSet.getItems();.
Here, the modified list contains duplicate files/names if it has been committed more than once during the current build window.
The CI server will show you the list of changes, if you are polling for changes and using SVN update. However, you seem to want to be changing the behaviour of the build depending on which files were modified. I don't think there is any out-of-the-box way to do that with Jenkins alone.
A post-commit hook is a reasonable idea. You could parameterize the job, and have your hook script launch the build with the parameter value set according to the changes committed. I'm not sure how difficult that might be for you.
However, you may want to consider splitting this into two separate jobs - one that runs on every commit, and a separate one for the long-running tests that you don't always need. Personally I prefer to keep job behaviour consistent between executions. Otherwise traceability suffers.
echo $SVN_REVISION
svn_last_successful_build_revision=`curl $JOB_URL'lastSuccessfulBuild/api/json' | python -c 'import json,sys;obj=json.loads(sys.stdin.read());print obj["'"changeSet"'"]["'"revisions"'"][0]["'"revision"'"]'`
diff=`svn di -r$SVN_REVISION:$svn_last_successful_build_revision --summarize`
You can use the Jenkins Remote Access API to get a machine-readable description of the current build, including its full change set. The subtlety here is that if you have a 'quiet period' configured, Jenkins may batch multiple commits to the same repository into a single build, so relying on a single revision number is a bit naive.
I like to keep my Subversion post-commit hooks relatively simple and hand things off to the CI server. To do this, I use wget to trigger the build, something like this...
/usr/bin/wget --output-document "-" --timeout=2 \
https://ci.example.com/jenkins/job/JOBID/build?token=MYTOKEN
The job is then configured on the Jenkins side to execute a Python script that leverages the BUILD_URL environment variable and constructs the URL for the API from that. The URL ends up looking like this:
https://ci.example.com/jenkins/job/JOBID/BUILDID/api/json/
Here's some sample Python code that could be run inside the shell script. I've left out any error handling or HTTP authentication stuff to keep things readable here.
import os
import json
import urllib2
# Make the URL
build_url = os.environ['BUILD_URL']
api = build_url + 'api/json/'
# Call the Jenkins server and figured out what changed
f = urllib2.urlopen(api)
build = json.loads(f.read())
change_set = build['changeSet']
items = change_set['items']
touched = []
for item in items:
touched += item['affectedPaths']
Using the Build Flow plugin and Git:
final changeSet = build.getChangeSet()
final changeSetIterator = changeSet.iterator()
while (changeSetIterator.hasNext()) {
final gitChangeSet = changeSetIterator.next()
for (final path : gitChangeSet.getPaths()) {
println path.getPath()
}
}
With Jenkins pipelines (pipeline supporting APIs plugin 2.2 or above), this solution is working for me:
def changeLogSets = currentBuild.changeSets
for (int i = 0; i < changeLogSets.size(); i++) {
def entries = changeLogSets[i].items
for (int j = 0; j < entries.length; j++) {
def entry = entries[j]
def files = new ArrayList(entry.affectedFiles)
for (int k = 0; k < files.size(); k++) {
def file = files[k]
println file.path
}
}
}
See How to access changelogs in a pipeline job.
Through Groovy:
<!-- CHANGE SET -->
<% changeSet = build.changeSet
if (changeSet != null) {
hadChanges = false %>
<h2>Changes</h2>
<ul>
<% changeSet.each { cs ->
hadChanges = true
aUser = cs.author %>
<li>Commit <b>${cs.revision}</b> by <b><%= aUser != null ? aUser.displayName : it.author.displayName %>:</b> (${cs.msg})
<ul>
<% cs.affectedFiles.each { %>
<li class="change-${it.editType.name}"><b>${it.editType.name}</b>: ${it.path} </li> <% } %> </ul> </li> <% }
if (!hadChanges) { %>
<li>No Changes !!</li>
<% } %> </ul> <% } %>
#!/bin/bash
set -e
job_name="whatever"
JOB_URL="http://myserver:8080/job/${job_name}/"
FILTER_PATH="path/to/folder/to/monitor"
python_func="import json, sys
obj = json.loads(sys.stdin.read())
ch_list = obj['changeSet']['items']
_list = [ j['affectedPaths'] for j in ch_list ]
for outer in _list:
for inner in outer:
print inner
"
_affected_files=`curl --silent ${JOB_URL}${BUILD_NUMBER}'/api/json' | python -c "$python_func"`
if [ -z "`echo \"$_affected_files\" | grep \"${FILTER_PATH}\"`" ]; then
echo "[INFO] no changes detected in ${FILTER_PATH}"
exit 0
else
echo "[INFO] changed files detected: "
for a_file in `echo "$_affected_files" | grep "${FILTER_PATH}"`; do
echo " $a_file"
done;
fi;
It is slightly different - I needed a script for Git on a particular folder...
So, I wrote a check based on jollychang.
It can be added directly to the job's exec shell script. If no files are detected it will exit 0, i.e. SUCCESS... this way you can always trigger on check-ins to the repository, but build when files in the folder of interest change.
But... If you wanted to build on-demand (i.e. clicking Build Now) with the changed from the last build.. you would change _affected_files to:
_affected_files=`curl --silent $JOB_URL'lastSuccessfulBuild/api/json' | python -c "$python_func"`
Note: You have to use Jenkins' own SVN client to get a change list. Doing it through a shell build step won't list the changes in the build.
It's simple, but this works for me:
$DirectoryA = "D:\Jenkins\jobs\projectName\builds" ####Jenkind directory
$firstfolder = Get-ChildItem -Path $DirectoryA | Where-Object {$_.PSIsContainer} | Sort-Object LastWriteTime -Descending | Select-Object -First 1
$DirectoryB = $DirectoryA + "\" + $firstfolder
$sVnLoGfIle = $DirectoryB + "\" + "changelog.xml"
write-host $sVnLoGfIle
I tried to add that to comments but code in comments is no way:
Just want to prettify code from heroin's answer:
def changedFiles = []
def changeLogSets = currentBuild.changeSets
for (entries in changeLogSets) {
for (entry in entries) {
for (file in entry.affectedFiles) {
echo "Found changed file: ${file.path}"
changedFiles += "${file.path}"
}
}
}
Keep in mind for some cases git plugin returns empty changeSet, like:
First run in newly created branch
'Build now' button build
Refer to https://issues.jenkins-ci.org/browse/JENKINS-26354 for more details.