I am trying to make a topojson file with csv data embedded using a makefile. I am using Mike Bostock's us-atlas as a guide.
topo/us-counties-10m-ungrouped.json: shp/us/counties.shp
mkdir -p $(dir $#)
topojson \
-o us_counties.json \
--no-pre-quantization \
--post-quantization=1e6 \
--external-properties=output.csv \
--id-property=FIPS \
--properties="County=County" \
--properties="PerChildrenPos=+PerChildrenPos" \
--simplify=7e-7 \
-- $<
It creates the topojson I need but completely ignores the output.csv file.
Here is a glimpse at what it returns.
{"type":"Polygon","id":"53051","properties":{"code":"53051"},"arcs":[[-22,79,80,-75,81]]}
Here's what I need it to return.
{"type":"Polygon","id":"53051","properties":{"code":"53051", "County":"Los Angeles", "PerChildrenPos": 10},"arcs":[[-22,79,80,-75,81]]}
Any ideas why it might be ignoring the csv file, I've tested moving it around to see if perhaps it was unaccessible or something?
Thanks in advance.
According to the documentation here: https://github.com/mbostock/topojson/wiki/Command-Line-Reference#external-properties
(If your CSV file uses a different column name for the feature identifier, you can specify multiple id properties, such as --id-property=+FIPS,+id.)
It seems that you need to change your --id-property=FIPS \ to something that corresponds to your CSV column names.
Also for --properties="County=County" \
I think it should be --properties County=+County \
Same for --properties="PerChildrenPos=+PerChildrenPos" \
Should be --properties PerChildrenPos=+PerChildrenPos \
For --external-properties=output.csv \
it should be --external-properties output.csv \
Basically the parameters do not need to be prefixed by an = sign.
Related
I'm sorry if this is a simple question, but I am just starting out with qemu and can't find a easy way to do this.
I am trying to somewhat automate my KVM deployment. I am currently running into the issue that I can't find a way to automatically set parameters for a filterref.
This is what my network option for virt-install currently looks like and that is working fine for now.
--network type=bridge,network=default,bridge=bridge0,model=e1000e,mac=$mac,filterref=clean-traffic
However I can't find anything to set a parameter to define the IP address it's supposed to be locked down to. This is the result that I want in the xml:
<filterref filter='clean-traffic'>
<parameter name='IP' value='XXX.XXX.XXX.XXX'/>
</filterref>
I am looking for a way to automatically add that parameter, preferably directly with virt-install or to an extent were I can just run a script, enter the few variables I want to set. And at this point the VM would already be running and waiting for the setup to be completed, with the filter loaded. Basically I want the parameter to be loaded before the first startup, so that there is no chance of anyone trying to mess with the ip address.
Is this possible?
This is the whole "script" I just copy into the console at the moment.
name=WindowsTest
mac=00:50:56:00:05:C5
size=70
ram=6000
vcpus=6
let cores=vcpus/2
virt-install \
--name=$name \
--ram=$ram \
--cpu=host \
--vcpus=$vcpus,maxvcpus=$vcpus,sockets=1,cores=$cores,threads=2 \
--os-type=windows \
--os-variant=win10 \
--disk path=/var/lib/libvirt/clutchImages/$name.qcow2,size=$size,format=qcow2,bus=virtio \
--cdrom /var/isos/Windows_20H2_English.iso \
--disk /var/isos/virtio-win-0.1.185.iso,device=cdrom \
--network type=bridge,network=default,bridge=bridge0,model=e1000e,mac=$mac,filterref=clean-traffic \
--graphics spice,listen=157.90.2.208 \
--graphics vnc
virsh version output:
virsh version
Compiled against library: libvirt 6.0.0
Using library: libvirt 6.0.0
Using API: QEMU 6.0.0
Running hypervisor: QEMU 4.2.0
I am on CentOS Linux release 8.3.2011.
Make arbitrary edits to virt-install's xml output
According to the man page you can make direct edits to the XML using XPath
syntax.
e.g.
virt-install \
#...
--network network="${net}",mac="${macaddr},filterref.filter=clean-traffic" \
--xml xpath.create=./devices/interface/filterref/parameter \
--xml xpath.set=./devices/interface/filterref/parameter/#name=IP \
--xml xpath.set=./devices/interface/filterref/parameter/#value=10.0.0.20
#...
virt-install man page excerpt:
man virt-install | grep -m1 -A40 '\-\-xml'
--xml
Syntax: --xml ARGS
Make direct edits to the generated XML using XPath syntax. Take an ex‐
ample like
virt-install --xml ./#foo=bar --xml ./newelement/subelement=1
This will alter the generated XML to contain:
<domain foo='bar' ...>
...
<newelement>
<subelement>1</subelement>
</newelement>
</domain>
The --xml option has 4 sub options:
--xml xpath.set=XPATH[=VALUE]
The default behavior if no explicit suboption is set. Takes the
form XPATH=VALUE unless paired with xpath.value . See below for
how value is interpreted.
--xml xpath.value=VALUE
xpath.set will be interpreted only as the XPath string, and
xpath.value will be used as the value to set. May help sidestep
problems if the string you need to set contains a '=' equals
sign.
If value is empty, it's treated as unsetting that particular
node.
--xml xpath.create=XPATH
Create the node as an empty element. Needed for boolean elements
like <readonly/>
--xml xpath.delete=XPATH
Delete the entire node specified by the xpath, and all its chil‐
dren
XML result
<interface type="network">
<!-- ... -->
<filterref filter="clean-traffic">
<parameter name="IP" value="10.0.0.20"/>
</filterref>
</interface>
virsh version output:
Compiled against library: libvirt 7.7.0
Using library: libvirt 7.7.0
Using API: QEMU 7.7.0
Running hypervisor: QEMU 6.2.0
Quick & dirty
name=WindowsTest
mac=00:50:56:00:05:C5
IP=xxx.yyy.zzz.qqq
size=70
ram=6000
vcpus=6
let cores=vcpus/2
virt-install \
--name=$name \
--ram=$ram \
--cpu=host \
--vcpus=$vcpus,maxvcpus=$vcpus,sockets=1,cores=$cores,threads=2 \
--os-type=windows \
--os-variant=win10 \
--disk path=/var/lib/libvirt/clutchImages/$name.qcow2,size=$size,format=qcow2,bus=virtio \
--cdrom /var/isos/Windows_20H2_English.iso \
--disk /var/isos/virtio-win-0.1.185.iso,device=cdrom \
--network type=bridge,network=default,bridge=bridge0,model=e1000e,mac=$mac,filterref=clean-traffic \
--graphics spice,listen=157.90.2.208 \
--graphics vnc
--print-xml > /tmp/{$name}.xml && \
sed -i "s/<filterref.*/<filterref filter='clean-traffic'>\n <parameter name='IP' value='${IP}'\/>\n <\/filterref>/g" /tmp/{$name}.xml && \
virsh create /tmp/{$name}.xml
This question already has answers here:
A better way to extract JSON value in bash script
(2 answers)
Closed 6 years ago.
I'm currently trying to grab and assign the N_596164000673190002 to a variable from a curl command.
This is the command:
curl -L -H 'X-Cisco-Meraki-API-Key: mykeygoeshere' -X POST -H'Content-Type: application/json' --data-binary '{"name":"'"$NETWORK_NAME"'", "type":"appliance", "timeZone":"'"$TIME_ZONE"'"}' 'https://dashboard.meraki.com/api/v0/organizations/foobar/networks'
This is the response:
{"id":"N_596164000673190002","organizationId":"foo","type":"appliance","name":"bar","timeZone":"America/Chicago","tags":""}
How do I successfully read and grab the variable after id (without the double quotes), while also simutaneously assigning it to a variable, $NETWORK_ID? I imagine this can all be done in one line.
If this is successful, echo $NETWORK_ID should return N_596164000673190002
To parse json in bash, people usually use jq as it is installed by default on most Unix distributions.
Try the following :
NETWORK_ID=$(my_curl_command | jq -r '.id')
Here, '.id' is a filter indicating we want to retrieve the value for the key id, and the -r flag is used to remove double quotes from the output.
Pipe the JSON output to python json to grab the id value you need, and use bash command substitution to assign the result to your NETWORK_ID environment variable.
NETWORK_ID=$(curl -L -H 'X-Cisco-Meraki-API-Key: mykeygoeshere' -X POST \
-H'Content-Type: application/json' \
--data-binary '{"name":"'"$NETWORK_NAME"'", "type":"appliance", \
"timeZone":"'"$TIME_ZONE"'"}' \
'https://dashboard.meraki.com/api/v0/organizations/foobar/networks' \
| python -c "import sys, json; print json.load(sys.stdin)['id']")
I'm using svnnotify to send notification email upon commits. I have the following script on my repo's hooks/post-commit:
#!/bin/sh
REPOS="$1"
REV="$2"
for address in $(/bin/cat /var/svn/teachbyapp/hooks/addressee.list)
do
/usr/local/bin/svnnotify \
-r $REV \
-C \
-d \
--diff-encoding utf8 \
-H HTML::ColorDiff \
-p $REPOS \
-t "$address" \
--from svn#factory.e-levelcom.com
done
It works but all I get is balck and white diff (no colors at all).
Added lines are underlined whereas removed lines have strike-through format. Nothing else, no color at all.
How can I get actually colored diff? Something like this
Ok I found it.
Thanks to this work
Also needed to append --css-inline option to svnnotify command in my script
Code
pandoc \
data.tex \
-f markdown \
-t html \
\
| grep -E '(^<|^$|^ *$)' \
\
| grep -v '^<p' \
\
| perl -pe 's#(?<!\\)%.*</#</#' \
\
| pandoc \
-f html \
-o vertical_output.pdf \
--latex-engine=xelatex
which gives tables on vertical layout as output, see the picture below.
Example case where horizontal table is needed
The content of the file data.tex is the following data.
Data.tex:
--------------------------------------------------------------------------------------------------------------------------------------------
Name Description Location Examples Start Peak Duration Appearance
--------------- --------------- -------------- ------------------- ---------------- -------------- --------------- -------------------
Prandial short-acting belly, Lispro (Humalog), 5-10 min before 30-60 minutes 2-4 hours Transparent
analogs abdomen Aspart (Novorapid), meal
= chemically Glulisine (Apidra)
sythesized;
Regular/short human insulin addomen Humulin R 30 min before 60 minutes 6-8 hours, Transparent
not analog meal hypoglycemia
not flexible risk
-------------------------------------------------------------------------------------------------------------------------------------------
Table: Diabetic drugs parndial and basal. Location, start, peak, duration and appearance.
giving by running the above code to the data
Comments
I would like to have a horizontal layout such that the table would be adjusted to it. I do not know to which stage I should affect to have it.
I think it should be possible in the last pandoc -process.
Too much space around sparse tables
One document has many types of tables.
The current accepted answer works well if tables are rather full, but badly with sparse tables.
Using the parameter of the accepted answer -V geometry:"paperwidth=22in, paperheight=210mm, margin=2cm", we get:
where the second table is extended too much; I would like to have it narrower; but not sure if this is possible dynamically in Pandoc.
How can you have a horizontal layout of pandocing?
Whenever you use Pandoc to produce PDF output, you can control the page size of the file with an addition to the command line. For example, generate a page with a width of 22 inches and a height of 210 Millimeters, while keeping each margin at 2 Centimeters, use this Pandoc parameter:
-V geometry:"paperwidth=22in, paperheight=210mm, margin=2cm"
I hope this example makes clear to you how you can influence the output paper size in different other ways.
However, I do not know if it answers your question. Your question is not very clear to me, I must admit.
My answer however may help other people when searching for "horizontal layout pandoc markdown".
Full commands to try
First, 12 inches width:
pandoc \
data.tex \
-f markdown \
-t html \
\
| grep -E '(^<|^$|^ *$)' \
\
| grep -v '^<p' \
\
| perl -pe 's#(?<!\\)%.*</#</#' \
\
| pandoc \
-f html \
-o vertical_output1.pdf \
--latex-engine=xelatex \
-V geometry:"paperwidth=12in, paperheight=80mm, margin=0.5cm"
Second, 14 inches width:
pandoc \
data.tex \
-f markdown \
-t html \
\
| grep -E '(^<|^$|^ *$)' \
\
| grep -v '^<p' \
\
| perl -pe 's#(?<!\\)%.*</#</#' \
\
| pandoc \
-f html \
-o vertical_output1.pdf \
--latex-engine=xelatex \
-V geometry:"paperwidth=14in, paperheight=80mm, margin=0.5cm"
Outputs (as PNG screenshots from the PDFs to make them visible here):
I am trying to setup OpenGrok to search through a few GB of code, mostly Java and Python projects. I use opengrok-0.12.1/bin/OpenGrok index $SRC_ROOT to build the index. I can see it indexing Java's "target" and Python's ".tox" directories which I don't need.
I searched online and found the same question in many forums, and the answer being to use -i. I have tried to use this option with both the OpenGrok wrapper script as well as opengrok.jar, but all I get is the help message (because the command line options were apparently wrong).
Could you give me an example command to build indices that ignore certain directories?
The solution is to use the -i flag. The best way to do this is to create a .conf file. For example, I have the following file defined as opengrok.conf:
OPENGROK_APP_SERVER=Tomcat
OPENGROK_TOMCAT_BASE=/usr/local/Cellar/tomcat/8.0.21/libexec
OPENGROK_SCAN_DEPTH=4
OPENGROK_VERBOSE=yes
OPENGROK_PROGRESS=yes
IGNORE_PATTERNS="-i f:foo.txt -i *.bar -i d:target -i d:.tox"
And run the indexing using:
OPENGROK_CONFIGURATION=opengrok.conf ./OpenGrok index $SRC_ROOT
It ignores indexing the file foo.txt, all files that match the pattern *.bar, and all files in directories named target or .tox.
Edit credits: mrenaud, pcas
Note that https://github.com/oracle/opengrok/pull/1841 renamed IGNORE_PATTERNS to OPENGROK_IGNORE_PATTERNS (in Oct 2017).
https://github.com/oracle/opengrok/wiki/How-to-setup-OpenGrok
java -jar opengrok-1.3.16/lib/opengrok.jar --help
-i, --ignore pattern
Ignore matching files (prefixed with 'f:' or no prefix) or directories
(prefixed with 'd:'). Pattern supports wildcards (example: -i '*.so'
-i d:'test*'). Option may be repeated.
with the new opengrok-tools, it looks like
opengrok-indexer \
--java_opts=-Djava.util.logging.config.file=$(OPENGROK_BASE)/etc/logging.properties \
--jar $(OPENGROK_DIST)/lib/opengrok.jar -- \
--ctags /usr/local/bin/ctags \
--source $(PWD) \
--dataRoot $(OPENGROK_BASE)/data \
--progress \
--history \
--assignTags \
--writeConfig $(OPENGROK_BASE)/etc/configuration.xml \
--analyzer .sc:ScalaAnalyzer \
--ignore 'd:zold' \
--ignore 'd:opengrok' \
--ignore 'd:tmp' \
--ignore 'd:.venv' \
--ignore 'd:.ipynb_checkpoints' \
--ignore 'd:.metals' \
# END