I have gotten JSON info from an open API using
(def station-info (clj-http.client/get statinfo {:as :json}))
I have spit that information into a .clj file defined as si. It's content look like this:
{:stations [{:station_id "1755", :name "Aker Brygge", :address "Aker Brygge",
:lat 59.91118372188379, :lon 10.730034556850455, :capacity 33}
{:station_id "1101", :name "Stortingstunellen", :address "Rådhusgata 34",
:lat 59.91065301806209, :lon 10.737365277561025, :capacity 24}]}
When I call the function (map :station_id (:stations si)) it returns an empty list "()".
But if I define a function with the same info in the REPL and then use the same function, it works!
Very strange.
EDIT: Fixed it by turning the string from the file into a data structure:
(def si-data-structure (edn/read-string (slurp si)))
Your function is right, so your data must be wrong. In particular, you will surely find that (:stations si) is also empty. Look at the si variable and see if it really contains what you expect: is it a map? Is :stations one of its keys?
I Fixed it by turning the string from the file into a data structure:
(def si-data-structure (edn/read-string (slurp si)))
Related
I am a newbie and making some exercises. How can I put a def with a list of sentences and a randomizer function inside a defn function? How does that work?
(def list["test1", "test2", "test3"]) - works fine
(rand-nth list) - works fine
How do I put it inside a function defn?
Thanks for help.
IIUC you just want to reimplement rand-nth, no?
(defn wrapped-rand-nth [a-list]
(rand-nth a-list))
If you want the list to be static (non-changing)
(defn randomize []
(rand-nth ["test1" "test2" "test3"]))
works, but it creates the vector upon each call, a better way is to
(let [the-list ["test1" "test2" "test3"]]
(defn randomize []
(rand-nth the-list)))
If I am trying to an array into the index.cljs.hl page how do I go about using the array in Clojurescript. I found that I can use:
(loop-tpl :bindings [single-data rpc/test-vector]
(h2 single-data))
In the hLisp part but if I want to use the array above where the html tag is I seem to run into problems. The array in the rpc.cljs page is as follows
(defc= test-vector ["Good" "Man" "Shoe"])
And I have tried using map without the data and just an array and it works:
(defn build-list
[]
(map #(h2 %) ["one" "two" "three"]))
But then if I try and use the array it no longer works and I have no clue why.
(defn build-list
[]
(map #(h2 %) rpc/test-vector))
Cheers
If you put '#rpc/test-vector' with the last example it works.
Cheers
UPDATE: See updated observations below.
ORIGINAL:
I have a usage question about the Clolure library Hickory. I want to use it to find code listings in an HTML page and do syntax highlighting on those pages. (This is a desktop app, not a web app.) Here is what I have tried.
(ns ....
(:require [clojure.data.zip.xml :as zx]
[clojure.zip :as zip]
[clygments.core :as cly]
[hickory.core :as hkc]
[hickory.render :as hkr]
[hickory.zip :as hkz])
...)
(defn hilite
"Return a version of the input HTML with syntax highlighted
code listings."
[html-in]
(let [hz (hkz/hickory-zip (hkc/as-hickory (hkc/parse html-in))) ; Convert to zipmap.
cz (zx/xml-> hz :html :body :pre :code)] ; Get array of zips containing code listings.
(doseq [code-z cz] ; For each code listing...
(let [code-n (zip/node code-z)] ; Convert to a zipmap to node.
(if-let [lang (get-in code-n [:attrs :class])] ; See if the node contains a class...
(when (.startsWith lang "language-") ; ...with a language declaration.
(let [language (str/replace lang "language-" "") ; Extract the language.
cntnt (:content code-n) ; Get the content (the code listing).
hili (cly/highlight cntnt (keyword language) :html {:styles "xcode"}) ; Do the highlighting.
prsed (first (hkc/parse-fragment hili))] ; Parse the highlighted code listing.
(zip/replace code-z prsed) ; Replace the original with the highighted version.
(zip/root code-z)))))) ; Unzip and apply the changes.
(hkr/hickory-to-html (zip/node hz)))) ; Return the html with highlighted code listings.
Things work fine up to the point where I try to replace the existing code listing with the highlighted version -- the code listings are found correctly and highlighted versions are generated correctly. But I just get back a copy of the original HTML, not the highlighted version. I assume I am getting something wrong with zip/replace and zip/root, but I don't know what. I have been looking at the docs and examples for hickory and clojure.zip, but still haven't gotten to a working version.
Do you see what I am doing wrong?
UPDATE: As was pointed out by Alan Thompson, replacing the contents of a node does not do the edit in-place but returns a copy with the modification, as is typical for Clojure. But there was still more wrong. The update itself was structured incorrectly. The structure of the new node intended to replace the existing node has to be fleshed out a bit more. Here is a revised version that sort of works.
(defn hilite
"Return a version of the input HTML with syntax highlighted
code listings."
[html-in]
(let [hz (hkz/hickory-zip (hkc/as-hickory (hkc/parse html-in))) ; Convert to zipmap.
cz (zx/xml-> hz :html :body :pre :code)] ; Get array of zips containing code listings.
(doseq [code-z cz] ; For each code listing...
(let [code-n (zip/node code-z)] ; Convert to a zipmap to node.
(if-let [lang (get-in code-n [:attrs :class])] ; See if the node contains a class...
(when (.startsWith lang "language-") ; ...with a language declaration.
(let [language (str/replace lang "language-" "") ; Extract the language.
cntnt (:content code-n) ; Get the content (the code listing).
hili (cly/highlight cntnt (keyword language) :html {:styles "xcode"}) ; Do the highlighting.
hck-hili (hkc/as-hickory (first (hkc/parse-fragment hili))) ; NEW
new-node {:type :element :attrs {:class (str "lang-" language)} ; NEW
:tag :code :content [hck-hili]} ; NEW
nz (zip/root (zip/replace code-z new-node))] ; Unzip and apply the changes. NEW
(hkr/hickory-to-html nz)))))))) ; Generate the html with highlighted code listings.
By the time the HTML is generated at the bottom of the function, it contains the highlighted code listing. There are still a few problems with this though. First, since this editing takes place in a doseq form, it returns nil, not the generated HTML. Second, if there is more than one listing to highlight, each pass through the doseq is editing the original HTML, not one that contains any previous edits. Sigh. Have to re-think the approach.
Not being familiar with Hickory, it looks like these 2 lines are the problem:
(zip/replace code-z prsed)
(zip/root code-z)
The return value of replace is not being used, so root is being called with the original value of code-z. You probably meant for something more like this:
(zip/root
(zip/replace code-z prsed))
so that root is begin called with the output of replace as its input. Since data structures in Clojure are immutable, any changes generate a new data structure as the return value of the function; the original input to the function remains unchanged.
After some additional research, I came across this blog post. The solution proposed involves parsing the html into a zipmap and walking the entire tree of nodes. Each node is checked for whether it should be edited, and if so an editing function is applied and used to replace the original node in the tree.
The tree editing is quite general. You give the tree editing function the root node of the zipmap, a predicate that determines if a particular node should be edited, and an editing function that returns a revised version of the node. Here is what I came up with for syntax highlighting code listings in an HTML page.
(ns ...
(:require [clojure.string :as str]
[clojure.zip :as zip]
[clygments.core :as cly]
[hickory.core :as hkc]
[hickory.render :as hkr]
[hickory.zip :as hkz]))
(defn is-highlightable?
"Return true if the node is of a type that this program can
highlight. Specifically, is the node wrapped in a :pre tag,
tagged as :code and does it include a language name."
[node]
(let [is-pre (= (:tag node) :pre)
fc (first (:content node)) ; Get inside the <pre> tag.
has-code (= (:tag fc) :code)
has-class (get-in fc [:attrs :class])
has-language (when has-class
(.startsWith has-class "language-"))]
(and is-pre has-code has-class has-language)))
(defn hilight-node
"Return a highlighted version of the source code in the node."
[node]
(let [fc (first (:content node)) ; Get inside the <pre> tag.
lang (get-in fc [:attrs :class])
language (str/replace lang "language-" "")
cntnt (first (:content fc))
hili (cly/highlight cntnt (keyword language) :html)
frag (hkc/parse-fragment hili)
hck-hili (hkc/as-hickory (first frag))
new-node {:type :element :attrs {:class (str "lang-" language)}
:tag :div :content [hck-hili]}]
new-node))
(defn edit-html-tree
"Take a zipper, a function that matches a pattern in the tree,
and a function that edits the current location in the tree. Examine the tree
nodes in depth-first order, determine whether the matcher matches, and if so
apply the editor."
[zipper matcher editor]
(loop [loc zipper]
(if (zip/end? loc)
(zip/root loc)
(if-let [matched (matcher (zip/node loc))]
(let [new-loc (zip/edit loc editor)]
(if (not= (zip/node new-loc) (zip/node loc))
(recur (zip/next new-loc))))
(recur (zip/next loc))))))
(defn hilight-syntax
"Return a version of the input HTML where the code listings have
been syntax highlighted."
[html-in]
(hkr/hickory-to-html (edit-html-tree
(hkz/hickory-zip (hkc/as-hickory (hkc/parse html-in)))
is-highlightable?
hilight-node)))
The code is specific to the way my input HTML is laid out to begin with, but could probably be used with few modifications. The edit-tree function is quite general, as mentioned above, and can be used for other, similar editing tasks just by supplying a different matching predicate and node editing function.
I'm new to Clojure and playing with it for fun.
I'm reading a CSV file and want to apply a different function to each column. What is an elegant (both concise and readable) way to do this?
I have explored two approaches:
Working on a vector of rows:
for each row...
(def row-1 ["John", "24"])
...I want to apply a different function to each element, and obtain this result:
["John", 24]
The function I want to use are:
(def converters-1 [identity, read-string])
Is there a simple way to apply the converters-1 functions to the row-1 elements?
Working on a map:
With this method I start by turning each row into a map:
(def row-2 {:name "John", :age "24"})
Edit: And I want to obtain this map:
{:name "John", :age 24}
The converters are also stored in a map:
(def converters-2 {:name identity, :age read-string})
Is there a simple way to apply the right converters to the row-2 elements?
I will be interested to read solutions for both approaches.
In the end I will put the data into a map. I'm just not sure whether I want to do the conversions before or after getting this map.
Use map for sequences; use merge-with for maps.
user=> (map #(% %2) converters-1 row-1)
("John" 24)
user=> (merge-with #(% %2) converters-2 row-2)
{:name "John", :age 24}
(map #(%1 %2) converters-1 row-1)
;; ("John" 24)
if (def row-2 {:name "John", :age "24"}) (it's an integer in your example)
(for [x (keys converters-2)] ((converters-2 x) (row-2 x)))
;; ("John" 24)
I found a solution for the map approach, but it's less elegant than Diego's solution with vectors.
(into {} (map (fn [[k v]]
[k ((converters-2 k) v)]
) row-2))
Is there a simpler solutions for this map-approach?
Am I missing a core function that would simplify it?
Edit: reading Diego's edit, I could also use keys:
(into {} (map #(
[% ((converters-2 %) (row-2 %))] ; the key, and the converted value
) (keys row-2)))
But I prefer the previous solution, because it does not need the comment: it is obvious what happens to the key, and to the value. Plus, in this solution I only need to write row-2 once.
Edit 2: If I write (converters-2 k identity), then I only need to indicate the columns who need a transformation. For the other columns (like :name here) identity is the default converter. That is an advantage, compared to the vector approach.
Edit 3: I found a another solution for the map approach, using update-in:
(reduce #(update-in %1 [%2] (converters-2 %2)) row-2 (keys row-2))
Well, now that it's written, to my novice eyes it's harder to read and understand.
So far the vector solution is still best. I'll keep my eyes open for a better map solution; it might come in handy some day.
I've stared at this so long I'm going in circles...
I'm using the rbvmomi gem, and in Pry, when I display an object, it recurses down thru the structure showing me the nested objects - but to_json seems to "dig down" into some objects, but just dump the reference for others> Here's an example:
[24] pry(main)> g
=> [GuestNicInfo(
connected: true,
deviceConfigId: 4000,
dynamicProperty: [],
ipAddress: ["10.102.155.146"],
ipConfig: NetIpConfigInfo(
dynamicProperty: [],
ipAddress: [NetIpConfigInfoIpAddress(
dynamicProperty: [],
ipAddress: "10.102.155.146",
prefixLength: 20,
state: "preferred"
)]
),
macAddress: "00:50:56:a0:56:9d",
network: "F5_Real_VM_IPs"
)]
[25] pry(main)> g.to_json
=> "[\"#<RbVmomi::VIM::GuestNicInfo:0x000000085ecc68>\"]"
Pry apparently just uses a souped-up pp, and while "pp g" gives me close to what I want, I'm kinda steering as hard as I can toward json so that I don't need a custom parser to load up and manipulate the results.
The question is - how can I get the json module to dig down like pp does? And if the answer is "you can't" - any other suggestions for achieving the goal? I'm not married to json - if I can get the data serialized and read it back later (without writing something to parse pp output... which may already exist and I should look for it), then it's all win.
My "real" goal here is to slurp up a bunch of info from our vsphere stuff via rbvmomi so that I can do some network/vm analysis on it, which is why I'd like to get it in a nice machine-parsed format. If I'm doing something stupid here and there's an easier way to go about this - lay it on me, I'm not proud. Thank you all for your time and attention.
Update: Based on Arnie's response, I added this monkeypatch to my script:
class RbVmomi::BasicTypes::DataObject
def to_json(*args)
h = self.props
m = h.merge({ JSON.create_id => self.class.name })
m.to_json(*args)
end
end
and now my to_json recurses down nicely. I'll see about submitting this (or the def, really) to the project.
The .to_json works in a recursive manner, the default behavior is defined as:
Converts this object to a string (calling to_s), converts it to a JSON string, and returns the result. This is a fallback, if no special method to_json was defined for some object.
json library has added some implementation for some common classes (check the left hand side of this documentation), such as Array, Range, DateTime.
For an array, to_json first convert all the elements to json object, concat then together, and then add the array mark [/].
For your case, you need to define your customized to_json method for GuestNicInfo, NetIpConfigInfo and NetIpConfigInfoIpAddress. I don't know your implementation about these three classes, so I wrote a example to demonstrate how to achieve this:
require 'json'
class MyClass
attr_accessor :a, :b
def initialize(a, b)
#a = a
#b = b
end
end
data = [MyClass.new(1, "foobar")]
puts data.to_json
#=> ["#<MyClass:0x007fb6626c7260>"]
class MyClass
def to_json(*args)
{
JSON.create_id => self.class.name,
:a => a,
:b => b
}.to_json(*args)
end
end
puts data.to_json
#=> [{"json_class":"MyClass","a":1,"b":"foobar"}]