I have been playing with the spotify API in an effort to get comfortable with common Lisp. The language is a large departure from what I am used to and I am afraid I may be missing some understanding of how to parse objects and lists.
When I poll the spotify API looking for a song by title. It returns a lot of json which I access using cl-json decode method. However in order to access any of the fields I find myself doing long nested car cdr combinations. The cl-json returns not a listp or lista but a giant list of nested cons. It seems like a nightmare to parse. I am attaching the code below as an example. So what I want to ask is, What am I missing? This code looks way to cumbersome and messy to be the way its done in LISP, and I assume there is a nice way to map all the JSON to some type of object or hierarchical tree? Which would make querying easier.
(ql:quickload '(:cl-json :drakma))
(defvar *auth-token*)
(defvar *results*)
(setq *url* "https://api.spotify.com/v1/search")
(push (cons "application" "json") drakma:*text-content-types*)
(setf drakma:*header-stream* *standard-output*)
(defun search-spotify (&optional (search-param "blue moon") (search-type "track") (limit 5))
"Perform a search on the API for track, artist, or album information"
(defvar complete-url "")
(defvar url nil)
(defvar params nil)
(setq url "https://api.spotify.com/v1/search")
(setq params (format nil "?q=~a&type=~a&limit=~a" (replace-all search-param " " "+") (replace-all search-type " " "+") limit))
(setq complete-url (format nil "~a~a" url params))
(print complete-url)
(setq *results* (cl-json:decode-json-from-string
(drakma:http-request complete-url
:method :get
:additional-headers `(("Authorization" . ,(format nil "Bearer ~a" (cdar *auth-token*))))
))))
(defun print-spotify-artist ()
(loop for result in (cdr (caddar *results*))
do (format t "Artist Name: ~a Spotify ID: ~a ~%" (cdadr (cddadr (caddar result))) (cdar (cdaadr (caddar result))))))
Running the above code and calling the print-spotify-artist function will print a list as follows:
Artist Name: Eve 6 Spotify ID:
https://open.spotify.com/artist/4Eqd24yS5YcxI8b6Xfuwr8 Artist Name:
Bill Murray Spotify ID:
https://open.spotify.com/artist/3wkZ8WTrs7WcfE13voUCK1 Artist Name:
Various Artists Spotify ID:
https://open.spotify.com/artist/0LyfQWJT6nXafLPZqxe9Of Artist Name:
Pat Martino Spotify ID:
https://open.spotify.com/artist/4DlMMgnldzX6OkCskmeGKz Artist Name:
Jess & Zeb Spotify ID:
https://open.spotify.com/artist/1oAndP8vmGtTlB6mbpieJs
Example JSON return, you get this returned for each song, for this example data I set the maximum results to 1. Notice that it is heavily nested. The below is what is returned from decode json function of cl-json.
((:TRACKS
(:HREF
. "https://api.spotify.com/v1/search?query=open%2Broad%2Bsong&type=track&offset=0&limit=1")
(:ITEMS
((:ALBUM (:ALBUM--TYPE . "album")
(:ARTISTS
((:EXTERNAL--URLS
(:SPOTIFY . "https://open.spotify.com/artist/4Eqd24yS5YcxI8b6Xfuwr8"))
(:HREF . "https://api.spotify.com/v1/artists/4Eqd24yS5YcxI8b6Xfuwr8")
(:ID . "4Eqd24yS5YcxI8b6Xfuwr8") (:NAME . "Eve 6") (:TYPE . "artist")
(:URI . "spotify:artist:4Eqd24yS5YcxI8b6Xfuwr8")))
(:AVAILABLE--MARKETS "AD" "AR" "AT" "AU" "BE" "BG" "BO" "BR" "CA" "CH"
"CL" "CO" "CR" "CY" "CZ" "DE" "DK" "DO" "EC" "EE" "ES" "FI" "FR" "GB"
"GR" "GT" "HK" "HN" "HU" "ID" "IE" "IL" "IS" "IT" "JP" "LI" "LT" "LU"
"LV" "MC" "MT" "MY" "NI" "NL" "NO" "NZ" "PA" "PE" "PH" "PL" "PT" "PY"
"RO" "SE" "SG" "SK" "SV" "TH" "TR" "TW" "US" "UY" "VN" "ZA")
(:EXTERNAL--URLS
(:SPOTIFY . "https://open.spotify.com/album/1qJOmC60ez9RNWPg4ELMBW"))
(:HREF . "https://api.spotify.com/v1/albums/1qJOmC60ez9RNWPg4ELMBW")
(:ID . "1qJOmC60ez9RNWPg4ELMBW")
(:IMAGES
((:HEIGHT . 639)
(:URL
. "https://i.scdn.co/image/3f7891ffa993954dd72ed50245280b15f5db5844")
(:WIDTH . 621))
((:HEIGHT . 300)
(:URL
. "https://i.scdn.co/image/16f1a6277b827fd97cb450c5dc246d29c2ed1d52")
(:WIDTH . 291))
((:HEIGHT . 64)
(:URL
. "https://i.scdn.co/image/935aca4d861213415fc64f780b0e9e5a0e8d865c")
(:WIDTH . 62)))
(:NAME . "Eve 6") (:RELEASE--DATE . "1998")
(:RELEASE--DATE--PRECISION . "year") (:TYPE . "album")
(:URI . "spotify:album:1qJOmC60ez9RNWPg4ELMBW"))
(:ARTISTS
((:EXTERNAL--URLS
(:SPOTIFY . "https://open.spotify.com/artist/4Eqd24yS5YcxI8b6Xfuwr8"))
(:HREF . "https://api.spotify.com/v1/artists/4Eqd24yS5YcxI8b6Xfuwr8")
(:ID . "4Eqd24yS5YcxI8b6Xfuwr8") (:NAME . "Eve 6") (:TYPE . "artist")
(:URI . "spotify:artist:4Eqd24yS5YcxI8b6Xfuwr8")))
(:AVAILABLE--MARKETS "AD" "AR" "AT" "AU" "BE" "BG" "BO" "BR" "CA" "CH" "CL"
"CO" "CR" "CY" "CZ" "DE" "DK" "DO" "EC" "EE" "ES" "FI" "FR" "GB" "GR" "GT"
"HK" "HN" "HU" "ID" "IE" "IL" "IS" "IT" "JP" "LI" "LT" "LU" "LV" "MC" "MT"
"MY" "NI" "NL" "NO" "NZ" "PA" "PE" "PH" "PL" "PT" "PY" "RO" "SE" "SG" "SK"
"SV" "TH" "TR" "TW" "US" "UY" "VN" "ZA")
(:DISC--NUMBER . 1) (:DURATION--MS . 194866) (:EXPLICIT)
(:EXTERNAL--IDS (:ISRC . "USRC19806851"))
(:EXTERNAL--URLS
(:SPOTIFY . "https://open.spotify.com/track/7kAKO1EYHt2MVlombUuoLN"))
(:HREF . "https://api.spotify.com/v1/tracks/7kAKO1EYHt2MVlombUuoLN")
(:ID . "7kAKO1EYHt2MVlombUuoLN") (:IS--LOCAL) (:NAME . "Open Road Song")
(:POPULARITY . 44)
(:PREVIEW--URL
. "https://p.scdn.co/mp3-preview/b793285aeeb4a1176b93dc739ffb361c6aabf4e5?cid=b067abcbc67f4ceba0d61e414926c9f5")
(:TRACK--NUMBER . 5) (:TYPE . "track")
(:URI . "spotify:track:7kAKO1EYHt2MVlombUuoLN")))
(:LIMIT . 1)
(:NEXT
. "https://api.spotify.com/v1/search?query=open%2Broad%2Bsong&type=track&offset=1&limit=1")
(:OFFSET . 0) (:PREVIOUS) (:TOTAL . 43)))
The Access library might help, it allows to access nested data structures.
(defparameter my-plist (list :foo "foo" :bar "bar"))
;; bar is a plist
(defclass obj-test ()
((foo :accessor foo :initarg :foo :initform :foo)
(bar :accessor bar :initarg :bar :initform (copy-list MY-PLIST))))
(defvar my-obj ((make-instance 'obj-test))
(accesses MY-OBJ 'bar 'foo) ;; => "foo"
There is a way to use dotted paths, if that is to your liking.
Access is battle tested, it is the heart of the Djula templating engine, one of the most downloaded libraries on Quicklisp.
Access also allows consistent access across data structures.
Deeper overview: https://lisp-journey.gitlab.io/blog/generice-consistent-access-of-data-structures-dotted-path/
Related
I'm trying to traverse some JSON response I'm getting from the OpenWeatherMap API but I'm getting some issues to retrieve some values. Here is my code:
{-# LANGUAGE OverloadedStrings #-}
import Control.Lens
import Data.Aeson.Lens (_String, key)
import Network.Wreq
myAPIKey :: String
myAPIKey = "my_api_key_here"
conditionsQuery :: String -> String -> String -> String
conditionsQuery city country key =
"https://api.openweathermap.org/data/2.5/forecast?q=" ++ city ++ "," ++ country ++ "&appid=" ++ key
main = do
print "What's the city?"
city <- getLine
print "And the country?"
country <- getLine
r <- get (conditionsQuery city country myAPIKey)
print $ r ^. responseBody . key "name" . _String
print $ r ^. responseBody . key "cod" . _String
print $ r ^. responseBody . key "id" . _String
The issue is that only the value of "cod" is returned ("200" in that case). The values for "name" and "id" appear as "", if we try with London,GB, Chicago, US (for instance). Yet the response body looks like:
{
...
"id": 2643743,
"name": "London",
"cod": 200
}
I first thought it was a type mismatch, but 200 is an Int there (unless I'm mistaken?) so I am not sure where the issue lies? "" seems to indicate that those 2 keys (id and name) do not exist, but they do.
Any ideas? Thanks in advance.
The response body does not look like that.
According to https://openweathermap.org/forecast5, the key "cod" appears at the outermost level of the JSON object, but "id" and "name" do not.
{
"city":{
"id":1851632,
"name":"Shuzenji",
...
}
"cod":"200",
...
}
I am new Clojurescript and want to hack arround with clojurescript and electron based on an small json file.
I am doing something like (with transit/cljs)
(def jsondata (t/read (t/reader :json) (.readFileSync fs path_to_file "utf8")))) )
first I check if status is ok, that works fine...
(let [json_status (get jsondata "status")]
(.log js/console "JSON Glossar Status:" json_status))
now, how can I access one of the maps in the pages array, or step through the map?
{"status":"ok",
"pages":[
{
"id":1,
"name":"name1",
"image":"imagename1.png",
"children":[
{
"id":1,
"copytext":"kdjsldjsljfl"
},
{
"id":2,
"copytext":"dksdöfksöfklsöf"
}
]
},
{
"id":2,
"name":"name1",
"image":"imagename1.png",
"children":[
{
"id":4,
"copytext":"kdjsldjsljfl"
},
{
"id":5,
"copytext":"dksdöfksöfklsöf"
}
]
}
]
}
You can use aget (i.e. "array get") for nested ClojureScript / JavaScript interop.
For example, if you wanted to access the second map item in your "pages" array, you could do this:
(def my-js-object
(clj->js {:status "ok"
:pages [{:id 1
:name "foo"
:children []}
{:id 2
:name "bar"
:children []}]}))
(aget my-js-object "pages" 1)
In the above code I'm simply using clj->js to construct a notional (and incomplete) representation of your JSON; I hope this is enough to make sense.
My REPL output was:
#js {:id 2, :name "bar", :children #js []}
If you wanted to do something more complex with each page item, e.g. "map over each page hashmap and pull out the name values", then you could make use of the .- JS property accessor
(->> (.-pages my-js-object)
(map #(.-name %)))
REPL output:
("foo" "bar")
To not answer the question, you could use
js->cljs, https://cljs.github.io/api/cljs.core/js-GTclj, to turn your json into a normal Clojure data structure and use Clojures normal fns to extract the data you want.
I'm using Clj Json library to parse JSON data in clojure. How do i access all the values itertively and print it in a text file?
I have read the JSON. Can someone help me in parsing it and writing it to a text file?
def all-records (json/read-json (slurp "file.json")));
(println all-records);
(println (get-in all-records [:entry]))
Update:
Here is a sample json file :
{"markers" : [{"point" :new GLatLng (40.266044, -74.718479),
"homeTeam" : "Lawrence Library",
"awayTeam" : "LUGip",
"markerImage" : "images/red.png",
"information" : "Linux users group meets second Wednesday of each month.",
"fixture" : "Wednesday 7pm",
"capacity" : "",
"previousScore" : ""},
{"point" :new GLatLng (40.211600, -74.695702),
"homeTeam" : "Hamilton Library",
"awayTeam" : "LUGip HW SIG",
"markerImage" : "images/white.png"]}
If you put the json in a file like so:
{"markers" : [{"point" :new GLatLng (40.266044, -74.718479),
"homeTeam" : "Lawrence Library",
"awayTeam" : "LUGip",
"markerImage" : "images/red.png",
"information" : "Linux users group meets second Wednesday of each month.",
"fixture" : "Wednesday 7pm",
"capacity" : "",
"previousScore" : ""},
{"point" :new GLatLng (40.211600, -74.695702),
"homeTeam" : "Hamilton Library",
"awayTeam" : "LUGip HW SIG",
"markerImage" : "images/white.png"]}
and then write a little code:
(ns ...
(:require [cheshire.core :as cc] ...))
(def json-data (slurp "resources/sample.json"))
(defn json->clj [arg]
"Shortcut to cheshire.core/parse-string"
(cc/parse-string arg true)) ; true => keywordize-keys
(pprint (json->clj json-data))
you get an error
Error refreshing environment: com.fasterxml.jackson.core.JsonParseException:
Unrecognized token 'new': was expecting 'null', 'true', 'false' or NaN
The problem is the 2 occurrances of new GLatLng (40.266044, -74.718479) in the JSON data. JSON data is only data, like strings & numbers. You cannot have a function call (in this case a constructor call) embedded in the JSON.
I have a problem parsing json data in a loop. Iam a clojure beginner and need some hint for looping through json data.
The data looks like this:
{"photoset" {"primary" "8455893107", "total" "2", "pages" 1, "perpage" 500, "page" 1,
"per_page" 500, "photo"
[{"id" "8455893107", "secret" "1a3236df06", "server" "8087",
"farm" 9, "title" "IMG_0137", "isprimary" "1"}
{"id" "8469482476", "secret" "4c1bf59214",
"server" "8235", "farm" 9, "title" "HippieBus", "isprimary" "0"}]
, "owner"
"93029076#N07", "id" "72157632724688181", "ownername" "clojureB5"}, "stat" "ok"}
What I want to do is loop through the two photos and build a new url with the id and farm value like http://www.flickr.com/farm/id
I know that I can get one value like this:
(-> (get-in (cheshire.core/parse-string (:body picList)) ["photoset" "photo"]) first (get "id"))
But I can I now loop through it?
You can simply use map.
(->> (get-in data ["photoset" "photo"])
(map #(str "http://www.flickr.com/" (get % "farm") "/" (get % "id"))))
It will yield the following list:
("http://www.flickr.com/9/8455893107" "http://www.flickr.com/9/8469482476")
I have an json response and would like to get the values out:
(cheshire.core/parse-string (:body picList))
{"photoset" {"primary" "8455893107", "total" "1", "pages" 1, "perpage" 500, "page" 1,
"per_page" 500, "photo" [{"id" "8455893107", "secret" "1a3236df06", "server" "8087",
"farm" 9, "title" "IMG_0137", "isprimary" "1"}], "owner" "93029076#N07", "id"
"72157632724688181", "ownername" "clojureB5"}, "stat" "ok"}
How can I get the different values like photoset->primary or photoset->photo->id ?
I tried something with (map #(get % "photoset")... but it doest work.
Thanks!
I think you're looking for clojure.core/get-in
(get-in your-parsed-json ["photoset" "primary"]) ;; "8455893107"
(-> (get-in your-parsed-json ["photoset" "photo"])
first
(get "id")) ;; "8455893107"
(get-in your-parsed-json ["photoset" "photo" 0 "id"]) ;; "8455893107"