Reagent not rendering as expected when adding a new item at end of reactive vector - clojurescript

Question

I'm working on a tree control in ClojureScript and Reagent. It can be used as a file system navigator, topic navigator, outliner, etc.
When a headline in an outline is selected and being edited, the traditional behavior when Return is tapped is to create a new headline (a child or sibling depending on the expansion state of the headline and whether it already has children or not) then focus it leaving it ready to edit. My control does that all correctly except in the case of editing the last sibling in a group.
In the problematic case, the headline is created as expected, but focusing the new control fails.
I created an MCVE using the figwheel template.
lein new figwheel test-reagent-vector -- --reagent
Here's a listing exhibiting the problem.
(ns test-reagent-vector.core
(:require [clojure.string :as s]
[reagent.core :as r]))
(def ^{:constant true} topic-separator \u02D1)
(def empty-test-topic {:topic "Empty Test Topic"})
(defonce global-state-with-hierarchy
(r/atom {:name "Global Application State, Inc."
:data {:one "one" :two 2 :three [3]}
:tree [{:topic "First Headline"}
{:topic "Middle Headline"}
{:topic "Last Headline"}]}))
(defn get-element-by-id
[id]
(.getElementById js/document id))
(defn event->target-element
[evt]
(.-target evt))
(defn event->target-value
[evt]
(.-value (event->target-element evt)))
(defn swap-style-property
"Swap the specified style settings for the two elements."
[first-id second-id property]
(let [style-declaration-of-first (.-style (get-element-by-id first-id))
style-declaration-of-second (.-style (get-element-by-id second-id))
value-of-first (.getPropertyValue style-declaration-of-first property)
value-of-second (.getPropertyValue style-declaration-of-second property)]
(.setProperty style-declaration-of-first property value-of-second)
(.setProperty style-declaration-of-second property value-of-first)))
(defn swap-display-properties
"Swap the display style properties for the two elements."
[first-id second-id]
(swap-style-property first-id second-id "display"))
;;------------------------------------------------------------------------------
;; Vector-related manipulations.
(defn delete-at
"Remove the nth element from the vector and return the result."
[v n]
(vec (concat (subvec v 0 n) (subvec v (inc n)))))
(defn remove-last
"Remove the last element in the vector and return the result."
[v]
(subvec v 0 (dec (count v))))
(defn remove-last-two
"Remove the last two elements in the vector and return the result."
[v]
(subvec v 0 (- (count v) 2)))
(defn insert-at
"Return a copy of the vector with new-item inserted at the given n. If
n is less than zero, the new item will be inserted at the beginning of
the vector. If n is greater than the length of the vector, the new item
will be inserted at the end of the vector."
[v n new-item]
(cond (< n 0) (into [new-item] v)
(>= n (count v)) (conj v new-item)
:default (vec (concat (conj (subvec v 0 n) new-item) (subvec v n)))))
(defn replace-at
"Replace the current element in the vector at index with the new-element
and return it."
[v index new-element]
(insert-at (delete-at v index) index new-element))
;;------------------------------------------------------------------------------
;; Tree id manipulation functions.
(defn tree-id->tree-id-parts
"Split a DOM id string (as used in this program) into its parts and return
a vector of the parts"
[id]
(s/split id topic-separator))
(defn tree-id-parts->tree-id-string
"Return a string formed by interposing the topic-separator between the
elements of the input vector."
[v]
(str (s/join topic-separator v)))
(defn increment-leaf-index
"Given the tree id of a leaf node, return an id with the node index
incremented."
[tree-id]
(let [parts (tree-id->tree-id-parts tree-id)
index-in-vector (- (count parts) 2)
leaf-index (int (nth parts index-in-vector))
new-parts (replace-at parts index-in-vector (inc leaf-index))]
(tree-id-parts->tree-id-string new-parts)))
(defn change-tree-id-type
"Change the 'type' of a tree DOM element id to something else."
[id new-type]
(let [parts (tree-id->tree-id-parts id)
shortened (remove-last parts)]
(str (tree-id-parts->tree-id-string shortened) (str topic-separator new-type))))
(defn tree-id->nav-vector-and-index
"Parse the id into a navigation path vector to the parent of the node and an
index within the vector of children. Return a map containing the two pieces
of data. Basically, parse the id into a vector of information to navigate
to the parent (a la get-n) and the index of the child encoded in the id."
[tree-id]
(let [string-vec (tree-id->tree-id-parts tree-id)
idx (int (nth string-vec (- (count string-vec) 2)))
without-last-2 (remove-last-two string-vec)
without-first (delete-at without-last-2 0)
index-vector (mapv int without-first)
interposed (interpose :children index-vector)]
{:path-to-parent (vec interposed) :child-index idx}))
;;------------------------------------------------------------------------------
;; Functions to manipulate the tree and subtrees.
(defn add-child!
"Insert the given topic at the specified index in the parents vector of
children. No data is deleted."
[parent-topic-ratom index topic-to-add]
(swap! parent-topic-ratom insert-at index topic-to-add))
(defn graft-topic!
"Add a new topic at the specified location in the tree. The topic is inserted
into the tree. No data is removed. Any existing information after the graft
is pushed down in the tree."
[root-ratom id-of-desired-node topic-to-graft]
(let [path-and-index (tree-id->nav-vector-and-index id-of-desired-node)]
(add-child! (r/cursor root-ratom (:path-to-parent path-and-index))
(:child-index path-and-index) topic-to-graft)))
;;;-----------------------------------------------------------------------------
;;; Functions to handle keystroke events.
(defn handle-enter-key-down!
"Handle a key-down event for the Enter/Return key. Insert a new headline
in the tree and focus it, ready for editing."
[root-ratom span-id]
(let [id-of-new-child (increment-leaf-index span-id)]
(graft-topic! root-ratom id-of-new-child empty-test-topic)
(let [id-of-new-editor (change-tree-id-type id-of-new-child "editor")
id-of-new-label (change-tree-id-type id-of-new-child "label")]
(swap-display-properties id-of-new-label id-of-new-editor)
(.focus (get-element-by-id id-of-new-editor)))))
(defn handle-key-down
"Detect key-down events and dispatch them to the appropriate handlers."
[evt root-ratom span-id]
(when
(= (.-key evt) "Enter") (handle-enter-key-down! root-ratom span-id)))
;;;-----------------------------------------------------------------------------
;;; Functions to build the control.
(defn build-topic-span
"Build the textual part of a topic/headline."
[root-ratom topic-ratom span-id]
(let [label-id (change-tree-id-type span-id "label")
editor-id (change-tree-id-type span-id "editor")]
[:span
[:label {:id label-id
:style {:display :initial}
:onClick (fn [e]
(swap-display-properties label-id editor-id)
(.focus (get-element-by-id editor-id))
(.stopPropagation e))}
#topic-ratom]
[:input {:type "text"
:id editor-id
:style {:display :none}
:onKeyDown #(handle-key-down % root-ratom span-id)
:onFocus #(.stopPropagation %)
:onBlur #(swap-display-properties label-id editor-id)
:onChange #(reset! topic-ratom (event->target-value %))
:value #topic-ratom}]]))
(defn tree->hiccup
"Given a data structure containing a hierarchical tree of topics, generate
hiccup to represent that tree. Also generates a unique, structure-based
id that is included in the hiccup so that the correct element in the
application state can be located when its corresponding HTML element is
clicked."
([root-ratom]
(tree->hiccup root-ratom root-ratom "root"))
([root-ratom sub-tree-ratom path-so-far]
[:ul
(doall
(for
[index (range (count #sub-tree-ratom))]
(let [t (r/cursor sub-tree-ratom [index])
topic-ratom (r/cursor t [:topic])
id-prefix (str path-so-far topic-separator index)
topic-id (str id-prefix topic-separator "topic")
span-id (str id-prefix topic-separator "span")]
^{:key topic-id}
[:li {:id topic-id}
[:div (build-topic-span root-ratom topic-ratom span-id)]])))]))
(defn home
"Return a function to layout the home (only) page."
[app-state-atom]
(fn [app-state-ratom]
[:div (tree->hiccup (r/cursor app-state-ratom [:tree]))]))
(r/render-component [home global-state-with-hierarchy]
(get-element-by-id "app"))
(I assume some of this is not relevant to the problem, like the tree id manipulation functions. They are just here to make building the example easier.)
The control uses a vector to contain siblings, something about inserting a new element at the end of the vector seems to cause the timing of rendering to change.
When the user has the last item selected and clicks Return, an error message appears in the browser console about a null argument being passed to get-element-by-id. This is triggered by the keyboard handling function handle-enter-key-down!.
The items in the list of headlines are really two HTML elements: a label that displays when the user is not editing it, and a text input that is shown during editing. When a new headline is created, the swap-display-properties function is called to make the editor visible, then it is focused.
When a headline is created at the end of a vector of siblings, the DOM identifiers for the new label and text input are not available to switch the visibility of the two elements. Thus the error message about a null argument to get-element-by-id.
But it works correctly for all of the other positions.
I've reproduced this
on a Mac
with OpenJDK 9 and 11
with the original versions of the dependencies used in the template and after updating them to current
on Safari and Firefox
I can force it to work by delaying the call to swap-display-properties by 25ms or longer.
;; Wait for rendering to catch up.
(js/setTimeout #(do (swap-display-properties id-of-new-label id-of-new-editor)
(.focus (get-element-by-id id-of-new-editor))) 25)
I assume I could do something with Reacts componentDidMount method, but I don't understand why there is a failure only when inserting a new headline at the end of a vector of siblings.
So...
Is it just a coincidence that the other cases work as expected?
Am I misunderstanding something about the Reagent workflow?
Is there some problem with Reagent?
Any ideas would be appreciated.

Answer 1

I think you have already identified the problem as a race condition between adding the new element in Reagent and it getting created in the DOM (where get-element-by-id is looking for it).
The simplest answer (besides adding 25 ms sleeps everywhere) is to use an event loop library like re-frame to schedule a "set-focus" event which will be processed on the next pass through the event loop.
As an aside, I never use concat or subvec. Keep it simple with take & drop, and always wrap the output of a fn with (vec ...) to force it in to a plain vector w/o any sneaky/problematic laziness.

Answer 2

Browser focus is difficult and confusing to maintain.
Here's what I think is happening when you hit enter
The keydown event fires
You add the new topic via graft-topic!
You switch the styles such that the input is showing and the label is
hidden
You focus on the next element in the list
then, after the keydown event is done, reagent rerenders
When this happens the element that is focused is replaced by the new element you created
In the case that the element you are hitting enter from is not the final element in the list
You are able to focus on the already existing element and then later reagent replaces this element with the new element created in graft-topic!
In the case that the element you are hitting enter from is the final element in the list
Focusing fails to happen because no element with that id exists yet
therefore no element is in focus when reagent rerenders with the newly created element
What the browser is doing
The new element you created is in the same place as the old focused element, so the browser keeps the focus in that place
You can test this with the following code snippet which switches the two inputs on every keydown.
Despite having different ids and being different components, focus remains in the same place it was even when swapping the two components
(defn test-comp []
(r/with-let [*test? (r/atom true)]
[:div
(if #*test?
[:div
[:input
{:value "test"
:id "test"
:on-key-down #(swap! *test? not)}]
[:input
{:value "not test"
:id "not test"
:on-key-down #(swap! *test? not)}]]
[:div
[:input
{:value "not test"
:id "not test"
:on-key-down #(swap! *test? not)}]
[:input
{:value "test"
:id "test"
:on-key-down #(swap! *test? not)}]])]))
(note: this will give you a warning about not having an on-change handler, but that's not important for this demo, just wanted to specify the value so you could see the two inputs swap places, but the focus remain in the same place)
As to how to fix this...
Don't rely on waiting a cycle or using js timeout to fix this, that just wastes precious time
I would recommend not using the browser to retain focus
The simple answer is to keep what index is focused in app-state, and then decide whether the label or the input is rendered based on what
Then add an auto-focus attribute to the input so that when it renders it will come into focus
Some pointers for how to use reagent
In your code you are resolving the reagent component with (), but what you should do is use []
This relates to how reagent decides when to rerender components, but since you resolve the whole tree, every time you change an atom that you derefed it will rerender your whole tree, not just the place you derefed the atom. (test this by adding a println to your code in the build-topic-span component)
Define cursors in a form-2 component (or use with-let), they only need to be defined once per component, so no need to have them be redefined on every subsequent render (not sure if this will lead to bugs, but it's good practice)
also you can use cursor like get-in, so instead of
t (r/cursor sub-tree-ratom [index])
topic-ratom (r/cursor t [:topic])
you can do
topic-ratom (r/cursor t [index :topic])
Some other notes
The swapping styles thing you're doing is confusing, if you keep track of what is focused, you can just render a different component depending on what is focused, no need to have both the label and the input in the dom at the same time.
passing around a bunch of string ids is very confusing, especially when calling graft-topic! you destructure the string back into the path. Data is much easier to work with, keep the path in a vector and make it a string only when it needs to be
This example refactored with these things in mind
(ns test-reagent-vector.core
(:require [clojure.string :as s]
[reagent.core :as r]))
(def ^{:constant true} topic-separator \u02D1)
(def empty-test-topic {:topic "Empty Test Topic"})
(defonce global-state-with-hierarchy
(r/atom {:name "Global Application State, Inc."
:focused-index nil
:data {:one "one" :two 2 :three [3]}
:tree [{:topic "First Headline"}
{:topic "Middle Headline"}
{:topic "Last Headline"}]}))
(defn get-element-by-id
[id]
(.getElementById js/document id))
(defn event->target-element
[evt]
(.-target evt))
(defn event->target-value
[evt]
(.-value (event->target-element evt)))
(defn swap-style-property
"Swap the specified style settings for the two elements."
[first-id second-id property]
(let [style-declaration-of-first (.-style (get-element-by-id first-id))
style-declaration-of-second (.-style (get-element-by-id second-id))
value-of-first (.getPropertyValue style-declaration-of-first property)
value-of-second (.getPropertyValue style-declaration-of-second property)]
(.setProperty style-declaration-of-first property value-of-second)
(.setProperty style-declaration-of-second property value-of-first)))
(defn swap-display-properties
"Swap the display style properties for the two elements."
[first-id second-id]
(swap-style-property first-id second-id "display"))
;;------------------------------------------------------------------------------
;; Vector-related manipulations.
(defn delete-at
"Remove the nth element from the vector and return the result."
[v n]
(vec (concat (subvec v 0 n) (subvec v (inc n)))))
(defn remove-last
"Remove the last element in the vector and return the result."
[v]
(subvec v 0 (dec (count v))))
(defn remove-last-two
"Remove the last two elements in the vector and return the result."
[v]
(subvec v 0 (- (count v) 2)))
(defn insert-at
"Return a copy of the vector with new-item inserted at the given n. If
n is less than zero, the new item will be inserted at the beginning of
the vector. If n is greater than the length of the vector, the new item
will be inserted at the end of the vector."
[v n new-item]
(cond (< n 0) (into [new-item] v)
(>= n (count v)) (conj v new-item)
:default (vec (concat (conj (subvec v 0 n) new-item) (subvec v n)))))
(defn replace-at
"Replace the current element in the vector at index with the new-element
and return it."
[v index new-element]
(insert-at (delete-at v index) index new-element))
;;------------------------------------------------------------------------------
;; Tree id manipulation functions.
(defn tree-id->tree-id-parts
"Split a DOM id string (as used in this program) into its parts and return
a vector of the parts"
[id]
(s/split id topic-separator))
(defn tree-id-parts->tree-id-string
"Return a string formed by interposing the topic-separator between the
elements of the input vector."
[v]
(str (s/join topic-separator v)))
(defn increment-leaf-index
"Given the tree id of a leaf node, return an id with the node index
incremented."
[tree-id]
(let [parts (tree-id->tree-id-parts tree-id)
index-in-vector (- (count parts) 2)
leaf-index (int (nth parts index-in-vector))
new-parts (replace-at parts index-in-vector (inc leaf-index))]
(tree-id-parts->tree-id-string new-parts)))
(defn change-tree-id-type
"Change the 'type' of a tree DOM element id to something else."
[id new-type]
(let [parts (tree-id->tree-id-parts id)
shortened (remove-last parts)]
(str (tree-id-parts->tree-id-string shortened) (str topic-separator new-type))))
(defn tree-id->nav-vector-and-index
"Parse the id into a navigation path vector to the parent of the node and an
index within the vector of children. Return a map containing the two pieces
of data. Basically, parse the id into a vector of information to navigate
to the parent (a la get-n) and the index of the child encoded in the id."
[tree-id]
(let [string-vec (tree-id->tree-id-parts tree-id)
idx (int (nth string-vec (- (count string-vec) 2)))
without-last-2 (remove-last-two string-vec)
without-first (delete-at without-last-2 0)
index-vector (mapv int without-first)
interposed (interpose :children index-vector)]
{:path-to-parent (vec interposed) :child-index idx}))
;;------------------------------------------------------------------------------
;; Functions to manipulate the tree and subtrees.
(defn add-child!
"Insert the given topic at the specified index in the parents vector of
children. No data is deleted."
[parent-topic-ratom index topic-to-add]
(swap! parent-topic-ratom insert-at index topic-to-add))
(defn graft-topic!
"Add a new topic at the specified location in the tree. The topic is inserted
into the tree. No data is removed. Any existing information after the graft
is pushed down in the tree."
[root-ratom id-of-desired-node topic-to-graft]
(let [path-and-index (tree-id->nav-vector-and-index id-of-desired-node)]
(add-child! (r/cursor root-ratom (:path-to-parent path-and-index))
(:child-index path-and-index) topic-to-graft)))
;;;-----------------------------------------------------------------------------
;;; Functions to handle keystroke events.
(defn handle-enter-key-down!
"Handle a key-down event for the Enter/Return key. Insert a new headline
in the tree and focus it, ready for editing."
[app-state root-ratom index]
(add-child! root-ratom (inc index) empty-test-topic)
(swap! app-state update :focused-index inc)
)
(defn handle-key-down
"Detect key-down events and dispatch them to the appropriate handlers."
[evt app-state root-ratom index]
(when (= (.-key evt) "Enter")
(handle-enter-key-down! app-state root-ratom index)))
;;;-----------------------------------------------------------------------------
;;; Functions to build the control.
(defn build-topic-span
"Build the textual part of a topic/headline."
[root-ratom index]
(r/with-let [topic-ratom (r/cursor root-ratom [index :topic])
focused-index (r/cursor global-state-with-hierarchy [:focused-index])]
(if-not (= index #focused-index)
[:label
{:onClick #(reset! focused-index index)}
#topic-ratom]
[:input {:type "text"
:auto-focus true
:onKeyDown #(handle-key-down % global-state-with-hierarchy root-ratom index)
:onChange #(reset! topic-ratom (event->target-value %))
:on-blur #(when (= index #focused-index)
(reset! focused-index nil))
:value #topic-ratom}])))
(defn tree->hiccup
"Given a data structure containing a hierarchical tree of topics, generate
hiccup to represent that tree. Also generates a unique, structure-based
id that is included in the hiccup so that the correct element in the
application state can be located when its corresponding HTML element is
clicked."
([root-ratom]
[tree->hiccup root-ratom root-ratom "root"])
([root-ratom sub-tree-ratom path-so-far]
[:ul
(doall
(for [index (range (count #sub-tree-ratom))]
^{:key (str index)}
[:li
[:div
[build-topic-span root-ratom index]]]
))]))
(defn home
"Return a function to layout the home (only) page."
[app-state-ratom]
(r/with-let [tree-ratom (r/cursor app-state-ratom [:tree])]
[:div
[tree->hiccup tree-ratom]]))
(r/render
[home global-state-with-hierarchy]
(get-element-by-id "app"))
I only changed home, tree→hiccup, build topic span and handle keydown.
In the future
The example I wrote up assumes this is a flat list, but seems like you're planning on making this a nested list in the future and if that's true I would recommend changing some things
associate a unique id to every topic and use that id to determine if that element is in focus
specify the path-so-far as a vector of the ids up to that point in the tree
don't specify the the key as a function of the index, what if the element switches places with another element in the tree? we don't want to rerender it. Base this off a unique id
investigate the reagent track! function to cut down on rerenders when asking if the current element is focused
Hope this helps
Feel free to message me if you have any more questions regarding how to build a nested interactive list :)

Answer 3

After the responses from Joshua Brown and Alan Thompson, I got to reviewing the API docs in Reagent again to understand what with-let did.
Then I noticed after-render, which was exactly what I needed. To fix the problem in my example, add after-render in the handle-enter-key-down! like this.
(defn handle-enter-key-down!
"Handle a key-down event for the Enter/Return key. Insert a new headline
in the tree and focus it, ready for editing."
[root-ratom span-id]
(let [id-of-new-child (increment-leaf-index span-id)]
(graft-topic! root-ratom id-of-new-child empty-test-topic)
(let [id-of-new-editor (change-tree-id-type id-of-new-child "editor")
id-of-new-label (change-tree-id-type id-of-new-child "label")]
(r/after-render
(fn []
(swap-display-properties id-of-new-label id-of-new-editor)
(.focus (get-element-by-id id-of-new-editor)))))))
Since the identifiers for the new label and text input exist after the render, swapping their display properties now works as expected and the newly visible input can be focused.
I believe this also fixes the potential race condition that existed before (but did not manifest) when inserting new headlines at other positions in the vector as well.