Org Mode Capture Templates and Time Tracking

Sharing My Workflow and Tooling

In this post I’ll share my workflow and the Emacs functions I use to facilitate my workflow. One highlight is better understanding how to use org-capture\’s (file+function "filename" function-finding-location) target element.

Context

One of my administrative tasks for my role at Software Services by Scientist.com is time tracking. It’s been over a decade since I last tracked my working hours. In my role I’m both coding and helping get team members unstuck on their tasks. This means on a given day, I jump between 5 to 7 projects. I find it invigorating helping get folks unstuck; either listening to their approach or digging in and pulling out a myriad of debugging and triage heuristics I’ve developed. To help me with keeping track of all of my hours and I work, I have begun leveraging even more of Emacs’s Org-Mode; a framework and toolkit for note taking and time tracking (and so much more).

My Current Workflow

At the start of my day, I review my existing todo items. This helps me remember where to start.

As I work on a todo item, I record time and take notes; which involves links and capturing further general documentation. Those notes will sometimes turn into blog posts or playbook articles. As I start a new project:

  • I start tracking my time.
  • Write a bit about what I’m working on.
  • And start taking notes.
  • For tasks that I don’t complete, I mark as todo.

As I wrap up a project’s task I go back to my todo items. The org-agenda function provides a menu of options to view my time and todo items. See the documentation At the end of the month I then go through my projects and record that time. I do all of this in my org-mode agenda file.

Code Supporting My Workflow

Before I started down this path I spent a month exploring, noting, and adjusting my workflow. As the month closed, I started to see the pattern I could use to extend my existing toolkit to better help my emerging workflow.

This section goes into the technical implementation.

Here’s my org-capture-templates. There are two of them:

project
The client’s project that I’m working on.
task
The task within a project.

For those reading along, here’s the documentation for org-mode’s the [capture template documentation](https://orgmode.org/manual/Template-elements.html).

Due to a present implementation constraint, which I’ll get to later, I need to first create the project’s parent day node. That is done via the file+olp+datetree directive; But I’m getting a bit ahead of myself.

(setq org-capture-templates
      '(;; Needed for the first project of the day; to ensure the datetree is
	;; properly generated.
	("p" "Project"
	 entry (file+olp+datetree jf/primary-agenda-filename-for-machine)
	 "* %(jf/org-mode-project-prompt) :project:\n\n%?"
	 :empty-lines-before 1
	 :empty-lines-after 1)
	("t" "Task"
	 ;; I tried this as a node, but that created headaches.  Instead I'm
	 ;; making the assumption about project/task depth.
	 plain (file+function jf/primary-agenda-filename-for-machine jf/org-mode-find-project-node)
	 ;; The five ***** is due to the assumptive depth of the projects and tasks.
	 "***** TODO %? :task:\n\n"
	 :empty-lines-before 1
	 :empty-lines-after 1)
	))

Anywhere in Emacs I can call org-capture (e.g. C-c c in Emacs dialect).

Begin Capturing Notes for the Project

The capture for the project positions the content in the following headline tree:

  • Year (e.g. 2022)
    • Month (e.g. 2022-09 September)
      • Day (e.g. 2022-09-03 Friday)
        • Project

The capture template for the project is (e.g. * %(jf/org-mode-project-prompt) :project:\n\n%?).

For the Project capture template, this:

  • creates a headline
  • prompts for the project
  • tags the node as a :project:
  • positions the cursor to begin note taking.

The following function prompts me to select an existing project or allows me to enter a new one.

(defun jf/org-mode-project-prompt ()
    "Prompt for project based on existing projects in agenda file.

Note: I tried this as interactive, but the capture templates
insist that it should not be interactive."
    (completing-read
     "Project: "
	 (sort
     (-distinct
      (org-map-entries
       (lambda ()
	 ;; Get the entry's title
	 (org-element-property :title (org-element-at-point)))
       ;; By convention project’s are:
       ;; - a level 4 headline
       ;; - tagged with :project:
       "+LEVEL=4+project"
       ;; Look within all agenda files
       'agenda))
     #'string<)))

When I started I thought I would need to create a local variable for projects. But I use org-map-entries to dynamically query the document for existing projects.

I also spent some time on the prompting function; in part because I thought it needed to be interactive. It does not.

Begin “Capturing” Notes for the Task

The “Task” capture template uses the file+function directive to find where in the document to insert the task.

The first parameter (e.g. jf/primary-agenda-filename-for-machine) specifies the agenda file for my machine. Those machines are work and personal; each with their own todo lists. The second parameter (e.g. jf/org-mode-find-project-node) is defined below; it finds and positions the cursor at the end of the given project within the give date.

;; Inspiration from https://gist.github.com/webbj74/0ab881ed0ce61153a82e
(cl-defun jf/org-mode-find-project-node (&key
					   (project (jf/org-mode-project-prompt))
					   ;; The `file+olp+datetree` directive creates
					   ;; a headline like “2022-09-03 Saturday”.
					   (within_headline (format-time-string "%Y-%m-%d %A")))
    "Find and position the cursor at the end of
the given PROJECT WITHIN_HEADLINE."
    ;; Ensure we’re using the right agenda file.
    (with-current-buffer (find-file-noselect jf/primary-agenda-filename-for-machine)
      (let ((existing-position (org-element-map
				   (org-element-parse-buffer)
				   'headline
				 ;; Finds the end position of:
				 ;; - a level 4 headline
				 ;; - that is tagged as a :project:
				 ;; - is titled as the given project
				 ;; - and is within the given headline
				 (lambda (hl)
				   (and (=(org-element-property :level hl) 4)
					;; I can't use the :title attribute as it is a
					;; more complicated structure; this gets me
					;; the raw string.
					(string= project (plist-get (cadr hl) :raw-value))
					(member "project" (org-element-property :tags hl))
					;; The element must have an ancestor with a headline of today
					(string= within_headline
						 (plist-get
						  ;; I want the raw title, no styling nor tags
						  (cadr (car (org-element-lineage hl))) :raw-value))
					(org-element-property :end hl)))
				 nil t)))
	(if existing-position
	    ;; Go to the existing position for this project
	    (goto-char existing-position)
	  (progn
	    ;; Go to the end of the file and append the project to the end
	    (end-of-buffer)
	    (insert (concat "\n**** " project " :project:\n\n")))))))

Current Implementation Constraint

My workflow does not need the “Project” capture. However the “Task” capture needs the headline structure that the “Project” capture creates. Future work that I could do would be for the “Task” capture to create the correct headline(s). But that’s a once a day inconvenience.

My Daily Task Sheet

Last the org-clock-report function provides a plain text tabular breakdown of my work days. Below is an anonymized example:

#+BEGIN: clocktable :scope subtree :maxlevel 5  :tcolumns 4
#+CAPTION: Clock summary at [2022-09-03 Sat 10:12]
| Headline                                           | Time    |       |      |      |
|----------------------------------------------------+---------+-------+------+------|
| *Total time*                                       | *14:30* |       |      |      |
|----------------------------------------------------+---------+-------+------+------|
| \_  2022-09 September                              |         | 14:30 |      |      |
| \_    2022-09-01 Thursday                          |         |       | 7:45 |      |
| \_      Client 1                                   |         |       |      | 0:30 |
| \_        Merge and Backport...                    |         |       |      | 0:30 |
| \_      Client 2                                   |         |       |      | 2:15 |
| \_        Get Bitnami SOLR Blocking Done           |         |       |      | 2:15 |
| \_      Learning Time                              |         |       |      | 1:30 |
| \_        Adjusting Time Tracking Automation...    |         |       |      | 0:30 |
| \_        Submit Proposal for Responsible...       |         |       |      | 0:30 |
| \_        Show and Tell                            |         |       |      | 0:30 |
| \_      Client 3                                   |         |       |      | 1:45 |
| \_        Pairing with A regarding Workflows       |         |       |      | 1:45 |
| \_      Client 4                                   |         |       |      | 1:15 |
| \_        Pairing on #138                          |         |       |      | 1:00 |
| \_        Reviewing...                             |         |       |      | 0:15 |
| \_      Client 5                                   |         |       |      | 0:30 |
| \_        Pairing with B on Collection Slugs       |         |       |      | 0:30 |
| \_    2022-09-02 Friday                            |         |       | 6:45 |      |
| \_      Client 6                                   |         |       |      | 0:15 |
| \_        Pairing with C regarding rebase...       |         |       |      | 0:15 |
| \_      Learning Time                              |         |       |      | 0:15 |
| \_        Writing About Emacs and Org-Mode Time... |         |       |      | 0:15 |
| \_      Client 1                                   |         |       |      | 2:15 |
| \_        Working on troubleshooting upstream...   |         |       |      | 0:30 |
| \_        Work on Documenting Hyrax’s IIIF...      |         |       |      | 1:45 |
| \_      Samvera                                    |         |       |      | 0:15 |
| \_        Reviewing PR for a Hyrax app without...  |         |       |      | 0:15 |
| \_      Client 2                                   |         |       |      | 1:30 |
| \_        Working on getting SOLR up and...        |         |       |      | 1:30 |
| \_      Client 7                                   |         |       |      | 1:15 |
| \_        Client 7 Upgrade Estimate                |         |       |      | 1:15 |
| \_      Client 5                                   |         |       |      | 1:00 |
| \_        Universal Viewer Overview                |         |       |      | 0:45 |
| \_        Working with D on Collections            |         |       |      | 0:15 |
#+END:

In the actual time sheet each of those lines link to the corresponding headline. The provides another way to navigate.

Conclusion

I never quite realized that I would appreciate time tracking. It helps me ensure that I’m not working more hours than I should. At other places, I’d work more hours. Here the time sheet helps set clear boundaries.

This workflow also helps me recover from context shifting. I want to help people get unstuck, but jumping in and out of that context does come with a cognitive cost. The underlying technical workflow provides the ritual/habit for re-orienting to what comes next.

As I mentioned earlier, my agenda file becomes a source for knowledge sharing; either with my future self or as a blog post. This article began as a quick note in my agenda file. And in that agenda file I’ve linked to this article.

Now to write a function to generate my daily stand-up “what did I do”; it should be rather straightforward based on my well structured time sheet and notes.

And as always, you can look to my dotemacs repository for even more regarding my Emacs configuration.