These Are the Tools I Know I Know

Have Toolbox, Will Share

Presented on at Samvera Virtual Connect 2023.

Overview

This presentation began it’s life as a Rails Conf 2023 two-hour workshop. And that workshop diverged but the content I developed was meant to fill more than 20 minutes.

In this presentation I’ll walk through:

  • Shell
  • Ruby
  • My Text Editor…hint it’s Emacs

A Little Wayfinding

For this presentation, I’ve enabled the keycast. The mode-line will show what keys I pressed and the commands this called.

Throughout this presentation you will see two types of blocks:

Bash scripts:

echo "Hello World"

Hello World

I’ll execute those live during the presentation (by way of Emacs’s org-babel package).

Margin notes:

Quick, to the presentation!

One Shell of a Game

In this section I’ll walk through the following commands:

tree
a directory/file visualization tool
ripgrep
a blazingly fast grep replacement (and more)
Ctrl+r
backwards history search
fzf
command-line fuzzy finder
bat
a cat replacement (with colors)

tree … to be you and me

The tree command can give you some insight into the structure of your directory. The -L 1 option means “Show only the first level.”

cd ~/git/org/denote/ ; tree -L 1

.
├── blog-posts
├── epigraphs
├── glossary
├── indices
├── melange
├── people
└── scientist

7 directories, 0 files

ripgrep: Grep is Dead Baby

Ripgrep, with its command rg, will “recursively search the current directory for lines matching a pattern.” It is a super charged grep replacement.

I use this command many times per day. More if you consider that I’ve baked it into my Emacs search functionality.

Common switches I use:

--before or -B
Include in this many lines before the match
--after or -A
Include in this many lines after the match
--files-with-matches
Only show the files that have matches
--only-matching or -o
Print only the matching parts (instead of the full line)
-r or --replace
Replace every match in the printed output with this text; you can use capture groups.

rg: Find Community Gem Versions for Project

Software Services by Scientist.com (SoftServ 📖) works on many projects that use similar Samvera community gems. When I’m working on one of those projects, it can be helpful to quickly see what versions we’re working with.

I’ll then check-out those versions of the code; so as I’m working on the project I can see the likely state of the upstream dependencies (and inspect their git history if things get interesting).

rg "^ +((bulk|hy)rax|rails|blacklight(-.*)?) \(\d+\.\d+\.\d+" \
   ~/git/britishlibrary/Gemfile.lock

rg: How Many Packages Do I Use in Emacs?

Sometimes I’m just curious how many “dependencies” I have in my Emacs configuration. This script simply tells me “How many packages do I have?”

rg "\(use-package ([^ ]*)" \
   ~/git/dotemacs \
   --only-matching --no-filename --no-line-number -r '$1' \
   | sort | uniq | wc -l

     154

rg: Where For Art Thou My Derivative Services?

I have a clone of most every Samvera repository on my machine; all in sub-directories of ~/git/.

The following will search all directories in ~/git , ignoring those pesky “dot” directories such as .git or .dassie. This command takes just under one second to run on an M1 Apple ARM chip. 

rg "Hyrax::FileSetDerivativesService" ~/git/ \
   --files-with-matches

/Users/jfriesen/git/org/denote/blog-posts/20221018T103408--responsible-and-sustainable-overrides-in-ruby-and-samvera-in-general__presentation_programming_samvera_scientist.org
/Users/jfriesen/git/org/denote/blog-posts/20230503T121711--these-are-the-tools-i-know-i-know__programming_ruby_samvera_text-editors.org
/Users/jfriesen/git/org/denote/melange/20230216T102133--researching-conditional-derivative-generation-in-the-samvera-stack__programming_samvera_scientist.org
/Users/jfriesen/git/org/denote/melange/20230317T093726--derivative-generation-in-hyrax__programming_samvera.org
/Users/jfriesen/git/org/denote/melange/20230328T093100--work-log-for-derivativezoo__programming_samvera_scientist.org
/Users/jfriesen/git/org/denote/blog-posts/20230502T173945--welcome-to-the-derivative-rodeo__programming_samvera.org
/Users/jfriesen/git/takeonrules.source/content/posts/2023/20221018T103408--responsible-and-sustainable-overrides-in-ruby-and-samvera-in-general__presentation_programming_samvera_scientist.md

How many repositories am I querying? Let’s find out.

ls ~/git | wc -l

     121

Switch ctrl+r

In the terminal, when you type Ctrl+r, you are prompted to search backwards through history. There are plugins to make this even more useful.

The default behavior of Ctrl+r is a reverse-i-search that doesn’t provide a lot of clues.

ctrl+r: Improving on the Default

There are several ways to improve the default experience:

Both mechanisms greatly improve on the default experience.

Leveraging fzf

I installed fzf via Homebrew (e.g. brew install fzf) and configured the following keys for my shell:

Ctrl+r
Backwards history search.
Ctrl+t
Find a file.
Alt+c
Find and cd into directory.

I’ll demonstrate these three functions.

Let’s look at this function in the terminal…

export FZF_CTRL_R_OPTS="
  --preview 'echo {}' --preview-window up:3:hidden:wrap
  --height 50%
  --bind 'ctrl-s:execute-silent(echo -n {2..} | e-send)'
  --bind 'ctrl-y:execute-silent(echo -n {2..} | pbcopy)'
  --bind 'ctrl-/:toggle-preview'
  --color header:italic
  --header 'Press CTRL-y to copy command into clipboard'"

Note the 3 key bindings:

Ctrl+s
Send the current line to my editor (via my e-send command).
Ctrl+y
Copy the current directory into my paste buffer.
Ctrl+/
Change where I’m showing the preview.

fzf: Find a File

Let’s look at this function in the terminal…

export FZF_CTRL_T_COMMAND="fd --type f ."
export FZF_CTRL_T_OPTS="
  --preview 'bat -n --color=always {}'
  --height 80%
  --bind 'ctrl-o:execute(editor {})'
  --bind 'ctrl-y:execute-silent(echo {} | pbcopy)'
  --bind 'ctrl-s:execute-silent(echo {} | e-send)'
  --bind 'ctrl-/:change-preview-window(down|hidden|)'
  --header 'Press CTRL-o to open in EDITOR; CTRL-y to copy to clipboard'"

Note the 4 key bindings:

Ctrl+e
Open the directory in my editor.
Ctrl+s
Send the current line to my editor (via my e-send command).
Ctrl+y
Copy the current directory into my paste buffer.
Ctrl+/
Change where I’m showing the preview.

fzf: Find a Directory

Let’s look at this function in the terminal…

export FZF_ALT_C_COMMAND="fd --type d . $HOME"
export FZF_ALT_C_OPTS="
  --preview 'tree -C {}'
  --bind 'ctrl-o:execute(editor {})'
  --bind 'ctrl-s:execute-silent(echo {} | e-send)'
  --bind 'ctrl-y:execute-silent(echo {} | pbcopy)'
  --header 'Press CTRL-o to open in EDITOR; CTRL-y to copy to clipboard'"

Note the 3 key bindings:

Ctrl+e
Open the directory in my editor.
Ctrl+s
Send the current line to my editor (via my e-send command).
Ctrl+y
Copy the current directory into my paste buffer.

bat in the attic

Where the venerable cat command gets the job done; bat supports syntax highlighting, Git integration, and viewing non-printing characters.

My Emacs magic won’t work for this command; we’ll instead need to look at the command line for this one.

I’m introducing this and moving on; I’ll later reference this command.

Ruby: a Gem of a Language

In this section, I’ll touch just a bit on things I do in Ruby.

Methods of My Madness
Exploring the Method object.
Make the Feedback Fast
Do you feel the need for speed?

Methods of My Madness

Every object has methods. Let’s check base-line Ruby to see how many methods a String object has:

"".methods.count

Methods of My Madness: Source

And we can grep through them:

"".methods.grep(/id/)

Methods of My Madness: Source Location

Where’s the location of a a method? (Due to the constraints of Emacs’s org-babel I need to write the file and require it.)

path = File.join(ENV['HOME'], "Desktop", "hello.rb")
File.open(path, "w") do |f|
  f.puts "module Hello"
  f.puts "  def self.world"
  f.puts "   :hello"
  f.puts "  end"
  f.puts "end"
end
require path

Hello.method(:world).source_location

Methods of My Madness: Source

The pry gem provides the super helpful Method#source method.

path = File.join(ENV['HOME'], "Desktop", "hello.rb")

require path
gem 'pry'
require 'pry'
Hello.method(:world).source

Make the Feedback Fast

This is something I cannot emphasize enough: seek fast feedback. When I’m writing code, I want fast tests. And sometimes, getting fast tests within an application is difficult. Regardless, try to find a way.

I’ve done the following:

  • Ignore the existing spec_helper.rb and do explicit requires.
  • Create a new gem and build out the logic without all the cruft.
  • Do the old if ___FILE___ == $0 trick.

Then when I’m done, incorporate the code and tests into the slower ecosystem.

I wrote about this in: https://takeonrules.com/2023/04/02/make-the-feedback-fast/

Wait the Old if ___FILE___ == $0 Trick?

Let’s say we have the following Ruby file at ~/file_trick.rb. If we require that file, it won’t run the FileTrick.see!. However if we use the following shell command: ruby file_trick.rb it will call the FileTrick.see! method.

module FileTrick
  def self.see!
    puts "I just ran #{__FILE__}"
  end
end
if __FILE__ == $0
  FileTrick.see!
end

I’ve used that process to include tests locally and run them separate from the larger ecosystem; or when I don’t yet know the shape of how I’ll use this nascent script. Caveat, you’ll need to remember to do all kinds of explicit requires. But again, it’s about getting fast feedback on potentially complex things.

Emacs: An Over the Hill Editor in a World of Sisyphean Antics

History
Quick overview of why Emacs.
Org-Mode
Foundational package for working
Collecting Information
A cool little for synthesis.
Projects
A bespoke indexing tool.
Tree Sitter
Intro to Tree Sitter.
Extensibility
We’re programmers, let’s get our tools working for us.
Ephemerality
Developing the courage to add and remove things on a whim.

A Brief History of My Foundational Tool

About 3 years ago I switched editors from Atom. When I switched to Atom I established an unwavering requirement: Any future text editor would need to be free and open source.

Leaving Atom I explored Vim and Visual Studio Code (VS Code 📖); and reluctantly settled on Emacs.

But that reluctancy shifted as I started from a simple foundation and built out what I needed (and wanted).

That tale is not for today, but instead a tour of a few things I love.

Why share this? I was “productive” after three weeks of Emacs. And with it’s extensibility, I’ve continually improved my “computering” skills and tools. Sometimes in tremendous leaps.

Let’s hop over to a few of those.

Org Mode, I Don’t Know What I Did Without You

You’ve seen me evaluate code inline using Org-Mode’s org-babel package.

I cannot emphasize enough how amazing org-mode is; I blather on about it on my blog: https://takeonrules.com/tags/emacs/

I use it for:

Information Hunting and Gathering

This will require a demonstration. And there are a lot of things in play.

First, I establish what I’m working on. Usually this means starting a clock (for time tracking).

Then, I move through my code. And file things away.

And when I’m done moving through code, I can go back and write up what I’ve found.

Let’s demonstrate.

I wrote about this in: https://takeonrules.com/2023/03/12/take-on-rules-blog-writing-and-publishing-playbook/

Any Sufficiently Advanced Hobby is Indistinguishable from Work

Due to the nature of my work (and play), I have many projects. I’ve created a dispatch tool in my editor. Let’s take a look.

Under the hood, I’m using Ripgrep, Org-Mode’s globaly properties, and Emacs data structures to create arbitrary menus.

My CMD+2 is an index of indices. And when I have a timer running for a project (see previous slide); the menu defaults to pre-selecting that project.

I wrote about this in: https://takeonrules.com/2022/12/18/walkthrough-of-my-project-workspace-functionality/

Sitting in a Tree…(K-I-S-S-I-N-G)

Tree Sitter “is a parser generator tool and an incremental parsing library.” It’s backing lots of the recent syntax highlighting and formating logic of text editors. It has a query language.

I wrote a silly little function jf/treesit/qualified_method_name; it copies my current scope and echoes it in my mini-buffer.

Let me show you.

I wrote about this in: https://takeonrules.com/2023/03/27/expanding-on-using-tree-sitter-in-emacs-for-getting-qualified-ruby-method-name/

Extensibility: Adding Features to Work with Me

I’ve long used a menu, mapped to CMD+1. Let’s take a look.

I use this menu as a reminder of some of the things I can do, as well as a quick launching point.

I regularly change this menu as I begin remember some things exist; or want to start remembering new things.

Ephemerality: Who Wants to Live Forever

Emacs is a Read-eval-print loop (REPL 📖) style editor, analog to Ruby’s Interactive Ruby (IRB 📖) or Ruby on Rails (Rails 📖)’s console. It’s super easy to add new functions and then discard them.

I wrote the following function because I was writing lots of pull requests that had the following information:

(defun jf/space_stone/related-to-text ()
  "A throw away function but one I'm going to be using for several days."
  (interactive)
  (insert
    "Related to:\n"
    "\n"
    "- https://github.com/scientist-softserv/iiif_print/issues/194\n"
    "- https://github.com/samvera-labs/bulkrax/issues/760\n"
    "- https://github.com/scientist-softserv/utk-hyku/issues/343\n"
    "- https://github.com/scientist-softserv/adventist-dl/issues/330\n"))

In other words, extending my editor doesn’t take all that much work.

Conclusion

In all of this, my hope is to demonstrate that I’m leveraging my tools around the concept of indexing; that is creating functions and processes that help me find the thing.

Modern programming Knowledge work is about needles and haystacks. It’s not only about documenting the knowledge gained, but also about creating means of stumbling upon it.

Post Script

I encourage you to watch Jay Dixit’s Emacs for Writers presentation. And most everything written by Protesilaos Stavrou; in particular his post Emacs: comments on the Rubber Duck Show of 2023-03-16.

The reality, as technologists, is that our lives are lived in text: code, emails, and documentation. Get good at interacting with text.

Ponder the Koan of Smalltalk: “Code is data is code.”

…And Ruby’s “Objects are classes are objects.”

…And Lisp’s “(Almost) Everything is a List.”

…And consider the advice of many table-top Role Playing Games (RPGs 📖): “Play to find out.”