Modification to Reading, Extracting And Storing Scholarly Information To Supercharge The Writing Process

In Reading, Extracting And Storing Scholarly Information To Supercharge The Writing Process, I wrote about how I extracted both highlights and full-text of entire manuscripts in order to give me granular access to information. Although I’ve continued my extraction of highlights, the extraction of full text (by highlighting the entire document) proved much too time consuming. Instead, I’ve been experimenting with an alternative that is much quicker (as suggested by Andrew in the comments of the entry)—saving the entire manuscript as single-page PDF documents. Here is what I’ve been doing.

After highlighting a manuscript in Highlights.app, I extract my highlights (along with color tags) to Devonthink Pro using the built in export function. By default, Highlights.app saves my extracted highlights files to the DTP Inbox. I move the folder from the DTP Inbox to my Desktop. Within the moved folder I make two new sub-folders: 1. HighlightsX and 2. PDFx. I then move the extracted markdown files to the HighlightsX sub-folder.Within Bookends, I export the annotated pdf to my desktop. There, I open the file with Adobe Acrobat (any app able to add headers and split documents will work).

In the left most header I put the Bookends citation (Bookends: Edit: Copy Citation), in the center I put the DOI number, in the right header, I put the Bookends link (Bookends: Edit: Copy Hypertext Link: Copy as Text). These headings are added to each page of the PDF. I then split the manuscript into multiple single-page documents. I save the split PDF documents to the PDFx sub-folder.

I then move the parent folder my Dropbox Writing folder and Index the folder using DTP. Within DTP I make sure both the main folder and the subfolders will have their tags included (option click on a folder in DTP and make sure “Exclude from Tagging” is unchecked)

.Although this method is faster—there are trade offs. The “Find Also” feature of DTP depends on the words in a document. A document with too many words dilutes the accuracy of the semantic search. A page of text has far more words than an extracted paragraph and thus is slightly less accurate in finding granular information. The other trade-off comes in the amount of text that must be read when searching. It is faster to scan a paragraph versus a whole page of text in a PDF. Regardless, the savings in time using this method far exceeds the trade-offs in accuracy.Let me know what you think.

Bookends Adds Floating Citation Window

I’d like to bring the latest update of BookEnds to your attention.This update fixed an error that would crash the Bookends when trying to obtain a reference (or link back to a reference in Bookends using a DOI link) from Highlights.app. The latest update fixes this error. Clicking on the reference link in Highlights now selects the reference in Bookends without having to modify the Highlights markdown file. This obviates the need to modify the Highlights markdown file as mentioned  in this entry.The latest update also added a floating citation window to Bookends. Like Papers, the floating citation window is invoked using customizable key combinations. So far, I’ve cited using the floating window in Scrivener, Ulysses, and Highlights.app. It works flawlessly.Well done Jon and Sonny Software!

Highlights.app Redux

I was too quick to judge the program Highlights. I gave Highlights a second chance, and although not perfect, the app is slowly winning me over. Highlights has greatly sped up the extraction of information from my reading.First a little background. I use Papers for my PDF and bibliographic management. I’ve written about how I extract highlights and annotations into individual files along with their relevant references. The reason to go to this trouble is to enable Devonthink’s artificial intelligence. The Devonthink AI uses word count to find related information in other files. I use this method extensively in my writing of grants and manuscripts. With my previous method, I found by adding references to each annotation (thus similar words) I was interfering with Devonthink’s ability to find relevant information. In order to improve Devonthink’s accuracy, I stopped adding references to each individual annotation. Instead, I merely include a Papers Link back to the original file.Highlights shows the annotations you make in the app's right-hand column. A pop-up interface within Highlights allows you to make comments, underline , change colors, etc. The killer feature of Highlights is single-command extraction of each annotation / comment to its own file. This feature allows me to extract highlights (and metadata) without having to invoke my Keyboard Maestro macros. When the data is extracted into Devonthink, the individual files are in Markdown. This makes the extracted information easy to view and edit.Although I’m more enthusiastic about Highlights, there are several quirks you should understand:

  1. When Highlights explodes your annotations into individual files, it prepends the original file’s name. If you use Highlights from within Papers (e.g. selecting Highlights as your PDF reader of choice from within Papers) you will end up with a ridiculous, machine based title in your metadata. This is not Highlights fault. The use of non-intuitive file names is one of my pet peeves about Papers. The workaround is to export a copy of the PDF to your Desktop and then launch the exported file using Highlights. Using this method, the author’s name and the title of the manuscript along with the year of publication are all prepended to each markdown file.

  2. After reading and highlighting, I edit the markdown headers in Highlights (choosing the edit tab in the annotations window). I erase all but the primary author’s name and then add the Paper’s Citation and a Papers Link back to original file (copied from Paper’s Edit Menu). There is another quirk here. In Highlights Edit Mode, if you don’t leave a space between the markdown coding and the information you add, your file will be reset back to its original state, erasing your modifications.

  3. Highlights has the ability to automatically look up DOI numbers. When it works, it’s great. It takes a single click to import a reference from the bibliography into your Paper’s Database. Unfortunately, this feature is flaky, especially with longer manuscripts. I often have to revert to my Launchbar scripts to capture the bibliographic information I need.

Once I’ve added the metadata I want to the master annotation file (and collected the references of interest), from within Highlights, I choose Export >> Devonthink. Highlights creates individual markdown files for each of the annotations. Each individual file contains the master file’s metadata. The data is copied to my Devonthink Global Inbox. I move the folder from the Devonthink Inbox to my Desktop and from there to my Annotations Folder using Launchbar.Highlights has significantly sped up the time it takes to process a manuscript. Using this method, I’m also having greater success with the “See Also” feature of Devonthink. I hope it works for you too.