Modification to Reading, Extracting And Storing Scholarly Information To Supercharge The Writing Process

In Reading, Extracting And Storing Scholarly Information To Supercharge The Writing Process, I wrote about how I extracted both highlights and full-text of entire manuscripts in order to give me granular access to information. Although I’ve continued my extraction of highlights, the extraction of full text (by highlighting the entire document) proved much too time consuming. Instead, I’ve been experimenting with an alternative that is much quicker (as suggested by Andrew in the comments of the entry)—saving the entire manuscript as single-page PDF documents. Here is what I’ve been doing.

After highlighting a manuscript in Highlights.app, I extract my highlights (along with color tags) to Devonthink Pro using the built in export function. By default, Highlights.app saves my extracted highlights files to the DTP Inbox. I move the folder from the DTP Inbox to my Desktop. Within the moved folder I make two new sub-folders: 1. HighlightsX and 2. PDFx. I then move the extracted markdown files to the HighlightsX sub-folder.Within Bookends, I export the annotated pdf to my desktop. There, I open the file with Adobe Acrobat (any app able to add headers and split documents will work).

In the left most header I put the Bookends citation (Bookends: Edit: Copy Citation), in the center I put the DOI number, in the right header, I put the Bookends link (Bookends: Edit: Copy Hypertext Link: Copy as Text). These headings are added to each page of the PDF. I then split the manuscript into multiple single-page documents. I save the split PDF documents to the PDFx sub-folder.

I then move the parent folder my Dropbox Writing folder and Index the folder using DTP. Within DTP I make sure both the main folder and the subfolders will have their tags included (option click on a folder in DTP and make sure “Exclude from Tagging” is unchecked)

.Although this method is faster—there are trade offs. The “Find Also” feature of DTP depends on the words in a document. A document with too many words dilutes the accuracy of the semantic search. A page of text has far more words than an extracted paragraph and thus is slightly less accurate in finding granular information. The other trade-off comes in the amount of text that must be read when searching. It is faster to scan a paragraph versus a whole page of text in a PDF. Regardless, the savings in time using this method far exceeds the trade-offs in accuracy.Let me know what you think.

Bookends Adds Floating Citation Window

I’d like to bring the latest update of BookEnds to your attention.This update fixed an error that would crash the Bookends when trying to obtain a reference (or link back to a reference in Bookends using a DOI link) from Highlights.app. The latest update fixes this error. Clicking on the reference link in Highlights now selects the reference in Bookends without having to modify the Highlights markdown file. This obviates the need to modify the Highlights markdown file as mentioned  in this entry.The latest update also added a floating citation window to Bookends. Like Papers, the floating citation window is invoked using customizable key combinations. So far, I’ve cited using the floating window in Scrivener, Ulysses, and Highlights.app. It works flawlessly.Well done Jon and Sonny Software!

My Migration from Papers to Bookends

I have dedicated a lot of time and energy squeaking every ounce of productivity from the Papers app. I’ve used Papers for more than a decade. Over the years, I built many scripts and work-arounds to address the shortcomings of the application.ReadCube purchased Papers in 2016. Because of the time I’ve spent in Papers, I started having angst when I read of certain changes to the software: a subscription model, the loss of Magic Citations, and the loss of integration with Scrivener.I started exploring alternatives to Papers. When I wrote about my interest in migrating away from Papers, several Wippp Readers suggested I check out Bookends. Several folks raved about the tight integration between Bookends and Devonthink as well as Bookends and Tinderbox—two other programs I use heavily in my academic life. Several people also mentioned the LEGENDARY support of SonnySoft, the company  behind Bookends.My initial evaluation of Bookends was far too cursory. Following publication of the blog entry, I received a very nice email from SonnySoft asking me to take a closer look at Bookends. After digging deep into the software, I decided to migrate from Papers to Bookends. I haven’t looked back. It’s been several months now and the more I use Bookends, the more I like it.Things I like about Bookends:

  • The interface, although not as modern as Papers, is cleaner and more organized.

  • The iOS and Desktop version are better integrated than Papers.

  • A PDF on my phone is drop-dead easy to import into Bookends. The PDF and the metadata I choose is automatically synced with the Mac app (although I have to rename the pdf when I get to my Mac).

  • All PDFs are saved to a single folder in iCloud, making them easy to access and for Devonthink to index.

  • Bookends integrates well with Scrivener (and many other writing clients)

  • Dragging and dropping citations into Tinderbox and Devonthink are both seamless. Tinderbox maintains metadata from Bookends. This obviates the need for the KM script I built to move citations from Papers to Tinderbox.

Things I don’t like:

  • It was difficult to import Papers library with PDF into Bookends (Bookends only imported about 1/3 of my PDFs).

  • I can't export or customize the format of the exported metadata.

  • My smart collections don’t transfer to iOS.

  • Although I can designate a “watch” folder to import new PDFs into Bookends, this only works with PDFs that are saved to the folder AFTER Bookends is open. This didn't work well with how I collect information so I decided to modify my Hazel script, changing it to launch Bookends instead of Papers.

  • I don't like the way duplicate references and / or PDFs are handled.

Over the next few months, I plan to write several entries about my migration and how Bookends has become a critical part of my augmented writing workflow.

Mourning the loss of Integration Between Papers and Scrivener

Early indications, including personal communication with ReadCube personnel, are that Magic Citations (now called SmartCite) will no longer integrate with Scrivener. The new Papers app will only work with Microsoft Word. Bibliography formatting is not an issue for me. The biggest loss is the ability to add citations on the fly in Scrivener without interupting my writing flow.The news of this impending feature loss (along with the announcement of an annual fee) had me scrambling to invesitgate my options for citation managers. During my search, I came across this Wikipedia article, Comparison of Reference Management Software with a great table that collates the majority of software out there.My needs are the following:

  1. Ability to organize and search through metadata and pdfs
  2. Integrated citation insertion with Scrivener and Ulysses
  3. Ability to insert citations while writing on an iOS device
  4. Ability to annotate PDFs and export each comment individually appended with the article's metadata
  5. Ability to export metadata (to enable my workflows for sense-making and export of annotations)

I looked at the following:

  • Zotero
  • Bookends
  • Mendeley
  • Endnote
  • Readcube
  • No product currently fills the void left by Papers, although the consensus of users (both those seeking alternatives to Papers and those who are being forced to leave Sente) seems to be to move to Bookends. I tried the demo version of Bookends and was not impressed. I am waiting impatiently for the release of ReadCube Papers. If the majority of features are retained, I will likely bite the bullet and pay the annual fee. I plan to figure out a work-around to add citations to Scrivener / Ulysses.I'd be interested in hearing your plans / thoughts on academic citation managers.