My Migration from Papers to Bookends

I have dedicated a lot of time and energy squeaking every ounce of productivity from the Papers app. I’ve used Papers for more than a decade. Over the years, I built many scripts and work-arounds to address the shortcomings of the application.ReadCube purchased Papers in 2016. Because of the time I’ve spent in Papers, I started having angst when I read of certain changes to the software: a subscription model, the loss of Magic Citations, and the loss of integration with Scrivener.I started exploring alternatives to Papers. When I wrote about my interest in migrating away from Papers, several Wippp Readers suggested I check out Bookends. Several folks raved about the tight integration between Bookends and Devonthink as well as Bookends and Tinderbox—two other programs I use heavily in my academic life. Several people also mentioned the LEGENDARY support of SonnySoft, the company  behind Bookends.My initial evaluation of Bookends was far too cursory. Following publication of the blog entry, I received a very nice email from SonnySoft asking me to take a closer look at Bookends. After digging deep into the software, I decided to migrate from Papers to Bookends. I haven’t looked back. It’s been several months now and the more I use Bookends, the more I like it.Things I like about Bookends:

  • The interface, although not as modern as Papers, is cleaner and more organized.

  • The iOS and Desktop version are better integrated than Papers.

  • A PDF on my phone is drop-dead easy to import into Bookends. The PDF and the metadata I choose is automatically synced with the Mac app (although I have to rename the pdf when I get to my Mac).

  • All PDFs are saved to a single folder in iCloud, making them easy to access and for Devonthink to index.

  • Bookends integrates well with Scrivener (and many other writing clients)

  • Dragging and dropping citations into Tinderbox and Devonthink are both seamless. Tinderbox maintains metadata from Bookends. This obviates the need for the KM script I built to move citations from Papers to Tinderbox.

Things I don’t like:

  • It was difficult to import Papers library with PDF into Bookends (Bookends only imported about 1/3 of my PDFs).

  • I can't export or customize the format of the exported metadata.

  • My smart collections don’t transfer to iOS.

  • Although I can designate a “watch” folder to import new PDFs into Bookends, this only works with PDFs that are saved to the folder AFTER Bookends is open. This didn't work well with how I collect information so I decided to modify my Hazel script, changing it to launch Bookends instead of Papers.

  • I don't like the way duplicate references and / or PDFs are handled.

Over the next few months, I plan to write several entries about my migration and how Bookends has become a critical part of my augmented writing workflow.

Folder Structure to Maximize Writing with Devonthink Pro

On several occasions I have written about how I use Devonthink Pro (DTP) for scholarly writing. Although at one point I had all my information on manuscripts in a single DTP database, over the last year I have maintained separate databases; one for PDFs, another for my annotations.I accomplish this by having separate folders in Dropbox. After extracting my annotations from Highlights.app, I place the exported files in their own folder. Once a month, I export all the PDFs in my Paper’s Library to their own Dropbox folder. I use a Hazel script to throw away any duplicate PDFs in the Dropbox folder. I index (not import) the annotations folder into one DTP database, and index my PDFs into another.This setup allows a fair amount of flexibility. Not only is this setup advantageous for writing with DTP (as I will cover in my next entry), it allows easy access to my PDFs for reading with Liquidtext or listening with Voice Dream.

Mourning the loss of Integration Between Papers and Scrivener

Early indications, including personal communication with ReadCube personnel, are that Magic Citations (now called SmartCite) will no longer integrate with Scrivener. The new Papers app will only work with Microsoft Word. Bibliography formatting is not an issue for me. The biggest loss is the ability to add citations on the fly in Scrivener without interupting my writing flow.The news of this impending feature loss (along with the announcement of an annual fee) had me scrambling to invesitgate my options for citation managers. During my search, I came across this Wikipedia article, Comparison of Reference Management Software with a great table that collates the majority of software out there.My needs are the following:

  1. Ability to organize and search through metadata and pdfs
  2. Integrated citation insertion with Scrivener and Ulysses
  3. Ability to insert citations while writing on an iOS device
  4. Ability to annotate PDFs and export each comment individually appended with the article's metadata
  5. Ability to export metadata (to enable my workflows for sense-making and export of annotations)

I looked at the following:

  • Zotero
  • Bookends
  • Mendeley
  • Endnote
  • Readcube
  • No product currently fills the void left by Papers, although the consensus of users (both those seeking alternatives to Papers and those who are being forced to leave Sente) seems to be to move to Bookends. I tried the demo version of Bookends and was not impressed. I am waiting impatiently for the release of ReadCube Papers. If the majority of features are retained, I will likely bite the bullet and pay the annual fee. I plan to figure out a work-around to add citations to Scrivener / Ulysses.I'd be interested in hearing your plans / thoughts on academic citation managers.

    ReadCube Release of Papers App

    If you’ve read my blog, you know I’m invested in Papers. The majority of my writing workflows use the app.I’ve been following news about the app with trepidation. Papers “teamed up” with RedCube in March of 2016. Readcube / Papers have been working on a new version of the app. Although the screenshots look reminiscent of Papers, there will be at least one major change; Papers is moving to a subscription model. I have not found pricing information yet.The combination of a new version, and unknown pricing model, and a distrust of traditional publishers has left me wanting to explore my options.I’m curious what app each of you is currently using for manuscript management and bibliography generation (and why). Please leave comments below.Addendum: Beware. Updating to Scrivener 3 breaks Magic Citations in Papers. From what I've read online, Readcube is not saying when (or if) this issue will be resolved.

    Highlights.app Redux

    I was too quick to judge the program Highlights. I gave Highlights a second chance, and although not perfect, the app is slowly winning me over. Highlights has greatly sped up the extraction of information from my reading.First a little background. I use Papers for my PDF and bibliographic management. I’ve written about how I extract highlights and annotations into individual files along with their relevant references. The reason to go to this trouble is to enable Devonthink’s artificial intelligence. The Devonthink AI uses word count to find related information in other files. I use this method extensively in my writing of grants and manuscripts. With my previous method, I found by adding references to each annotation (thus similar words) I was interfering with Devonthink’s ability to find relevant information. In order to improve Devonthink’s accuracy, I stopped adding references to each individual annotation. Instead, I merely include a Papers Link back to the original file.Highlights shows the annotations you make in the app's right-hand column. A pop-up interface within Highlights allows you to make comments, underline , change colors, etc. The killer feature of Highlights is single-command extraction of each annotation / comment to its own file. This feature allows me to extract highlights (and metadata) without having to invoke my Keyboard Maestro macros. When the data is extracted into Devonthink, the individual files are in Markdown. This makes the extracted information easy to view and edit.Although I’m more enthusiastic about Highlights, there are several quirks you should understand:

    1. When Highlights explodes your annotations into individual files, it prepends the original file’s name. If you use Highlights from within Papers (e.g. selecting Highlights as your PDF reader of choice from within Papers) you will end up with a ridiculous, machine based title in your metadata. This is not Highlights fault. The use of non-intuitive file names is one of my pet peeves about Papers. The workaround is to export a copy of the PDF to your Desktop and then launch the exported file using Highlights. Using this method, the author’s name and the title of the manuscript along with the year of publication are all prepended to each markdown file.

    2. After reading and highlighting, I edit the markdown headers in Highlights (choosing the edit tab in the annotations window). I erase all but the primary author’s name and then add the Paper’s Citation and a Papers Link back to original file (copied from Paper’s Edit Menu). There is another quirk here. In Highlights Edit Mode, if you don’t leave a space between the markdown coding and the information you add, your file will be reset back to its original state, erasing your modifications.

    3. Highlights has the ability to automatically look up DOI numbers. When it works, it’s great. It takes a single click to import a reference from the bibliography into your Paper’s Database. Unfortunately, this feature is flaky, especially with longer manuscripts. I often have to revert to my Launchbar scripts to capture the bibliographic information I need.

    Once I’ve added the metadata I want to the master annotation file (and collected the references of interest), from within Highlights, I choose Export >> Devonthink. Highlights creates individual markdown files for each of the annotations. Each individual file contains the master file’s metadata. The data is copied to my Devonthink Global Inbox. I move the folder from the Devonthink Inbox to my Desktop and from there to my Annotations Folder using Launchbar.Highlights has significantly sped up the time it takes to process a manuscript. Using this method, I’m also having greater success with the “See Also” feature of Devonthink. I hope it works for you too.

    LiquidText PDF Reader

    I have experimented with different PDF readers on my iPad. Over the past several months I’ve exclusively used a program called LiquidText.LiquidText has a unique, award-winning, interface that allows me to drag, drop, and manipulate text using my finger. LiquidText was named “Most Innovative iPad App of the Year” by Apple in 2015. LiquidText feels much more natural than any other PDF reader I’ve used on iOS--and I've used many. After highlighting with my finger, the exerpts / highlights / comments are extracted into their own area. The excerpts may be dragged around, linked, and repositioned at will.When reading and highlighting, I either drag text from the PDF into the notes area, or simply hit “AutoExcerpt” and the text is extracted for me. I can comment on the excerpted information.I can also view the place in the paper the highlight came from by merely touching the excerpt in the notes area.LiquidText reads and writes to a whole range of cloud services, but has limited range of export document choices.An Enterprise version is available for $9.99 that adds features like multi-document search and the ability to comment on multiple documents simultaneously. I've been using the app so much, I upgraded primarily to support the developer.There are a few features that are not (yet) available—the most critical for me is the inability to export highlights into a text file—LiquidText only exports highlights to Microsoft Word. This, unfortunately, is not compatible with my workflow for extracting highlights, but I like Liquidtext so much, I figured out a workaround using Skim—here is my entire workflow:

    • I set up a Dropbox LiquidText Folder to serve as a bridge with my desktop.
    • I open the entry in Papers on iOS.
    • From within Papers I select "open in". -this brings up action menu.
    • I open, read, and annotate in Liquidtext. This includes linking the full-text of references to each excerpt (as suggested by reader GH).
    • I export the PDF and Notes from Liquidtext to my Dropbox Folder.
    • When I get to my Mac, I open the PDF in Skim.
    • Under the File Menu in Skim, I choose “Convert Notes” making my highlights into Skim Notes.
    • I then process the annotations as discussed in this blog entry.
    • (If you want to take the time on your desktop you can also replace the Papers PDF with the LiquidText highlighted version).

    Give Liquidtext a try and please let me know what you think.

    Using Skitch and Papers to Capture Figures and Tables

    Few would argue that the most important point of a scholarly manuscript is made in its figures and tables. I am going to share with you how I capture figures while reading scholarly information on my Mac. This workflow uses:SkitchPapersInstall Skitch and make sure, in Skitch Preferences, to enable the “keep Skitch Helper running in background when I quit,” and “Start Skitch Helper when I log in to my computer.”Screen Shot 2016-05-28 at 9.12.13 AMAs I’m reading scholarly literature and come across a table or figure I want to save I do the following. I make the figure as big as possible on my screen. Then, from the Skitch menu in my menubar, I select the Crosshair Snapshot. I then select the figure (and sometimes the caption) trying to balance the white space surrounding the figure.Next, I go to Papers, select the reference in my Papers Library, and then from the Edit Menu:Copy As:Reference.I return to Skitch and double-click at the bottom of the figure then paste the reference text. I then balance the text. The height of the Skitch figure will expand to accomodate the new text.Screen Shot 2016-05-28 at 9.14.40 AMWhen I want to refer to or use the table or figure I view it directly in Skitch or find it in Evernote. Using this method, I can also search for words in the reference (e.g. the author’s name or the title of the manuscript) and sometimes even the words in the figure itself.If I want to use the figure in a presentation or to send it to a colleague or trainee I can export the figure from the Skitch File Menu.Using this method I’ve captured hundreds of figures. I hope this workflow helps you too.

    Retrieval, Annotation, and Capture of Highlights (2016)

    I have a unique way of processing scholarly information, I am convinced it gives me an advantage when writing grants and manuscripts--allowing me to find specific notes and related information very quickly and efficiently without having to rely purely on memory. I’ve written a bit about this before, but the process has evolved, so I am dedicating an entry to my updated workflow. The software and utlities integral to this workflow are:

    Highlighting in Papers 3Despite lacking the outstanding features of Skim, I am back to reading/highlighting in Papers. I switched back to Papers primarily for synchronization. I do a fair amount of reading on my iPad. Any highlights I make on my Mac become immediately available on iOS. Anyhing I highlight on my iPad shows up on my Mac. Although I prefer Skim, the simplification afforded by doing everything in Papers was irresistable. Here is the manuscript we’ll be working with, highlighted in Papers.highlighting-in-papers-3Export NotesOnce I’m finished highlighting, it's time to process. Within Papers I go to File->Export->Notes and am provided with the following menu box. (1) I usually choose to save the file to my Desktop, as it needs to be easily accessible for processing. (2) I choose "Selected Papers Only" or you I export notes from my whole collection. (3) I choose Plain Text.export-notesFirst Stage of Processing the Untitled Notes File on DesktopHere is the untitled notes file saved to my desktop.first-stage-of-processing-the-untitled-notes-file-on-desktopContents of the unprocessed notes Filecontents-of-the-unprocessed-notes-fileKeyboard Maestro Script to Add Citation + Link and Save FileI have written two Keyboard Maestro scripts to help process my exported notes. The first script adds metadata to the text of the notes file we just exported, then renames the file using the unique Papers Citation. In order to use my script, in Papers, I make sure the correct paper is highlighted. Next, in the Finder, I select the exported note file. I invoke the script and it prepends the unique Papers Citation followed by a Papers Link to the top of the file.keyboard-maestro-script-to-add-citation---link-and-save-fileKeyboard Maestro Script to Add Citation + Link and Save File ContinuedThe script continues on to a save dialogue box where it replaces "Untitled" with the unique Papers Citation. I have to manually remove the curly brackets (as they are disallowed in file naming). I save the file to the Desktop.Download the script here…. You'll have to remove the .txt from the end of the file to use.keyboard-maestro-script-to-add-citation---link-and-save-file-continuedUniquely named Notes File Saved to Desktop.This note I just saved contains ALL of the exported highlights from the manuscript in a single file. As I've talked about in previous entries, I find it more efficient to access a single concept rather than having to dig through an entire papers worth of notes. I developed (along with a few others) a second KM script that copies the Metadata at the top of the note file, parses the single note file into a collection of notes files--one highlight per file, and appends the metadata to each file.Processing highlights in this way has revolutionized my grant and manuscript writing efficiency.Keyboard Maestro Script to Parse Highlights and Append metadata to each notekeyboard-maestro-script-to-parse-highlights-and-append-metadata-to-each-noteDownload the script here…. You'll have to remove the .txt from the end of the file to use.Moving the FolderI highlight the folder on my Desktop, then double click my Shift Key--this invokes a Launchbar action.Screenshot 2015-12-19 16.54.54Launchbar Moves FolderUsing Launchbar I move the text notes to the folder where I keep all the extracted highlights from all the manuscripts I'v read.Screenshot 2016-01-05 11.25.23Ulysses to finish the processAs I've mentioned previously, i use Ulysses extensively in my personal and professional life. I use Ulysses to finish the processing of my highlights, but any text editor will do. (1) Ulysses is pointed to read all the files in my Notes folder--here you see all the individual files extracted from this manuscript. (2) is the text of a single highlight, (3) is the metada added to each file by the KM script. With some notes I will add my own comments, other times, not. (4) are the references that support the statement....these are added manually by reading through the statement and copying each of the corresponding references from the original PDF. I don't go back and forth between the text file and the PDF...instead, I take advantage of Launchbar's Clipboard History.ulysses-to-finish-the-processLaunchbar Clipboard HistoryI merely highlight each reference mentioned, then in bulk I paste each of the references into the note.launchbar-clipboard-historyAppearance of NoteI showed you what this file looked like in Ulysses. Here is its appearance as a free standing text file. Depending on my mood (and time) I may or may not append refernces to each note file.appearance-of-noteCopy PDF to Indexed FolderAfter I've processed everything, the last step is to export a copy of my PDF to a location Devonthink can access.copy-pdf-to-indexed-folderUsing Devonthink to Find Concepts and Related InformationYou may aak yourself why I go to such lengths processing the information I read. The answer is Devonthink. I use the "Artificial Intelligence" of Devonthink quite extensively in my writing. As I mentioned, all the processed highlights, copies of PDFs go into a set of folders on Dropbox. I then Index (not import) the information in these folders (1) so they are "seen" by Devonthink. Why is that important? When I conduct a search (2), I can find the idea I'm looking for (3). By clicking on the "See also" function (4) in Devonthink I can see every piece of related information in my scholarly library (5), including things I have previously written. This ability has revolutionized the way I write.using-devonthink-to-find-concepts-and-related-informationConclusionIn this entry, I have shown you how I read and process scholarly my annotations from manuscripts. Although the process requires an extra ten minutes or so per manuscript, the payoff is immense.Please comment below.

    Papers Links

    In previous entries I've written about my love of Papers and how I use it for all my grants and scholarly writing.One of the features I use constantly is called Papers Links. This is a brief code (looks like a URL) that links back to a particular manuscript in your personal library. Here is an example:papers3://publication/doi/10.1097/AIA.0b013e3181eace73Papers Links are available in Papers by highlighting the manuscript, then using "Copy As" under the Edit Menu, right clicking on the paper of interest, or using the keys: Shift- Command-L.
    Combining Papers Links with Launchbar is nothing short of awesome. I annotate everything I extract from a manuscript with a Papers Link and a Papers Citation. When writing, I can highlight any Papers Link, hit my Shift Key twice to invoke Launchbar Actions (as I discussed in this entry). The double shift copies the highlighted text to Launchbar--the orange tab shows that Launchbar is awaiting an Action Command.Screenshot 2015-12-19 16.54.54I hit Tab Key to bring up the context sensitive menu.Screenshot 2015-12-19 16.55.02I select Papers and hit return. Papers launches directly to the paper of interest.Screenshot 2015-12-19 16.55.25I use Paper Links in all my extracted annotations—it makes it lightning-fast to use a reference when writing, and to be able to dig deeper into the original reference as needed.Screenshot 2015-12-13 16.23.22I also use Papers Links when I copy figures or tables from a manuscript using Skitch / Evernote. The Papers Link allows me to quickly find the manuscript with the embedded media element when writing or preparing presentations.I use Papers Links constantly when writing. I hope you find them helpful too.

    Papers 3 Overview

    Papers 3 has been re-written from the ground up. There are changes in virtually every facet of the program, from importing to matching manuscripts to storage and synchronization. Before I explain my workflow, I thought it would be important to familiarize you with some of these changes.

    You can download a 30 day trial of Papers 3 and follow along with me. Papers 2 and 3 can exist side-by-side without altering your data.

    Go ahead and install Papers 3. Once that’s done, let’s take a look at some of the preferences.

    Preferences General:

    I choose to open (and annotate) my manuscripts in Skim (as I did with Papers 2). For a short time, I experimented with the built in highlighting features of Papers, but they didn’t meet my needs.

    Preferences general

    Preference: Sync

    I want my library to sync across my computers and iOS devices. Here, I’ve turned on Dropbox Sync and checked automatically sync new changes.

     

    Preference sync

    Preferences: Library

    Here are my Library Preference Settings:

    • (1) denotes the location of my Papers Library in Dropbox
    • Hitting the Browse Button (2) places a virtual disk on your desktop that allows access to your entire Papers Library (via paper, author, year, etc)
    • I keep the Copy files to Paper Library folder after import (3) checked
    • I check Organize library folders by subdirectories (4). I use the convention Category Author Year
    • I check Rename Files in Library Folder (5): I use the convention Author Title Source Year

    Preferences library

    Main Screen

    This is the main Papers 3 screen. The terminology I use here is primarily my own–may deviate from what the company calls a specific area.

    • The Library (1) organizes all the media in the program. My Collections includes both manually created folders and Smart Folders (folders created by metadata of the media. Shared Collections allows you to share selected folders with others.
    • The Search Bar (2) allows you limit your searches or change the presentation of your Library
    • The Filter Bar (3) lets you (from left to right) search for a document, view your library, view by Labels, view library by author, view by Type, and view previous database entries.
    • The Viewing Pane (4) is where you view your media. I typically use the “Column” view, but there are other choices as well.
    • Metadata (5) for the paper may be viewed (from left to right) by Overview, Information, Notes, or Activity
    • Metadata Inspector (6) is where you view Metadata and related information
    • PDF and Supplement Window (7) lets you visually see the first page of the actual paper as it exists in the library.
    Main screen

    Annotations and Highlights in Skim

    When I double click on an entry in Papers, the manuscript opens in Skim.

    Annotations in skim

    My Move to Papers 3 (End of Endnote Redux)

    I was a big fan of Papers 2 as my manuscript and citation management software. After reading the early reviews of Papers 3, I was hesitant to upgrade and trepidatious about the future of the software. I went back and took another look at Sente, Mendeley, Endnote, and Zotero. In the end, I decided to stick with Papers.

    Fortunately for me, despite the rocky start, the folks at Mekentosj have continued to work on Papers 3. About a month ago, I decided to make the transition from Papers 2 to Papers 3. This was not an easy decision, as changes in Papers 3 have forced me to revise major portions of my writing workflow. Although the transition was painful, I’m convinced what I have now is more robust and flexible than my previous system.

    Over the next few weeks, I’ll write about my transition to Papers 3 (both Mac and iOS). I’ll cover the Papers 3 new interface, how I continue to leverage my Papers 2 data (despite the interface and data storage changes), and how I continue to integrate Papers 3 with Devonthink.

    I hope you find my experience useful.

    Writing and Reading Workflow: Batching and Automating Full-Text Retrieval

    In my work at Duke University on simulation, games-based learning, and learning technology, I frequently read and write scientific information. I’ve developed a series of workflows to help me efficiently deal with information when writing papers and grants. In a previous entry I talked about how I try to batch operations, doing similar things at the same time. This entry will be about my workflow for reading scientific manuscripts.

    I use Skim for scientific reading, typically using the bottom pane of the “Split PDF” feature to look at the title of references as I come across them in the body of the manuscript. On average, I’m interested in downloading about 10% of these references to my personal library.

    Although I love Skim, one feature I have not been able to figure out is how to highlight items in the bottom split screen. Thus, I’ve created a workaround. Before I start reading an article in Skim, I create a note called “Get Manuscripts” As I’m reading, I merely record the references number of articles I want to fetch in my “Get References” note. Because I create this note first it’s always located at the top of my Skim Notes. When I’m done reading the current article and processing my annotations, I retrieve the supporting articles I identified while reading. 

    To do this, I use a phenomenal program called KeyBoard Maestro. Keyboard Maestro lets me type a single keyboard command to trigger an entire workflow. One by one, I work through my “Get References” numbers, highlighting the title of each corresponding article. After highlighting each title, Keyboard Maestro does the rest (invoked by Command-Option-R).

    Keyboard Maestro automatically  (1) copies my highlighted text, (2) surfs to the Duke Library literature search page, (3) pastes the text string into the appropriate field, and (4) submits the search. Thus, with a single key combination, I am able to download the full text of supporting literature. Once I find the full text I'm interested in (usually a PDF), I save it to my Download Folder, where Hazel takes over and imports the paper into my Papers Library.

    If I'm not ready to retrieve the articles, I create an Omnifocus Action for each to be collected at a later time.

    My macro is not limited to Skim, it works with any text I’ve highlighted, be it on the web, in a manuscript, or somewhere else. Here is a picture of the programming, but obviously, you’ll have to modify it for your own favorite full-text repository. Enjoy!

    Maestro Automate Full Text Retrieval

     

    Addendum (January 14, 2014): If you’re searching at Duke, you’ve probably noticed the library changed its search interface. I’ve updated the Keyboard Maestro Script accordingly. It’s only configured to search for peer-reviewed articles. Download the script and import it into Keyboard Maestro (will only work for Duke Libraries).

    Meeting Workflows: Processing Meeting Notes and Discovering Linkages

    It was a great deal of fun to join my friends, David Sparks - MacSparky, Katie Floyd, and Brett Terpstra (hosted by Dan Miller) on stage at MacWorld Live to discuss my Meeting Workflow. The session was webcast live from the MacWorld site. Following the session, I received many questions as to how I process my notes once they’ve been collected. Here is what I do…..

    As I mentioned, I write and capture all my notes in Plaintext (using Drafts on my iPad and iPhone and nvALT on my Mac). I covered the way I capture ideas (IdeaX) using TextExpander. I use a similar method for capturing meeting notes (MeetX), chunks of writing (ScribbleX), quotations (QuoteX), and random thoughts (ThoughtX). Using Simplenote Notesy, I’m able to keep my notes on my iPad and iPhone in sync with my Mac. A single folder indexed by nvALT serves as the repository for everything, regardless of their content. In a pinch, I can search my notes on my iPad or iPhone using SimpleNote Notesy, but the true power of this workflow is realized when I return to my Mac.

    I have alluded to my love of Devonthink Pro Office. I’m especially fond of DT's ability to find related notes using its artificial intelligence. For the purpose of this workflow, I have a single database that indexes three things: (1) my collection of manuscripts (from Papers2), (2) my web clippings, and (3) my nvALT notes. My web clippings are added directly to my DT database, but my Papers folder and nvALT folder are INDEXED, not imported into the same database. Indexing in DT is done by choosing File:Index… and navigating to the folder of interest on your computer (this is only during the initial set-up). You will have to manually update the index your folders each time you add new information. This is quick and simple– done by choosing the appropriate folder in DT and choosing File:Update Indexed Items.

    When I’m writing (or searching for information), I can select a particular note and find all related information in my library-whether it’s a scholarly article, a newspaper clipping, a web site, or a note I’ve taken. Often times, DT finds linkages for me that weren’t readily apparent. This speeds my writing process immensely and makes me look like a magician to my peers. I hope it works for you too.

    Appended March 2, 2013: Read this entry on why I switched from SimpleNote to Notesy.

    Writing Workflows: Capturing Annotations to Enhance Scientific Writing and Knowledge Retrieval

    As I mentioned in a previous post, reading and writing are central to my academic career. I am a voracious reader. I not only have to keep current in my clinical specialty, Anesthesiology, but also in my research areas, Simulation and Games Based Learning. Because of this, I’ve developed multiple workflows that make reading and annotating more enjoyable, efficient, and useful.I’ve mentioned my preference for Papers2 app to keep track of scientific publications. What I didn’t mention is that I’m a prodigious highlighter-it is not uncommon for me to mark up a single manuscript with dozens of highlights and notes. Keeping track of the information in manuscripts AND my thoughts about what I read used to be a real chore. Before Papers included highlighting, I had a complex workflow to accomplish this feat. Now it’s relatively simple.As I mentioned, when working with scientific literature, I like to batch operations. I go on paper collecting binges where I do little (if any) reading. Only after I’ve completed my collecting, is it time to read. I actually schedule time on my calendar to read, annotate, and synthesize ideas.This entry is about my annotation workflow–and how I use technology to speed up the process of retrieving concepts when writing and citing.I do most of my reading (and annotating) on one of two devices: my Mac or my iPad. I read so much, it’s hard to keep track of all the ideas. There are few things more frustrating than knowing I’ve read something, but being unable to find the source. This workflow alleviates the stress of finding relevant information in my library. But even more than that, using artificial intelligence in a tool called Devonthink, I can find links between concepts that were not initially apparent to me.This workflow has been a personal success. Before implementing this workflow, I could spend hours trying to track down an appropriate reference. Since implementing this workflow the same task takes seconds.I also wanted to thank Derek Van Ittersum (Kent State | blog) for inspiring me to polish this workflow–I had a chance to collaborate with Derek on the Mac Power Users 100 show.Programs needed:  

    Setting Things Up

    I’m assuming you’ve already installed DropBox on both your Mac and you iPad. As I discussed in a previous entry, I configure Papers save PDFs to a designated folder Dropbox. I set this up by navigating to the preference menu in Papers and designated my preferred folder you in Dropbox. I do this to have the PDFs in my Papers collection accessible from anywhere and by any tool.How Papers handles PDFsWithin the Papers preferences, I’ve configured Papers to launch a PDF in a new tab (using Papers itself, NOT an external PDF reader).Launch PDF within Papers

    Annotating

    Papers on the Mac and iPad have built in highlighting tools (on the Mac, invoked by hitting the control key over highlighted text, or on the iPad by holding a finger down and dragging). As I highlight or make notes in Papers , a separate layer is created in the PDF that keeps track of annotation information. As I annotate, each highlight and note appears under the “Notes” tab in Papers.Notes Tab in PapersIf I find a paper I’d like to read that’s not in my library I copy and paste the title and author to my OmniFocus Inbox-to be gathered during a future collecting binge. This way I don’t interrupt what I’m doing.After I’m done highlighting and note taking, I write a single summary note that captures the essence of the paper. At this point I also tag the paper Papers Keywords. Once I’ve completed my summary, if I’ve been reading on my iPad, I synchronize the PDF back to Papers on my Mac. After syncing, the annotations I made on the iPad are available on the Mac (and visa versa). If I’ve been reading on my Mac, I don’t need to synchronize to perform the next step.Once the annotated PDF is in the Papers collection on my Mac, I go to File:Export:Notes within Papers (make sure the “export selected paper” is highlighted and RTF is selected) to export a Notes file to a folder on my Desktop. RTF format is important–Devonthink relies on it.Export Notes from PapersA limitation of Papers is that it can’t export each note or highlight separately–it dumps all my annotations into a single file. I open the file and cut and paste each individual note (that represents a single highlight) into its own file. I then comment on the highlighted passage. Finally, I use the “Magic Manuscript” feature of Papers (invoked by hitting my Option Key twice) to append the citation reference to each statement.Extracted Note with Magic Manuscript CitationFinally, I drag each RTF into to the “Supplemental Files” tab of the Papers. This saves each comment in a supplemental folder residing side-by-side with the manuscript in the Papers Dropbox hierarchy.Adding RTF to Supplemental Files in PapersThe next step of the workflow uses the “Indexing” feature of Devonthink Pro Office. Using Devonthink, it is possible to index (or reference) any folder on your hard drive. This makes the contents of PDFs and RTFs available to Devonthinks Artificial Intelligence without directly importing the information into a Devonthink Database.I created a Devonthink Literature Database that indexes (does not import) my Papers2 Folder. I did this the first time by opening the File:Index… Menu item in Devonthink and navigating to the Papers Folder in Dropbox. All subsequent updates are done by opening my Literature Database, highlighting the indexed “Papers2” Folder, and navigating to File:Update Indexed Items. Indexing can takes quite some time–be patient.Devonthink IndexingThis all sounds complicated, but trust me, it’s worth it. What this allows me to do is open Devonthink and find any statement, concept, or related item quickly. I can then cherry pick comments related to my search and paste them into a draft document with little modification.When I’m writing, I take all related concepts and paste all of them into a single card in Scrivener. This allows me to arrange, and rearrange information to help support my argument. Because I took the time up front to include the Magic Manuscript Index from Papers, no further searching or citation work is needed, until I’m ready to format my bibliography.Scrivener for iOS is under development–it is likely my workflow will change when it is released. Instead of using Scrivener, one might consider using Tinderbox –Derek’s tool of choice or the newly announced Scapple (by the makers of Scrivener). I will cover the next step of the writing process using these tools in an upcoming entry.

    Closing

    In this entry I’ve covered my annotation workflow and how I make use of my annotations when writing. Using Dropbox, I can access my highlighted manuscripts from anywhere. Using a combination of Papers and Devonthink I can make use of my annotations saving me countless hours of time when writing.

    Writing Workflows: Collecting Information in Papers2 (MacPowerUsers Show 100)

    I had a great time as a guest on David Sparks and Katie Floyd’s milestone MacPowerUser Show 100 yesterday. Their podcast has been a staple on my iPod and iPhone for years. To celebrate they invited 10 guests to discuss workflows in their professional lives. Over the past several years, I have learned an incredible amount from Katie and David. It was nice to be able to give something back.I had the pleasure of discussing a portion of my writing workflow–how I get information into my academic research library-Papers2. I’ve talked about why I use Papers in a previous entry. If you’re not familiar with Papers you should read my previous post first. Go ahead. I’ll wait.——————————- Okay, welcome back.When tackling a new writing project I pass through several stages:

    1. collecting
    2. organizing
    3. reading
    4. synthesizing
    5. citing

    The workflow I covered on the MacPowerUsers had to do with collecting information. I usually obtain scientific literature in one of three ways:

    • as a recipient of an Endnote library (when writing collaboratively)
    • through personal searches on the web
    • as an attachment to a colleague’s email.

    I’ll cover each of these contingencies below.Endnote still has quite a foothold in academics. I often find my collaborators have not yet made the switch to Papers. Many times, when writing collaboratively, we must share libraries between the two applications (Endnote and Papers). Fortunately Papers can import (and export) Endnote libraries seamlessly (as long as they’re in Endnote XML format). Easy and no need for a workflow!However, the other two situations are more complex (searching myself or receiving attachments to emails) and require a workflow–making use of the wonderful program called Hazel. Hazel monitors folders on my computer and acts on individual files according to rules I create. Hazel can even peek at the content of files and “recognize” what's inside. I have Hazel monitoring about a dozen folders on my Mac, but my Downloads Folder keeps Hazel the busiest. I have about two dozen rules running on my Downloads Folder alone.For this particular writing workflow I ask Hazel to look at the contents of every PDF in my Downloads Folder and match files that have words unique to scholarly publications.I’ve found having Hazel search for the word “References” within each PDF works the best. When a PDF whose contents contain the word “References” is matched, Hazel automatically launches Papers and imports the manuscript. While importing, Papers fetches metadata, renames the PDF to the convention I’ve specified, and files the manuscript in a specified hierarchy on DropBox–all automatically. Another preference in Papers erases the original file once it’s imported.I’ve tried other search words for Hazel including “abstract,” “methods,” “results,” and “discussion”–none work as well as references. Most every scholarly manuscripts has references (unfortunately, even the word “references” will not be 100% reliable–in a minority of publications, references will be called a "bibliographies" or "citations").You are probably wondering why I don’t just use the built in unified search window in Papers. My answer: it is faster and less frustrating to find full-text directly through our Library’s web page. Papers unified search will work through a fire wall (using proxy URLs in the search interface) – but it’s hit-or-miss whether a link in Papers will lead to a full-text article or merely a frustrating publisher’s login screen–most usually the latter. My hit rate is much higher on the web. I use Papers built-in search engine primarily to retrieve metadata (after the full text pdf has already been imported).When I’m collecting full-text articles I save EVERYTHING to my Downloads Folder and let Hazel do the rest. When she finds a match, she launches Papers and imports each manuscript without any additional effort on my part. Using this method, I can conduct my search for scholarly information with minimal interruptions - Hazel does the rest.Searching on my own is the most common way I get information into papers, but occasionally a colleague will mail me something they think I should read. For these situations here’s what I do. I have Hazel monitor my Mail Downloads folder (~/Library/Containers/com.apple.mail/Data/Library/Mail Downloads ) and copy EVERY attachment to my Downloads Folder (I do this for a host of reasons–not just publications–I’ll talk about why in a future entry) Once the paper is in the Downloads folder Hazel can work her magic using the rule mentioned earlier. Apple now hides the Library Folder by default. Here is a quick tutorial on how to find it on your hard drive (works for Lion or Mountain Lion).So there it is: how I use Hazel to speed up the collection of information when writing or researching. I hope it’s helpful to you. In a future entry I’ll talk about an emerging trend in research paper management: social networking.Cheers, Jeff