Modification to Reading, Extracting And Storing Scholarly Information To Supercharge The Writing Process

In Reading, Extracting And Storing Scholarly Information To Supercharge The Writing Process, I wrote about how I extracted both highlights and full-text of entire manuscripts in order to give me granular access to information. Although I’ve continued my extraction of highlights, the extraction of full text (by highlighting the entire document) proved much too time consuming. Instead, I’ve been experimenting with an alternative that is much quicker (as suggested by Andrew in the comments of the entry)—saving the entire manuscript as single-page PDF documents. Here is what I’ve been doing.

After highlighting a manuscript in Highlights.app, I extract my highlights (along with color tags) to Devonthink Pro using the built in export function. By default, Highlights.app saves my extracted highlights files to the DTP Inbox. I move the folder from the DTP Inbox to my Desktop. Within the moved folder I make two new sub-folders: 1. HighlightsX and 2. PDFx. I then move the extracted markdown files to the HighlightsX sub-folder.Within Bookends, I export the annotated pdf to my desktop. There, I open the file with Adobe Acrobat (any app able to add headers and split documents will work).

In the left most header I put the Bookends citation (Bookends: Edit: Copy Citation), in the center I put the DOI number, in the right header, I put the Bookends link (Bookends: Edit: Copy Hypertext Link: Copy as Text). These headings are added to each page of the PDF. I then split the manuscript into multiple single-page documents. I save the split PDF documents to the PDFx sub-folder.

I then move the parent folder my Dropbox Writing folder and Index the folder using DTP. Within DTP I make sure both the main folder and the subfolders will have their tags included (option click on a folder in DTP and make sure “Exclude from Tagging” is unchecked)

.Although this method is faster—there are trade offs. The “Find Also” feature of DTP depends on the words in a document. A document with too many words dilutes the accuracy of the semantic search. A page of text has far more words than an extracted paragraph and thus is slightly less accurate in finding granular information. The other trade-off comes in the amount of text that must be read when searching. It is faster to scan a paragraph versus a whole page of text in a PDF. Regardless, the savings in time using this method far exceeds the trade-offs in accuracy.Let me know what you think.

Deckset: Rapid Presentation Generation using MarkDown

My workflows have changed since I wrote my entry: “Why I Use Plaintext” in June 2012. The entry was about avoiding the distraction of Microsoft Word and Apple Pages while writing. At the time I wrote the entry there was no Ulysses. Now I use Ulysses for almost all my short-form writing. In 2012, there was no iOS version of Scrivener. Now I use Scrivener for all my long-form writing. Although technology has changed, some things have not. I still dislike bloated software packages that get in the way of my final goal. I still love Plaintext / Markdown.There is another requirement of my job that relies just as heavily on bloated software-presentations. Speeding up the development of a presentation is the focus of this entry.Presentations are used throughout business and academia to transmit ideas. Some influential thinkers, such as Edward Tufte, argue bulleted presentations shouldn’t be used at all. The reality is that presentations are deeply ingrained in business and academia. Many presenters rely too heavily on the glitz offered by the software at the expense of content. It’s easy to get lost in software features—spending hours tweaking backgrounds, transitions, and text placement.Which brings me to Deckset. Deckset is an amazing application for iOS that converts Markdown files into presentations. As this review of Deckset 1 in Macworld says: “It’s designed for the average person who needs to make beautiful slides without the muss and fuss of Keynote or PowerPoint.” With the release of Deckset 2 and its outstanding features, the application has become my go-to application for rapid preparation of presentations. Because Deckset uses Markdown, I find myself concentrating less on making my slides look attractive and more on content. Deckset works seamlessly with Ulysses, allowing me to quickly edit my presentations on my phone, my tablet, or my Mac. (You can use any text editor with Deckset. If you plan to use Ulysses, check out these helpful tips).I recently used Deckset to develop a new 30 minute presentation. I estimate the presentation took about 1/4 the amount of time to develop versus Keynote. The cost for Deckset 2 is $29 (with educational discounts available).Here is a link to the Deckset manual.

Highlights.app Redux

I was too quick to judge the program Highlights. I gave Highlights a second chance, and although not perfect, the app is slowly winning me over. Highlights has greatly sped up the extraction of information from my reading.First a little background. I use Papers for my PDF and bibliographic management. I’ve written about how I extract highlights and annotations into individual files along with their relevant references. The reason to go to this trouble is to enable Devonthink’s artificial intelligence. The Devonthink AI uses word count to find related information in other files. I use this method extensively in my writing of grants and manuscripts. With my previous method, I found by adding references to each annotation (thus similar words) I was interfering with Devonthink’s ability to find relevant information. In order to improve Devonthink’s accuracy, I stopped adding references to each individual annotation. Instead, I merely include a Papers Link back to the original file.Highlights shows the annotations you make in the app's right-hand column. A pop-up interface within Highlights allows you to make comments, underline , change colors, etc. The killer feature of Highlights is single-command extraction of each annotation / comment to its own file. This feature allows me to extract highlights (and metadata) without having to invoke my Keyboard Maestro macros. When the data is extracted into Devonthink, the individual files are in Markdown. This makes the extracted information easy to view and edit.Although I’m more enthusiastic about Highlights, there are several quirks you should understand:

  1. When Highlights explodes your annotations into individual files, it prepends the original file’s name. If you use Highlights from within Papers (e.g. selecting Highlights as your PDF reader of choice from within Papers) you will end up with a ridiculous, machine based title in your metadata. This is not Highlights fault. The use of non-intuitive file names is one of my pet peeves about Papers. The workaround is to export a copy of the PDF to your Desktop and then launch the exported file using Highlights. Using this method, the author’s name and the title of the manuscript along with the year of publication are all prepended to each markdown file.

  2. After reading and highlighting, I edit the markdown headers in Highlights (choosing the edit tab in the annotations window). I erase all but the primary author’s name and then add the Paper’s Citation and a Papers Link back to original file (copied from Paper’s Edit Menu). There is another quirk here. In Highlights Edit Mode, if you don’t leave a space between the markdown coding and the information you add, your file will be reset back to its original state, erasing your modifications.

  3. Highlights has the ability to automatically look up DOI numbers. When it works, it’s great. It takes a single click to import a reference from the bibliography into your Paper’s Database. Unfortunately, this feature is flaky, especially with longer manuscripts. I often have to revert to my Launchbar scripts to capture the bibliographic information I need.

Once I’ve added the metadata I want to the master annotation file (and collected the references of interest), from within Highlights, I choose Export >> Devonthink. Highlights creates individual markdown files for each of the annotations. Each individual file contains the master file’s metadata. The data is copied to my Devonthink Global Inbox. I move the folder from the Devonthink Inbox to my Desktop and from there to my Annotations Folder using Launchbar.Highlights has significantly sped up the time it takes to process a manuscript. Using this method, I’m also having greater success with the “See Also” feature of Devonthink. I hope it works for you too.

Highlights.app not ready for prime time

On recommendation of several people, I decided to try the annotation app, Highlights. Although I was intrigued with several of the features, after extensive use, I can’t recommend the app. The program still needs work before I could reliably use it in my writing workflow.I found two of Highlights features especially attractive:

  1. The ability to extract each highlight or comment as its own markdown file.
  2. The ability to underline references in the PDF and have those references automatically appended/linked to extracted notes.

I trialed Highlights for about a week. Ultimately, even with the intriguing features, The appnwas far too buggy for me to adopt.Here are some of the issues I faced:

  1. References would not reliably link to a note. I could find no rhyme or reason for this behavior. Sometimes the feature worked, sometimes it didn’t. No matter what I tried, I was unable to remedy this issue. My attempt at editing markdown files led to frustration—my edits were often erased.
  2. Even when the reference extraction worked, there is a bug that alters the markdown file, adding additional markdown to each reference. With many references, this bug makes each file unreadable.
  3. I found no way to configure the order, type, and appearance of the metadata.
  4. Extraction of figures from the manuscript were buggy and unreliable.
  5. Metadata was impossible to change. Initially, I set Up the app to automatically retrieve metadata. Unfortunately, several of the paPers the software Retrieved had erroneous metadata. Once imported, I found the metadata impossible to change. Since I add the title, author, And other metadata to every one of my extracted notes, this was The fatal flaw that caused me to end my trial.

Although I am intrigued by several of the features of the Highlights App, I will continue to use my tried and true method of note extraction using Skim. I plan keeping an eye peeled for these issues to be fixed within Highlights. With some improvements, I could see the app becoming my Mac PDF reader of choice.

Importing Microsoft Word files into Ulysses

I subscribe to the newsletter from Soulmen, the makers of my favorite text / markdown writing app, Ulysses. From the newsletter, I learned it is now possible to import Microsoft Word .docx documents. The article said it was possible from any device, but I could only do it using the instructions for iPhone (not on my Mac).In order to import a Word file, it must be in a folder Ulysses can access. Within Ulysses iOS, choose the group where you’d like your imported document to live. Then at the bottom right of screen, choose ‘import’ and select your file. The Word document is converted to MarkdownXL with your formatting intact.Happy writing!

Why I Use Plaintext

As academic physician, a large part of my work involves writing. I write emails, meeting notes, drafts of manuscripts and books, blogs, ideas, grant applications, and more. I've owned every version of Microsoft Word for the Mac. But over the last two years, my workflow for writing has changed dramatically.

For me, writing can be very difficult. I find I write best with minimal distractions. I need to be in a position to get into a flow state and type whatever comes to mind. I’ve found it best to get the stream of ideas down on the page as quickly as possible, then come back later to edit. Software features, whether for formatting, editing, or reviewing often become a distraction. Over time, I've found I do my most productive writing in a program that offers few options (and thus very few opportunities for distraction).  Knowing this about me, you won't be surprised to learn that I have all but forsaken Microsoft Word (and Apple Pages) in favor of plaintext.

You might find this odd coming from someone who writes manuscripts and grants for a living. You're probably wondering how an academic can survive without Word? And why would a tech-savvy nerd like me choose something so basic as plaintext?

Please let me explain. If left to my basic inclinations, I might flail for hours trying to construct the perfect sentence. For a long time, this "perfectionism" would leave me stuck-often on the first page of a large writing project. Only recently did I learn to overcome this impediment. For me, the key to productive writing is for me to minimize distractions.  I need to control distractions in my environment, distractions on my device, and distractions within the writing program itself. Once I start writing, I try to keep my words flowing freely, saving the editing for my second (and third, and fourth) pass.

Word no longer met the demands of my workflow. I resented Word for its proprietary format, the size of its documents, its complexity, and the worst of all: auto-formatting. I commonly found Word getting in the way of my writing rather than getting out of the way and letting me write. Pages was not much better. And with either program it was difficult to keep versions synchronized between my Mac, my iPhone, and my iPad. After I wrote something I wanted the text to be immediately available on every device I owned.

I started looking for alternatives to Word and Pages. And there was a plethora of minimalist writing programs for the Mac and iOS to choose from. Most had only a few features–just enough to enable writing and to do minimal formatting. My favorite programs seamlessly synchronized to the Cloud, allowing me to write once, but access the text on any device. Because I write text bound either for a word processor or the web, I needed a single way to write in in words and in HTML. Plaintext combined with Markdown-a plaintext formatting syntax developed by John Gruber-allows me to write once and then move my words either to a word processor or to the web. Fletcher Penny’s MultiMarkdown adds many additional capabilities to Markdown and is especially handy for academics. If my note isn't going to stay in its native plaintext form, I usually run it through Marked-a program that imports Markdown/Multimarkdown and exports formatted text in the end format I desire (whether it's for a word processor or the web).

These days, I do the majority of my writing on my iPad or Mac using use a Markdown-enabled text editor. I use Word or Pages only in the very final stages of a writing project (after the bulk of the writing and editing is complete).

But there’s more. I spend about 30 minutes commuting in my car each day. I now occasionally use this time for “writing.” The addition of voice recognition to the iPhone 4s gave me the ability to dictate during my commute. I dictate into SimpleNote Notesy or the Dragon Dictation app on my iPhone. If I use Simplenote Notesy, the text automatically syncs using Dropbox and is immediately available on my Mac or iPad. If I use Dragon, all I have to do is mail myself the dictation and import the text into my writing application. I find myself drafting emails, letters, and even beginning long manuscripts while commuting. Then, when I arrive at the office, I just need to polish my prose rather than start writing from scratch.

Once I’ve written the document, plaintext is easy to share across operating systems (Windows, Mac, Linux) and devices-negating the need to buy proprietary software or other converters. Plaintext documents are tiny, taking up a fraction of the space of a proprietary word processing file. Plaintext is flexible-serving as notes in its native form, or as the early stage writing / editing platform for larger writing projects.

My current workflow for most short documents is to write or dictate them into plaintext (I use Writing Kit on my iPad, SimpleNoteNotesy or Dragon Dictation for dictation on my iPhone, and Byword on my mac). Although I can’t dictate Markdown on my iPhone, it’s a simple task to add once I reach my destination. I use Dropbox to keep all my notes in sync. If I’m writing a document, I can used Marked to copy my formatted text directly to Pages or to Word for final formatting. For a blog entry or web page I can export my Markdown document to HTML using Marked.

There are numerous writing apps for the Mac and iOS. I suggest you try several (most are $5 or less each) to find the best fit for your personal style. Brett Terpstra recently wrote a blog entry comparing the different features of writing programs for iOS. Features important to me in the editors were: (1) built-in TextExpander, (2) import Markdown and Multimarkdown files, (3) synchronizes to the Cloud or to Dropbox.

For large academic projects, I have a more complex workflow that includes several other tools. I’ll cover this more complex writing workflow in a future entry.

Cheers, Jeff

Appended June 17, 2012: I was taking a look at @MacSparky's Twitter feed. He recommended this blog post by @yuvizalkow : I’m A Failed Writer #13: Beyond Microsoft Word… Or Not on the same topic.

Appended March 2, 2013: Read this entry on why I switched from SimpleNote to Notesy.

Enhanced by Zemanta