Modification to Reading, Extracting And Storing Scholarly Information To Supercharge The Writing Process

In Reading, Extracting And Storing Scholarly Information To Supercharge The Writing Process, I wrote about how I extracted both highlights and full-text of entire manuscripts in order to give me granular access to information. Although I’ve continued my extraction of highlights, the extraction of full text (by highlighting the entire document) proved much too time consuming. Instead, I’ve been experimenting with an alternative that is much quicker (as suggested by Andrew in the comments of the entry)—saving the entire manuscript as single-page PDF documents. Here is what I’ve been doing.

After highlighting a manuscript in, I extract my highlights (along with color tags) to Devonthink Pro using the built in export function. By default, saves my extracted highlights files to the DTP Inbox. I move the folder from the DTP Inbox to my Desktop. Within the moved folder I make two new sub-folders: 1. HighlightsX and 2. PDFx. I then move the extracted markdown files to the HighlightsX sub-folder.Within Bookends, I export the annotated pdf to my desktop. There, I open the file with Adobe Acrobat (any app able to add headers and split documents will work).

In the left most header I put the Bookends citation (Bookends: Edit: Copy Citation), in the center I put the DOI number, in the right header, I put the Bookends link (Bookends: Edit: Copy Hypertext Link: Copy as Text). These headings are added to each page of the PDF. I then split the manuscript into multiple single-page documents. I save the split PDF documents to the PDFx sub-folder.

I then move the parent folder my Dropbox Writing folder and Index the folder using DTP. Within DTP I make sure both the main folder and the subfolders will have their tags included (option click on a folder in DTP and make sure “Exclude from Tagging” is unchecked)

.Although this method is faster—there are trade offs. The “Find Also” feature of DTP depends on the words in a document. A document with too many words dilutes the accuracy of the semantic search. A page of text has far more words than an extracted paragraph and thus is slightly less accurate in finding granular information. The other trade-off comes in the amount of text that must be read when searching. It is faster to scan a paragraph versus a whole page of text in a PDF. Regardless, the savings in time using this method far exceeds the trade-offs in accuracy.Let me know what you think.

Bookends Adds Floating Citation Window

I’d like to bring the latest update of BookEnds to your attention.This update fixed an error that would crash the Bookends when trying to obtain a reference (or link back to a reference in Bookends using a DOI link) from The latest update fixes this error. Clicking on the reference link in Highlights now selects the reference in Bookends without having to modify the Highlights markdown file. This obviates the need to modify the Highlights markdown file as mentioned  in this entry.The latest update also added a floating citation window to Bookends. Like Papers, the floating citation window is invoked using customizable key combinations. So far, I’ve cited using the floating window in Scrivener, Ulysses, and It works flawlessly.Well done Jon and Sonny Software!

Mourning the loss of Integration Between Papers and Scrivener

Early indications, including personal communication with ReadCube personnel, are that Magic Citations (now called SmartCite) will no longer integrate with Scrivener. The new Papers app will only work with Microsoft Word. Bibliography formatting is not an issue for me. The biggest loss is the ability to add citations on the fly in Scrivener without interupting my writing flow.The news of this impending feature loss (along with the announcement of an annual fee) had me scrambling to invesitgate my options for citation managers. During my search, I came across this Wikipedia article, Comparison of Reference Management Software with a great table that collates the majority of software out there.My needs are the following:

  1. Ability to organize and search through metadata and pdfs
  2. Integrated citation insertion with Scrivener and Ulysses
  3. Ability to insert citations while writing on an iOS device
  4. Ability to annotate PDFs and export each comment individually appended with the article's metadata
  5. Ability to export metadata (to enable my workflows for sense-making and export of annotations)

I looked at the following:

  • Zotero
  • Bookends
  • Mendeley
  • Endnote
  • Readcube
  • No product currently fills the void left by Papers, although the consensus of users (both those seeking alternatives to Papers and those who are being forced to leave Sente) seems to be to move to Bookends. I tried the demo version of Bookends and was not impressed. I am waiting impatiently for the release of ReadCube Papers. If the majority of features are retained, I will likely bite the bullet and pay the annual fee. I plan to figure out a work-around to add citations to Scrivener / Ulysses.I'd be interested in hearing your plans / thoughts on academic citation managers.

    ReadCube Release of Papers App

    If you’ve read my blog, you know I’m invested in Papers. The majority of my writing workflows use the app.I’ve been following news about the app with trepidation. Papers “teamed up” with RedCube in March of 2016. Readcube / Papers have been working on a new version of the app. Although the screenshots look reminiscent of Papers, there will be at least one major change; Papers is moving to a subscription model. I have not found pricing information yet.The combination of a new version, and unknown pricing model, and a distrust of traditional publishers has left me wanting to explore my options.I’m curious what app each of you is currently using for manuscript management and bibliography generation (and why). Please leave comments below.Addendum: Beware. Updating to Scrivener 3 breaks Magic Citations in Papers. From what I've read online, Readcube is not saying when (or if) this issue will be resolved. Redux

    I was too quick to judge the program Highlights. I gave Highlights a second chance, and although not perfect, the app is slowly winning me over. Highlights has greatly sped up the extraction of information from my reading.First a little background. I use Papers for my PDF and bibliographic management. I’ve written about how I extract highlights and annotations into individual files along with their relevant references. The reason to go to this trouble is to enable Devonthink’s artificial intelligence. The Devonthink AI uses word count to find related information in other files. I use this method extensively in my writing of grants and manuscripts. With my previous method, I found by adding references to each annotation (thus similar words) I was interfering with Devonthink’s ability to find relevant information. In order to improve Devonthink’s accuracy, I stopped adding references to each individual annotation. Instead, I merely include a Papers Link back to the original file.Highlights shows the annotations you make in the app's right-hand column. A pop-up interface within Highlights allows you to make comments, underline , change colors, etc. The killer feature of Highlights is single-command extraction of each annotation / comment to its own file. This feature allows me to extract highlights (and metadata) without having to invoke my Keyboard Maestro macros. When the data is extracted into Devonthink, the individual files are in Markdown. This makes the extracted information easy to view and edit.Although I’m more enthusiastic about Highlights, there are several quirks you should understand:

    1. When Highlights explodes your annotations into individual files, it prepends the original file’s name. If you use Highlights from within Papers (e.g. selecting Highlights as your PDF reader of choice from within Papers) you will end up with a ridiculous, machine based title in your metadata. This is not Highlights fault. The use of non-intuitive file names is one of my pet peeves about Papers. The workaround is to export a copy of the PDF to your Desktop and then launch the exported file using Highlights. Using this method, the author’s name and the title of the manuscript along with the year of publication are all prepended to each markdown file.

    2. After reading and highlighting, I edit the markdown headers in Highlights (choosing the edit tab in the annotations window). I erase all but the primary author’s name and then add the Paper’s Citation and a Papers Link back to original file (copied from Paper’s Edit Menu). There is another quirk here. In Highlights Edit Mode, if you don’t leave a space between the markdown coding and the information you add, your file will be reset back to its original state, erasing your modifications.

    3. Highlights has the ability to automatically look up DOI numbers. When it works, it’s great. It takes a single click to import a reference from the bibliography into your Paper’s Database. Unfortunately, this feature is flaky, especially with longer manuscripts. I often have to revert to my Launchbar scripts to capture the bibliographic information I need.

    Once I’ve added the metadata I want to the master annotation file (and collected the references of interest), from within Highlights, I choose Export >> Devonthink. Highlights creates individual markdown files for each of the annotations. Each individual file contains the master file’s metadata. The data is copied to my Devonthink Global Inbox. I move the folder from the Devonthink Inbox to my Desktop and from there to my Annotations Folder using Launchbar.Highlights has significantly sped up the time it takes to process a manuscript. Using this method, I’m also having greater success with the “See Also” feature of Devonthink. I hope it works for you too.

    Grantome - Website offers new insights into Successful NIH Funding

    I recently came across a new (at least to me) website called Grantome. The site was developed by Cleveland, Ohio data scientists. Granthome’s mission is to use data to drive the discovery of new knowledge about scientific research grants. Grantome extracts data from various places and combines it into a single data source that offers insight (and, they claim, a competitive advantage) in procuring grant funding.Currently, only the U.S. National Institute of Health (NIH) is included. The NIH information is updated weekly. There are plans to expand Grantome to include the U.S. National Science Foundation (NSF), the U.S. Department of Energy, and federal grant organizations in Canada and Europe.I ran Grantome through its paces, conducting a search on the topic of the proposal I’m currently writing. The interface is simple, clean and intuitive. Granthome quickly spit out interesting information on my topic including: 1) the number of grants issued per year on my topic, 2) the authors of the successful grants on my topic, 3) the institution of the winning proposals, 4) the institute where the funding originated, as well as 5) the study sections that approved each of the successful grants.I will keep a close eye on Grantome and plan to use it with future funding proposals. I hope I’m able to meet up with this team during one of my frequent visits to Cleveland.Check Grantome out and let me know what you think! not ready for prime time

    On recommendation of several people, I decided to try the annotation app, Highlights. Although I was intrigued with several of the features, after extensive use, I can’t recommend the app. The program still needs work before I could reliably use it in my writing workflow.I found two of Highlights features especially attractive:

    1. The ability to extract each highlight or comment as its own markdown file.
    2. The ability to underline references in the PDF and have those references automatically appended/linked to extracted notes.

    I trialed Highlights for about a week. Ultimately, even with the intriguing features, The appnwas far too buggy for me to adopt.Here are some of the issues I faced:

    1. References would not reliably link to a note. I could find no rhyme or reason for this behavior. Sometimes the feature worked, sometimes it didn’t. No matter what I tried, I was unable to remedy this issue. My attempt at editing markdown files led to frustration—my edits were often erased.
    2. Even when the reference extraction worked, there is a bug that alters the markdown file, adding additional markdown to each reference. With many references, this bug makes each file unreadable.
    3. I found no way to configure the order, type, and appearance of the metadata.
    4. Extraction of figures from the manuscript were buggy and unreliable.
    5. Metadata was impossible to change. Initially, I set Up the app to automatically retrieve metadata. Unfortunately, several of the paPers the software Retrieved had erroneous metadata. Once imported, I found the metadata impossible to change. Since I add the title, author, And other metadata to every one of my extracted notes, this was The fatal flaw that caused me to end my trial.

    Although I am intrigued by several of the features of the Highlights App, I will continue to use my tried and true method of note extraction using Skim. I plan keeping an eye peeled for these issues to be fixed within Highlights. With some improvements, I could see the app becoming my Mac PDF reader of choice.

    Skim Split Screen Workaround

    When reading PDFs on my Mac, I use Skim. Although I love the program, Skim has one annoying quirk. I cannot select (or highlight) text in a split screen. This makes it difficult to highlight or download references in real-time.I found a workaround that is fast and efficient. After opening a PDF in Skim, I’ll select “Print” from the File Menu and then, from the bottom left drop down menu, I’ll select “Open PDF in Preview.” This opens a second editable version of the manuscript side-by-side with the original.Let me know how this works for you.

    Scrivener iOS coming on July 20!

    I have been quite impressed with the Scriver iOS Beta--it's capabilities far exceeded my expectations both on my iPhone and iPad. It has already simplified my Writing Workflow. I'm not alone in my exuberance. Check out this glowing review. The wait for The official release is almost over....the official date Is July 20 at a price of $19.99. There will be a simultaneous release of a new version of Scrivener for Mac.

    New NIH Grant Requirements - Critical Evaluation of Existing Literature

    For those of you that write NIH grants for a living, you are well aware of the changes required for new grant applications in 2016. In an effort to improve rigor and transparency as well as to increase reproducibility, The NIH now requires the grants to be written in a whole new way. A major part of these changes has to do with critical evaluation of existing literature.In light of these changes I have been formally appending my manuscripts a new way. In previous entries I’ve written how I take notes while reading, then extract these highlights and notes to their own individual files. Now, as I'm reading, I make and effort to record perceived strengths weaknesses of each study appending my comments with the text “StrengthX” or “WeaknessX.” I then extract each comment as its own text file. When writing grants or manuscripts, using Devonthink, I’m able to find similar notes to the one I’m reading. By appending StrengthX or WeaknessX, I’m able to single out my own comments instead of a seeing every instance of each word.In a future entry, I’ll talk more about the ways I’ve updated finding extracted information using Devonthink (and Tinderbox). A good portion of my writing workflow from 2015 has changed. I will write an updated writing workflow after the official release of Scrivener iOS.I'd be interested to hear how other academics are dealing with the changes at the NIH. Please leave a note in the comments below.

    Using Skitch and Papers to Capture Figures and Tables

    Few would argue that the most important point of a scholarly manuscript is made in its figures and tables. I am going to share with you how I capture figures while reading scholarly information on my Mac. This workflow uses:SkitchPapersInstall Skitch and make sure, in Skitch Preferences, to enable the “keep Skitch Helper running in background when I quit,” and “Start Skitch Helper when I log in to my computer.”Screen Shot 2016-05-28 at 9.12.13 AMAs I’m reading scholarly literature and come across a table or figure I want to save I do the following. I make the figure as big as possible on my screen. Then, from the Skitch menu in my menubar, I select the Crosshair Snapshot. I then select the figure (and sometimes the caption) trying to balance the white space surrounding the figure.Next, I go to Papers, select the reference in my Papers Library, and then from the Edit Menu:Copy As:Reference.I return to Skitch and double-click at the bottom of the figure then paste the reference text. I then balance the text. The height of the Skitch figure will expand to accomodate the new text.Screen Shot 2016-05-28 at 9.14.40 AMWhen I want to refer to or use the table or figure I view it directly in Skitch or find it in Evernote. Using this method, I can also search for words in the reference (e.g. the author’s name or the title of the manuscript) and sometimes even the words in the figure itself.If I want to use the figure in a presentation or to send it to a colleague or trainee I can export the figure from the Skitch File Menu.Using this method I’ve captured hundreds of figures. I hope this workflow helps you too.

    Scrivener iOS: From Alpha to Beta

    It looks like our long wait may be over. A post on the Literature and Latte blog this week said the iOS version of Scrivener will move from in-house Alpha testing to wider spread Beta testing. From the article, looks like they will choose a small number of primary Beta testers to kick the tires....once the egregious bugs have been found they will move to more wide-spread testing. Regardless, I can hardly wait. Scrivener iOS is going to simplify my writing workflow quite significantly.

    Importing Microsoft Word files into Ulysses

    I subscribe to the newsletter from Soulmen, the makers of my favorite text / markdown writing app, Ulysses. From the newsletter, I learned it is now possible to import Microsoft Word .docx documents. The article said it was possible from any device, but I could only do it using the instructions for iPhone (not on my Mac).In order to import a Word file, it must be in a folder Ulysses can access. Within Ulysses iOS, choose the group where you’d like your imported document to live. Then at the bottom right of screen, choose ‘import’ and select your file. The Word document is converted to MarkdownXL with your formatting intact.Happy writing!

    The World's Most Dangerous Writing App

    I’ve read plenty on writing. Most resources suggest, when writing a first draft, you simply write whatever comes to mind—writing the whole first draft in one long stream of conciousness. Yesterday, I read a Wired article about The Most Dangerous Writing App, a web tool to help writers get over writer’s block.The app is simple, if you stop writing for more than 5 seconds, The Most Dangerous Writing App erases ALL your work. Not a few sentences, not a paragraph, but everything you’ve written. Now that’s incentive to keep writing (especially if you’ve chosen a longer writing period of 30 minutes to an hour)!I’m not sure if The Most Dangerous Writing App will help me or frustrate me, but am willing to give it a try. If you decide to try it too, please leave your comments below.