Workflow: Computing, Notes and References Organization

A long while back, I posted an article on my more-analog-than-digital workflow. Compared to then, I now spend a lot more time on producing my own research rather than on hyper-absorption of information necessary for quals and orals preparation and research area definition in the first and second years of my PhD program. My research now incorporates both statistical and physical-simulation based hydrological modeling. While I had done a lot of work in statistical and econometric modeling in my first and second years, the physical simulation modeling requires a lot more computing power. Specifically, I am applying a 3D, variably saturated overland-subsurface coupled, high-resolution model that takes advantage of parallel computing resources. My workflow has therefore evolved to incorporate multiple operating systems and organization of a lot more information and data.

The vast majority of high level scientific computing uses Linux operating systems, so my research and computing workflows now have to incorporate sharing file structure and hardware between Windows (what I used before) and Linux operating systems. My laptop (Lenovo X230, with a 500GB SSD switched in about a year ago) is now a dual boot system, with Windows 7 running on a 320GB partition (with 100 MB System Reserve) and Ubuntu on 145GB (including 5GB swap space). Now that I have to access two different operating systems however, I have a new problem: accessibility of data from within either of the two systems. Data stored on my Windows partition can be accessed as a mounted media drive from Linux, but Linux data cannot be accessed when I boot into Windows.

Mostly the kinds of files that I would like convenient access to from both operating systems are notes/task files and my references.

Notes and Tasks

The volume of information I must now manage has mostly outgrown my previous note-taking system. I now use plain text files to manage all the bits of information both for research and life. With my previous Midori Traveler’s Notebook-based system and to-do lists, I would eventually loose track of information I needed on hand because I would have to retire notebooks after they were filled, and because to-do tasks required different time frames for completion.

Example of my Wunderlist interface. On the left are major areas of organization. Each area contains a list. The list can be a set of to-dos, and each to-do can contain subtasks, notes, with due dates and reminders. Checking a to-do off is accompanied by a satisfying chime sound.
Example of my Wunderlist interface. On the left are major areas of organization. Each area contains a list. The list can be a set of to-dos, and each to-do can contain subtasks, notes, with due dates and reminders. Checking a to-do off is accompanied by a satisfying chime sound.

For awhile after the TN-based system, I also worked with an app called Wunderlist, which is a productivity app that could be synced between Windows and iOS (my iPhone), helped me organize tasks into major projects, as well as keep track of long-term and short-term milestones, and stray bits of information (like when you by chance stumble across an article that you want to read, but know that if you did, you’d start to get distracted from the task at hand). There were a couple of things that were inconvenient to me about Wunderlist. First, there is no Linux download available. Admittedly, this is not a huge problem, since when working in Linux, I could just open up my web browser and sign in to my Wunderlist account from there. The larger inconvenience was the inflexibility of expanding a particular task and organizing associated information along with that task. There is a feature to add subtasks underneath a task, but taking notes for each subtask is difficult in the limited space. Searching half-completed tasks and their associated notes is also difficult. I do like the calendar-due date and checklist features of Wunderlist however, so I am still open to using this app in the future. But for now, the majority of my workflow is now migrating to plain text files.

Plain text files are convenient because they can be as simple or complex as you like. To get plain text files synced and accessible across multiple platforms (Windows, Linux, iOS) using DropBox and takes up a trivial amount of space. I gain much inspiration from the website Plaintext Productivity, which clearly explains the rationale for using plain text files to organize todo lists, notes and coherent writing (what we’re ultimately trying to get towards as academics, right?). More on how this system works for me later.

References

In a previous post, I talked about mobile reference management using Zotero from within FirefoxPortable installed on a USB drive. What is good about this system is that if I am away from my personal laptop, I still have immediate access to all my citations and associated pdfs from any Windows computer, regardless of whether or not Firefox is installed, and without having to log on to Zotero, or re-download pdfs. There are a couple of inconveniences to using Zotero from FirefoxPortable. First, my FirefoxPortable cannot be booted up in a Linux environment from my USB key. (There are FirefoxPortable versions that are compatible with Linux, but that would require having duplicated libraries on one USB key. Considering there are few situations where I’d be away from my computer using someone else’s Linux system, duplicating my Zotero library on the USB key is a waste of space). Second, when I am writing on my laptop and have to be mobile, it was a pain to close out of all pdfs, close FirefoxPortable, and eject the USB key (laptop can’t be placed into my bag with the USB inserted). My solution now is to use the Zotero extension from Firefox installed on each of the operating systems on my computer. This does take up non-trivial duplicate space on my machine (right now 6 GB of space on each operating system), but for now, I haven’t found a better solution. I will update my Zotero USB key every once in awhile to make sure it has all the pdfs downloaded as well. Seems redundant, but for now, worth it.

Since my last post on reference management, I have found the more references (right now 1,405) I have in my library, the more difficult it has become to remember which citations make what arguments. Collections, sub-collections and notes taken in Zotero help with that, but in addition, I need to write literature reviews to remember how citations relate to each other to define a gap for more research. I now use handwritten notes much less frequently than before, and have started taking notes organized in Excel spreadsheets. Each Excel workbook contains a broad area of the literature review, for example: Urbanization’s Impacts on Hydrological Regimes. The workbook contains multiple worksheets: A Search Record, All Sources, and several subsets of All Sources that contain records that can be grouped together. All Sources has broad column headings that can be used to group the citations different ways, for example, for Hydrological Regimes: Type of Study (Empirical, Simulated, etc), Author, Date, Title, Publication, Key Words, Scale, Intensity, Response Focus, Causal Focus, and Notes. Subsets of references may add more in-depth column headings such as Method of Quantification of Causal Focus, Estimation Technique, Other Controls, Goodness of Fit, Length of Study, n, and Conclusions. Using an Excel workbook is preferable to taking notes in Zotero or in a notebook because I can easily sort and group citations by the categories in the column headings and search (CTRL-F) and see notes for multiple references simultaneously.

Example of a literature review-based organization of citations and notes in Excel. Multiple tabs allow me to keep track of searches performed, general findings of all relevant citations, and more specific information on subsets of citations
Example of a literature review-based organization of citations and notes in Excel. Multiple tabs allow me to keep track of searches performed, general findings of all relevant citations, and more specific information on subsets of citations

The more I use Ubuntu (my chosen distribution of Linux) however, the more time I believe I could spend exclusively in a Linux-based workflow. Right now, the major thing holding me back is my GIS work. I am comfortable using ArcGIS software, which I need Windows to run on. My labmates have switched over to GRASS GIS in Linux, which is free and open source, so probably better in the long-run, but I just haven’t devoted the time to making the switch yet. 

Leave a Reply

Your email address will not be published. Required fields are marked *