What do you do with big piles of technical and hobby articles collected during several decades?

My library of books is nicely organised and catalogued using LibraryThing, but the magazines and journals are a mess. There are a lot of things I don't want to throw away, but I never look at because finding the right item involves getting down dusty archive boxes (or worse: sorting through a random pile in a corner somewhere. And the items I have as PDFs aren't much better. Most are just collected in folders by publication, year and month. I need to do some sorting and indexing.

 

I've looked at a few tools designed for academic researchers, e.g. Zotero and Mendeley. They work on the assumption that most of the literature you are using is from current academic journals available online, and they focus on the process of importing the bibliographic data from the standard academic search tools. In my case a lot of the stuff I have is either too old or too hobbyish to be retrieved in this way. Another drawback is that most expect you to store your library on their server - and pay for storage if you go beyond the small free allocation the give you. As I already have many gigabytes of scanned PDFs that wouldn't be cheap. But they do look after the indexing and searching once they have the data, and allow you to organise work with keywords and things. 

After some thought, I've decided that the Papers app is the one I will go for. Unlike some of the others it isn't free, but it is based on local storage, and it has the possibility to sync to an iPad over wifi, so I can store and index my collection on the desktop and read items on the iPad. I got the free trial version and successfully indexed fifty or sixty articles in it before committing myself.

The workflow I've come to is something like this at the moment, but I may still refine it a bit:

  • if not already done, scan a year's worth of the magazine making a separate file for each issue (32 or 40 pages is a manageable size and lets you spot page-turn errors quite easily)
  • convert to searchable PDFs and give them meaningful file names
  • store on my NAS in a publication/year folder tree
  • using Preview, open each issue, identify the articles for indexing (I skip the news, letters and diary columns, etc.)
  • copy and paste a range of pages for each article into a new file and import it to Papers (overlapping pages where necessary)
  • In Papers, add the global bibliographic data for the publication, year, language, etc., then go through the individual articles to correct the title, author and page numbers as required
  • add some keywords for the content

In theory, there will be at least some article where Papers manages to retrieve the data itself from Google Scholar or somewhere, but so far I haven't got that to work for those I've tried. Modern Tramway and its successors seem to be retrievable, but the quality of the data it gets back isn't satisfactory. And my bound journals from the thirties aren't recognised at all. 

(I got very frustrated with the iPad sync not working the second time I tried it, and couldn't work out what was wrong until it dawned on me that the iPad had lost its wifi connection somehow. Once that was fixed it worked perfectly, but maybe it would be nicer if the app popped up a warning to say it couldn't  sync without a network...)