Bill Gardner / Bibliographic Tools
[RefWorks: new IEEE - URL output style prints "Anonymous" for web pages w/o author]
[In general, we still need to improve the output styles, both RefWorks & .bst (re URL treatment)]
Every researcher needs a way to collect a personal set of bibliographic references, share them with others, cite them in articles and theses, and have them print out in some standard formats. Relying on hand editing of bibliographies is a recipe for producing messy, unattractive, and inaccurate publications, that nonetheless consume a great deal of time in achieving these poor results.
What you want is the freedom to cite at will any reference you have accumulated, and have both the citation and the corresponding reference formatted properly according to the publisher's demands. If you edit the document for a different publisher, these formats should change automatically without hand editing. If you remove or insert citations, you should not have to massage the reference list by hand to bring it up to date, nor change the other citations to match the updated reference list.
The problem is to find a format for storing references that is easy to share, and is possible to transfer among the citation tools needed for use with the different word processing programs that we like to employ. It is not feasible to impose a single word processor, since we all have our habits and are unwilling to climb the learning curve of an unfamiliar/distasteful product. Still, some publishers impose specific requirements that take precedence over our habits.
Requirements for a solution
This suggested solution is believed to satisfy the requirements. As a compromise, it is more awkward than a narrowly-targeted solution, but pays off in sharing, flexibility, and portability. We start with a dataflow diagram of the solution:
The components are explained in the next two subsections, followed by a special section on how to reference URLs. Finally, good practices are suggested for formatting citations and references.
The bibliographic database: JabRef, .bib files, RefWorks
The bibliographic database (the references) will be maintained as BibTeX .bib files using a GUI-based tool, JabRef , that helps identify which fields have to be filled in for various types of references. BibTeX files use a classic human-readable format that shows no sign of becoming obsolete, and they are easy to transfer among different tools. The preferred encoding is UTF-8, in order to store accented characters without requiring LaTeX escape conventions. This is to facilitate transferring to/from tools that recognize UTF-8 but not LaTeX escapes.
There are other GUIs for manipulating BibTeX files, and even a simple text editor will do. But JabRef is a cross-platform Java program, and has a "custom export" feature needed to massage BibTeX files into the format required downstream.
Filling in BibTex entries
Get help here. Note on entry types:
Note that if your references do not use special characters, UTF-8 is equivalent to ASCII and you can name the file .bib , implying that it is ready to use as ASCII input. But if it does have special characters, name it .bub ("u" for UTF), implying that it must be converted before the LaTeX-based downstream tools can process it correctly. The utf2any script is used to convert UTF-8 files to ASCII with LaTeX special-character escapes.
You also need to put {braces} around any acronyms and proper nouns that you don't want converted to lower case by BibTeX, if this wasn't done by your citation source (ACM does it properly; IEEE does not). Those escapes are transparent to utf2any.
utf2any utility
For Win32, download utf2any.exe and maps/latex.map. If you need to compile it for Unix, there's a .c file and makefile (uses flex and gcc).
To execute: utf2any -8 -f latex.map mybib.bub > mybib.bib
where mybib.bub stands for the BibTeX file with UTF-8 encoding, and mybib.bib is the converted file with LaTeX escapes. CAUTION: utf2any does not like long filenames for its input files!
ASCII .bib files are directly usable with LaTeX/BibTeX and with FrameMaker (via the CiteMaker plugin).
To use BibTeX format files with MS Word , another reference maintenance tool is needed. Our university has a free site license to RefWorks and runs free training seminars, so that is preferred, but other tools, such as EndNote, can be adopted. To be sure, RefWorks overlaps the functionality of JabRef, but experimentation shows that using RefWorks for the primary database makes it impossible to get accurate exports of BibTeX files, whereas the reverse route is dependable. JabRef can export a RefWorks Tagged file (.rwt) which can be imported by RefWorks with the correct results. Accented characters encoded in UTF-8 will stay correct in the RefWorks entry, and any extra { } for BibTeX capitalization will be stripped off automatically. This custom export operation requires additional "layout files" and Java classes, which we have created.
The following table explains how to obtain and configure JabRef to support a primary UTF-8 database in .bib files, and export a derivative database in .rwt files for RefWorks:
|
Download JabRef (2.2 or higher) and install. Windows: After installing, go to the Start Menu and find JabRef. Right-click on it to get Properties. Then, if necessary, change "Start in" to match the directory where the JabRef files are located. Linux: Ensure that the directory where the JabRef files are located is in the classpath that is in effect when JabRef is launched. |
The program needs to be started in its own directory in order to find the extra Java classes that will be compiled in the next step. NOTE: The key symptom of setup problems, especially with the classpath, is the error message "Formatter Not Found" when attempting to export RefWorks tagged format. |
|
Download additional files here, and unzip them in same directory as JabRef's .jar file. In that directory, invoke the Java compiler with this command: javac -classpath *.jar -d . *.java A subdir called "net" should appear, and the classes should be in net/sf/jabref/export/layout/format. |
The JabRef *.layout files implement the custom export to RefWorks tagged format (.rwt) for these entry types: Phdthesis, Techreport, and Webpage The layout files invoke the Java classes to carry out some text filtering. |
|
(Try to replace our GetEnd/StartPage formatter classes with new builtin First/LastPage formatters.) This step is not needed for 2.5 and above. Still in the same directory, invoke the Java archive manager with this command: |
The jarfile supplied with JabRef has no classpath defined. This means that it cannot find the Java classes you just compiled in the previous step, and you will get the "Formatter Not Found" error. This command updates the jarfile's "manifest" to include the classpath "." meaning the startup directory that contains subdir "net". The manifest update is contained in cp.mf. |
|
Options > Manage custom exports > Add new: |
This will add a menu item under File > Custom export to output entries in RefWorks "tagged" format (.rwt file). The layout files explicitly map BibTeX fields into certain RefWorks fields, so that they will be displayed properly. In contrast, the default BibTeX import of RefWorks is unpredictable and unhelpful. |
|
Options > Customize entry types: |
BibTeX does not define a "webpage" entry type, so we create one. The supplied layout files know how to export this type to RefWorks. |
|
Maintains .bib (.bub) files in the format needed by RefWorks without use of LaTeX special-characters escapes. |
|
|
The following settings are suggested, not mandatory:
Options > Preferences > BibTeX key generator: |
This is to allow for automatic key generation by JabRef for references imported from any tool that supplies its own (useless) citation keys. The generated keys will come out in an easy-to-anticipate form. If you use the Proceedings type, you should create a key by hand, e.g., RSP05, CCECE04. This place-holding auto-generated key will remind you to do that. |
Making citations and building a bibliography: MS Word, LaTeX, FrameMaker
MS Word (Windows or Mac)
For MS Word , follow the instructions in the table below to download the RefWorks Word plugin , and configure your RefWorks account.
|
Read about RefWorks, create an account, and login. |
The plugin provides a convenient interface to the backend server that holds your RefWorks account. |
|
Download this file, RWOutputStyles.rwb. In RefWorks, select Tools > Backup/Restore Click Restore and browse to where you saved RWOutputStyles.rwb. Be sure to check Include Output Styles. |
This loads the modified output styles that are capable of printing URLs as stored by JabRef:
Harvard - British Standard - URL2 These two will show up in red in the list of output styles when you select Bibliography. |
Then, to use a JabRef-maintained BibTeX database in RefWorks, select References > Import with "RefWorks Tagged Format" to import the UTF-8 .rwt file exported from JabRef. Each reference will automatically be given an internal RefWorks ID number, unique within your RefWorks account. (If you are using EndNote or other such product, you may try JabRef's File > Export, or simply import the .bib BibTeX file directly.) You can now use the plugin tool to insert RefWorks citations in your document. If you are off-line, you can type the citations by hand, but you must know the ID numbers! (You can print them out in advance for off-line reference.)
Use RefWorks to generate a bibliography for the citations and insert it into a new document, whose name is formed by prefixing Final- to your original document. Since the output styles built into RefWorks do not handle URLs well, we have prepared some output styles that do. These were imported to your account via the backup/restore feature (directions above).
Since RefWorks maintains its own database of references on a central server, it is necessary to keep it in sync with your primary BibTeX file. It is a good idea to use JabRef's "mark" feature (Ctrl-M or "Mark" button) to mark any item you are exporting to RefWorks. This will make it easier for you to keep track of what has been imported and what has not.
Your choices re synchronizing are (1) to edit both databases in parallel, or (2) to edit the BibTeX file, re-export a .rwt file, and re-import it to RefWorks. Minor imports can be done quickly by copying a JabRef entry (use Ctrl-C to put a BibTeX record on the clipboard) and pasting it into the RefWorks window "Import Data from the following Text" (after setting the Import Filter to BibTeX).
A problem with strategy (2) is that the re-imported references will unfortunately have new ID numbers, creating duplicates and near-duplicates with the old entries. Duplicates either have to be pruned, or one can avoid creating them by purging RefWorks before reimporting. Then in the Word document one must find and change the old citations to the new ones. This is such an unpleasant chore that it is best to either leave inserting citations until the end of document editing (meanwhile perhaps using unchanging BibTeX indexes as place holders for the future RefWorks ID numbers) or else do the parallel editing of strategy (1) once you have built up a substantial amount of citations.
LaTeX
For LaTeX , citations can simply be inserted using "\cite" and the respective JabRef/BibTeX index. If the reference database is a UTF-8 .bub file, first convert its special characters into LaTeX escape conventions by running utf2any. Finally, running BibTeX will pull the citations out of the database file and format them according to the specified bib style (.bst file). A selection of URL-enhanced .bst files are here.
NOTE: If you find that some escape sequences are showing up in plain text in your output, or are not producing the expected accented characters, read the comments in latex.map . You may need to insert a \usepackage in your input file.
FrameMaker
FrameMaker users should obtain the plugin CiteMaker, which will install a new menu. This tool inserts variables into your document containing BibTeX indexes. As with LaTeX, a .bub file must first be converted to LaTeX conventions via utf2any. (Unfortunately, there is no way to insert a \usepackage if the accented characters are not coming out right.)
To begin using CiteMaker with a new document, you need to select 3 items from the menu:
When CiteMaker generates a new bibliography, it replaces the variables with properly formatted citations and puts the bibliography into a file whose name is formed by suffixing your original document name by -bib.fm . The ideal way to import this file is to insert it one time via File > Import > File , by reference, into your original document where you want the bibliography to appear. Later, whenever the bibliography is regenerated, the reference will automatically be updated.
TIP for importing bib file by reference: Citemaker will output the word "References" in BibHeader paragraph format, followed by all the references in BibEntry format, followed by two more blank lines in BibHeader format. You cannot change this behaviour. The very last blank line will assume the paragraph format of the paragraph where the importation marker is anchored. If this is the title of a chapter that uses "Start new page," then the references will end with a page break that seems impossible to remove. Furthermore, one will struggle in vain to eliminate the word "References." You can work around these facts of life as follows:
The following table explains how to obtain and configure CiteMaker:
|
CiteMaker comes bundled with its own copy of BibTeX and a selection of .bst style files. CiteMaker 1.2 is written for FM 5.5, but it does work fine with 7.0. Caution: CiteMaker does not like spaces in pathnames, so install it under C:\ (not Program Files) and place your .bib and .bst files in your file system without intervening space-filled directory names. There are some problems with CiteMaker's readme.txt, so follow the instructions below. |
|
|
Download CiteMaker, and run the installer. |
If the site is not available, there is a backup here. |
|
Locate your FM directory and open maker.ini for editing. Find the section called [APIClients]. At the end of this section, add the single line (adapted according to where you installed CiteMaker): CiteMaker=Standard, CiteMaker Utility, c:\CiteMaker\Frame55\bin\citemaker.dll, all |
Adds "CiteMaker" menu to FM's menu bar, connected to citemaker.dll. |
|
See the next section for additional "URL-enabled" .bst style files that you can download and store in CiteMaker\Frame55\bstfiles. |
|
Other word processors
For other word processors, such as OpenOffice , you can copy/paste citations created by RefWorks (since the Word plugin won't apply), save your document as .doc or .rtf, and generate the bibliography by passing it through RefWorks. JabRef does have an export format called "OpenOffice Calc," but its use has not been explored.
The need for including URLs in references is constantly expanding, but tool support is still lacking or introduces incompatibilities. For the recommended tool flow, this requires work to set up, but it only has to be done once and the results are worth it.
There are two issues: (1) how to insert URLs into .bib entries so that they are accessible to both BibTeX and RefWorks (when imported); and (2) how to make BibTeX and RefWorks pull them out of the entries and print them nicely. The solution to the first issue is easy: Just use the "url" field on JabRef's General tab. You can even drag/drop a URL from your browser into the url field. When imported by RefWorks, this field will end up under Links. It is recommended not to add any LaTeX escapes (such as \url{...}), because they will not be stripped off by RefWorks and will end up printing out verbatim. If BibTeX ends up breaking long URLs awkwardly or inserting "%" characters, you can arbitrarily insert blanks to help it break the URL in reasonable places.
NOTE that you can actually replace CiteMaker's minimal BibTeX with a full-featured one having LaTeX support! On Windows, for example, if you obtain MiKTeX, go to miktex\bin and copy bibtex.exe to \Citemaker\Frame55\bin, replacing the bibtex.exe you find there. Now CiteMaker will invoke MiKTeX's program instead, and you may be able to take advantage of more advanced style files.
The above instructions apply to adding URLs to conventional reference types (article, book, etc.). If the reference is purely a web page, as has become common, you can use the Webpage type if you configured it in JabRef. Otherwise, select the Misc type, and invent a key (unless it will be derived automatically from the author or page title). In Optional Fields, fill in the Title of the page, and Author, if any. Put a "freshness date" (last checked) in Year. In General, paste the URL in Url. Misc entries are imported into RefWorks as Generic, so you'll need to edit the output style's Generic type to make it print the Links field.
If you want to record a "freshness date," put it in the new "lastchecked" optional field, which needs to be added to any type you want to use it with.
If you want to record a URL with a reference for your personal information, but not have it print out, go to the General tab and enter it in some field other than Url, say Comment.
Citation and reference formats
It is important to appreciate that the needs of article-readers and thesis-readers differ. Articles are usually short, and the desire is to be able to look up a citation rapidly, therefore a numeric format is ideal. Theses, on the other hand, are long, with many citations. In reading through the thesis, it is often enough to know the cited author's name, without looking up the citation in detail, so a name/year format is good. Furthermore, one often wants to consult a thesis' reference list as a comprehensive bibliography on the research area, so it should be organized alphabetically by author. For a thesis, numeric citations and references listed in order of occurrence greatly limit the usability, at least in this author's opinion.