Indexes: Part 7 — Lessons Learned in Using DEXembed for the First Time

Editor’s note: This version of the post incorporates corrections made by the author to the Options and Advice sections.

Ælfwine Mischler

I recently created an embedded index in Word for a book that will be published as an ebook and in print. I chose to use DEXembed because colleagues advised that its syntax — a space between the curly brackets and the enclosed text — will work better when the text is converted to an ebook.

A quick explanation of an embedded index: For a print book, the index is written after the book has been designed, using a PDF file of the final pages and page numbers as locators. This is changing, and many publishers are now asking for embedded indexes. For an embedded index, the indexer uses something else as locators. Depending on the program used, this could be paragraph numbers, word numbers, or temporary bookmarks. After indexing, the program embeds the entries by inserting field codes that look like this: { XE “main entry:subentry” }. The index is then generated from the field codes so the pages numbers are displayed. In an ebook, they may also be linked to the location in the text. If the book is designed as hardcover and paperback with different pagination, the embedded index entries will give the correct page numbers for each edition.

Embedded indexes are more work for the indexers, so most of us will charge more for an embedded index.

Options in DEXembed

DEXembed (available from the Editorium) is a Word add-on that allows the indexer to use dedicated indexing software rather than Word’s clunky built-in indexing function. DEXembed can use paragraphs, words, or numbers as locators — but only one type in a given document. Paragraph number was the best choice for this project, but the author had sometimes used auto-spacing and other times had used Enter twice between paragraphs. I told him repeatedly that he had to remove the extra Enters and make the spacing between paragraphs consistent (which he did) and that he could not change the paragraphing after I had started indexing. (More on that in the second part of this article in February 2019.)

Experienced colleagues in the Digital Publications Indexing Special Interest Group (DPI SIG) say that Word does not handle ranges of locators well. It is therefore better to mark only the beginnings of entries that are less than two pages long. DEXembed offers three options for ranges: Mark them with bookmarks, mark them with beginning and end codes, or do not mark them. The documentation for DEXembed says that publishers usually prefer begin and end codes.

Before starting my index, I sent two small sample indexes to my author’s publisher — one using bookmarks and one using begin and end codes — and asked which worked better for them. They got better results with the bookmarks, which also meant one less step for me in the end. Hurray!

I Won’t Talk to You

DPI SIG members also advised me that Word and InDesign use different syntax for some things, and I had to take this into consideration while indexing. I also found that my Sky indexing software and Word do not always communicate well.

This index required a separate scripture index of Qur’an verses. In Word, you can use an f-switch that is coded with \f followed by a name to make two indexes at once { “heading1” \f “subject” } and { “heading1” \f “quran” } (See Seth A. Maislin’s blog for more.) However, my colleagues advised that InDesign will reject XE fields with a backslash.

A suggested solution that I followed was to use two levels of subentries, with the main entries for the two indexes. That is, I had only two main entries, for which I used bold text, and my first level of subentry was the real main entry I wanted. The sub-subentry was the real subentry I wanted. The designers can adjust the indentation and spacing to make these appear as two separate indexes:

The chapter and verse numbers presented two other problems of their own. How to write something like 2:10? First, Word signals heading levels with a colon, so I had to use a backslash before the colon to tell Word that this was a literal colon, not a subheading signal. I admit that at that point, I had forgotten the warnings of my colleagues that InDesign would reject these entries.

As of this writing, I am waiting for the author’s comments and corrections, and the results of a small test index for the publisher: three entries using a backslash and colon, and three using a plus sign to be replaced by a colon in the generated index. If I do indeed have to remove \: from the index, I want to be sure that + is not a signal for something else in InDesign.

A second problem in writing chapter and verse numbers was the sorting. I knew that in Sky, I had to enter one- and two-digit chapter numbers with preceding zeros so they would sort properly. Thus, Chapter 2 was entered as 002 and Chapter 16 as 016. The verse numbers following the colons, however, sorted properly in Sky without additional zeros.

Word was not happy with that, but I could only learn that at the end. I finished my index, embedded the entries, generated the index, and then found that Word had mis-sorted the verses so that, for example, 18:70 came before 18:7. I had to open Sky, add the zeros to the verses, re-embed the entries, generate the new index, and remove the extra zeros from the generated index.

Maybe I’ll Talk a Little Bit

Another difference between Sky and Word is how they handle text to be ignored in sorting. Sky’s sorting automatically ignores prepositions at the beginning of subentries, but  Word’s does not. Sky also allows the indexer to code other things to be ignored in sorting. I commonly do this with the al- that begins many Arabic names.

For the embedded index, I had to enclose items to be ignored in angle brackets, but then in Sky, they all sorted to the top because they started with symbols. I was not sure that Word would put <al->Bukhari, <al->Ghazali, <al->Tabari, etc., in the proper places in the generated index. On this, I did have success, but I had to go back to the few subentries that begin with prepositions and enclose the prepositions in angle brackets.

DEXembed uses a text file to embed the entries, and all the bold and italics are lost in the process, although their coding remains. Once the entries were embedded, I had to edit the XE fields to get the bold and italic formatting back. (See Sue Klefstad’s blog post for details.) This was not difficult with a Find and Replace using wildcards (but be sure to turn off Tracked Changes!), but it was an extra step to perform.

Advice for Embedded Indexing

It is important to communicate with the author and publisher before beginning an embedded index. Learn how the Word manuscript will be handled after indexing and how it will be published. (There is more information on the resources page of the DPI SIG website.)

Once you have written your index in your dedicated indexing software, always embed in a copy of the document. Always keep the original “clean” and do not embed in it. Sometimes Word does not embed the entries properly and you might have to try again. DEXembed does have a function to remove embedded entries, but if Word gives you run-time errors as it did to me (see the second part of my February 2019 column), you will want to try again in a clean copy so there is no chance of stray coding in the file.

My thanks to colleagues Sue Klefstad and Seth A. Maislin for their invaluable blog posts, and to other colleagues in the DPI SIG for their advice in e-mail messages.

Ælfwine Mischler is an American copyeditor and indexer in Cairo, Egypt, who has been the head copyeditor at a large Islamic website and a senior editor for an EFL textbook publisher. She often edits and indexes books on Islamic studies, Middle East studies, and Egyptology.

Lyonizing Word: Helping Authors Write

Jack Lyon

In my previous post, Lyonizing Word: Workflow for Writing, I suggested some tools that would help authors write without the problems that are almost inevitable when working in Microsoft Word. These include inconsistent and meaningless formatting, document corruption, fouled-up footnotes, incorrect AutoCorrect “corrections,” and so on. Unfortunately, most authors already use Word and aren’t likely to change. How can we, as editors, help them create Word documents that are well-structured and clean, thus reducing our own workload?

Word itself includes a feature that helps make this possible, although I doubt that many editors or authors are even aware of it: Restrict Editing. You’ll find this feature on Word’s Ribbon interface under the Review tab.

What does it do? It prevents authors from using arbitrary, meaningless formatting, applying various fonts in various sizes higgledy-piggledy all over the place as authors are wont to do. The only formatting they can do is with styles — and then only with the styles that you allow. You will like this. And your designer will like this. And your typesetter will like this.

At first, your authors will not like this. But once they understand how it works, they should find great relief in not having to design as well as write. All they have to do — all they can do — is apply a heading style to headings, a block quotation style to block quotations, and so on. They can get on with actually writing, rather than worrying about whether this heading should be bold and that one italic, whether poetry should use Garamond or Palatino. As technical writer Brendan Rowland notes in comment 153 on the blog Charlie’s Diary, “When you’ve worked with locked/protected docs in Word, you’ll never want to work any other way. Life becomes so much easier. No more user-created spaghetti formatting — this becomes a distant memory.”

Restricting Editing

Here’s how to set up a document that restricts editing in Microsoft Word:

1. In Word, create a new document.
2. Click the Review tab.
3. Click the Restrict Editing icon (far right).
4. Put a check in the box labeled “Limit formatting to a selection of styles.”
5. Just below that, click Settings.
6. Put a check in the new box labeled “Limit formatting to a selection of styles.”
7. Put a check in the box next to each style that you want your authors to be able to use. For recommendations on what those styles might be, see my article “But What Styles?”
8. Under the Formatting heading, make sure the first box is unchecked and the last two are.
9. Click the OK button.
10. Now, in the task pane on the right, click the button labeled “Yes, Start Enforcing Protection.”
11. To enforce protection, enter a password, confirm it, and click OK. The password doesn’t need to be long and complex; it just needs to be something your authors won’t guess and that you will remember. In fact, something as simple as your initials will do. After you’ve entered a password, your authors can’t turn off protection, so it really is protection.
12. Save the document.
13. Give the document to your authors, instructing them to write their masterpieces in that document and no other.

Creating Character Styles

There is a problem with this system, however, and it’s a serious one. When you restrict formatting to a selection of styles, Word no longer allows you to use directly applied formatting like italic and bold — styles only, so no CTRL + I for you! The only way around this is to use character styles (not paragraph styles) that are set to use italic, bold, or whatever you need. And here, in my opinion, is what you need:

• Italic.

• Superscript.

• Subscript.

• Strikethrough.

What, no bold? Not unless you’re working with an author whose field requires bold — some branches of math or medicine, perhaps. But for most authors, access to bold means they’ll try to use it to format headings when they should be using a heading style, such as Heading 2 or Heading 3.

What, no underline? Again, not unless you’re working with an author whose field requires it. Otherwise, some authors will use underlining when they should be using italic — a holdover from the days of the typewriter.

Now you need to add the character styles to your document. Here’s how:

1. For the time being, stop enforcing protection on the document. Otherwise, you won’t be able to create a new style. You remember your password, right?
2. Click the little arrow at the bottom right of Home > Styles to open the Styles task pane on the right.
3. At the bottom of the task pane, click the little New Style icon on the bottom left.
4. Give your style a name, such as Italic.
5. In the box labeled “Style type,” click the dropdown arrow and select Character. This is key to making this work.
6. Under Formatting, click the Italic button.
7. Click the OK button.
8. Repeat the process for any other character styles your authors will need.
9. Again enforce protection for the document.

A side benefit to using character styles is that they can be imported into InDesign, where they can be set to use whatever formatting is needed — something that isn’t possible with directly applied formatting like italic or bold.

Creating Keyboard Shortcuts

So now the character styles are available, but only from the Styles task pane. Not very convenient; your authors are going to want their CTRL + I back. Here’s how to provide it:

1. Under the File tab, click Options > Customize Ribbon.
2. Click the button labeled “Keyboard shortcuts: Customize” on the bottom left.
3. In the Categories box on the left, scroll to the bottom and select Styles.
4. In the Styles box on the right, select the style you created earlier (such as Italic).
5. Put your cursor in the box labeled “Press new shortcut key” and, well, press a new shortcut key. Let’s use CTRL + I for our italic character style.
6. Click the dropdown arrow in the box labeled “Save changes in:” and select your document. Now your keyboard shortcut will be saved in the document rather than in your Normal template. Don’t skip this step!
7. Click the Assign button on the lower left.
8. Click the Close button on the lower right.
9. Click the OK button.
10. Save your document.
11. Give the document to your authors.

Now when your authors select some text and press CTRL + I, the Italic character style will be applied, so they can work without using the mouse to select the Italic style in the Styles task pane. Easy, intuitive, perfect. Rinse and repeat, with the appropriate keyboard shortcuts, for your other character styles.

At this point, you may be wondering why I didn’t just create this document for you. Stay tuned; next time I will, with a few little extras to make your life easier. But if you ever need to do all of this yourself, now you know how.

Jack Lyon (editor@editorium.com) owns and operates the Editorium, which provides macros and information to help editors and publishers do mundane tasks quickly and efficiently. He is the author of Microsoft Word for Publishing Professionals, Wildcard Cookbook for Microsoft Word, and of Macro Cookbook for Microsoft Word. Both books will help you learn more about macros and how to use them.

Lyonizing Word: Workflow for Writing

by Jack Lyon

I do a lot of writing, and over the years I’ve investigated many a tool that’s supposed to help with that process. The most prominent of these, of course, is the bloated but powerful Microsoft Word. With my various add-ins at the Editorium, it can be a terrific editing tool. But for writing, something else is needed. Why? Because (as with most word processors) writing in Word is like scribbling on a scroll. Access to text is sequential rather than random (as I explained in my essay, “Changing Formats: From Scroll to Codex to eBooks”, although if you’ve used Word’s built-in heading styles, it’s possible to jump to those headings using the navigation window.

Rather than scrolling (or jumping) around a long, long document, I prefer to write in bits and pieces and then combine selected bits and pieces into a single document ready for editing. It’s possible to do this (kind of) in Scrivener using its “corkboard” feature (on both Mac and PC). Unfortunately, like Word, Scrivener strikes me as clunky, uncooperative, and overly complex.

Notebox Disorganizer

I’ve tried nearly every writing program out there, and the best solution I’ve found is the idiosyncratic and free Notebox Disorganizer from the Squirrel Technologist. (Sorry, Windows only — but please keep reading, as the other tools I’ll be discussing here work on Macintosh or Linux as well as Windows, and they’re well worth having.)

Notebox Disorganizer is a sort of spreadsheet for writers. It looks like this:

Notebook Disorganizer

The top part of the screen consists of boxes divided among rows and columns. Each box represents a separate document (although all of the documents are in the same file). We can move the cursor to the box we want to use and press ENTER. The cursor jumps to the document at the bottom, and we’re ready to write. To return to the boxes, we hit the ESCAPE key.

With Notebox Disorganizer, we can see the entire structure of our book laid out in a grid. Here, the book is broken up into parts that include the various chapters, but we could just as easily have each column be a chapter, and the boxes in that column be scenes. For nonfiction, each column could be a chapter, and the boxes could be sections of the chapter.

We can move boxes and columns around as needed. If we realize that scene 4 in chapter 2 should really be in chapter 8, we can cut the box and then paste it where it belongs. If we see that scene 4 should actually be scene 5, we can move it down. The program offers lots of flexibility. If you’d like to see the Notebox Disorganizer file in which I wrote this article, you can download it from the Editorium’s website.

(Note: The source code for Notebox Disorganizer is in the public domain and can be downloaded from the Squirrel Technologist website. So if you’re interested in customizing the program or incorporating its ideas into something else, the developer, Forrest Leeson, encourages you to do so.)

Markdown Syntax

Out of the box, Notebox Disorganizer uses Rich Text Format (.rtf) which means we can apply various fonts in various sizes and colors. Unfortunately, that encourages us to apply various fonts in various sizes and colors, when what is really needed is a proper document structure: headings need to be identified as headings, block quotes as block quotes, and so on. Directly applied formatting, no matter how beautiful, won’t supply that. To make that happen (and to keep writing rather than fussing with formatting), we can do two things:

1. Change Notebox Disorganizer’s preferences (under Tools > Set Preferences > Misc > Forbid Formatted Text) so that it uses plain text only — no formatting allowed.
2. Use Markdown syntax to specify (rather than apply) formatting — for example, use *asterisks* to indicate italic. Heading levels are specified with cross-hatches: # Heading 1, ## Heading 2, ### Heading 3, and so on. A complete reference for Markdown syntax (which is intuitive, human readable, and platform and program agnostic) is available as a downloadable PDF or online from GitHub.

Making a Manuscript

After we’ve written the various sections that make up chapters, it’s time to combine the text in all those boxes into a single document. To do that, we add boxes to the program’s “outbox” by selecting them and then pressing the spacebar. The result looks like this:

Outbox

If there are certain boxes we don’t want to include (research notes, for example), we just don’t include them in the outbox. After we’ve finished with our selection, we click File > Export the Outbox and give the document a name. Under “Files of type,” we select “Text.” Then we click OK, and the text is exported as a single text file, with Markup codes intact.

Turning Markdown into Formatting

Now that our document is finished, we need to turn it into a Word document. Why? Because that’s what publishers seem to want, unfortunately. But because it’s properly structured and marked up, we can just as easily turn it into a web page, a PDF, or just about anything else using the marvelous and (again) free Pandoc. (Pandoc works on Mac, Windows, or Linux.)

Pandoc is a tool that every writer and editor should have, as it can turn almost any document format into almost any other document format, which is something you might need to do sometime. For that reason, I’m going to ask you to try an experiment with me. It’s not hard, and I think you’ll like the results. Do this, in this order:

1. Download and install Pandoc.
2. Download and install Typora. (Typora, too, works on Mac, Windows, or Linux. Click the little arrow at the bottom of the home page; then click Download on the upper right.) Typora is an editing and rendering program for Markdown.

Have you finished installing? Great, then download from the Editorium website the Markdown document I created after writing this article. Put it on your desktop and then double-click it to open it in Typora.

Beautiful, no? Nice formatting and proper document structure. Just for fun, try some of the alternative CSS themes (click Theme) — or open the file in a plain old text editor to see the Markdown codes.

You can actually use Typora on its own to write just about anything (note the document outline on the left). As soon as you type something (using Markdown syntax), Typora renders it into an appropriate format. But we need a Word document, right? Well, one of the beautiful things about Typora is that it works automatically with Pandoc, so we can easily export our document as a Word file. To see this in action, click File > Export > Word (.docx). Now open the Word file (same folder and name as your Markdown document) and marvel at the result — a nicely formatted and structured document that any editor would be pleased to work on and any designer would be happy to import into InDesign. Please take a moment to contemplate how revolutionary that actually is.

Authors and Styles and Fonts, Oh My!

Now, if we could just get authors to write using Markdown, what a wonderful world it would be! Here’s why:

As you’ve seen, editors can easily convert a Markdown document into a Word document for editing, with all of Word’s tools at their disposal. The Markdown codes will be appropriately converted into Microsoft Word paragraph styles, with no extraneous formatting or messed-up footnotes to be cleaned up. Wouldn’t that be nice!

But what about authors? Why should they work in Markdown when they could just as easily work in Word? The reasons are many:

1. They can’t just as easily work in Word. In fact, most authors have no clue about how to properly do so. Word makes it easy for authors to mess up a document almost beyond belief, with inconsistent and meaningless formatting, document corruption, fouled-up footnotes, incorrect AutoCorrect “corrections,” and on and on and on. Editors are left to clean up all that stuff.
2. Microsoft Word is expensive — $149.99 for Office Home & Student 2016 (but doesn’t include newer versions as they’re released); if you go with Office 365 Personal (which does include new versions), you’re looking at $69 per year; for Office 365 Home, $99 per year. And those years add up.
3. Markdown is intuitive — easy to learn, read, and use.
4. Authors can create or read Markdown documents in any text editor or word processor (even Word) on any platform — Mac, Windows, Linux, Android, iPhone, whatever, without problems of compatibility.
5. Markdown documents can easily be converted into all kinds of properly structured and formatted documents, including Word, XML, HTML, LaTeX, and PDF — true single-source publishing.
6. Markdown documents will be readable and usable as long as text files are readable and usable — which is to say, forever.
7. As Markdown documents are nothing but text, they’re small, taking up very little room on a hard drive or thumb drive, and they’re easy to send by email. In fact, you can use Markdown to write email.
8. Perhaps most important, Markdown allows authors to simply write, without worrying about formatting and other complexities, thus increasing their productivity — which is something that benefits everyone.
9. If you can persuade your authors to write with Markdown, the benefits should be great for all concerned. Well, for all except Microsoft:

Imagine there’s no Redmond;
It’s easy if you try.
No styles or wonky footnotes—
Something easy on the eye.
Imagine all the people
Writing stuff in peace! (No “helpful” automatic formatting, AutoCorrect, etc.)
You may say I’m a dreamer,
But I’m not the only one. (There are lots of Markdown editing and rendering programs out there.)
Just try to write with Markdown,
And you’ll see it can be done!

(Apologies to John Lennon.)

Jack Lyon (editor@editorium.com) owns and operates the Editorium, which provides macros and information to help editors and publishers do mundane tasks quickly and efficiently. He is the author of Microsoft Word for Publishing Professionals, Wildcard Cookbook for Microsoft Word, and of Macro Cookbook for Microsoft Word. Both books will help you learn more about macros and how to use them.

Lyonizing Word: Taming Styles in Microsoft Word

by Jack Lyon

Microsoft Word includes a powerful feature for marking the various levels of a manuscript (such as headings, block quotations, poetry, and so on). That feature is styles, which are valuable for many reasons, including:

• They make it possible to reformat a whole document simply by redefining styles or applying a different template using those styles.
• They make it possible to find and replace only text using a certain style. For example, you might want to find source citations by searching for parentheses in text styled as block quotations.
• They make it possible to generate a table of contents based on specified styles.

So styles are very useful. The problem is that Microsoft Word, in its usual “helpful” way, tries to manage which styles are available, in which document, and how those styles can be accessed. Finally growing tired of this nonsense, I decided to take the matter firmly in hand by writing this article.

My first gripe is that Word decides which styles to show in the Styles area of the Home ribbon, which decision seems to be based on nothing that makes any sense. Right now, it’s showing the following:

Quick Style Gallery

Of the styles available, I use Normal and Heading 1. But Strong? Subtle Emphasis? Intense Emphasis? Who makes this stuff up? Not an actual writer or editor, that’s for sure. So the first thing to do is get rid of the icons for the styles I never use:

1. Right-click the icon (such as that for Strong).
2. Click “Remove from Quick Style Gallery” (which, evidently is what the Styles area is called).

Remove from Quick Style Gallery

Now, the question is, when I restart Word or create a new document, does the Strong icon come back? Let’s find out. (Now restarting Word.)

Ha! It’s gone! But what happens if I create a new document? (Now creating a new document.)

Shoot, Strong is back again. So we can conclude that removing a style from the Quick Style Gallery applies only to the document in which we remove the style.

I could get rid of Strong and then save what I’ve done as a Quick Style Set:

Save as Quick Style Set

But I’d like to get rid of Strong once and for all. How can I do that?

Well, I’ll start by showing Word’s task pane (by clicking the little arrow at the bottom right of the Styles area):

Word’s task pane

Now I should be able to click the drop-down arrow next to Strong and delete it, right? Nope. Word won’t let me. How annoying!

Delete Strong

Well, then, where does the Strong style live? In Word’s Normal.dotm template, of course. Can I get rid of it there? I open the folder where the template lives, which on my computer is here:

C:\Users\Jack\AppData\Roaming\Microsoft\Templates

Then I open the Normal.dotm template. Now can I delete the Strong style?

No, I can’t; same problem as before. Word really, really, really wants to keep its built-in styles — which is why they’re called “built-in,” I guess. So my only recourse is to (1) set how the style will be displayed and then (2) tell Word which styles to display. Here’s how:

1. Open the Normal.dotm template, which is where your default styles are stored.
2. Under Style Pane Options (the blue “Options” link at the bottom of the task pane), set “Styles to Show” as “Recommended.” Select “New documents based on this template.”

Show styles as recommended

3. Under Manage Styles (the third button at the bottom of the task pane), set all styles to “Hide” or “Hide until used” except those you want to show. (Even now, Word won’t let you hide everything.) Select “New documents based on this template.”

Hide Strong

4. Make any other adjustments you’d like, such as the order in which the styles will appear in the task pane.
5. Save and close the Normal.dotm template.

After you’ve done that, every time you start Word or create a new document, you’ll get only the styles you want to see. I think. I hope. Maybe.

How about you? Do you have any helpful hints about how to tame Word’s styles? If so, I’d love to hear from you.

Jack Lyon (editor@editorium.com) owns and operates the Editorium, which provides macros and information to help editors and publishers do mundane tasks quickly and efficiently. He is the author of Microsoft Word for Publishing Professionals, Wildcard Cookbook for Microsoft Word, and of Macro Cookbook for Microsoft Word. Both books will help you learn more about macros and how to use them.