Mr. Edward Bear (pseudonym) shared an awesome secret for sussing out quote issues in a previous post. He’s back with a nifty way to help spot spelling issues. It helped me spot a number of inconsistencies in Bad Penny, which I recently fixed. Take it away, Mr. Bear.
* * * *
Augmented Proofreading 2
Last time, I mentioned checking spelling and spelling consistency, and this note is to fulfill my promise.
To begin with, I’ll admit freely I do not recommend just running a spieling chucker over the text. And yes, I’m using the phrase deliberately. The spelling checker in the program I’m using to write this just waltzed right past it, despite the fact that neither word makes sense in this context. I got started trying to get some mileage out of spelling checkers a while back, when I realized that OCR scanners will work like hell trying to find a word and probably will come up with a word, but it will be the wrong word. Over at Project Gutenberg, they use the term “scanos” as an analogue of “typos.”
The other adversary is consistency of spelling and usage. Examples might include “canceled” and “cancelled” or variant possessives, such as “Jones’” and “Jones’s” in the same text, or simply variant spellings of the same character’s name, such as “Erik” and “Eric.”
So what do I do for these sorts of problems?
What you need is a list of all the unique words in the story, and it’s easy with a dollop of regex. (See the first post on this topic. The one I’ve taken to using is the following:
[^-‘‘’\w\d\r\n]
This one I can explain, since I wrote it. 🙂
What it says is to look for any individual characters within the brackets([]). The leading caret (^) says “Nope, they should not be in this list.” Following that are the hyphen(-), the straight quote(‘), and the opening and closing curly single quotes(‘’). After that, it’s any character that would normally be in a word (\w), any digit (\d), any carriage return(\r) or any new line aka paragraph terminator (\n).
And what you do with the above is to set Textpad (see previous post again) to replace all of them with a newline (\n) character.
Here’s what the search and replace box in Textpad looks like for this:

Notice that the “Regular expression” box is checked. And then you choose “Replace All”.
The result is to turn this:
Once upon a night we’ll wake to the carnival of life
The beauty of this ride ahead such an incredible high
It’s hard to light a candle, easy to curse the dark instead
This moment the dawn of humanity
The last ride of the day
into this:
Once
upon
a
night
we’ll
wake
to
the
carnival
of
life
The
beauty
of
this
ride
ahead
such
an
incredible
high
It’s
hard
to
light
a
candle
easy
to
curse
the
dark
instead
This
moment
the
dawn
of
humanity
The
last
ride
of
the
day
For a book, this means what you have as a multi-thousand line long document and LOTS of repeated words. The next step is to lose the repetitions by sorting the list. For TextPad, you’ll find Tools->Sort on the menu bar, and I generally set the sort up as the following:

This is a case-sensitive sort and “Delete duplicate lines”, of course, gets rid of all the duplications of words like “a”, “an”, and “the”.
For the above text, you get:
It’s
Once
The
This
a
ahead
an
beauty
candle
carnival
curse
dark
dawn
day
easy
hard
high
humanity
incredible
instead
last
life
light
moment
night
of
ride
such
the
this
to
upon
wake
we’ll
34 unique words.
And here’s the payoff for all this funky text munging: Run a spelling checker over this, and it will find typos, of course, but it will also generally complain about most names, which aren’t generally kept in spelling checker dictionaries, and it will also show you words and their variants, nestled fairly closely together. Here’s an example from one of my projects:
Swiss-born
Take
Talking
Tallmadge
Tallmadge’s
Tarelton
Tarleton
Tarleton’s
Tavern
Tecumseh
Tell
Ten
Thames
The spelling checker landed on Tallmadge and its possessive form, but do you notice the “Tarelton” on the list? That’s the time for the Aha!, and you can fix the “Tarelton” form. The same thing applies to possessive variants and any other variant form you find.
As with the last post, I’ll be peeking in on the comments and answering any questions that come up.
Personally, I always thought at least one factor for the decline was the end of the space race. Without such a big science fiction draw in the public eye to spur interest, they slackened off.
Hum, interesting theory.
I personally let my subscription lapse when I was so busy that I found I was almost 1 year behind in reading the issues. Maybe my situation was unique or perhaps culturally we are reading less? I also wonder about the younger generation. Is there a sufficient draw to attract that demographic into the genre? If not, then perhaps there is a certain degree of natural attrition at work. (As Colton alluded to, do kids dream about being astronauts anymore?)
I don’t know if they do. But I can tell you they dreamed of going to Hogwarts the last decade.
And literacy rates have been rising for the last years. Rowling had a lot to do with that in the younger ages in the late 90’s and early 2000’s. Stephenie Meyer continued it and spawned a whole new YA Paranomal Romance genre, which has become big enough that Barnes & Noble now breaks it out. Go to the stores and you see Teen Fiction, YA Paranormal Romance, and YA Fantasy and Adventure. Those categories are growing.
Is it that the culture has no demand for SFF in short form, or that these magazines cater to an adult market? And maybe one that eschews the very stories that YA readers love.
Kris Rusch raises that question in her essay “Barbarians at the Gate” in Asimov’s: http://www.asimovs.com/_issue_0612/thoughtexperiments.shtml
Lots of speculation. No data.
Literacy rates: http://johndbrown.com/2009/03/literary-reading-rates-on-the-rise/
Total speculation here, BUT… I wonder if some of the decline is due to the prevalence of science fiction-ish technology in our lives these days. Tiny communication devices, for example, are no longer something to oooh and ahhh at on the big screen or in the pages of a magazine; we don’t really get the same sense of wonder from them anymore. (Though that same prevalence of technology is perhaps all the more reason and need for us to have people writing about the possible future effects and consequences of that technology… more need, but less people reading it?)
Another possible factor is the blurring of the scifi and fantasy lines, how so much fantasy seems to have a scientific (or pseudo-scientific) underlay. I’d be interested to hear further thoughts on all this.
S. Jeanne,
I don’t know why the magazines made their big slide, but I don’t think SF & Fantasy has gone out of fashion or become less popular. There are boom-and-bust cycles with fiction genres. But we’re not looking at the genre as a whole here, just magazine subscriptions.
There were lots of big grossing SF movies in the 1990s and 2000s.
http://en.wikipedia.org/wiki/List_of_science-fiction_films_of_the_1990s
http://en.wikipedia.org/wiki/List_of_science-fiction_films_of_the_2000s
I don’t know about SF book sales, but if you look at books like the big selling HUNGER GAMES and UGLIES series (and others), SF has not gone away.
Potter started a up a huge fantasy boom cycle. Meyer started up a huge paranormal romance boom. Hunger games started a SF dystopia boom. Not as big as those two, but significant.
So I don’t know that we can say that SF and Fantasy is less popular. We can say that the magazine format is. But that may have more to do with the format than anything else. And maybe the timing–a recession, the rise of the internet, pc, and video games–all helped hasten the magazine bust. But I don’t know if that says anything about SF interest in general.