爱学习的站长www.mmic.net.cn

3.2 Strings: Text Processing at the Lowest Level  字符串：最底层的文本处理

 

PS:个人认为这部分很重要，字符串处理是NLP里最基本的部分，各位童鞋好好看，老鸟略过...

It’s time to study a fundamental data type that we’ve been studiously（故意地） avoiding so far. In earlier chapters we focused on a text as a list of words. We didn’t look too closely at words and how they are handled in the programming language. By using NLTK’s corpus interface we were able to ignore the files that these texts had come from. The contents of a word, and of a file, are represented by programming languages as a fundamental data type known as a string. In this section, we explore strings in detail, and show the connection between strings, words, texts, and files.

 

Basic Operations with Strings 字符串的基本操作

 

Strings are specified using single quotes ①or double quotes②, as shown in the following code example. If a string contains a single quote, we must backslash-escape the quote③ so Python knows a literal quote character is intended, or else put the string in double quotes②. Otherwise, the quote inside the string④will be interpreted as a close quote, and the Python interpreter will report a syntax error:

 

Sometimes strings go over several lines. Python provides us with various ways of entering them. In the next example, a sequence of two strings is joined into a single string. We need to use backslash ① or parentheses ② so that the interpreter knows that the statement is not complete after the first line.

 

 

Unfortunately these methods do not give us a newline between the two lines of the sonnet(十四行诗). Instead, we can use a triple-quoted string as follows:

 

 

 

Now that we can define strings, we can try some simple operations on them. First let’s look at the + operation, known as concatenation ① . It produces a new string that is a copy of the two original strings pasted together end-to-end（首尾相连）. Notice that concatenation doesn’t do anything clever like insert a space between the words. We can even multiply strings②:

 

 

Your Turn: Try running the following code, then try to use your understanding of the string + and * operations to figure out how it works. Be careful to distinguish between the string ' ', which is a single whitespace character, and '', which is the empty string.

 

 

 

We’ve seen that the addition and multiplication operations apply to strings, not just numbers. However, note that we cannot use subtraction or division with strings:

 

 

These error messages are another example of Python telling us that we have got our data types in a muddle（困惑）. In the first case, we are told that the operation of subtraction (i.e., -) cannot apply to objects of type str (strings), while in the second, we are told that division cannot take str and int as its two operands.

 

Printing Strings 打印字符串

 

So far, when we have wanted to look at the contents of a variable or see the result of a calculation, we have just typed the variable name into the interpreter. We can also see the contents of a variable using the print statement:

 

 

Notice that there are no quotation marks this time. When we inspect a variable by typing its name in the interpreter, the interpreter prints the Python representation of its value. Since it’s a string, the result is quoted. However, when we tell the interpreter to print the contents of the variable, we don’t see quotation characters, since there are none inside the string.

The print statement allows us to display more than one item on a line in various ways,

as shown here:

 

 

 

Accessing Individual Characters 访问单独的字符

 

As we saw in Section 1.2 for lists, strings are indexed, starting from zero. When we index a string, we get one of its characters (or letters). A single character is nothing special—it’s just a string of length 1.

As with lists, if we try to access an index that is outside of the string, we get an error:

 

Again as with lists, we can use negative indexes for strings, where -1 is the index of the last character①. Positive and negative indexes give us two ways to refer to any position in a string. In this case, when the string had a length of 12, indexes 5 and -7 both refer to the same character (a space). (Notice that 5 = len(monty) - 7.)

We can write for loops to iterate over the characters in strings. This print statement ends with a trailing comma, which is how we tell Python not to print a newline at the end.

We can count individual characters as well. We should ignore the case distinction by normalizing everything to lowercase, and filter out non-alphabetic characters:

 

 

This g

引自：ic pdf http://pdflist.mmic.net.cn

ives us the letters of the alphabet, with the most frequently occurring letters listed first (this is quite complicated and we’ll explain it more carefully later). You might like to visualize the distribution using fdist.plot(). The relative character frequencies of a text can be used in automatically identifying the language of the text.