Komodo User Guide

Regular Expressions Primer

The Regular Expressions Primer is a tutorial for those completely new to regular expressions. To familiarize you with regular expressions, this primer starts with the simple building blocks of the syntax and through examples, builds to construct expressions useful for solving real every-day problems. The primer later discusses how to search for and replace text with regular expression syntax.

Regular expressions are embedded in programs to parse text. For example, a Perl program might contain the following segment:

while (<DATA>) {
        if (/(\w*):\s+(.*)/) {
                $headers{$1} = $2;
        }
}

In this example, a regular expression is used as the condition for an "if" statement.

Komodo's Rx Toolkit is used to build and evaluate regular expressions. See Using Rx Toolkit for more information.

Top

About Regular Expressions

Regular expressions are used to describe patterns of characters that match against text strings. They can be used as a tool to search for and replace text, manipulate data, or test for a certain condition in a string of characters. Many everyday tasks can be accomplished with regular expressions, such as checking for the occurrence of a specific word or phrase in the body of an e-mail message, or finding specific file types, such as .txt files, in a folder or directory. Regular expressions are often called "regex", "regexes", "regexps", and "RE". This primer uses the terms "regular expressions", "regex", and "regexes" equally.

Top

About Regex Syntax

Regular expressions use syntax elements comprised of alphanumeric characters and symbols. For example, the regex (2) searches for the number 2, while the regex ([1-9][0-9]{2}-[0-9]{4}) matches a regular 7-digit phone number.

There are many flavors and types of regular expression syntax. These variations are found in various tools, languages and operating systems. For example, Perl, Python, grep, sed, VI, and Unix all use variations on standard regex syntax. This primer focuses on standard regex patterns not tied to a specific language or tool. This standard syntax can be later applied to the specific language, tool or application of your choice.

Top

Building Simple Patterns

Complete regular expressions are constructed using characters as small building block units. Each building block is in itself simple, but since these units can be combined in an infinite number of ways, knowing how to combine them to achieve a goal takes some practice. This section shows you how to build regexes through examples ranging from the simple to the more complex.

Matching Simple Strings

The simplest and most common type of regex is an alphanumeric string that matches itself, called a "literal text match". A literal text regex matches anywhere along a string. For example, a literal string matches itself when placed alone, and at the beginning, middle, or end of a larger string. Literal text matches are case sensitive.

Using regexes to search for simple strings.

Example 1: Search for the string "at".

  • Regex:
        at
        
  • Matches:
        at
        math
        hat
        ate
        
  • Doesn't Match:
        it
        a-t
        At
        

Example 2: Search for the string "email".

  • Regex:
        email
        
  • Matches:
        email
        emailing
        many_emails
        
  • Doesn't Match:
        Email
        EMAILing
        e-mails
        

Example 3: Search for the alphanumeric string "abcdE567".

  • Regex:
        abcdE567
        
  • Matches:
        abcdE567
        AabcdE567ing
        text_abcdE567
        
  • Doesn't Match:
        SPAMabCdE567
        ABCDe567
        

Note: Regular expressions are case sensitive unless case is deliberately modified.

Top

Searching with Wildcards

In the previous examples, regular expressions are constructed with literal characters that match themselves. There are other characters in regex syntax that match in a more generalized way. These are called "metacharacters". Metacharacters do not match themselves, but rather perform a specific task when used in a regular expression. One such metacharacter is the dot ".", or wildcard. When used in a regular expression, the wildcard can match any single character.

Using the wildcard to match any character.

Example 1: Use a wildcard to search for any one character before the string "ubject:".

  • Regex:
        .ubject:
        
  • Matches:
        Subject:
        subject:
        Fubject:
        
  • Doesn't Match:
        Subject
        subject
        

Example 2: Use three dots "..." to search for any three characters within a string.

  • Regex:
        t...s
        
  • Matches:
        trees
        tEENs
        t345s
        t-4-s
        
  • Doesn't Match:
       Trees
       twentys
       t1234s
        

Example 3: Use several wildcards to match characters throughout a string.

  • Regex:
        .a.a.a
        
  • Matches:
        Canada
        alabama
        banana
        3a4a5a
        
  • Doesn't Match:
        aaa
        
Top

Searching for Special Characters

In regular expression syntax, most non-alphanumerical characters are treated as special characters. These characters, called "metacharacters", include asterisks, question marks, dots, slashes, etc. In order to search for a metacharacter without using its special attribute, precede it with a backslash "\" to change it into a literal character. For example, to build a regex to search for a .txt file, precede the dot with a backslash \.txt to prevent the dot's special function, a wildcard search. The backslash, called an "escape character" in regex terminology, turns metacharacters into literal characters.

Precede the following metacharacters with a backslash "\" to search for them as literal characters:

^ $ + * ? . | ( ) { } [ ] \

Using the backslash "\" to escape special characters in a regular expression.

Example 1: Escape the dollar sign "$" to find the alphanumeric string "$100".

  • Regex:
        \$100
        
  • Matches:
        $100
        $1000
        
  • Doesn't Match:
        \$100
        100
        

Example 2: Use the dot "." as a literal character to find a file called "email.txt".

  • Regex:
        email\.txt
        
  • Matches:
        email.txt
        
  • Doesn't Match:
        email
        txt
        email_txt
        

Example 3: Escape the backslash "\" character to search for a Windows file.

  • Regex:
        c:\\readme\.txt
        
  • Matches:
        c:\readme.txt
        
  • Doesn't Match:
        c:\\readme.txt
        d:\readme.txt
        c:/readme.txt
        
Top

Ranges and Repetition

Regex syntax includes metacharacters which specify the number of times a particular character or string must match. This group of metacharacters is called "quantifiers"; they influence the quantity of matches found. Quantifiers act on the element immediately preceding them, which could be a digit, a letter, or another metacharacter (including spaces as metacharacters not previously defined and the dot "."). This section demonstrates how quantifiers search using ranges and repetition.

Ranges, {min, max}

Ranges are considered a "counting qualifier" in regular expressions. This is because they specify the minimum number of matches to find and the maximum number of matches to allow. Use ranges in regex searches when a bound, or a limit, should be placed on search results. For example, the range {3,5} matches an item at least 3 times, but not more than 5 times. When this range is combined with the regex, a{3,5}, the strings "aaa", "aaaa", and "aaaaa" are successfully matched. If only a single number is expressed within curly braces {3}, the pattern matches exactly three items. For example, the regex b{3} matches the string "bbb".

Using ranges to identify search patterns.

Example 1: Match the preceding "0" at least 3 times with a maximum of 5 times.

  • Regex:
        60{3,5} years
        
  • Matches:
        6000 years
        60000 years
        600000 years
        
  • Doesn't Match:
        60 years
        600 years
        6003 years
        6000000 years
        

Example 2: Using the "." wildcard to match any character sequence two or three characters long.

  • Regex:
        .{2,3}
        
  • Matches:
        404
        44
        com
        w3
        
  • Doesn't Match:
        4
        a
        aaaa
        

Example 3: Match the preceding "e" exactly twice.

  • Regex:
        be{2}t
        
  • Matches:
        beet
        
  • Doesn't Match:
        bet
        beat
        eee
        

Example 4: Match the preceding "w" exactly three times.

  • Regex:
        w{3}\.mydomain\.com
        
  • Matches:
        www.mydomain.com
        
  • Doesn't Match:
        web.mydomain.com
        w3.mydomain.com
        
Top

Repetition, ?*+

Unlike range quantifiers, the repetition quantifiers (question mark "?", asterisk "*", and plus "+") have few limits when performing regex searches. This is significant because these quantifiers settle for the minimum number of required matches, but always attempt to match as many times as possible, up to the maximum allowed. For example, the question mark "?" matches any preceding character 0 or 1 times, the asterisk "*" matches the preceding character 0 or more times, and the plus "+" matches the preceding character 1 or more times. Use repetition quantifiers in regex searches when large numbers of results are desired.

Using repetition to search for repeated characters with few limits.

Example 1: Use "?" to match the "u" character 0 or 1 times.

  • Regex:
        colou?r
        
  • Matches:
        colour
        color
        
  • Doesn't Match:
        colouur
        Colour
        

Example 2: Use "*" to match the preceding item 0 or more times; use "." to match any character.

  • Regex:
        www\.my.*\.com
        
  • Matches:
        www.mysite.com
        www.mypage.com
        www.my.com
        
  • Doesn't Match:
        www.oursite.com
        mypage.com
        

Example 3: Use "+" to match the preceding "5" at least once.

  • Regex:
        bob5+@foo\.com
        
  • Matches:
        bob5@foo.com
        bob5555@foo.com
        
  • Doesn't Match:
        bob@foo.com
        bob65555@foo.com
        
Top

Quantifier Summary

The following table defines the various regex quantifiers. Note that each quantifier is unique and will perform a varying minimum and maximum number of matches in order to search successfully.

Quantifier Description
{num} Matches the preceding element num times.
(min, max} Matches the preceding element at least min times, but not more than max times.
? Matches any preceding element 0 or 1 times.
* Matches the preceding element 0 or more times.
+ Matches the preceding element 1 or more times.

Top

Using Conditional Expressions

Conditional expressions help qualify and restrict regex searches, increasing the probability of a desirable match. The vertical bar "|" symbol, meaning "OR", places a condition on the regex to search for either one character in a string or another. Because the regex has a list of alternate choices to evaluate, this regex technique is called "alternation". To search for either one character or another, insert a vertical bar "|" between the desired characters.

Example 1: Use "|" to alternate a search for various spellings of a string.

  • Regex:
        gray|grey
        
  • Matches:
        gray
        grey 
        
  • Doesn't Match:
        GREY
        Gray
        

Example 2: Use "|" to alternate a search for either email or Email or EMAIL or e-mail.

  • Regex:
        email|Email|EMAIL|e-mail
        
  • Matches:
        email
        Email
        EMAIL
        e-mail
        
  • Doesn't Match:
        EmAiL
        E-Mail
        
Top

Grouping Similar Items in Parentheses

Use parentheses to enclose a group of related search elements. Parentheses limit scope on alternation and create substrings to enhance searches with metacharacters. For example, use parentheses to group the expression (abc), then apply the range quantifier {3} to find instances of the string "abcabcabc".

Using parentheses to group regular expressions.

Example 1: Use parentheses and a range quantifier to find instances of the string "abcabcabc".

  • Regex:
        (abc){3}
        
  • Matches:
        abcabcabc
        abcabcabcabc
        
  • Doesn't Match:
        abc
        abcabc
        

Example 2: Use parentheses to limit the scope of alternative matches on the words gray and grey.

  • Regex:
        gr(a|e)y
        
  • Matches:
        gray
        grey
        
  • Doesn't Match:
        gry
        graey
        

Example 3: Use parentheses and "|" to locate past correspondence in a mail-filtering program. This regex finds a 'To:' or a 'From:' line followed by a space and then either the word 'Smith' or the word 'Chan'.

  • Regex:
        (To:|From:)(Smith|Chan)
        
  • Matches:
        To:Smith
        To:Chan
        From:Smith
        To:Smith, Chan
        To:Smithe
        From:Channel4News
        
  • Doesn't Match:
        To:smith
        To:All
        To:Schmidt
        
Top

Matching Sequences

You can build a regular expression to match a sequence of characters. These sequences, called "character classes", simply place a set of characters side-by-side within square brackets "[]". An item in a character class can be either an ordinary character, representing itself, or a metacharacter, performing a special function. This primer covers how to build simple character classes, prevent matches with character classes, and construct compound character classes with metacharacters.

Building Simple Character Classes

The most basic type of character class is a set of characters placed side-by-side within square brackets "[]". For example, the regular expression [bcr]at, matches the words "bat", "cat", or "rat" because it uses a character class (that includes "b","c", or "r") as its first character. Character classes only match singular characters unless a quantifier is placed after the closing bracket. For examples using quantifiers with character classes, see Compound Character Classes. The following table shows how to use simple character classes in regex searches.

Note: When placed inside a character class, the hyphen "-" metacharacter denotes a continuous sequence of letters or numbers in a range. For example, [a-d] is a range of letters denoting the continuous sequence of a,b,c and d. When a hyphen is otherwise used in a regex, it matches a literal hyphen.

Using simple character classes to perform regex searches.

Example 1: Use a character class to match all cases of the letter "s".

  • Regex:
        Java[Ss]cript
        
  • Matches:
        JavaScript
        Javascript
        
  • Doesn't Match:
        javascript
        javaScript
        

Example 2: Use a character class to limit the scope of alternative matches on the words gray and grey.

  • Regex:
        gr[ae]y
        
  • Matches:
        gray
        grey
        
  • Doesn't Match:
        gry
        graey
        

Example 3: Use a character class to match any one digit in the list.

  • Regex:
        [0123456789]
        
  • Matches:
        5 
        0
        9
        
  • Doesn't Match:
        x
        ?
        F
        

Example 4: To simplify the previous example, use a hyphen "-" within a character class to denote a range for matching any one digit in the list.

  • Regex:
        [0-9]
        
  • Matches:
        5
        0
        9
        
  • Doesn't Match:
        234
        42
        

Example 5: Use a hyphen "-" within a character class to denote an alphabetic range for matching various words ending in "mail".

  • Regex:
        [A-Z]mail
        
  • Matches:
        Email
        Xmail
        Zmail
        
  • Doesn't Match:
        email
        mail
        

Example 6: Match any three or more digits listed in the character class.

  • Regex:
        [0-9]{3,}
        
  • Matches:
        012
        1234
        555
        98754378623
        
  • Doesn't Match:
        10
        7
        
Top

Preventing Matches with Character Classes

Previous examples used character classes to specify exact sequences to match. Character classes can also be used to prevent, or negate, matches with undesirable strings. To prevent a match, use a leading caret "^" (meaning NOT), within square brackets,[^...]. For example, the regex [^a] matches any single character except the letter "a".

Note: The caret symbol must be the first character within the square brackets to negate a character class.

Using character classes to prevent a sequence from matching.

Example 1: Prevent a match on any numeric string. Use the "*" to match an item 0 or more times.

  • Regex:
        [^0-9]*
        
  • Matches:
        abc
        c
        Mail
        u-see
        a4a
        
  • Doesn't Match:
        1
        42
        100
        23000000
        

Example 2: Search for a text file beginning with any character not a lower-case letter.

  • Regex:
        [^a-z]\.txt
        
  • Matches:
        A.txt
        4.txt
        Z.txt
        
  • Doesn't Match:
        r.txt
        a.txt
        Aa.txt
        

Example 3: Prevent a match on the numbers "10" and "12".

  • Regex:
        1[^02]
        
  • Matches:
        13
        11
        19
        17
        1a
        
  • Doesn't Match:
        10
        12
        42
        a1
        
Top

Compound Character Classes

Character classes are a versatile tool when combined with various pieces of the regex syntax. Compound character classes can help clarify and define sophisticated searches, test for certain conditions in a program, and filter wanted e-mail from spam. This section uses compound character classes to build meaningful expressions with the regex syntax.

Using compound character classes with the regex syntax.

Example 1: Find a partial e-mail address. Use a character class to denote a match for any number between 0 and 9. Use a range to restrict the number of times a digit matches.

  • Regex:
        smith[0-9]{2}@
        
  • Matches:
        smith44@
        smith42@
        
  • Doesn't Match:
        Smith34
        smith6
        Smith0a
        

Example 2: Search an HTML file to find each instance of a header tag. Allow matches on whitespace after the tag but before the ">".

  • Regex:
        (<[Hh][1-6] *>)
        
  • Matches:
         <H1>
         <h6>
         <H3  >
         <h2    >
        
  • Doesn't Match:
        <H1
        <   h2>
        <a1>
        

Example 3: Match a regular 7-digit phone number. Prevent the digit "0" from leading the string.

  • Regex:
        ([1-9][0-9]{2}-[0-9]{4})
        
  • Matches:
         555-5555
         123-4567 
        
  • Doesn't Match:
        555.5555
        1234-567
        023-1234
        

Example 4: Match a valid web-based protocol. Escape the two front slashes.

  • Regex:
        [a-z]+:\/\/
        
  • Matches:
        http://
        ftp://
        https://
        
  • Doesn't Match:
        http
        http:
        ptf:// 
        

Example 5: Match a valid e-mail address.

  • Regex:
        [a-z0-9_-]+(\.[a-z0-9_-]+)*@[a-z0-9_-]+(\.[a-z0-9_-]+)+
        
  • Matches:
        j_smith@foo.com
        j.smith@bc.canada.ca
        smith99@foo.co.uk
        1234@mydomain.net
        
  • Doesn't Match:
        @foo.com
        .smith@foo.net
        smith.@foo.org
        www.myemail.com
        
Top

Character Class Summary

The following table defines various character class sequences. Use these alphanumeric patterns to simplify your regex searches.

Character Class Description
[0-9] Matches any digit from 0 to 9.
[a-zA-z] Matches any alphabetic character.
[a-zA-z0-9] Matches any alphanumeric character.
[^0-9] Matches any non-digit.
[^a-zA-z] Matches any non-alphabetic character.
Top

Matching Locations within a String

At times, the pattern to be matched appears at either the very beginning or end of a string. In these cases, use a caret "^" to match a desired pattern at the beginning of a string, and a dollar sign "$" for the end of the string. For example, the regular expression email matches anywhere along the following strings: "email", "emailing", "bogus_emails", and "smithsemailaddress". However, the regex ^email only matches the strings "email" and "emailing". The caret "^" in this example is used to effectively anchor the match to the start of the string. For this reason, both the caret "^" and dollar sign "$" are referred to as anchors in the regex syntax.

Note: The caret "^" has many meanings in regular expressions. Its function is determined by its context. The caret can be used as an anchor to match patterns at the beginning of a string, for example:(^File). The caret can also be used as a logical "NOT" to negate content in a character class, for example: [^...].

Using anchors to match at the beginning or end of a string.

Example 1: Use "$" to match the ".com" pattern at the end of a string.

  • Regex:
        .*\.com$
        
  • Matches:
        mydomain.com 
        a.b.c.com
        
  • Doesn't Match:
        mydomain.org 
        mydomain.com.org
        

Example 2: Use "^" to match "inter" at the beginning of a string, "$" to match "ion" at the end of a string, and ".*" to match any number of characters within the string.

  • Regex:
        ^inter.*ion$
        
  • Matches:
        internationalization
        internalization
        
  • Doesn't Match:
        reinternationalization
        

Example 3: Use "^" inside parentheses to match "To" and "From" at the beginning of the string.

  • Regex:
        (^To:|^From:)(Smith|Chan)
        
  • Matches:
        From:Chan
        To:Smith
        From:Smith 
        To:Chan 
        
  • Doesn't Match:
        From: Chan
        from:Smith
        To Chan
        

Example 4: Performing the same search as #3, place the caret "^" outside the parentheses this time for similar results.

  • Regex:
        ^(From|Subject|Date):(Smith|Chan|Today)
        
  • Matches:
        From:Smith
        Subject:Chan 
        Date:Today
        
  • Doesn't Match:
        X-Subject:
        date:Today
        
Top

Searching and Replacing

Regular expressions are often used to search and replace text strings. Up until this point, the preceding examples have centered on matching a string using regex syntax. This section examines the search and replace operation as a prominent feature of regular expressions and solves standard problems using the substitution syntax.

Like with building regular expressions, there are many variations on substitution syntax depending on the language used. This primer focuses on general search and replace syntax. This standard syntax can be later applied to the specific language, tool or application of your choice.

Top

Building Simple Substitution Searches

Substitution searches search for and replace a pattern of text. Substitutions are performed using the s/// operator, "s" standing for substitution. The s/// operator takes a regular expression between the first and second front slashes, while the second and third front slashes take the replacement text.

For example:

s/<regex>/<substitution-string>/


Use the s/// operator to search for and replace a simple text string. Note: these searches only replace the first instance of the string found.

Example 1: Search for the string "email" and replace it with "e-mail".

  • Regex Substitution:
        s/email/e-mail
        
  • Search for:
        email
        
  • Replace with:
        e-mail
        

Example 2: Search for an old domain name and replace it with the new domain name. Using regex syntax, escape "." and "/" characters.

  • Regex Substitution:
        s/http:\/\/www\.old-domain\.com/http:\/\/www\/new-domain\.com/
        
  • Search for:
        www.old-domain.com
        
  • Replace with:
        www.new-domain.com
        

Example 3: Search for a single string starting with any lowercase letter and ending with "mail". Replace the string with "Email".

  • Regex Substitution:
        s/[a-z]mail/Email
        
  • Search for:
        email
        zmail
        xmail
        
  • Replace with:
        Email
        
Top

Modifying Substitution Searches

The previous substitution examples focused on small searches, such as replacing a single lower-case word in a single line of text. Extend the scope and flexibility of substitution searches through the use of modifiers. The modifier parameter is appended to the end of the s/// operator as follows:

s/<regex>/<substitution-string>/<modifier>

Use the modifier "i" to ignore case in alphabetic searches, "m" to allow multiple lines in a string, "s" to treat a pattern as a single line, "x" to allow for whitespace and comments, and "g" for global searching all occurances of the pattern in a file and not just the first instance found.

Use various modifiers with the s/// operator to search for and replace text strings.

Example 1: Using the "g" modifier, search globally through all .htm instances in a file and replace them with ".html". Using "$", only substitute the ".htm" string when it appears at the end of a line. An example file where this substitution succeeds:

/manual/mod_python/pythonapi.htm
/manual/mod_python/more-comp.htm
/manual/mod_python/overview.htm
  • Regex Substitution:
        s/\.htm$/\.html/g
        
  • Search for:
        .htm
        
  • Replace with:
        .html
        

Example 2: Using the "g" modifier, remove all html tags in a file and replace the tags with an empty string.

  • Regex Substitution:
        s/<[^>]+>//g
        
  • Search for:
        <code>Tag</code>
        
  • Replace with:
        Tag
        

Example 3: Perform a case insensitive search for various instances of "login" and replace with the string "password".

  • Regex Substitution:
        s/LOGIN/password/i
        
  • Search for:
        LOGIN
        login
        LoGiN
        Login
        
  • Replace with:
        password
        
Top

Substitution Modifier Summary

The following table defines various modifiers for the substitution operator. Modifiers change how a match is performed. Use these modifiers to expand the scope and versatility of your substitutions.

Modifier Meaning
i Ignore case when matching exact strings.
m Treat string as multiple lines. Allow "^'' and "$'' to match next to newline characters.
s Treat string as single line. Allow ".'' to match a newline character.
x Ignore whitespace and newline characters in the regular expression. Allow comments. 
o Compile regular expression once only.
g Match all instances of the pattern in the target string.
Top

More Regex Resources

Internet Web Sites:

Beginner:

Intermediate:

Advanced:

Top