strutils - Text manipulation

So much practical programming involves string manipulation, which Python readily accomodates. Still, there are dozens of basic and common capabilities missing from the standard library, several of them provided by strutils.

boltons.strutils.camel2under(camel_string)

Converts a camelcased string to underscores. Useful for turning a class name into a function name.

>>> camel2under('BasicParseTest')
'basic_parse_test'

boltons.strutils.under2camel(under_string)

Converts an underscored string to camelcased. Useful for turning a function name into a class name.

>>> under2camel('complex_tokenizer')
'ComplexTokenizer'

boltons.strutils.slugify(text, delim='_', lower=True, ascii=False)

A basic function that turns text full of scary characters (i.e., punctuation and whitespace), into a relatively safe lowercased string separated only by the delimiter specified by delim, which defaults to _.

The ascii convenience flag will asciify() the slug if you require ascii-only slugs.

>>> slugify('First post! Hi!!!!~1    ')
'first_post_hi_1'

>>> slugify("Kurt Gödel's pretty cool.", ascii=True) ==         b'kurt_goedel_s_pretty_cool'
True

boltons.strutils.split_punct_ws(text)

While str.split() will split on whitespace, split_punct_ws() will split on punctuation and whitespace. This used internally by slugify(), above.

>>> split_punct_ws('First post! Hi!!!!~1    ')
['First', 'post', 'Hi', '1']

boltons.strutils.unit_len(sized_iterable, unit_noun='item')

Returns a plain-English description of an iterable’s len(), conditionally pluralized with cardinalize(), detailed below.

>>> print(unit_len(range(10), 'number'))
10 numbers
>>> print(unit_len('aeiou', 'vowel'))
5 vowels
>>> print(unit_len([], 'worry'))
No worries

boltons.strutils.ordinalize(number, ext_only=False)

Turns number into its cardinal form, i.e., 1st, 2nd, 3rd, 4th, etc. If the last character isn’t a digit, it returns the string value unchanged.

Parameters:

• number (int or str) – Number to be cardinalized.
• ext_only (bool) – Whether to return only the suffix. Default False.

>>> print(ordinalize(1))
1st
>>> print(ordinalize(3694839230))
3694839230th
>>> print(ordinalize('hi'))
hi
>>> print(ordinalize(1515))
1515th

boltons.strutils.cardinalize(unit_noun, count)

Conditionally pluralizes a singular word unit_noun if count is not one, preserving case when possible.

>>> vowels = 'aeiou'
>>> print(len(vowels), cardinalize('vowel', len(vowels)))
5 vowels
>>> print(3, cardinalize('Wish', 3))
3 Wishes

boltons.strutils.pluralize(word)

Semi-intelligently converts an English word from singular form to plural, preserving case pattern.

>>> pluralize('friend')
'friends'
>>> pluralize('enemy')
'enemies'
>>> pluralize('Sheep')
'Sheep'

boltons.strutils.singularize(word)

Semi-intelligently converts an English plural word to its singular form, preserving case pattern.

>>> singularize('records')
'record'
>>> singularize('FEET')
'FOOT'

boltons.strutils.asciify(text, ignore=False)

Converts a unicode or bytestring, text, into a bytestring with just ascii characters. Performs basic deaccenting for all you Europhiles out there.

Also, a gentle reminder that this is a utility, primarily meant for slugification. Whenever possible, make your application work with unicode, not against it.

Parameters:

• text (str or unicode) – The string to be asciified.
• ignore (bool) – Configures final encoding to ignore remaining unasciified unicode instead of replacing it.

>>> asciify('Beyoncé') == b'Beyonce'
True

boltons.strutils.is_ascii(text)

Check if a unicode or bytestring, text, is composed of ascii characters only. Raises ValueError if argument is not text.

Parameters:text (str or unicode) – The string to be checked.

>>> is_ascii('Beyoncé')
False
>>> is_ascii('Beyonce')
True

boltons.strutils.is_uuid(obj, version=4)

Check the argument is either a valid UUID object or string.

Parameters:

• obj (object) – The test target. Strings and UUID objects supported.
• version (int) – The target UUID version, set to 0 to skip version check.

>>> is_uuid('e682ccca-5a4c-4ef2-9711-73f9ad1e15ea')
True
>>> is_uuid('0221f0d9-d4b9-11e5-a478-10ddb1c2feb9')
False
>>> is_uuid('0221f0d9-d4b9-11e5-a478-10ddb1c2feb9', version=1)
True

boltons.strutils.html2text(html)

Strips tags from HTML text, returning markup-free text. Also, does a best effort replacement of entities like “ ”

>>> r = html2text(u'<a href="#">Test &amp;<em>(\u0394&#x03b7;&#956;&#x03CE;)</em></a>')
>>> r == u'Test &(\u0394\u03b7\u03bc\u03ce)'
True

boltons.strutils.strip_ansi(text)

Strips ANSI escape codes from text. Useful for the occasional time when a log or redirected output accidentally captures console color codes and the like.

>>> strip_ansi('artÜ')
'art'

The test above is an excerpt from ANSI art on sixteencolors.net. This function does not interpret or render ANSI art, but you can do so with ansi2img or escapes.js.

boltons.strutils.bytes2human(nbytes, ndigits=0)

Turns an integer value of nbytes into a human readable format. Set ndigits to control how many digits after the decimal point should be shown (default 0).

>>> bytes2human(128991)
'126K'
>>> bytes2human(100001221)
'95M'
>>> bytes2human(0, 2)
'0.00B'

boltons.strutils.find_hashtags(string)

Finds and returns all hashtags in a string, with the hashmark removed. Supports full-width hashmarks for Asian languages and does not false-positive on URL anchors.

>>> find_hashtags('#atag http://asite/#ananchor')
['atag']

find_hashtags also works with unicode hashtags.

boltons.strutils.a10n(string)

That thing where “internationalization” becomes “i18n”, what’s it called? Abbreviation? Oh wait, no: a10n. (It’s actually a form of numeronym.)

>>> a10n('abbreviation')
'a10n'
>>> a10n('internationalization')
'i18n'
>>> a10n('')
''

boltons.strutils.gunzip_bytes(bytestring)

The gzip module is great if you have a file or file-like object, but what if you just have bytes. StringIO is one possibility, but it’s often faster, easier, and simpler to just use this one-liner. Use this tried-and-true utility function to decompress gzip from bytes.

>>> gunzip_bytes(_EMPTY_GZIP_BYTES) == b''
True
>>> gunzip_bytes(_NON_EMPTY_GZIP_BYTES).rstrip() == b'bytesahoy!'
True

boltons.strutils.iter_splitlines(text)

Like str.splitlines(), but returns an iterator of lines instead of a list. Also similar to file.next(), as that also lazily reads and yields lines from a file.

This function works with a variety of line endings, but as always, be careful when mixing line endings within a file.

>>> list(iter_splitlines('\nhi\nbye\n'))
['', 'hi', 'bye', '']
>>> list(iter_splitlines('\r\nhi\rbye\r\n'))
['', 'hi', 'bye', '']
>>> list(iter_splitlines(''))
[]

boltons.strutils.indent(text, margin, newline='\n', key=<type 'bool'>)

The missing counterpart to the built-in textwrap.dedent().

Parameters:

• text (str) – The text to indent.
• margin (str) – The string to prepend to each line.
• newline (str) – The newline used to rejoin the lines (default: \n)
• key (callable) – Called on each line to determine whether to indent it. Default: bool, to ensure that empty lines do not get whitespace added.

boltons.strutils.escape_shell_args(args, sep=' ', style=None)

Returns an escaped version of each string in args, according to style.

Parameters:

• args (list) – A list of arguments to escape and join together
• sep (str) – The separator used to join the escaped arguments.
• style (str) – The style of escaping to use. Can be one of cmd or sh, geared toward Windows and Linux/BSD/etc., respectively. If style is None, then it is picked according to the system platform.

See args2cmd() and args2sh() for details and example output for each style.

boltons.strutils.args2cmd(args, sep=' ')

Return a shell-escaped string version of args, separated by sep, using the same rules as the Microsoft C runtime.

>>> print(args2cmd(['aa', '[bb]', "cc'cc", 'dd"dd']))
aa [bb] cc'cc dd\"dd

As you can see, escaping is through backslashing and not quoting, and double quotes are the only special character. See the comment in the code for more details. Based on internal code from the subprocess module.

boltons.strutils.args2sh(args, sep=' ')

Return a shell-escaped string version of args, separated by sep, based on the rules of sh, bash, and other shells in the Linux/BSD/MacOS ecosystem.

>>> print(args2sh(['aa', '[bb]', "cc'cc", 'dd"dd']))
aa '[bb]' 'cc'"'"'cc' 'dd"dd'

As you can see, arguments with no special characters are not escaped, arguments with special characters are quoted with single quotes, and single quotes themselves are quoted with double quotes. Double quotes are handled like any other special character.

Based on code from the pipes/shlex modules. Also note that shlex and argparse have functions to split and parse strings escaped in this manner.

boltons.strutils.parse_int_list(range_string, delim=', ', range_delim='-')

Returns a sorted list of positive integers based on range_string. Reverse of format_int_list().

Parameters:

• range_string (str) – String of comma separated positive integers or ranges (e.g. ‘1,2,4-6,8’). Typical of a custom page range string used in printer dialogs.
• delim (char) – Defaults to ‘,’. Separates integers and contiguous ranges of integers.
• range_delim (char) – Defaults to ‘-‘. Indicates a contiguous range of integers.

>>> parse_int_list('1,3,5-8,10-11,15')
[1, 3, 5, 6, 7, 8, 10, 11, 15]

boltons.strutils.format_int_list(int_list, delim=', ', range_delim='-', delim_space=False)

Returns a sorted range string from a list of positive integers (int_list). Contiguous ranges of integers are collapsed to min and max values. Reverse of parse_int_list().

Parameters:

• int_list (list) – List of positive integers to be converted into a range string (e.g. [1,2,4,5,6,8]).
• delim (char) – Defaults to ‘,’. Separates integers and contiguous ranges of integers.
• range_delim (char) – Defaults to ‘-‘. Indicates a contiguous range of integers.
• delim_space (bool) – Defaults to False. If True, adds a space after all delim characters.

>>> format_int_list([1,3,5,6,7,8,10,11,15])
'1,3,5-8,10-11,15'