surrogateescape largely missing from documentation

[jimjjewett]

BPO	14015
Nosy	@akuchling, @ezio-melotti, @JimJJewett
Files	patch14015.txt

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2013-06-16.17:00:53.447>
created_at = <Date 2012-02-14.18:56:29.593>
labels = ['expert-unicode', 'docs']
title = 'surrogateescape largely missing from documentation'
updated_at = <Date 2013-06-16.17:00:53.435>
user = 'https://github.com/JimJJewett'

bugs.python.org fields:

activity = <Date 2013-06-16.17:00:53.435>
actor = 'python-dev'
assignee = 'docs@python'
closed = True
closed_date = <Date 2013-06-16.17:00:53.447>
closer = 'python-dev'
components = ['Documentation', 'Unicode']
creation = <Date 2012-02-14.18:56:29.593>
creator = 'Jim.Jewett'
dependencies = []
files = ['30514']
hgrepos = []
issue_num = 14015
keywords = []
message_count = 3.0
messages = ['153359', '190860', '191273']
nosy_count = 6.0
nosy_names = ['akuchling', 'ezio.melotti', 'cvrebert', 'docs@python', 'python-dev', 'Jim.Jewett']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue14015'
versions = ['Python 3.2', 'Python 3.3']

[jimjjewett]

Recent discussion on the mailing lists and in http://bugs.python.org/issue13997 make it clear that the best way to get python2 results for "ASCII-in-the-parts-I-might-process-or-change" is to replace

    f = open(fname)
with
    f = open(fname, encoding="ascii", errors="surrogateescape")

Unfortunately, surrogateescape (let alone this recipe) is not easily discoverable.

http://docs.python.org/dev/library/functions.html#open lists 5 error-handlers -- but not this one. It says that other error handlers are possible if they are registered with http://docs.python.org/dev/library/codecs.html#codecs.register_error but I haven't found a way to determine which error handlers are already registered.

The codecs.register (as opposed to register_error) documentation does list it as a possible value, but that is the only reference.

The other 5 error handlers are also available as module-level functions within the codecs module, and have their own documenation sections within http://docs.python.org/dev/library/codecs.html

Neither help(open) nor import codecs; help(codecs) provides any hints of the existence of surrogateescape. Both explicitly suggest that it does not exist, by enumerating other values.

[akuchling]

Here's a proposed patch that touches the Sphinx documentation and a docstring in codecs.py. The text is slightly revised from my current revisions to the Unicode howto.

help(open) says "See the documentation for codecs.register for a list of the permitted encoding error strings". This is strictly correct: the Sphinx documentation features this info. But help(codecs.register) doesn't; it's more helpful to look at help(codecs.Codec).

So maybe the docstring for open() should say: "See help(codecs.Codec) for a list of the permitted..." instead.

[python-dev]

New changeset 55f611f55952 by Andrew Kuchling in branch '3.3':
Describe 'surrogateescape' in the documentation.
http://hg.python.org/cpython/rev/55f611f55952