Repository navigation
Standardised mechanism for stripping importlib frames from tracebacks #59691
Description
| BPO | 15486 |
|---|---|
| Nosy | @brettcannon, @birkenfeld, @jcea, @atsuoishimoto, @amauryfa, @ncoghlan, @pitrou, @meadori, @cjerdonek, @ericsnowcurrently |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/birkenfeld'
closed_at = <Date 2012-07-31.11:14:32.635>
created_at = <Date 2012-07-29.10:46:43.208>
labels = ['interpreter-core', 'type-bug', 'library', 'release-blocker']
title = 'Standardised mechanism for stripping importlib frames from tracebacks'
updated_at = <Date 2012-08-03.15:38:37.331>
user = 'https://github.com/ncoghlan'bugs.python.org fields:
activity = <Date 2012-08-03.15:38:37.331>
actor = 'jcea'
assignee = 'georg.brandl'
closed = True
closed_date = <Date 2012-07-31.11:14:32.635>
closer = 'python-dev'
components = ['Interpreter Core', 'Library (Lib)']
creation = <Date 2012-07-29.10:46:43.208>
creator = 'ncoghlan'
dependencies = []
files = ['26597']
hgrepos = []
issue_num = 15486
keywords = ['patch']
message_count = 11.0
messages = ['166745', '166748', '166760', '166770', '166798', '166846', '166875', '166880', '166913', '166947', '166988']
nosy_count = 12.0
nosy_names = ['brett.cannon', 'georg.brandl', 'jcea', 'ishimoto', 'amaury.forgeotdarc', 'ncoghlan', 'pitrou', 'Arfrever', 'meador.inge', 'chris.jerdonek', 'python-dev', 'eric.snow']
pr_nums = []
priority = 'release blocker'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue15486'
versions = ['Python 3.3']
Activity
ncoghlan commentedon Jul 29, 2012
Thanks to issue bpo-15425, the number of special cases in the importlib frame stripping is growing. This patch tweaks the frame stripping mechanism to make it easy to selectively strip frames in importlib._bootstrap by creating a _call_with_frames_removed helper function.
The rules in remove_importlib_frames then change to remove any sequence of importlib frames which ends with that helper and all importlib frames in the ImportError case.
ncoghlan commentedon Jul 29, 2012
In trying to find a new case that wasn't already covered by the test suite, I found an error which I'm not even sure should be an error. Doesn't 3.2 suppress failures that occur when attempting to implicitly cache the .pyc?
Anyway, given the rest of the patch, trimming this traceback will be easy if we decide that's the right thing to do.
[-]Standardise the mechanisms for stripping importlib frames[/-][+]Standardised mechanism for stripping importlib frames from tracebacks[/+]cjerdonek commentedon Jul 29, 2012
The test cases uploaded to bpo-7559 ("TestLoader.loadTestsFromName swallows import errors") a while back contain a number of distinct cases (four, I think) in which an import error can be raised. IIRC, these include a couple recursive scenarios (cyclical import from within __init__.py, etc).
Those test cases were designed specifically to cover different scenarios in which unittest's loader was swallowing import errors. Might be worth taking a look.
pitrou commentedon Jul 29, 2012
This is an elegant solution, +1 from me.
pitrou commentedon Jul 29, 2012
That said, given the nature of the patch (a cleanup without any functional impact), I don't think it should be a release blocker.
ncoghlan commentedon Jul 30, 2012
The "release blocker" status comes from the test case I added in order to demonstrate the ability to strip a new frame sequence without needing to modify the C code.
Currently that test (failing to write the PYC file) fails with an IsADirectory traceback that includes a lot of importlib frames.
There are two possible solutions:
I need to check if failing to write the .pyc was a fatal error in 3.2 before I decide how to fix it (unless someone else remembers off the top of their head)
ncoghlan commentedon Jul 30, 2012
OK, after a bit of experimentation, it appears both 3.2 and 3.3 eventually get annoyed if you mess about too much with __pycache__.
They're both fine if __pycache__ is entirely unwritable (they just silently skip caching the bytecode)
3.2 throws EOFError if you replace the cache entry with an empty file, 3.3 silently rewrites it with a valid cached version
3.2 throws EOFError if you replace the cache entry with a directory, 3.3 throws a more accurate IsADirectory error
That means my chosen test case is a valid one, and I can just update the offending call in importlib._bootrap to use the new frame stripping hook as I originally planned.
ncoghlan commentedon Jul 30, 2012
Looking at the actual code - I don't think it actually makes sense to strip the frames in this case. Unlike the previous examples, it only comes up if someone is doing something deliberately pathological, at which point a little noise in the traceback is the least of your worries.
Updated patch just drops that test case entirely and only has the changes that simplify the frame removal code. Since Georg isn't on IRC right now, assigning him to decide if he wants this version in beta 2 rather than the current "multiple exit points" approach.
birkenfeld commentedon Jul 30, 2012
Looks good and simplifies things. Green light :)
ericsnowcurrently commentedon Jul 31, 2012
patch LGTM. Nice and clean.
python-dev commentedon Jul 31, 2012
New changeset 62033490ca0f by Nick Coghlan in branch 'default':
Close bpo-15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise
http://hg.python.org/cpython/rev/62033490ca0f