gh-128045: Mark unknown opcodes as deopting to themselves

[DinoV]

When accessing co_code on a code object we'll first run through to do the de-opt https://github.com/python/cpython/blob/main/Objects/codeobject.c#L1706

This removes any unknown opcodes. Instead the deopt table should just recognize unknown opcodes as deopting to themselves, allowing the extensible interpreter loop to consume unknown opcodes.

• Issue: code objects remove unknown opcodes from the instruction stream when accessing co_code #128045

[iritkatriel]

Since we're not testing this at all in cpython, I'd suggest we at least add a couple of assertions to make sure this is correctly covering the range of opcodes:

Suggested change

	for name, deopt in sorted(deopts):
	out.emit(f"[{name}] = {deopt},\n")
	defined = set(analysis.opmap.values())
	for i in range(256):
	if i not in defined:
	out.emit(f"[{i}] = {i},\n")
	defined = set(analysis.opmap.values())
	for i in range(256):
	if i not in defined:
	deopts.append((f'{i}', f'{i}'))
	assert len(deopts) == 256
	assert len(set(x[0] for x in deopts)) == 256
	for name, deopt in sorted(deopts):
	out.emit(f"[{name}] = {deopt},\n")

[markshannon]

I think we regard undefined instructions an error.
If you want to pass custom bytecodes to your custom interpreter, a more robust approach might be needed.
We can do the deopt thing for now, but it seems fragile.

For example, one thing I have considered is storing bytecodes in a compact format on disk: Instructions without an oparg would take 1 byte, those with an oparg would take 2. We would then combine the unmarshalling and quickening steps to create the full in-memory form in a single pass. Custom instructions would not survive this process.

[DinoV]

Took me a while to get back to this but I'm finally back :) I applied the changes suggested by @iritkatriel.

I think we regard undefined instructions an error. If you want to pass custom bytecodes to your custom interpreter, a more robust approach might be needed. We can do the deopt thing for now, but it seems fragile.

For example, one thing I have considered is storing bytecodes in a compact format on disk: Instructions without an oparg would take 1 byte, those with an oparg would take 2. We would then combine the unmarshalling and quickening steps to create the full in-memory form in a single pass. Custom instructions would not survive this process.

I think as long as we could still have some way to construct a code object ourselves that would be fine, we'd just may need to implement our own custom unmarshaling logic that could support our opcodes. Obviously that doesn't cover all the ways that things could change in the future but we can try and figure out how to adapt to other potential changes :)

[markshannon]

Looks good

[miss-islington-app]

Thanks @DinoV for the PR 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

[bedevere-app]

3.14 branch.