✕
Skip to content

[AbstractTypeRefining] Skip unreachable casts - #9358

Open
tlively wants to merge 5 commits into
mainfrom
abstract-type-refining-unreachable
Open

tlively wants to merge 5 commits into
mainfrom
abstract-type-refining-unreachable

Conversation

@tlively

@tlively tlively commented Nov 6, 2026

Copy link
Copy Markdown
Member

Skip Preoptimizer::visitRefCast, visitRefGetDesc, and visitBrOn when an operand is unreachable (either already unreachable in the input or replaced with an unreachable block earlier in Preoptimizer), updating curr->type to Type::unreachable so parent expressions also see the unreachability and ReFinalize cleans up unreachable branches.

This fixes a bug where Preoptimizer was replacing unreachable expressions with none-typed expressions, leading to validation failures.

Skip `Preoptimizer::visitRefCast`, `visitRefGetDesc`, and `visitBrOn` when an operand is unreachable (either already unreachable in the input or replaced with an unreachable block earlier in `Preoptimizer`), updating `curr->type` to `Type::unreachable` so parent expressions also see the unreachability and `ReFinalize` cleans up unreachable branches.

This fixes a bug where Preoptimizer was replacing unreachable expressions with none-typed expressions, leading to validation failures.
@tlively
tlively requested a review from a team as a code owner November 6, 2026 23:46

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the AbstractTypeRefining pass to correctly handle cases where operands of RefCast, RefGetDesc, or BrOnCast instructions are optimized to unreachable. In these scenarios, the instruction's type is now explicitly set to unreachable to prevent validation or optimization issues. Corresponding test cases have been added to verify these changes. I have no feedback to provide as there are no review comments.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Google Antigravity Code Review

I have reviewed the changes in this Pull Request and left inline feedback.

if (curr->op == BrOnNull || curr->op == BrOnNonNull) {
return;
}
if (curr->ref->type == Type::unreachable ||

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[HIGH | Confidence: 5/5 | BUG]

The unreachable check in visitBrOn is placed after the early return for BrOnNull and BrOnNonNull:

Suggested change
if (curr->ref->type == Type::unreachable ||
if (curr->op == BrOnNull || curr->op == BrOnNonNull) {
return;
}
if (curr->ref->type == Type::unreachable ||
(curr->desc && curr->desc->type == Type::unreachable)) {
curr->type = Type::unreachable;
return;
}

If the ref operand of a BrOnNull or BrOnNonNull became unreachable during the post-order traversal (e.g., from an uninstantiated RefGetDesc replaced with an unreachable sequence), line 442 returns early without updating curr->type to Type::unreachable.

If this BrOn expression is subsequently visited as the ref child of an enclosing RefCast, RefGetDesc, or another BrOn, the enclosing expression will observe curr->ref->type as a reachable concrete type instead of unreachable. This bypasses the enclosing expression's unreachable guard and causes it to perform invalid transformations (e.g., localizeChildren extracting the branch to a local, or makeRefCast), leading to invalid IR or assertion failures during finalization.

To ensure consistent type propagation matching BrOn::finalize(), move the unreachable check before the operation check:

Suggested change
if (curr->ref->type == Type::unreachable ||
void visitBrOn(BrOn* curr) {
if (curr->ref->type == Type::unreachable ||
(curr->desc && curr->desc->type == Type::unreachable)) {
curr->type = Type::unreachable;
return;
}
if (curr->op == BrOnNull || curr->op == BrOnNonNull) {
return;
}
auto optimized = getOptimized(curr->castType);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't quite get the comment here, but it at least makes sense to me to move the check up like it suggests? Although maybe refinalize would do this anyway

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, seems reasonable.

@tlively
tlively removed the request for review from a team November 7, 2026 00:18
void visitRefCast(RefCast* curr) {
if (curr->ref->type == Type::unreachable ||
(curr->desc && curr->desc->type == Type::unreachable)) {
curr->type = Type::unreachable;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this needed? Did we modify the code at this point but not call refinalize yet? If so it seems worth a comment as in general this should not be necessary.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added comments.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants