From 70c431c53da9aa8d86ced8c95856ad7070c30120 Mon Sep 17 00:00:00 2001
From: "Flavio S. Glock"
Date: Wed, 9 Sep 2026 14:22:53 +0200
Subject: [PATCH 1/8] docs: plan PPR compatibility completion
Record the known PPR 0.001010 failures and the staged acceptance plan for
issue #1318 before implementation begins.
Generated with Codex (https://openai.com/codex)
Co-Authored-By: Codex
---
dev/design/ppr-compatibility.md | 107 ++++++++++++++++++++++++++++++++
1 file changed, 107 insertions(+)
create mode 100644 dev/design/ppr-compatibility.md
diff --git a/dev/design/ppr-compatibility.md b/dev/design/ppr-compatibility.md
new file mode 100644
index 000000000..3ad520b80
--- /dev/null
+++ b/dev/design/ppr-compatibility.md
@@ -0,0 +1,107 @@
+# PPR 0.001010 Compatibility
+
+Issue: #1318
+
+## Objective
+
+Make the complete non-optional test suite for CPAN distribution
+`DCONWAY/PPR-0.001010` pass under PerlOnJava. The final integration gate is a
+bounded `./jcpan -t PPR` run with no failed test files or stuck harness
+processes.
+
+This work follows #1299, which fixed the Joni analysis hang while loading
+PPR's large dynamic named-subexpression grammar. Loading the module is no
+longer the acceptance boundary: PPR grammar matching must now be compatible.
+
+## Known Baseline
+
+On the merged #1317 source, `timeout 1800 ./jcpan -t PPR` configured and built
+PPR successfully, then exposed these failures:
+
+| Test | Observed symptom | Initial ownership hypothesis |
+| --- | --- | --- |
+| `t/blocks.t` | `Range [4, 0) out of bounds for length 61` | Joni matcher offset/capture state |
+| `t/control.t` | `Range [7, 0) out of bounds for length 49` | Joni matcher offset/capture state |
+| `t/for_ref_iterator.t` | `Range [1, 0) out of bounds for length 43` | Joni matcher offset/capture state |
+| `t/format.t` | `Range [1, 0) out of bounds for length 427` | Joni matcher offset/capture state |
+| `t/heredoc.t` | second heredoc block does not match; harness then stops advancing | grammar matching / progression state |
+
+The current baseline also proves that `t/00.load.t`, `t/decomment.t`,
+`t/decomment_heredoc_large.t`, `t/document_self.t`, `t/erudil.t`, and
+`t/eyedrops.t` passed before the stalled test. `t/disapproval.t` is an
+explicit optional-dependency skip.
+
+## Design Constraints
+
+- Keep native regex semantics in the Joni fork. Do not special-case PPR source
+ text, rewrite its patterns, or introduce a Java-regex fallback.
+- Reduce each failure into a project-owned test under
+ `src/test/resources/unit` before its fix is considered complete.
+- Run each new Perl-level test on system Perl first, then on JVM and
+ interpreter backends. Preserve positive and negative controls.
+- Treat a timeout or a `0/0` test record as a failure, not a skip.
+- Do not modify imported CPAN test files.
+
+## Work Plan
+
+### Phase 1: Establish the range-error owner
+
+1. Reduce the first `t/blocks.t` grammar fragment to a small named-subpattern
+ match.
+2. Capture JVM debug stack traces and compare JVM/interpreter outcomes.
+3. Locate the earliest invalid begin/end offset in Joni matcher, capture
+ publication, or PerlOnJava's byte-to-character conversion.
+4. Add the permanent reducer and correct the shared root cause.
+
+### Phase 2: Complete adjacent grammar failures
+
+1. Verify whether `control`, `for_ref_iterator`, and `format` share the Phase
+ 1 root cause.
+2. Add distinct reducers for any non-shared behavior.
+3. Test both matching and non-matching paths, including nested and empty
+ recursive paths where relevant.
+
+### Phase 3: Heredoc correctness and progress
+
+1. Reduce PPR's second `t/heredoc.t` fixture unchanged in semantic form.
+2. Establish the system Perl oracle and locate the first divergent regex stage.
+3. Fix the mismatch and any subsequent non-progress condition separately if
+ they have different causes.
+
+### Phase 4: Distribution acceptance
+
+1. Run focused PPR tests on both PerlOnJava backends.
+2. Run the complete bounded `./jcpan -t PPR` suite and inspect its full log.
+3. Run `make`, scan for warnings, and validate the exact clean candidate head.
+
+## Progress Tracking
+
+### Current Status: Phase 1 in progress
+
+### Completed Phases
+
+- [x] Phase 0: #1299 load-time analysis hang (2026-09-09)
+ - Replaced exponential named-subexpression recursion analysis with graph
+ analysis and retained dynamic-callout safety handling.
+ - Added `src/test/resources/unit/regex_large_named_grammar.t`.
+ - Merged in PR #1317.
+
+### Next Steps
+
+1. Capture the `t/blocks.t` JVM stack trace and create its smallest valid
+ project-owned reproducer.
+2. Compare the reducer on system Perl, JVM, and interpreter backends.
+3. Identify and repair the narrowest shared owner before proceeding to the
+ remaining PPR test files.
+
+### Open Questions
+
+- Are the four range errors one invalid region/capture-publication defect or
+ multiple matcher defects?
+- Does the `t/heredoc.t` stall remain after fixing the range-error family?
+
+## Related Work
+
+- Issue #1299: PPR grammar load-time Joni analysis hang.
+- Issue #1318: PPR compatibility follow-up and acceptance tracking.
+- `.agents/skills/debug-regex-engine/SKILL.md`.
From dbfc1ee14a51b04f3dba06db80d58ab3d8ee8925 Mon Sep 17 00:00:00 2001
From: "Flavio S. Glock"
Date: Wed, 9 Sep 2026 14:51:31 +0200
Subject: [PATCH 2/8] fix(regex): safely publish recursive unmatched captures
Treat unordered native capture offsets as nonparticipating captures and keep
recursive-call detection on the active call-frame chain. This clears PPR's
initial range-error family while preserving the remaining heredoc work for
issue #1318.
Generated with Codex (https://openai.com/codex)
Co-Authored-By: Codex
---
dev/design/ppr-compatibility.md | 28 +++++++++++++------
.../runtime/regex/JoniRegexPattern.java | 15 ++++++++--
.../runtime/regex/JoniRegexPatternTest.java | 7 +++++
.../joni/src/org/joni/ByteCodeMachine.java | 2 +-
third_party/joni/src/org/joni/StackEntry.java | 18 ++++++++++++
.../joni/src/org/joni/StackMachine.java | 26 ++++++++---------
6 files changed, 70 insertions(+), 26 deletions(-)
diff --git a/dev/design/ppr-compatibility.md b/dev/design/ppr-compatibility.md
index 3ad520b80..ef938b2eb 100644
--- a/dev/design/ppr-compatibility.md
+++ b/dev/design/ppr-compatibility.md
@@ -76,7 +76,7 @@ explicit optional-dependency skip.
## Progress Tracking
-### Current Status: Phase 1 in progress
+### Current Status: Phase 2 in progress
### Completed Phases
@@ -85,20 +85,30 @@ explicit optional-dependency skip.
analysis and retained dynamic-callout safety handling.
- Added `src/test/resources/unit/regex_large_named_grammar.t`.
- Merged in PR #1317.
+- [x] Phase 1: Capture-range publication (2026-09-09)
+ - Identified Joni's stale nonnegative begin / zero end sentinel as an
+ unmatched capture in PerlOnJava's adapter.
+ - Added direct adapter coverage and published it as `undef` rather than an
+ invalid Java substring range.
+ - This clears PPR's `blocks`, `control`, `for_ref_iterator`, and `format`
+ range-error family.
### Next Steps
-1. Capture the `t/blocks.t` JVM stack trace and create its smallest valid
- project-owned reproducer.
-2. Compare the reducer on system Perl, JVM, and interpreter backends.
-3. Identify and repair the narrowest shared owner before proceeding to the
- remaining PPR test files.
+1. Complete Phase 2 by running the remaining range-family PPR tests and
+ separating any non-shared failures.
+2. Reduce the second `t/heredoc.t` fixture, which now reports its expected
+ assertion failure and then times out in `ByteCodeMachine.opCall`.
+3. Compare that reducer on system Perl, JVM, and interpreter backends.
+4. Fix the remaining recursive-call execution path without weakening PPR's
+ grammar or introducing source-specific handling.
### Open Questions
-- Are the four range errors one invalid region/capture-publication defect or
- multiple matcher defects?
-- Does the `t/heredoc.t` stall remain after fixing the range-error family?
+- Why does the second heredoc fixture fail before the subsequent recursive-call
+ execution timeout?
+- Does the heredoc execution path need an additional semantic guard distinct
+ from call-frame lookup performance?
## Related Work
diff --git a/src/main/java/org/perlonjava/runtime/regex/JoniRegexPattern.java b/src/main/java/org/perlonjava/runtime/regex/JoniRegexPattern.java
index c3e040c17..08169ac97 100644
--- a/src/main/java/org/perlonjava/runtime/regex/JoniRegexPattern.java
+++ b/src/main/java/org/perlonjava/runtime/regex/JoniRegexPattern.java
@@ -53,6 +53,16 @@
/** Sole production adapter from Perl regex operations to the vendored Joni fork. */
final class JoniRegexPattern {
+ /**
+ * Joni records an unmatched capture with negative offsets in ordinary
+ * patterns. Deep recursive patterns can also leave a stale begin offset
+ * paired with the region's zero end sentinel. Perl exposes both forms as
+ * an unmatched capture, never as an invalid substring range.
+ */
+ static boolean isParticipatingCapture(int begin, int end) {
+ return begin >= 0 && end >= begin;
+ }
+
record DeferredPropertyFact(String name, String displayName,
CharacterPropertyResolver.Context context, int option,
int position, boolean negated) {}
@@ -1199,8 +1209,7 @@ public String group(int index) {
requireMatch();
int begin = index == 0 ? matcher.getBegin() : captures.getBeg(index);
int end = index == 0 ? matcher.getEnd() : captures.getEnd(index);
- if (begin < 0 || end < 0) return null;
- if (index == 0 && begin > end) return null;
+ if (!JoniRegexPattern.isParticipatingCapture(begin, end)) return null;
return input.substring(toCharOffset(begin), toCharOffset(end));
}
@@ -1211,7 +1220,7 @@ public String group(String name) {
if (physical == null) return group(namedGroupNumber(name));
int begin = matcher.physicalNamedCaptureBegin(physical);
int end = matcher.physicalNamedCaptureEnd(physical);
- if (begin < 0 || end < 0) return null;
+ if (!JoniRegexPattern.isParticipatingCapture(begin, end)) return null;
return input.substring(toCharOffset(begin), toCharOffset(end));
}
diff --git a/src/test/java/org/perlonjava/runtime/regex/JoniRegexPatternTest.java b/src/test/java/org/perlonjava/runtime/regex/JoniRegexPatternTest.java
index 26d8bdc13..0bb7552e5 100644
--- a/src/test/java/org/perlonjava/runtime/regex/JoniRegexPatternTest.java
+++ b/src/test/java/org/perlonjava/runtime/regex/JoniRegexPatternTest.java
@@ -16,6 +16,13 @@
class JoniRegexPatternTest {
private static final RegexFlags FLAGS = RegexFlags.fromModifiers("", "");
+ @Test
+ void malformedNativeCaptureRangeIsPublishedAsUnmatched() {
+ assertTrue(JoniRegexPattern.isParticipatingCapture(4, 4));
+ assertFalse(JoniRegexPattern.isParticipatingCapture(-1, -1));
+ assertFalse(JoniRegexPattern.isParticipatingCapture(4, 0));
+ }
+
@Test
void nativeDynamicCalloutSourceAndMetadataRemainVisible() {
String source = "(?{=DYNAMIC:0})";
diff --git a/third_party/joni/src/org/joni/ByteCodeMachine.java b/third_party/joni/src/org/joni/ByteCodeMachine.java
index a4c66d0c0..4626418d7 100644
--- a/third_party/joni/src/org/joni/ByteCodeMachine.java
+++ b/third_party/joni/src/org/joni/ByteCodeMachine.java
@@ -3943,7 +3943,7 @@ private void opReturn() {
StackEntry frame = returnFrame();
restoreCallFrameCaptureSnapshot(frame);
ip = frame.getCallFrameRetAddr();
- pushReturn();
+ pushReturn(frame);
}
private void opFail() {
diff --git a/third_party/joni/src/org/joni/StackEntry.java b/third_party/joni/src/org/joni/StackEntry.java
index c53c0afb7..786af5fbb 100644
--- a/third_party/joni/src/org/joni/StackEntry.java
+++ b/third_party/joni/src/org/joni/StackEntry.java
@@ -21,6 +21,8 @@
class StackEntry {
int type;
+ private int activeCallFrameHead = -1;
+ private int callFramePreviousHead = -1;
private int E1, E2, E3, E4;
private Object calloutToken;
private ByteCodeMachine.DynamicContinuation dynamicContinuation;
@@ -205,6 +207,22 @@ boolean getCallFrameRecursiveVisibility() {
return callFrameRecursiveVisibility;
}
+ void setActiveCallFrameHead(int head) {
+ activeCallFrameHead = head;
+ }
+
+ int getActiveCallFrameHead() {
+ return activeCallFrameHead;
+ }
+
+ void setCallFramePreviousHead(int head) {
+ callFramePreviousHead = head;
+ }
+
+ int getCallFramePreviousHead() {
+ return callFramePreviousHead;
+ }
+
/* absent position */
void setAbsentStr(int pos) {
E1 = pos;
diff --git a/third_party/joni/src/org/joni/StackMachine.java b/third_party/joni/src/org/joni/StackMachine.java
index b534095df..94cb567d6 100644
--- a/third_party/joni/src/org/joni/StackMachine.java
+++ b/third_party/joni/src/org/joni/StackMachine.java
@@ -162,6 +162,7 @@ private final StackEntry ensure1() {
if (stk >= stack.length) doubleStack();
StackEntry e = stack[stk];
if (e == null) stack[stk] = e = USE_CEC ? new SCStackEntry() : new StackEntry();
+ e.setActiveCallFrameHead(stk == 0 ? -1 : stack[stk - 1].getActiveCallFrameHead());
return e;
}
@@ -239,6 +240,7 @@ private void push(int type, int pat, int s, int prev, int pkeep) {
private final void pushEnsured(int type, int pat) {
StackEntry e = stack[stk];
+ e.setActiveCallFrameHead(stk == 0 ? -1 : stack[stk - 1].getActiveCallFrameHead());
e.type = type;
e.setStatePCode(pat);
if (USE_CEC) ((SCStackEntry)e).setStateCheck(0);
@@ -436,6 +438,8 @@ protected final void pushNullCheckEnd(int cnum) {
protected final void pushCallFrame(int pat, int groupNum, boolean snapshotCaptures,
boolean restoreCallerCaptures, boolean recursiveVisibility) {
StackEntry e = ensure1();
+ e.setCallFramePreviousHead(e.getActiveCallFrameHead());
+ e.setActiveCallFrameHead(stk);
e.type = CALL_FRAME;
e.setCallFrameRetAddr(pat);
e.setCallFrameNum(groupNum);
@@ -491,26 +495,22 @@ protected final void restoreCallFrameCaptureSnapshot(StackEntry frame) {
}
protected final boolean isInsideSubexpCall(int groupNum) {
- int returned = 0;
- for (int i = stk - 1; i >= 0; i--) {
- StackEntry e = stack[i];
- if (e.type == RETURN) {
- returned++;
- } else if (e.type == CALL_FRAME) {
- if (returned > 0) {
- returned--;
- } else if (e.getCallFrameNum() >= 0
- && (groupNum == 0 || e.getCallFrameNum() == groupNum)) {
- return true;
- }
+ int callFrame = stk == 0 ? -1 : stack[stk - 1].getActiveCallFrameHead();
+ while (callFrame >= 0) {
+ StackEntry e = stack[callFrame];
+ if (e.getCallFrameNum() >= 0
+ && (groupNum == 0 || e.getCallFrameNum() == groupNum)) {
+ return true;
}
+ callFrame = e.getCallFramePreviousHead();
}
return false;
}
- protected final void pushReturn() {
+ protected final void pushReturn(StackEntry frame) {
StackEntry e = ensure1();
e.type = RETURN;
+ e.setActiveCallFrameHead(frame.getCallFramePreviousHead());
stk++;
}
From 697b80e20c7727a8c56f0471376e832bdfd528e1 Mon Sep 17 00:00:00 2001
From: "Flavio S. Glock"
Date: Wed, 9 Sep 2026 19:24:27 +0200
Subject: [PATCH 3/8] fix(regex): bound pathological backtracking memory
Retain PPR's recursive grammar callbacks and capture state while bounding
Joni's heap-backed backtracking stack. Exhaustion now follows the existing
Perl recursion-exhaustion behavior rather than consuming the JVM heap.
Generated with Codex (https://openai.com/codex/)
Co-Authored-By: Codex
---
.../runtime/regex/JoniRegexPattern.java | 176 +++++++++++++-----
.../runtime/regex/RuntimeRegex.java | 22 +++
.../runtimetypes/HashSpecialVariable.java | 44 ++++-
.../runtime/runtimetypes/RegexState.java | 6 +
.../runtimetypes/RuntimeRegexState.java | 16 ++
.../regex_callout_provisional_capture_state.t | 19 ++
.../joni/src/org/joni/ByteCodeMachine.java | 131 ++++++++++++-
third_party/joni/src/org/joni/Config.java | 4 +
third_party/joni/src/org/joni/MatchView.java | 12 +-
third_party/joni/src/org/joni/StackEntry.java | 17 +-
.../joni/src/org/joni/StackMachine.java | 32 ++--
11 files changed, 391 insertions(+), 88 deletions(-)
create mode 100644 src/test/resources/unit/regex_callout_provisional_capture_state.t
diff --git a/src/main/java/org/perlonjava/runtime/regex/JoniRegexPattern.java b/src/main/java/org/perlonjava/runtime/regex/JoniRegexPattern.java
index 08169ac97..641025537 100644
--- a/src/main/java/org/perlonjava/runtime/regex/JoniRegexPattern.java
+++ b/src/main/java/org/perlonjava/runtime/regex/JoniRegexPattern.java
@@ -38,6 +38,7 @@
import java.util.ArrayDeque;
import java.util.Iterator;
import java.util.ArrayList;
+import java.util.Arrays;
import java.util.Collections;
import java.util.LinkedHashMap;
import java.util.Map;
@@ -72,6 +73,7 @@ record DeferredPropertyFact(String name, String displayName,
"Both or neither range ends should be Unicode";
private static final int INPUT_ENCODING_CACHE_ENTRIES = 512;
private static final int INPUT_ENCODING_CACHE_MAX_LENGTH = 8_192;
+ private static final int DYNAMIC_PATTERN_CACHE_ENTRIES = 128;
private static final Map INPUT_ENCODINGS = inputEncodingCache();
private static final Map BYTE_INPUT_ENCODINGS = inputEncodingCache();
private static final Map SUBJECT_INPUT_ENCODINGS =
@@ -953,7 +955,8 @@ private boolean find(int option, boolean anchored) {
}
if (!callbacks.isEmpty()) {
calloutHandler = new PerlCalloutHandler(
- input, byteToChar, callbacks, flags, hasControlVerbState, byteMode, subject);
+ input, byteToChar, callbacks, namedGroups, flags,
+ hasControlVerbState, byteMode, subject);
matcher.setCalloutHandler(calloutHandler);
}
int result;
@@ -976,7 +979,7 @@ private boolean find(int option, boolean anchored) {
}
if (!callbacks.isEmpty()) {
calloutHandler = new PerlCalloutHandler(
- input, byteToChar, callbacks, flags,
+ input, byteToChar, callbacks, namedGroups, flags,
hasControlVerbState, byteMode, subject);
matcher.setCalloutHandler(calloutHandler);
}
@@ -1319,34 +1322,81 @@ private static int[] buildByteToChar(String input, int byteLength, int[] charToB
}
private static final class PerlCalloutHandler implements CalloutHandler {
+ private record DynamicPatternCacheKey(String source, RegexFlags flags,
+ boolean compileAsBytes,
+ String userPropertyPackage) {}
+
private record Token(int localLevel, RegexState regexState, RuntimeScalar previousR,
RuntimeScalar result, boolean block, boolean dynamic,
CaptureSnapshot previousDynamicView) {}
- private record CaptureSnapshot(int position, int[] begins, int[] ends,
- int lastClosed, String controlMark) implements MatchView {
+ /**
+ * Dynamic callbacks only need to remember captures that were open at
+ * the callback position. Retaining every offset made large recursive
+ * grammars copy and resolve all captures twice per callback.
+ */
+ private record CaptureSnapshot(int position, int[] openAtPosition) {
+ boolean hasOpenCapture(int capture) {
+ for (int openCapture : openAtPosition) {
+ if (openCapture == capture) return true;
+ }
+ return false;
+ }
+
static CaptureSnapshot of(MatchView match) {
int count = match.captureCount();
- int[] begins = new int[count + 1];
- int[] ends = new int[count + 1];
- for (int capture = 0; capture <= count; capture++) {
- begins[capture] = match.captureBegin(capture);
- ends[capture] = match.captureEnd(capture);
+ int position = match.currentBytePosition();
+ int[] directOpenCaptures = match.openCapturesAtCurrentPosition();
+ if (directOpenCaptures != null) {
+ return new CaptureSnapshot(position, directOpenCaptures);
+ }
+ int[] openAtPosition = new int[Math.min(count, 8)];
+ int openCount = 0;
+ MatchView.CaptureOffsets[] allOffsets = match.captureOffsets();
+ for (int capture = 1; capture <= count; capture++) {
+ MatchView.CaptureOffsets offsets = allOffsets == null
+ ? match.captureOffsets(capture) : allOffsets[capture];
+ int begin = offsets.begin();
+ int end = offsets.end();
+ if (begin == position && (end < 0 || end == position)) {
+ if (openCount == openAtPosition.length) {
+ openAtPosition = Arrays.copyOf(openAtPosition,
+ openAtPosition.length << 1);
+ }
+ openAtPosition[openCount++] = capture;
+ }
}
- return new CaptureSnapshot(match.currentBytePosition(), begins, ends,
- match.lastClosedCapture(), match.controlMark());
+ return new CaptureSnapshot(position,
+ openCount == openAtPosition.length ? openAtPosition
+ : Arrays.copyOf(openAtPosition, openCount));
+ }
+ }
+
+ private record DynamicCaptureView(MatchView current, CaptureSnapshot previous)
+ implements MatchView {
+ private boolean restoresPreviousStart(int capture) {
+ int position = current.currentBytePosition();
+ return current.captureBegin(capture) == position
+ && current.captureEnd(capture) == position
+ && previous.hasOpenCapture(capture);
}
- @Override public int currentBytePosition() { return position; }
- @Override public int captureCount() { return begins.length - 1; }
- @Override public int captureBegin(int capture) { return begins[capture]; }
- @Override public int captureEnd(int capture) { return ends[capture]; }
- @Override public int lastClosedCapture() { return lastClosed; }
+ @Override public int currentBytePosition() { return current.currentBytePosition(); }
+ @Override public int captureCount() { return current.captureCount(); }
+ @Override public int captureBegin(int capture) {
+ return restoresPreviousStart(capture) ? previous.position()
+ : current.captureBegin(capture);
+ }
+ @Override public int captureEnd(int capture) { return current.captureEnd(capture); }
+ @Override public int lastClosedCapture() { return current.lastClosedCapture(); }
+ @Override public String controlMark() { return current.controlMark(); }
}
private final String input;
private final int[] byteToChar;
private final List callbacks;
+ private final Map namedGroups;
+ private final Map> provisionalNamedCaptureGroups;
private final RegexFlags outerFlags;
private final boolean publishesControlVerbState;
private final boolean byteMode;
@@ -1364,21 +1414,33 @@ static CaptureSnapshot of(MatchView match) {
private final ArrayDeque callbackMutations =
new ArrayDeque<>();
private boolean preserveCallbackMutations;
+ private final Map dynamicPatternCache =
+ new LinkedHashMap<>(16, 0.75f, true) {
+ @Override
+ protected boolean removeEldestEntry(
+ Map.Entry eldest) {
+ return size() > DYNAMIC_PATTERN_CACHE_ENTRIES;
+ }
+ };
PerlCalloutHandler(String input, int[] byteToChar, List callbacks,
+ Map namedGroups,
RegexFlags outerFlags, boolean publishesControlVerbState,
boolean byteMode, RuntimeScalar subject) {
- this(input, byteToChar, callbacks, outerFlags, publishesControlVerbState,
+ this(input, byteToChar, callbacks, namedGroups, outerFlags, publishesControlVerbState,
byteMode, subject, null);
}
private PerlCalloutHandler(
String input, int[] byteToChar, List callbacks,
+ Map namedGroups,
RegexFlags outerFlags, boolean publishesControlVerbState,
boolean byteMode, RuntimeScalar subject, PerlCalloutHandler parent) {
this.input = input;
this.byteToChar = byteToChar;
this.callbacks = callbacks;
+ this.namedGroups = namedGroups;
+ this.provisionalNamedCaptureGroups = buildProvisionalNamedCaptureGroups(namedGroups);
this.outerFlags = outerFlags;
this.publishesControlVerbState = publishesControlVerbState;
this.byteMode = byteMode;
@@ -1470,11 +1532,17 @@ public DynamicPatternResult executeDynamic(
boolean compileAsBytes = byteMode
&& (byteBackedDynamic || latin1Dynamic);
inputEncodingCompatible = !byteMode || latin1Dynamic;
- nestedPattern = UnicodeResolver.withUserPropertyPackage(
- dynamicPackage,
- () -> new JoniRegexPattern(dynamicSource,
- scopedFlags, 0, compileAsBytes,
- compileAsBytes, compileAsBytes));
+ DynamicPatternCacheKey cacheKey = new DynamicPatternCacheKey(
+ dynamicSource, scopedFlags, compileAsBytes, dynamicPackage);
+ nestedPattern = dynamicPatternCache.get(cacheKey);
+ if (nestedPattern == null) {
+ nestedPattern = UnicodeResolver.withUserPropertyPackage(
+ dynamicPackage,
+ () -> new JoniRegexPattern(dynamicSource,
+ scopedFlags, 0, compileAsBytes,
+ compileAsBytes, compileAsBytes));
+ dynamicPatternCache.put(cacheKey, nestedPattern);
+ }
} catch (SyntaxException exception) {
String message = exception.getMessage();
if (message != null && (message.contains("premature end of char-class")
@@ -1490,6 +1558,7 @@ public DynamicPatternResult executeDynamic(
nestedPattern.materializeDefinedDeferredProperties();
CalloutHandler nestedHandler = nestedCallbacks.isEmpty() ? null
: new PerlCalloutHandler(input, byteToChar, nestedCallbacks,
+ nestedPattern.namedGroups,
value.value instanceof RuntimeRegex runtimeRegex
&& runtimeRegex.getRegexFlags() != null
? runtimeRegex.getRegexFlags() : scopedFlags,
@@ -1545,14 +1614,18 @@ private Evaluation evaluate(RuntimeRegexCallback callback, MatchView match) {
}
}
CaptureSnapshot priorDynamicView = previousDynamicView;
+ boolean dynamicReadsCaptures = callback.kind == RuntimeRegexCallback.Kind.DYNAMIC
+ && callbackReadsCaptures(callback);
if (callback.kind.isBlock() && parent == null) {
callbackMutations.addLast(RegexCallbackMutationSnapshot.capture(callback.code));
}
- MatchView provisional = callback.kind == RuntimeRegexCallback.Kind.DYNAMIC
+ MatchView provisional = dynamicReadsCaptures
? dynamicCaptureView(match, priorDynamicView) : match;
publishProvisional(provisional);
- if (callback.kind == RuntimeRegexCallback.Kind.DYNAMIC) {
+ if (dynamicReadsCaptures) {
previousDynamicView = CaptureSnapshot.of(match);
+ } else if (callback.kind == RuntimeRegexCallback.Kind.DYNAMIC) {
+ previousDynamicView = null;
}
var callbackLocations = PerlRuntime.current().executionState()
.activeRegexCallbackLocations;
@@ -1619,6 +1692,27 @@ private Evaluation evaluate(RuntimeRegexCallback callback, MatchView match) {
}
}
+ private static boolean callbackReadsCaptures(RuntimeRegexCallback callback) {
+ String source = callback.source;
+ if (source == null || source.isEmpty()) return false;
+ return source.matches("(?s).*\\$(?:[0-9]|\\^N|[&`']|[+\\-]\\{|[+\\-](?![A-Za-z_])).*")
+ || source.contains("@-") || source.contains("@+")
+ || source.contains("%-") || source.contains("%+");
+ }
+
+ private static Map> buildProvisionalNamedCaptureGroups(
+ Map namedGroups) {
+ Map> groups = new LinkedHashMap<>();
+ for (Map.Entry entry : namedGroups.entrySet()) {
+ String encodedName = entry.getKey();
+ if (CaptureNameEncoder.isInternalCapture(encodedName)) continue;
+ groups.computeIfAbsent(CaptureNameEncoder.decodeGroupName(encodedName),
+ ignored -> new ArrayList<>()).add(entry.getValue());
+ }
+ return groups;
+ }
+
+
@Override
public void unwind(Object value) {
restore((Token) value, false);
@@ -1708,19 +1802,8 @@ private void restoreCallbackMutations() {
private static MatchView dynamicCaptureView(
MatchView current, CaptureSnapshot previous) {
- CaptureSnapshot adjusted = CaptureSnapshot.of(current);
- if (previous == null || previous.position() >= adjusted.position()) return adjusted;
- for (int capture = 1; capture <= adjusted.captureCount(); capture++) {
- if (adjusted.begins()[capture] == adjusted.position()
- && adjusted.ends()[capture] == adjusted.position()
- && previous.begins()[capture] == previous.position()
- && (previous.ends()[capture] < 0
- || previous.ends()[capture] == previous.position())) {
- adjusted.begins()[capture] = previous.position();
- adjusted.ends()[capture] = adjusted.position();
- }
- }
- return adjusted;
+ if (previous == null || previous.position() >= current.currentBytePosition()) return current;
+ return new DynamicCaptureView(current, previous);
}
private static void restoreCallbackScope(int localLevel, RegexState regexState,
@@ -1770,22 +1853,21 @@ private void publishProvisional(MatchView match) {
state.lastCaptureGroups = new String[count];
state.manualCaptureStarts = new int[count];
state.manualCaptureEnds = new int[count];
- for (int group = 1; group <= count; group++) {
+ Arrays.fill(state.manualCaptureStarts, Integer.MIN_VALUE);
+ Arrays.fill(state.manualCaptureEnds, Integer.MIN_VALUE);
+ state.provisionalCaptureResolver = group -> {
int begin = charOffset(match.captureBegin(group));
int end = charOffset(match.captureEnd(group));
if (begin < 0 || end < begin) {
- state.manualCaptureStarts[group - 1] = -1;
- state.manualCaptureEnds[group - 1] = -1;
- state.lastCaptureGroups[group - 1] = null;
- } else {
- state.manualCaptureStarts[group - 1] = begin;
- state.manualCaptureEnds[group - 1] = end;
- state.lastCaptureGroups[group - 1] = input.substring(begin, end);
+ return new RuntimeRegexState.ProvisionalCapture(null, -1, -1);
}
- }
+ return new RuntimeRegexState.ProvisionalCapture(input.substring(begin, end), begin, end);
+ };
+ state.lastNamedCaptureGroups = new LinkedHashMap<>();
+ state.provisionalNamedCaptureGroups = provisionalNamedCaptureGroups;
int lastClosed = match.lastClosedCapture();
state.lastClosedCapture = lastClosed > 0 && lastClosed <= count
- ? state.lastCaptureGroups[lastClosed - 1] : null;
+ ? RuntimeRegex.captureString(lastClosed) : null;
if (publishesControlVerbState || match.controlMark() != null) {
RuntimeRegex.updateControlVerbVariables(match.controlMark(), null);
}
diff --git a/src/main/java/org/perlonjava/runtime/regex/RuntimeRegex.java b/src/main/java/org/perlonjava/runtime/regex/RuntimeRegex.java
index 9b0a2b14c..675d68e51 100644
--- a/src/main/java/org/perlonjava/runtime/regex/RuntimeRegex.java
+++ b/src/main/java/org/perlonjava/runtime/regex/RuntimeRegex.java
@@ -3122,6 +3122,7 @@ public static RuntimeBase matchRegexBytes(RuntimeScalar quotedRegex, RuntimeScal
private static void updateLastNamedCaptureGroups(RegexMatcher matcher) {
RuntimeRegexState regexState = state();
+ regexState.provisionalNamedCaptureGroups = null;
Map namedGroups = matcher.namedGroups();
Map> byPerlName = new LinkedHashMap<>();
if (namedGroups == null || namedGroups.isEmpty()) {
@@ -3159,6 +3160,8 @@ private static void updateNumberedCaptureGroups(RegexMatcher matcher) {
regexState.lastParenMatchOverride = null;
regexState.manualCaptureStarts = null;
regexState.manualCaptureEnds = null;
+ regexState.provisionalCaptureResolver = null;
+ regexState.provisionalNamedCaptureGroups = null;
int captureCount = matcher.groupCount();
int lastClosedCapture = matcher.lastClosedCapture();
regexState.lastClosedCapture = lastClosedCapture > 0
@@ -4094,6 +4097,7 @@ public static String captureString(int group) {
if (state().lastCaptureGroups == null || group > state().lastCaptureGroups.length) {
return null;
}
+ materializeProvisionalCapture(group);
return state().lastCaptureGroups[group - 1];
}
@@ -4105,6 +4109,7 @@ public static String lastCaptureString() {
// in the match (i.e., is non-null). Non-participating groups in alternations
// have null values from Java's Matcher.group().
for (int i = state().lastCaptureGroups.length - 1; i >= 0; i--) {
+ materializeProvisionalCapture(i + 1);
if (state().lastCaptureGroups[i] != null) {
return state().lastCaptureGroups[i];
}
@@ -4133,6 +4138,7 @@ public static RuntimeScalar matcherStart(int group) {
return publicMatcherOffset(state().lastMatchStart);
}
if (state().manualCaptureStarts != null && group > 0 && group <= state().manualCaptureStarts.length) {
+ materializeProvisionalCapture(group);
return publicMatcherOffset(state().manualCaptureStarts[group - 1]);
}
if (state().globalMatcher == null) {
@@ -4157,6 +4163,7 @@ public static RuntimeScalar matcherEnd(int group) {
return publicMatcherOffset(state().lastMatchEnd);
}
if (state().manualCaptureEnds != null && group > 0 && group <= state().manualCaptureEnds.length) {
+ materializeProvisionalCapture(group);
return publicMatcherOffset(state().manualCaptureEnds[group - 1]);
}
if (state().globalMatcher == null) {
@@ -4200,6 +4207,21 @@ public static int matcherSize() {
return size + 1;
}
+ private static void materializeProvisionalCapture(int group) {
+ RuntimeRegexState regexState = state();
+ RuntimeRegexState.ProvisionalCaptureResolver resolver =
+ regexState.provisionalCaptureResolver;
+ if (resolver == null || regexState.manualCaptureStarts == null
+ || group <= 0 || group > regexState.manualCaptureStarts.length
+ || regexState.manualCaptureStarts[group - 1] != Integer.MIN_VALUE) {
+ return;
+ }
+ RuntimeRegexState.ProvisionalCapture capture = resolver.resolve(group);
+ regexState.lastCaptureGroups[group - 1] = capture.value();
+ regexState.manualCaptureStarts[group - 1] = capture.start();
+ regexState.manualCaptureEnds[group - 1] = capture.end();
+ }
+
/** Perl trims trailing non-participating captures from {@code @-}, but not {@code @+}. */
public static int matcherStartSize() {
int size = matcherSize();
diff --git a/src/main/java/org/perlonjava/runtime/runtimetypes/HashSpecialVariable.java b/src/main/java/org/perlonjava/runtime/runtimetypes/HashSpecialVariable.java
index 7598ae3ae..39b5485d3 100644
--- a/src/main/java/org/perlonjava/runtime/runtimetypes/HashSpecialVariable.java
+++ b/src/main/java/org/perlonjava/runtime/runtimetypes/HashSpecialVariable.java
@@ -83,7 +83,7 @@ public static RuntimeHash getStash(String namespace) {
public Set> entrySet() {
Set> entries = new HashSet<>();
if (this.mode == Id.CAPTURE_ALL || this.mode == Id.CAPTURE) {
- Map> namedCaptures = PerlRuntime.current().regexState.lastNamedCaptureGroups;
+ Map> namedCaptures = namedCaptures();
if (namedCaptures != null) {
for (Map.Entry> e : namedCaptures.entrySet()) {
if (this.mode == Id.CAPTURE_ALL) {
@@ -119,9 +119,8 @@ public Set> entrySet() {
@Override
public RuntimeScalar get(Object key) {
if (this.mode == Id.CAPTURE_ALL || this.mode == Id.CAPTURE) {
- Map> namedCaptures = PerlRuntime.current().regexState.lastNamedCaptureGroups;
- if (namedCaptures != null && key instanceof String name) {
- List captures = namedCaptures.get(name);
+ if (key instanceof String name) {
+ List captures = namedCapturesFor(name);
if (captures == null) return scalarUndef;
if (this.mode == Id.CAPTURE_ALL) {
return captureAllArrayRef(captures);
@@ -148,15 +147,17 @@ public RuntimeScalar get(Object key) {
@Override
public boolean containsKey(Object key) {
if (this.mode == Id.CAPTURE_ALL) {
- // For %-, all named groups exist (even non-participating ones)
- Map> namedCaptures = PerlRuntime.current().regexState.lastNamedCaptureGroups;
- return namedCaptures != null && key instanceof String name && namedCaptures.containsKey(name);
+ // For %-, all named groups exist (even non-participating ones).
+ if (!(key instanceof String name)) return false;
+ RuntimeRegexState state = PerlRuntime.current().regexState;
+ return namedCapturesFor(name) != null
+ || state.provisionalNamedCaptureGroups != null
+ && state.provisionalNamedCaptureGroups.containsKey(name);
}
if (this.mode == Id.CAPTURE) {
// For %+, only groups that actually captured
- Map> namedCaptures = PerlRuntime.current().regexState.lastNamedCaptureGroups;
- if (namedCaptures != null && key instanceof String name) {
- List captures = namedCaptures.get(name);
+ if (key instanceof String name) {
+ List captures = namedCapturesFor(name);
return captures != null && captures.stream().anyMatch(v -> v != null);
}
return false;
@@ -168,6 +169,29 @@ public boolean containsKey(Object key) {
return super.containsKey(key);
}
+ private Map> namedCaptures() {
+ RuntimeRegexState state = PerlRuntime.current().regexState;
+ if (state.provisionalNamedCaptureGroups != null) {
+ for (String name : state.provisionalNamedCaptureGroups.keySet()) {
+ namedCapturesFor(name);
+ }
+ }
+ return state.lastNamedCaptureGroups;
+ }
+
+ private List namedCapturesFor(String name) {
+ RuntimeRegexState state = PerlRuntime.current().regexState;
+ List captures = state.lastNamedCaptureGroups == null
+ ? null : state.lastNamedCaptureGroups.get(name);
+ if (captures != null || state.provisionalNamedCaptureGroups == null) return captures;
+ List groups = state.provisionalNamedCaptureGroups.get(name);
+ if (groups == null) return null;
+ captures = new ArrayList<>(groups.size());
+ for (int group : groups) captures.add(RuntimeRegex.captureString(group));
+ state.lastNamedCaptureGroups.put(name, captures);
+ return captures;
+ }
+
@Override
public Set keySet() {
if (this.mode != Id.STASH) {
diff --git a/src/main/java/org/perlonjava/runtime/runtimetypes/RegexState.java b/src/main/java/org/perlonjava/runtime/runtimetypes/RegexState.java
index 8aa93d9ff..9c85df25e 100644
--- a/src/main/java/org/perlonjava/runtime/runtimetypes/RegexState.java
+++ b/src/main/java/org/perlonjava/runtime/runtimetypes/RegexState.java
@@ -26,10 +26,12 @@ public class RegexState implements DynamicState {
private final boolean lastParenMatchOverrideActive;
private final String lastParenMatchOverride;
private final Map> lastNamedCaptureGroups;
+ private final Map> provisionalNamedCaptureGroups;
private final boolean lastMatchWasByteString;
private final boolean lastMatchResultsTainted;
private final int[] manualCaptureStarts;
private final int[] manualCaptureEnds;
+ private final RuntimeRegexState.ProvisionalCaptureResolver provisionalCaptureResolver;
public RegexState() {
owner = PerlRuntime.current();
@@ -51,10 +53,12 @@ public RegexState() {
lastParenMatchOverrideActive = state.lastParenMatchOverrideActive;
lastParenMatchOverride = state.lastParenMatchOverride;
lastNamedCaptureGroups = state.lastNamedCaptureGroups;
+ provisionalNamedCaptureGroups = state.provisionalNamedCaptureGroups;
lastMatchWasByteString = state.lastMatchWasByteString;
lastMatchResultsTainted = state.lastMatchResultsTainted;
manualCaptureStarts = state.manualCaptureStarts;
manualCaptureEnds = state.manualCaptureEnds;
+ provisionalCaptureResolver = state.provisionalCaptureResolver;
}
public static void save() {
@@ -98,9 +102,11 @@ public void dynamicRestoreState() {
state.lastParenMatchOverrideActive = lastParenMatchOverrideActive;
state.lastParenMatchOverride = lastParenMatchOverride;
state.lastNamedCaptureGroups = lastNamedCaptureGroups;
+ state.provisionalNamedCaptureGroups = provisionalNamedCaptureGroups;
state.lastMatchWasByteString = lastMatchWasByteString;
state.lastMatchResultsTainted = lastMatchResultsTainted;
state.manualCaptureStarts = manualCaptureStarts;
state.manualCaptureEnds = manualCaptureEnds;
+ state.provisionalCaptureResolver = provisionalCaptureResolver;
}
}
diff --git a/src/main/java/org/perlonjava/runtime/runtimetypes/RuntimeRegexState.java b/src/main/java/org/perlonjava/runtime/runtimetypes/RuntimeRegexState.java
index 357667501..9407d7575 100644
--- a/src/main/java/org/perlonjava/runtime/runtimetypes/RuntimeRegexState.java
+++ b/src/main/java/org/perlonjava/runtime/runtimetypes/RuntimeRegexState.java
@@ -18,6 +18,17 @@
* or {@code /o} state.
*/
public final class RuntimeRegexState {
+ /**
+ * Resolves a capture while a regex callback is executing. The matcher is
+ * paused during that callback, so individual captures can be materialized
+ * only when Perl code actually reads them.
+ */
+ public interface ProvisionalCaptureResolver {
+ ProvisionalCapture resolve(int group);
+ }
+
+ public record ProvisionalCapture(String value, int start, int end) {}
+
public static final int MAX_REGEX_CACHE_SIZE = 1000;
static final int MAX_POSITION_CACHE_SIZE = 1000;
@@ -38,10 +49,13 @@ public final class RuntimeRegexState {
public boolean lastParenMatchOverrideActive;
public String lastParenMatchOverride;
public Map> lastNamedCaptureGroups;
+ /** Named groups available from a paused callback matcher, by Perl name. */
+ public Map> provisionalNamedCaptureGroups;
public boolean lastMatchWasByteString;
public boolean lastMatchResultsTainted;
public int[] manualCaptureStarts;
public int[] manualCaptureEnds;
+ public ProvisionalCaptureResolver provisionalCaptureResolver;
/** Per-runtime locale publication used by matcher-time /l resolution. */
public final RuntimeLocaleState localeState = new RuntimeLocaleState();
@@ -109,10 +123,12 @@ public void clearMatchState() {
lastParenMatchOverrideActive = false;
lastParenMatchOverride = null;
lastNamedCaptureGroups = null;
+ provisionalNamedCaptureGroups = null;
lastMatchWasByteString = false;
lastMatchResultsTainted = false;
manualCaptureStarts = null;
manualCaptureEnds = null;
+ provisionalCaptureResolver = null;
}
/**
diff --git a/src/test/resources/unit/regex_callout_provisional_capture_state.t b/src/test/resources/unit/regex_callout_provisional_capture_state.t
new file mode 100644
index 000000000..9e5c9edb7
--- /dev/null
+++ b/src/test/resources/unit/regex_callout_provisional_capture_state.t
@@ -0,0 +1,19 @@
+use strict;
+use warnings;
+use Test::More;
+use re 'eval';
+
+my ($named, $numbered, $start, $end);
+ok 'ok' =~ /(?ok)(?{
+ $named = $+{word};
+ $numbered = $1;
+ $start = $-[1];
+ $end = $+[1];
+})/, 'dynamic callback matches';
+
+is $named, 'ok', 'dynamic callback sees its named capture';
+is $numbered, 'ok', 'dynamic callback sees its numbered capture';
+is $start, 0, 'dynamic callback sees capture start';
+is $end, 2, 'dynamic callback sees capture end';
+
+done_testing;
diff --git a/third_party/joni/src/org/joni/ByteCodeMachine.java b/third_party/joni/src/org/joni/ByteCodeMachine.java
index 4626418d7..5ab2ddc6e 100644
--- a/third_party/joni/src/org/joni/ByteCodeMachine.java
+++ b/third_party/joni/src/org/joni/ByteCodeMachine.java
@@ -3458,12 +3458,7 @@ private void opCall() {
}
private int activeSubexpCallDepth() {
- int depth = 0;
- for (int i = 0; i < stk; i++) {
- if (stack[i].type == CALL_FRAME) depth++;
- else if (stack[i].type == RETURN) depth--;
- }
- return depth;
+ return stk == 0 ? 0 : stack[stk - 1].getActiveCallDepth();
}
private void opCallout() {
@@ -3756,6 +3751,113 @@ public int captureEnd(int capture) {
return (bsAt(regex.btMemEnd, capture) ? stack[value].getMemPStr() : value) - str;
}
+ @Override
+ public MatchView.CaptureOffsets captureOffsets(int capture) {
+ checkCapture(capture);
+ if (capture == 0) return new MatchView.CaptureOffsets(sstart - str, s - str);
+ // A completed recursive call has caller-snapshot visibility rules;
+ // retain the established individual resolver in that representation.
+ if (completedRecursiveCall() != null) {
+ return new MatchView.CaptureOffsets(captureBegin(capture), captureEnd(capture));
+ }
+ int begin = repeatStk[memStartStk + capture];
+ int end = repeatStk[memEndStk + capture];
+ if (end == INVALID_INDEX && begin != INVALID_INDEX) {
+ CapturePointers previous = previousClosedCapturePointers(capture);
+ if (previous.begin != INVALID_INDEX) begin = previous.begin;
+ if (previous.end != INVALID_INDEX) end = previous.end;
+ }
+ int beginOffset = begin == INVALID_INDEX ? INVALID_INDEX
+ : (bsAt(regex.btMemStart, capture) ? stack[begin].getMemPStr() : begin) - str;
+ int endOffset = end == INVALID_INDEX ? INVALID_INDEX
+ : (bsAt(regex.btMemEnd, capture) ? stack[end].getMemPStr() : end) - str;
+ return new MatchView.CaptureOffsets(beginOffset, endOffset);
+ }
+
+ @Override
+ public MatchView.CaptureOffsets[] captureOffsets() {
+ if (completedRecursiveCall() != null) return null;
+ int count = regex.numMem;
+ int[] previousStarts = new int[count + 1];
+ int[] previousEnds = new int[count + 1];
+ java.util.Arrays.fill(previousStarts, INVALID_INDEX);
+ java.util.Arrays.fill(previousEnds, INVALID_INDEX);
+ boolean[] needsPrevious = new boolean[count + 1];
+ int unresolvedPrevious = 0;
+ for (int capture = 1; capture <= count; capture++) {
+ needsPrevious[capture] = repeatStk[memEndStk + capture] == INVALID_INDEX
+ && repeatStk[memStartStk + capture] != INVALID_INDEX;
+ if (needsPrevious[capture]) unresolvedPrevious++;
+ }
+ for (int i = stk - 1; i >= 0; i--) {
+ StackEntry entry = stack[i];
+ int capture = entry.type == MEM_END || entry.type == MEM_START
+ ? entry.getMemNum() : -1;
+ if (capture <= 0 || capture > count || !needsPrevious[capture]) continue;
+ if (entry.type == MEM_END && previousEnds[capture] == INVALID_INDEX) {
+ previousEnds[capture] = i;
+ } else if (entry.type == MEM_START && previousEnds[capture] != INVALID_INDEX
+ && previousStarts[capture] == INVALID_INDEX) {
+ previousStarts[capture] = i;
+ if (--unresolvedPrevious == 0) break;
+ }
+ }
+ MatchView.CaptureOffsets[] offsets = new MatchView.CaptureOffsets[count + 1];
+ offsets[0] = new MatchView.CaptureOffsets(sstart - str, s - str);
+ for (int capture = 1; capture <= count; capture++) {
+ int begin = repeatStk[memStartStk + capture];
+ int end = repeatStk[memEndStk + capture];
+ if (previousStarts[capture] != INVALID_INDEX) begin = previousStarts[capture];
+ if (previousEnds[capture] != INVALID_INDEX) end = previousEnds[capture];
+ int beginOffset = begin == INVALID_INDEX ? INVALID_INDEX
+ : (bsAt(regex.btMemStart, capture) ? stack[begin].getMemPStr() : begin) - str;
+ int endOffset = end == INVALID_INDEX ? INVALID_INDEX
+ : (bsAt(regex.btMemEnd, capture) ? stack[end].getMemPStr() : end) - str;
+ offsets[capture] = new MatchView.CaptureOffsets(beginOffset, endOffset);
+ }
+ return offsets;
+ }
+
+ @Override
+ public int[] openCapturesAtCurrentPosition() {
+ if (completedRecursiveCall() != null) return null;
+ int position = s - str;
+ int[] open = new int[Math.min(regex.numMem, 8)];
+ int openCount = 0;
+ for (int capture = 1; capture <= regex.numMem; capture++) {
+ int begin = repeatStk[memStartStk + capture];
+ int end = repeatStk[memEndStk + capture];
+ if (end == INVALID_INDEX && begin != INVALID_INDEX
+ && bsAt(regex.btMemStart, capture)) {
+ // The active MEM_START records the visible capture state that
+ // preceded this open iteration. It is the same state the
+ // general resolver finds by walking back to the prior matched
+ // MEM_END/MEM_START pair, but is available in constant time.
+ StackEntry activeStart = stack[begin];
+ if (activeStart.type == MEM_START
+ && activeStart.getMemEnd() != INVALID_INDEX) {
+ begin = activeStart.getMemStart();
+ end = activeStart.getMemEnd();
+ }
+ }
+ int beginOffset = capturePointerOffset(capture, begin, true);
+ int endOffset = capturePointerOffset(capture, end, false);
+ if (beginOffset == position && (endOffset < 0 || endOffset == position)) {
+ if (openCount == open.length) {
+ open = java.util.Arrays.copyOf(open, open.length << 1);
+ }
+ open[openCount++] = capture;
+ }
+ }
+ return openCount == open.length ? open : java.util.Arrays.copyOf(open, openCount);
+ }
+
+ private int capturePointerOffset(int capture, int pointer, boolean begin) {
+ if (pointer == INVALID_INDEX) return INVALID_INDEX;
+ return (bsAt(begin ? regex.btMemStart : regex.btMemEnd, capture)
+ ? stack[pointer].getMemPStr() : pointer) - str;
+ }
+
private int committedCaptureOffset(int capture, boolean begin) {
CompletedRecursiveCall completed = completedRecursiveCall();
if (completed == null || msaRegion == null || isFindLongest(regex.options | msaOptions)
@@ -3889,6 +3991,23 @@ private int previousClosedCapturePointer(int capture, boolean begin) {
return INVALID_INDEX;
}
+ private record CapturePointers(int begin, int end) {}
+
+ private CapturePointers previousClosedCapturePointers(int capture) {
+ int endPointer = INVALID_INDEX;
+ for (int i = stk - 1; i >= 0; i--) {
+ StackEntry entry = stack[i];
+ if (endPointer == INVALID_INDEX) {
+ if (entry.type == MEM_END && entry.getMemNum() == capture) {
+ endPointer = i;
+ }
+ } else if (entry.type == MEM_START && entry.getMemNum() == capture) {
+ return new CapturePointers(i, endPointer);
+ }
+ }
+ return new CapturePointers(INVALID_INDEX, endPointer);
+ }
+
private boolean captureClosedAfterReturn(int capture, int returnIndex) {
for (int i = stk - 1; i > returnIndex; i--) {
StackEntry entry = stack[i];
diff --git a/third_party/joni/src/org/joni/Config.java b/third_party/joni/src/org/joni/Config.java
index 849e4fe08..1a58c7395 100644
--- a/third_party/joni/src/org/joni/Config.java
+++ b/third_party/joni/src/org/joni/Config.java
@@ -60,6 +60,10 @@ public interface Config extends org.jcodings.Config {
boolean USE_QTFR_PEEK_NEXT = ConfigSupport.getBoolean("joni.use_qtfr_peek_next", true);
int INIT_MATCH_STACK_SIZE = ConfigSupport.getInt("joni.init_match_stack_size", 64);
+ // The backtracking stack is heap-backed. Keep an explicit ceiling so a
+ // pathological failed match is reported through Perl's existing recursion
+ // exhaustion path instead of exhausting the whole JVM heap.
+ int MAX_MATCH_STACK_SIZE = ConfigSupport.getInt("joni.max_match_stack_size", 1 << 20);
boolean OPTIMIZE = ConfigSupport.getBoolean("joni.optimize", true);
@Deprecated boolean DONT_OPTIMIZE = !OPTIMIZE;
diff --git a/third_party/joni/src/org/joni/MatchView.java b/third_party/joni/src/org/joni/MatchView.java
index 9a1f6a143..da7024dc1 100644
--- a/third_party/joni/src/org/joni/MatchView.java
+++ b/third_party/joni/src/org/joni/MatchView.java
@@ -18,9 +18,9 @@
* SOFTWARE.
*/
package org.joni;
-
/** Read-only provisional matcher state, valid only during a callout. */
public interface MatchView {
+ record CaptureOffsets(int begin, int end) {}
int currentBytePosition();
int captureCount();
@@ -29,6 +29,16 @@ public interface MatchView {
int captureEnd(int capture);
+ default CaptureOffsets captureOffsets(int capture) {
+ return new CaptureOffsets(captureBegin(capture), captureEnd(capture));
+ }
+
+ /** All capture offsets, or {@code null} when no bulk view is available. */
+ default CaptureOffsets[] captureOffsets() { return null; }
+
+ /** Capture IDs visibly open at the current byte position, or {@code null}. */
+ default int[] openCapturesAtCurrentPosition() { return null; }
+
/** Number of the most recently closed active capture, or -1 if none. */
int lastClosedCapture();
diff --git a/third_party/joni/src/org/joni/StackEntry.java b/third_party/joni/src/org/joni/StackEntry.java
index 786af5fbb..757d072c1 100644
--- a/third_party/joni/src/org/joni/StackEntry.java
+++ b/third_party/joni/src/org/joni/StackEntry.java
@@ -22,6 +22,7 @@
class StackEntry {
int type;
private int activeCallFrameHead = -1;
+ private int activeCallDepth;
private int callFramePreviousHead = -1;
private int E1, E2, E3, E4;
private Object calloutToken;
@@ -181,13 +182,6 @@ void setCallFrameNum(int num) {
int getCallFrameNum() {
return E2;
}
- /* string position */
- void setCallFramePStr(int pstr) {
- E3 = pstr;
- }
- int getCallFramePStr() {
- return E3;
- }
void setCallFrameCaptureSnapshot(int[] snapshot) {
callFrameCaptureSnapshot = snapshot;
}
@@ -215,6 +209,15 @@ int getActiveCallFrameHead() {
return activeCallFrameHead;
}
+ void setActiveCallDepth(int depth) {
+ activeCallDepth = depth;
+ }
+
+ int getActiveCallDepth() {
+ return activeCallDepth;
+ }
+
+
void setCallFramePreviousHead(int head) {
callFramePreviousHead = head;
}
diff --git a/third_party/joni/src/org/joni/StackMachine.java b/third_party/joni/src/org/joni/StackMachine.java
index 94cb567d6..72e3e491a 100644
--- a/third_party/joni/src/org/joni/StackMachine.java
+++ b/third_party/joni/src/org/joni/StackMachine.java
@@ -135,7 +135,11 @@ protected final void leaveMatcherExecution() {
}
private void doubleStack() {
- StackEntry[] newStack = new StackEntry[stack.length << 1];
+ if (stack.length >= Config.MAX_MATCH_STACK_SIZE) {
+ throw new StackOverflowError("regex match stack limit exceeded");
+ }
+ int newLength = Math.min(stack.length << 1, Config.MAX_MATCH_STACK_SIZE);
+ StackEntry[] newStack = new StackEntry[newLength];
System.arraycopy(stack, 0, newStack, 0, stack.length);
stack = newStack;
}
@@ -163,6 +167,7 @@ private final StackEntry ensure1() {
StackEntry e = stack[stk];
if (e == null) stack[stk] = e = USE_CEC ? new SCStackEntry() : new StackEntry();
e.setActiveCallFrameHead(stk == 0 ? -1 : stack[stk - 1].getActiveCallFrameHead());
+ e.setActiveCallDepth(stk == 0 ? 0 : stack[stk - 1].getActiveCallDepth());
return e;
}
@@ -241,6 +246,7 @@ private void push(int type, int pat, int s, int prev, int pkeep) {
private final void pushEnsured(int type, int pat) {
StackEntry e = stack[stk];
e.setActiveCallFrameHead(stk == 0 ? -1 : stack[stk - 1].getActiveCallFrameHead());
+ e.setActiveCallDepth(stk == 0 ? 0 : stack[stk - 1].getActiveCallDepth());
e.type = type;
e.setStatePCode(pat);
if (USE_CEC) ((SCStackEntry)e).setStateCheck(0);
@@ -440,6 +446,7 @@ protected final void pushCallFrame(int pat, int groupNum, boolean snapshotCaptur
StackEntry e = ensure1();
e.setCallFramePreviousHead(e.getActiveCallFrameHead());
e.setActiveCallFrameHead(stk);
+ e.setActiveCallDepth(e.getActiveCallDepth() + 1);
e.type = CALL_FRAME;
e.setCallFrameRetAddr(pat);
e.setCallFrameNum(groupNum);
@@ -511,6 +518,7 @@ protected final void pushReturn(StackEntry frame) {
StackEntry e = ensure1();
e.type = RETURN;
e.setActiveCallFrameHead(frame.getCallFramePreviousHead());
+ e.setActiveCallDepth(frame.getActiveCallDepth() - 1);
stk++;
}
@@ -1027,21 +1035,11 @@ protected final int sreturn() {
}
protected final StackEntry returnFrame() {
- int level = 0;
- int k = stk;
- while (true) {
- k--;
- StackEntry e = stack[k];
-
- if (e.type == CALL_FRAME) {
- if (level == 0) {
- return e;
- } else {
- level--;
- }
- } else if (e.type == RETURN) {
- level++;
- }
- }
+ // Every stack entry inherits the active call-frame head when pushed;
+ // RETURN entries explicitly advance it to the caller. Looking it up
+ // directly avoids rescanning deep recursive grammar stacks at every
+ // subexpression return.
+ int callFrame = stack[stk - 1].getActiveCallFrameHead();
+ return stack[callFrame];
}
}
From d034f04253fe8fa2a1c0a1638192d130c86e8ded Mon Sep 17 00:00:00 2001
From: "Flavio S. Glock"
Date: Thu, 10 Sep 2026 02:42:49 +0200
Subject: [PATCH 4/8] fix(regex): complete PPR compatibility
Correct recursive capture and nullable-call behavior in Joni, preserve
callback-bearing regexes during empty-pattern reuse, and cover the resulting
PPR grammar paths with focused regressions.
Generated with Codex (https://openai.com/codex/)
Co-Authored-By: Codex
---
dev/design/ppr-compatibility.md | 25 ++---
docs/about/changelog.md | 3 +
.../perlonjava/frontend/parser/Parser.java | 14 +++
.../runtime/regex/RuntimeRegex.java | 17 +++-
.../runtimetypes/ErrorMessageUtil.java | 25 ++++-
.../regex/duplicate_named_backref_selection.t | 18 ++++
.../unit/regex/empty_pattern_callback_reuse.t | 13 +++
.../recursive_false_condition_empty_branch.t | 21 ++++
.../unit/regex/recursive_nullable_repeat.t | 16 ++++
...egex_callback_immediate_fail_side_effect.t | 11 +++
.../unit/syntax_error_colon_context.t | 9 ++
.../unit/unmatched_right_curly_eval.t | 10 ++
.../joni/src/org/joni/ArrayCompiler.java | 8 ++
.../joni/src/org/joni/ByteCodeMachine.java | 96 ++++++++-----------
.../joni/src/org/joni/ByteCodePrinter.java | 14 +++
third_party/joni/src/org/joni/Regex.java | 14 ++-
.../joni/src/org/joni/StackMachine.java | 30 +++++-
17 files changed, 270 insertions(+), 74 deletions(-)
create mode 100644 src/test/resources/unit/regex/duplicate_named_backref_selection.t
create mode 100644 src/test/resources/unit/regex/empty_pattern_callback_reuse.t
create mode 100644 src/test/resources/unit/regex/recursive_false_condition_empty_branch.t
create mode 100644 src/test/resources/unit/regex/recursive_nullable_repeat.t
create mode 100644 src/test/resources/unit/regex_callback_immediate_fail_side_effect.t
create mode 100644 src/test/resources/unit/syntax_error_colon_context.t
create mode 100644 src/test/resources/unit/unmatched_right_curly_eval.t
diff --git a/dev/design/ppr-compatibility.md b/dev/design/ppr-compatibility.md
index ef938b2eb..4a409f685 100644
--- a/dev/design/ppr-compatibility.md
+++ b/dev/design/ppr-compatibility.md
@@ -76,7 +76,7 @@ explicit optional-dependency skip.
## Progress Tracking
-### Current Status: Phase 2 in progress
+### Current Status: Complete (2026-09-10)
### Completed Phases
@@ -92,23 +92,24 @@ explicit optional-dependency skip.
invalid Java substring range.
- This clears PPR's `blocks`, `control`, `for_ref_iterator`, and `format`
range-error family.
+- [x] Phases 2–4: Recursive grammar execution and distribution acceptance (2026-09-10)
+ - Corrected nullable recursive-call empty checks, recursive capture
+ restoration for duplicate named groups, and duplicate-name backreference
+ selection in Joni.
+ - Kept callback-bearing regex programs intact for empty-pattern reuse with
+ changed modifiers, allowing PPR's unpunctuated JAPH to execute.
+ - Added focused coverage in `src/test/resources/unit/regex/` and validated
+ it on system Perl plus both PerlOnJava backends.
+ - `timeout 1800 ./jcpan -t PPR` passes: 75 files, 1,255 tests.
### Next Steps
-1. Complete Phase 2 by running the remaining range-family PPR tests and
- separating any non-shared failures.
-2. Reduce the second `t/heredoc.t` fixture, which now reports its expected
- assertion failure and then times out in `ByteCodeMachine.opCall`.
-3. Compare that reducer on system Perl, JVM, and interpreter backends.
-4. Fix the remaining recursive-call execution path without weakening PPR's
- grammar or introducing source-specific handling.
+1. Monitor the PR checks and review feedback.
+2. Keep PPR in the CPAN compatibility acceptance rotation.
### Open Questions
-- Why does the second heredoc fixture fail before the subsequent recursive-call
- execution timeout?
-- Does the heredoc execution path need an additional semantic guard distinct
- from call-frame lookup performance?
+- None for the current PPR acceptance scope.
## Related Work
diff --git a/docs/about/changelog.md b/docs/about/changelog.md
index 1a7162550..d8432b09c 100644
--- a/docs/about/changelog.md
+++ b/docs/about/changelog.md
@@ -26,6 +26,9 @@ priorities and future plans.
- Fixed large dynamic named-subexpression grammars hanging during regex compilation.
+- Restore PPR's complete suite by correcting recursive duplicate-name captures,
+ nullable recursion checks, and callback regex reuse.
+
- Preserve Data::Dumper's pure-Perl numeric-string behavior for
Test::Differences, including copied `qw` values and numeric zero fixtures.
diff --git a/src/main/java/org/perlonjava/frontend/parser/Parser.java b/src/main/java/org/perlonjava/frontend/parser/Parser.java
index 5f3fafc75..44864ffa3 100644
--- a/src/main/java/org/perlonjava/frontend/parser/Parser.java
+++ b/src/main/java/org/perlonjava/frontend/parser/Parser.java
@@ -223,6 +223,20 @@ public Node parse() {
} finally {
compilationState.unitcheckQueueStack.get().pop();
}
+ // ParseBlock stops before a closing brace so callers parsing a nested
+ // block can consume it. At file/eval scope there is no such caller:
+ // leaving it accepted makes `eval 'sub {} }'` silently compile.
+ LexerToken remaining = TokenUtils.peek(this);
+ if (remaining.type == LexerTokenType.OPERATOR && "}".equals(remaining.text)) {
+ ErrorMessageUtil.SourceLocation loc = ctx.errorUtil
+ .getSourceLocationAccurate(tokenIndex);
+ String message = "Unmatched right curly bracket at " + loc.fileName()
+ + " line " + loc.lineNumber() + ", at end of line\n"
+ + ctx.errorUtil.errorMessage(tokenIndex, "syntax error")
+ + "Execution of " + loc.fileName()
+ + " aborted due to compilation errors.\n";
+ throw new PerlCompilerException(message);
+ }
// Mark the AST as a top-level file block for proper bare block return value handling
// This annotation is checked in EmitBlock to handle RUNTIME context bare blocks
if (!isTopLevelScript && ast instanceof AbstractNode) {
diff --git a/src/main/java/org/perlonjava/runtime/regex/RuntimeRegex.java b/src/main/java/org/perlonjava/runtime/regex/RuntimeRegex.java
index 675d68e51..19b77736d 100644
--- a/src/main/java/org/perlonjava/runtime/regex/RuntimeRegex.java
+++ b/src/main/java/org/perlonjava/runtime/regex/RuntimeRegex.java
@@ -2499,7 +2499,8 @@ && scanExecutableSource(sourcePattern, extendedSource,
true, true).executable();
if (executableSource || unterminatedClassExecutableCandidate) {
if (RuntimeRegexSourceCompiler.isCompilingRuntimeSource()
- && unterminatedClassExecutableCandidate) {
+ && unterminatedClassExecutableCandidate
+ && hasInitiallyClosedCharacterClass(sourcePattern)) {
String recursionDiagnostic = unterminatedExecutableSequence(
sourcePattern);
if (recursionDiagnostic != null) {
@@ -2514,6 +2515,7 @@ && scanExecutableSource(sourcePattern, extendedSource,
return RuntimeRegexSourceCompiler.compile(
patternString, rawModifierStr,
executableSource && eagerInitialClassExecutableCandidate
+ && hasInitiallyClosedCharacterClass(sourcePattern)
? unterminatedExecutableSequence(sourcePattern)
: null,
!executableSource || sourcePolicy.admitRuntimeEval())
@@ -2547,6 +2549,12 @@ static boolean containsExecutableSource(String pattern, boolean extended) {
false, false).executable();
}
+ /** Diagnostic used when malformed synthetic source attempts to re-enter itself. */
+ private static boolean hasInitiallyClosedCharacterClass(String pattern) {
+ return pattern != null
+ && (pattern.contains("[](?{") || pattern.contains("[^](?{"));
+ }
+
/** Diagnostic used when malformed synthetic source attempts to re-enter itself. */
private static String unterminatedExecutableSequence(String pattern) {
int dynamic = pattern == null ? -1 : pattern.lastIndexOf("(??{");
@@ -3191,7 +3199,12 @@ private static RuntimeRegex emptyPatternReuse(RuntimeRegex previous,
boolean lexicalReStrict,
RuntimeScalar replacement) {
RuntimeRegex reused;
- if (previous != null && (flags == null || flags.equals(previous.regexFlags))) {
+ if (previous != null && (flags == null || flags.equals(previous.regexFlags)
+ // Runtime executable callbacks are compiled into the native
+ // program, not recoverable from the marker-bearing source.
+ // Empty-pattern reuse must therefore retain that program even
+ // when the new operation supplies modifiers.
+ || !previous.executableCallbacks.isEmpty())) {
reused = previous.cloneTracked();
} else {
String source = previous == null ? "" : previous.patternString;
diff --git a/src/main/java/org/perlonjava/runtime/runtimetypes/ErrorMessageUtil.java b/src/main/java/org/perlonjava/runtime/runtimetypes/ErrorMessageUtil.java
index 7f1206c0a..7a71b1f2e 100644
--- a/src/main/java/org/perlonjava/runtime/runtimetypes/ErrorMessageUtil.java
+++ b/src/main/java/org/perlonjava/runtime/runtimetypes/ErrorMessageUtil.java
@@ -362,10 +362,30 @@ private String buildNearString(int index, String message) {
}
}
+ // Perl includes the completed operand immediately before a stray
+ // colon in its syntax context (for example, `near "2:"`), rather
+ // than starting the excerpt at the unexpected punctuation.
+ int start = index;
+ boolean trimTrailingWhitespace = false;
+ if ("syntax error".equals(message)
+ && index > 0
+ && ":".equals(tokens.get(index).text)) {
+ int previous = index - 1;
+ while (previous >= 0
+ && tokens.get(previous).type == LexerTokenType.WHITESPACE) {
+ previous--;
+ }
+ if (previous >= 0
+ && tokens.get(previous).type != LexerTokenType.NEWLINE) {
+ start = previous;
+ trimTrailingWhitespace = true;
+ }
+ }
+
int end = Math.min(tokens.size() - 1, index + 5);
StringBuilder sb = new StringBuilder();
int nonWsCount = 0;
- for (int i = index; i <= end; i++) {
+ for (int i = start; i <= end; i++) {
LexerToken tok = tokens.get(i);
if (tok.type == LexerTokenType.EOF || tok.type == LexerTokenType.NEWLINE) break;
if (tok.text.equals("{") || tok.text.equals("}")) break;
@@ -377,6 +397,9 @@ private String buildNearString(int index, String message) {
}
String near = sb.toString();
near = near.replaceAll("^\\s+", "");
+ if (trimTrailingWhitespace) {
+ near = near.replaceAll("\\s+$", "");
+ }
return near;
}
diff --git a/src/test/resources/unit/regex/duplicate_named_backref_selection.t b/src/test/resources/unit/regex/duplicate_named_backref_selection.t
new file mode 100644
index 000000000..e734ad6dd
--- /dev/null
+++ b/src/test/resources/unit/regex/duplicate_named_backref_selection.t
@@ -0,0 +1,18 @@
+use strict;
+use warnings;
+use Test::More;
+
+my $pattern = qr/\A(?a)(?b)\k\z/;
+
+ok('aba' =~ $pattern,
+ 'duplicate named backreference uses the oldest participating capture');
+ok('abb' !~ $pattern,
+ 'duplicate named backreference does not retry a newer capture');
+
+my $case_insensitive = qr/\A(?a)(?b)\k\z/i;
+ok('aBa' =~ $case_insensitive,
+ 'case-insensitive duplicate named backreference uses the oldest capture');
+ok('aBB' !~ $case_insensitive,
+ 'case-insensitive duplicate named backreference does not retry a newer capture');
+
+done_testing;
diff --git a/src/test/resources/unit/regex/empty_pattern_callback_reuse.t b/src/test/resources/unit/regex/empty_pattern_callback_reuse.t
new file mode 100644
index 000000000..aa2699a8d
--- /dev/null
+++ b/src/test/resources/unit/regex/empty_pattern_callback_reuse.t
@@ -0,0 +1,13 @@
+use strict;
+use warnings;
+use re 'eval';
+use Test::More;
+
+my $callback_pattern = qr/(?{ 1 })x/;
+ok 'x' =~ $callback_pattern, 'callback-bearing pattern matches';
+
+my $result = eval q{'x' =~ s//y/r};
+is $@, '', 'empty-pattern reuse recompiles without an error';
+is $result, 'y', 'empty-pattern reuse retains the prior callback regex';
+
+done_testing;
diff --git a/src/test/resources/unit/regex/recursive_false_condition_empty_branch.t b/src/test/resources/unit/regex/recursive_false_condition_empty_branch.t
new file mode 100644
index 000000000..2aac957bf
--- /dev/null
+++ b/src/test/resources/unit/regex/recursive_false_condition_empty_branch.t
@@ -0,0 +1,21 @@
+use strict;
+use warnings;
+use Test::More tests => 4;
+
+our $error = 'previous parse error';
+
+my $grammar = qr{
+ (?(DEFINE)
+ (? (?&statement)+ )
+ (?
+ a
+ | (?(?{ !defined $error }) b (?!))
+ )
+ )
+ \A (?&document) \z
+}x;
+
+ok('a' =~ $grammar, 'one non-empty statement matches after failed condition');
+ok('aa' =~ $grammar, 'two non-empty statements match after failed condition');
+ok('aaa' =~ $grammar, 'three non-empty statements match after failed condition');
+ok('b' !~ $grammar, 'failed condition does not admit its branch');
diff --git a/src/test/resources/unit/regex/recursive_nullable_repeat.t b/src/test/resources/unit/regex/recursive_nullable_repeat.t
new file mode 100644
index 000000000..c3df15825
--- /dev/null
+++ b/src/test/resources/unit/regex/recursive_nullable_repeat.t
@@ -0,0 +1,16 @@
+use strict;
+use warnings;
+
+my $grammar = qr{
+ (?(DEFINE)
+ (? (?&statement)+ )
+ (? a | )
+ )
+ \A (?&document) \z
+}x;
+
+for my $source ('a', 'aa', 'aaa') {
+ print $source =~ $grammar
+ ? "ok - nullable recursive repeat matches $source\n"
+ : "not ok - nullable recursive repeat rejects $source\n";
+}
diff --git a/src/test/resources/unit/regex_callback_immediate_fail_side_effect.t b/src/test/resources/unit/regex_callback_immediate_fail_side_effect.t
new file mode 100644
index 000000000..c3f91382f
--- /dev/null
+++ b/src/test/resources/unit/regex_callback_immediate_fail_side_effect.t
@@ -0,0 +1,11 @@
+use strict;
+use warnings;
+use Test::More;
+
+my $value = 1;
+ok !('a' =~ /a(?{ $value = 3 })(?!)/),
+ 'an immediate negative assertion fails the match';
+is $value, 3,
+ 'callback assignment survives an immediate zero-width failure';
+
+done_testing;
diff --git a/src/test/resources/unit/syntax_error_colon_context.t b/src/test/resources/unit/syntax_error_colon_context.t
new file mode 100644
index 000000000..c0cbd53cd
--- /dev/null
+++ b/src/test/resources/unit/syntax_error_colon_context.t
@@ -0,0 +1,9 @@
+use strict;
+use warnings;
+use Test::More;
+
+my $error = eval q{sub example { my $value = 2: }};
+ok !defined $error, 'malformed source does not compile';
+like $@, qr/syntax error.*near "2:"/, 'syntax diagnostic retains operand before colon';
+
+done_testing;
diff --git a/src/test/resources/unit/unmatched_right_curly_eval.t b/src/test/resources/unit/unmatched_right_curly_eval.t
new file mode 100644
index 000000000..a7be5f3ac
--- /dev/null
+++ b/src/test/resources/unit/unmatched_right_curly_eval.t
@@ -0,0 +1,10 @@
+use strict;
+use warnings;
+use Test::More;
+
+my $compiled = eval q!sub example { } }!;
+ok !defined $compiled, 'unmatched right curly does not compile';
+like $@, qr/^Unmatched right curly bracket at /,
+ 'eval reports an unmatched right curly diagnostic';
+
+done_testing;
diff --git a/third_party/joni/src/org/joni/ArrayCompiler.java b/third_party/joni/src/org/joni/ArrayCompiler.java
index e5a15a291..8c847877b 100644
--- a/third_party/joni/src/org/joni/ArrayCompiler.java
+++ b/third_party/joni/src/org/joni/ArrayCompiler.java
@@ -255,6 +255,14 @@ private int selectStrOpcode(int mbLength, int byteLength, boolean ignoreCase) {
private void compileTreeEmptyCheck(Node node, int emptyInfo) {
int savedNumNullCheck = regex.numNullCheck;
+ // A subexpression call can update its captures through an empty
+ // alternative without consuming input. Repeating that call must stop
+ // on input position alone; the capture-sensitive guard would otherwise
+ // keep revisiting the same empty call indefinitely.
+ if (node.getType() == NodeType.CALL && emptyInfo == TargetInfo.IS_EMPTY_MEM) {
+ emptyInfo = TargetInfo.IS_EMPTY;
+ }
+
if (emptyInfo != 0) {
regex.requireStack = true;
addOpcode(OPCode.NULL_CHECK_START);
diff --git a/third_party/joni/src/org/joni/ByteCodeMachine.java b/third_party/joni/src/org/joni/ByteCodeMachine.java
index 5ab2ddc6e..14467211b 100644
--- a/third_party/joni/src/org/joni/ByteCodeMachine.java
+++ b/third_party/joni/src/org/joni/ByteCodeMachine.java
@@ -2900,70 +2900,50 @@ private void opBackRefPreviousIC() {
}
private void opBackRefMulti() {
- int tlen = code[ip++];
-
- int i;
- loop:for (i=0; i range) continue;
+ int mem = selectBackrefMultiCapture();
+ if (mem < 0) {opFail(); return;}
+ backref(mem);
+ }
- sprev = s;
- int swork = s;
+ private void opBackRefMultiIC() {
+ int mem = selectBackrefMultiCapture();
+ if (mem < 0) {opFail(); return;}
- while (n-- > 0) {
- if (bytes[pstart++] != bytes[swork++]) continue loop;
- }
+ int pstart = backrefStart(mem);
+ int pend = backrefEnd(mem);
+ int n = pend - pstart;
+ if (s + n > range) {opFail(); return;}
- s = swork;
+ sprev = s;
+ value = s;
+ if (!backrefStringCmpIC(currentCaseFoldFlag(), pstart, this, n, end)) {
+ opFail();
+ return;
+ }
+ s = value;
+ if (sprev < range) {
int len;
-
- // beyond string check
- if (sprev < range) {
- while (sprev + (len = enc.length(bytes, sprev, end)) < s) sprev += len;
- }
-
- ip += tlen - i - 1; // * SIZE_MEMNUM (1)
- break; /* success */
+ while (sprev + (len = enc.length(bytes, sprev, end)) < s) sprev += len;
}
- if (i == tlen) {opFail(); return;}
}
- private void opBackRefMultiIC() {
- int tlen = code[ip++];
-
- int i;
- loop:for (i=0; i range) continue;
-
- sprev = s;
-
- value = s;
- if (!backrefStringCmpIC(currentCaseFoldFlag(), pstart, this, n, end)) continue loop; // STRING_CMP_VALUE_IC
- s = value;
-
- int len;
- if (sprev < range) {
- while (sprev + (len = enc.length(bytes, sprev, end)) < s) sprev += len;
+ if (!backrefInvalid(mem)) {
+ selected = mem;
}
-
- ip += tlen - i - 1; // * SIZE_MEMNUM (1)
- break; /* success */
}
- if (i == tlen) {opFail(); return;}
+ return selected;
}
private boolean memIsInMemp(int mem, int num, int memp) {
@@ -3116,7 +3096,8 @@ private void opNullCheckEndMemSTPush() {
int isNull;
if (Config.USE_MONOMANIAC_CHECK_CAPTURES_IN_ENDLESS_REPEAT) {
- isNull = nullCheckMemStRec(mem, s);
+ int positional = nullCheckRecIfPresent(mem, s);
+ isNull = positional == Integer.MIN_VALUE ? nullCheckMemStRec(mem, s) : positional;
} else {
isNull = nullCheckRec(mem, s);
}
@@ -3680,6 +3661,13 @@ protected boolean calloutSamePositionFailureCommits() {
case OPCode.NULL_CHECK_START:
cursor += OPSize.NULL_CHECK_START;
break;
+ case OPCode.PUSH_POS_NOT:
+ // `(?! )` is compiled as an empty negative assertion:
+ // PUSH_POS_NOT followed immediately by FAIL_POS. Perl
+ // retains a preceding callback's assignment through that
+ // explicit, zero-width failure.
+ return cursor + OPSize.PUSH_POS_NOT < code.length
+ && code[cursor + OPSize.PUSH_POS_NOT] == OPCode.FAIL_POS;
default:
return false;
}
diff --git a/third_party/joni/src/org/joni/ByteCodePrinter.java b/third_party/joni/src/org/joni/ByteCodePrinter.java
index 651b95d63..2825882d6 100644
--- a/third_party/joni/src/org/joni/ByteCodePrinter.java
+++ b/third_party/joni/src/org/joni/ByteCodePrinter.java
@@ -409,6 +409,20 @@ public int compiledByteCodeToString(StringBuilder sb, int bp) {
sb.append(':').append(addr).append(':').append(mem);
break;
+ case OPCode.CALLOUT:
+ mem = code[bp];
+ bp += OPSize.MEMNUM;
+ sb.append(':').append(mem);
+ break;
+
+ case OPCode.CALLOUT_CONDITION:
+ mem = code[bp];
+ bp += OPSize.MEMNUM;
+ addr = code[bp];
+ bp += OPSize.RELADDR;
+ sb.append(':').append(mem).append(":(").append(addr).append(')');
+ break;
+
case OPCode.DYNAMIC_CALLOUT:
mem = code[bp];
bp += OPSize.MEMNUM;
diff --git a/third_party/joni/src/org/joni/Regex.java b/third_party/joni/src/org/joni/Regex.java
index 3ed01b8c9..0a22d29d1 100644
--- a/third_party/joni/src/org/joni/Regex.java
+++ b/third_party/joni/src/org/joni/Regex.java
@@ -30,6 +30,7 @@
import java.util.Arrays;
import java.util.Collections;
import java.util.EnumSet;
+import java.util.HashSet;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
@@ -111,6 +112,7 @@ static ParsedProgramMetadata copyOf(
int numMem; /* used memory(...) num counted from 1 */
int numPhysicalNamedCaptures;
+ private final Set multiplexNamedGroups = new HashSet<>();
int numRepeat; /* OP_REPEAT/OP_REPEAT_NG id-counter */
int numNullCheck; /* OP_NULL_CHECK_START/END id counter */
int numCombExpCheck; /* combination explosion check */
@@ -450,8 +452,12 @@ int nameAdd(byte[]name, int nameP, int nameEnd, int backRef, Syntax syntax) {
// dup the name here as oni does ?, what for ? (it has to manage it, we don't)
e = new NameEntry(name, nameP, nameEnd);
nameTable.putDirect(name, nameP, nameEnd, e);
- } else if (e.backNum >= 1 && !syntax.allowMultiplexDefinitionName()) {
- throw new ValueException(ErrorMessages.MULTIPLEX_DEFINED_NAME, new String(name, nameP, nameEnd - nameP));
+ } else if (e.backNum >= 1) {
+ if (!syntax.allowMultiplexDefinitionName()) {
+ throw new ValueException(ErrorMessages.MULTIPLEX_DEFINED_NAME, new String(name, nameP, nameEnd - nameP));
+ }
+ multiplexNamedGroups.add(backRef);
+ for (int existing : e.getBackRefs()) multiplexNamedGroups.add(existing);
}
int physicalRef = ++numPhysicalNamedCaptures;
@@ -463,6 +469,10 @@ public int numberOfPhysicalNamedCaptures() {
return numPhysicalNamedCaptures;
}
+ boolean isMultiplexNamedGroup(int group) {
+ return multiplexNamedGroups.contains(group);
+ }
+
NameEntry nameToGroupNumbers(byte[]name, int nameP, int nameEnd) {
return nameFind(name, nameP, nameEnd);
}
diff --git a/third_party/joni/src/org/joni/StackMachine.java b/third_party/joni/src/org/joni/StackMachine.java
index 72e3e491a..f1ef33b46 100644
--- a/third_party/joni/src/org/joni/StackMachine.java
+++ b/third_party/joni/src/org/joni/StackMachine.java
@@ -485,13 +485,19 @@ protected final void restoreCallFrameCaptureSnapshot(StackEntry frame) {
// A nested call reuses physical capture slots. Closed values belonged
// to the caller before the call and must be visible again after return
// (for example, a palindrome's enclosing character backreference).
- // Deliberately do not restore open or unset slots: their final state
- // belongs to the successful nested path.
+ // Deliberately do not restore ordinary open or unset slots: their
+ // final state belongs to the successful nested path. Multiplex named
+ // definitions are different: a value introduced only by the nested
+ // call must not remain a candidate for a caller's same-name
+ // backreference after that call returns.
for (int mem = 1; mem < count; mem++) {
if (snapshot[mem] != INVALID_INDEX
&& snapshot[count + mem] != INVALID_INDEX) {
repeatStk[memStartStk + mem] = snapshot[mem];
repeatStk[memEndStk + mem] = snapshot[count + mem];
+ } else if (regex.isMultiplexNamedGroup(mem)) {
+ repeatStk[memStartStk + mem] = INVALID_INDEX;
+ repeatStk[memEndStk + mem] = INVALID_INDEX;
}
}
@@ -917,6 +923,24 @@ protected final int nullCheckRec(int id, int s) {
}
}
+ /**
+ * Recursive null check that reports {@link Integer#MIN_VALUE} when an
+ * older program has no matching start marker on the stack.
+ */
+ protected final int nullCheckRecIfPresent(int id, int s) {
+ int level = 0;
+ for (int k = stk - 1; k >= 0; k--) {
+ StackEntry e = stack[k];
+ if (e.type == NULL_CHECK_START && e.getNullCheckNum() == id) {
+ if (level == 0) return e.getNullCheckPStr() == s ? 1 : 0;
+ level--;
+ } else if (e.type == NULL_CHECK_END && e.getNullCheckNum() == id) {
+ level++;
+ }
+ }
+ return Integer.MIN_VALUE;
+ }
+
protected final int nullCheckMemSt(int id, int s) {
int k = stk;
int isNull;
@@ -1036,7 +1060,7 @@ protected final int sreturn() {
protected final StackEntry returnFrame() {
// Every stack entry inherits the active call-frame head when pushed;
- // RETURN entries explicitly advance it to the caller. Looking it up
+ // RETURN entries explicitly advance it to the caller. Looking it up
// directly avoids rescanning deep recursive grammar stacks at every
// subexpression return.
int callFrame = stack[stk - 1].getActiveCallFrameHead();
From 5f33ca2a006cf48ce18d383e3572c864c04a22ab Mon Sep 17 00:00:00 2001
From: "Flavio S. Glock"
Date: Thu, 10 Sep 2026 10:37:04 +0200
Subject: [PATCH 5/8] fix(io): route argumentless readline through ARGV
Preserve Perl's localized diamond-reader behavior for argumentless readline,
preventing PPR document_self warnings on the JVM backend.
Generated with Codex (https://openai.com/codex/)
Co-Authored-By: Codex
---
docs/about/changelog.md | 3 +++
.../perlonjava/backend/jvm/EmitOperator.java | 14 ++++++++++++++
.../frontend/parser/OperatorParser.java | 8 +++++++-
.../resources/unit/implicit_argv_readline.t | 18 ++++++++++++++++++
4 files changed, 42 insertions(+), 1 deletion(-)
create mode 100644 src/test/resources/unit/implicit_argv_readline.t
diff --git a/docs/about/changelog.md b/docs/about/changelog.md
index d8432b09c..5c4bc6b4b 100644
--- a/docs/about/changelog.md
+++ b/docs/about/changelog.md
@@ -29,6 +29,9 @@ priorities and future plans.
- Restore PPR's complete suite by correcting recursive duplicate-name captures,
nullable recursion checks, and callback regex reuse.
+- Route argumentless `readline` through localized `@ARGV`, matching Perl's
+ diamond-reader behavior and keeping PPR's self-document test warning-free.
+
- Preserve Data::Dumper's pure-Perl numeric-string behavior for
Test::Differences, including copied `qw` values and numeric zero fixtures.
diff --git a/src/main/java/org/perlonjava/backend/jvm/EmitOperator.java b/src/main/java/org/perlonjava/backend/jvm/EmitOperator.java
index 1e820620e..1d4476b8e 100644
--- a/src/main/java/org/perlonjava/backend/jvm/EmitOperator.java
+++ b/src/main/java/org/perlonjava/backend/jvm/EmitOperator.java
@@ -182,6 +182,20 @@ static void handleReadlineOperator(EmitterVisitor emitterVisitor, BinaryOperator
if (operator.equals("readline")) {
emitterVisitor.pushCallContext();
+ if (Boolean.TRUE.equals(node.getAnnotation("implicitArgvReadline"))) {
+ emitterVisitor.ctx.mv.visitMethodInsn(Opcodes.INVOKESTATIC,
+ "org/perlonjava/runtime/runtimetypes/DiamondIO",
+ "readline",
+ "(Lorg/perlonjava/runtime/runtimetypes/RuntimeScalar;I)"
+ + "Lorg/perlonjava/runtime/runtimetypes/RuntimeBase;",
+ false);
+ if (emitterVisitor.ctx.contextType == RuntimeContextType.VOID) {
+ handleVoidContext(emitterVisitor);
+ } else if (emitterVisitor.ctx.contextType == RuntimeContextType.SCALAR) {
+ handleScalarContext(emitterVisitor, node);
+ }
+ return;
+ }
}
emitOperator(node, emitterVisitor);
}
diff --git a/src/main/java/org/perlonjava/frontend/parser/OperatorParser.java b/src/main/java/org/perlonjava/frontend/parser/OperatorParser.java
index 40644a97b..9867836b8 100644
--- a/src/main/java/org/perlonjava/frontend/parser/OperatorParser.java
+++ b/src/main/java/org/perlonjava/frontend/parser/OperatorParser.java
@@ -1145,6 +1145,7 @@ static BinaryOperatorNode parseReadline(Parser parser, LexerToken token, int cur
// Handle file-related operators with special handling for default handles
ListNode operand = ListParser.parseZeroOrMoreList(parser, 0, false, true, false, false);
Node handle;
+ boolean implicitArgvReadline = false;
if (operand.elements.isEmpty()) {
String defaultHandle = switch (operator) {
case "readline" -> "main::ARGV";
@@ -1158,6 +1159,7 @@ static BinaryOperatorNode parseReadline(Parser parser, LexerToken token, int cur
handle = new OperatorNode("undef", null, currentIndex);
} else {
handle = new IdentifierNode(defaultHandle, currentIndex);
+ implicitArgvReadline = operator.equals("readline");
}
} else {
handle = operand.elements.removeFirst();
@@ -1173,7 +1175,11 @@ static BinaryOperatorNode parseReadline(Parser parser, LexerToken token, int cur
}
}
}
- return new BinaryOperatorNode(operator, handle, operand, currentIndex);
+ BinaryOperatorNode result = new BinaryOperatorNode(operator, handle, operand, currentIndex);
+ if (implicitArgvReadline) {
+ result.setAnnotation("implicitArgvReadline", true);
+ }
+ return result;
}
static BinaryOperatorNode parseSplit(Parser parser, LexerToken token, int currentIndex) {
diff --git a/src/test/resources/unit/implicit_argv_readline.t b/src/test/resources/unit/implicit_argv_readline.t
new file mode 100644
index 000000000..e266abd14
--- /dev/null
+++ b/src/test/resources/unit/implicit_argv_readline.t
@@ -0,0 +1,18 @@
+use strict;
+use warnings;
+use File::Temp qw(tempfile);
+use Test::More;
+
+my ($fh, $filename) = tempfile();
+print {$fh} "first\nsecond\n";
+close $fh;
+
+my $source = do {
+ local (@ARGV, $/) = $filename;
+ readline;
+};
+
+is $source, "first\nsecond\n",
+ 'argumentless readline slurps the localized @ARGV file';
+
+done_testing;
From 55e666e4227b211f7fa9a0254e39c12ad244a0b8 Mon Sep 17 00:00:00 2001
From: "Flavio S. Glock"
Date: Thu, 10 Sep 2026 13:17:38 +0200
Subject: [PATCH 6/8] fix(io): restore diamond ARGV compatibility
Preserve diamond and double-diamond semantics across both backends, including
ARGV lifecycle, EOF handling, diagnostics, and ordinary-diamond fork warnings.
Generated with Codex (https://openai.com/codex/)
Co-Authored-By: Codex
---
docs/about/changelog.md | 3 +
.../backend/bytecode/CompileOperator.java | 10 +-
.../bytecode/OpcodeHandlerExtended.java | 12 +-
.../perlonjava/backend/jvm/EmitOperator.java | 4 +-
.../frontend/parser/OperatorParser.java | 16 ++
.../frontend/parser/StringParser.java | 9 +-
.../runtime/io/CustomFileChannel.java | 13 +-
.../runtime/operators/IOOperator.java | 8 +
.../perlonjava/runtime/operators/WarnDie.java | 3 +
.../runtime/runtimetypes/DiamondIO.java | 172 +++++++++++++++++-
.../resources/unit/diamond_argv_lifecycle.t | 49 +++++
11 files changed, 280 insertions(+), 19 deletions(-)
create mode 100644 src/test/resources/unit/diamond_argv_lifecycle.t
diff --git a/docs/about/changelog.md b/docs/about/changelog.md
index 5c4bc6b4b..2cf1e35cc 100644
--- a/docs/about/changelog.md
+++ b/docs/about/changelog.md
@@ -12,6 +12,9 @@ priorities and future plans.
same-named constant calls, restoring `Types::Numbers` loading through
`Data::Float`.
+- Restore Perl-compatible `<>` and `<<>>` ARGV traversal, `eof()` behavior,
+ diagnostics, and warning handling on both execution backends.
+
- Implement undef-aware experimental equality operators (`===`, `!==`, `equ`,
and `neu`) with lexical warnings and single-evaluation chained comparisons.
diff --git a/src/main/java/org/perlonjava/backend/bytecode/CompileOperator.java b/src/main/java/org/perlonjava/backend/bytecode/CompileOperator.java
index 6c03d26a5..ea371d42b 100644
--- a/src/main/java/org/perlonjava/backend/bytecode/CompileOperator.java
+++ b/src/main/java/org/perlonjava/backend/bytecode/CompileOperator.java
@@ -1794,7 +1794,15 @@ private static void visitDiamond(BytecodeCompiler bc, OperatorNode node) {
bc.compileNode(node.operand, -1, RuntimeContextType.SCALAR);
int fhReg = bc.lastResultReg;
int rd = bc.allocateOutputRegister();
- bc.emit(Opcodes.READLINE); bc.emitReg(rd); bc.emitReg(fhReg); bc.emit(bc.currentCallContext);
+ // Preserve that this READLINE originated from <>/<<>>. The runtime
+ // value can be a glob (not merely the empty-string marker), so it
+ // cannot reliably infer diamond semantics from the filehandle.
+ bc.emit(Opcodes.READLINE); bc.emitReg(rd); bc.emitReg(fhReg);
+ int diamondFlags = 0x100;
+ if (Boolean.TRUE.equals(node.getAnnotation("doubleDiamond"))) {
+ diamondFlags |= 0x200;
+ }
+ bc.emit(bc.currentCallContext | diamondFlags);
bc.lastResultReg = rd;
} else {
OperatorNode globNode = new OperatorNode("glob", node.operand, node.tokenIndex);
diff --git a/src/main/java/org/perlonjava/backend/bytecode/OpcodeHandlerExtended.java b/src/main/java/org/perlonjava/backend/bytecode/OpcodeHandlerExtended.java
index e4d85d783..f9a901ec8 100644
--- a/src/main/java/org/perlonjava/backend/bytecode/OpcodeHandlerExtended.java
+++ b/src/main/java/org/perlonjava/backend/bytecode/OpcodeHandlerExtended.java
@@ -919,18 +919,18 @@ public static int executeReadline(int[] bytecode, int pc, RuntimeBase[] register
int rd = bytecode[pc++];
int fhReg = bytecode[pc++];
int ctx = bytecode[pc++];
+ boolean diamond = (ctx & 0x100) != 0;
+ boolean doubleDiamond = (ctx & 0x200) != 0;
+ ctx &= ~(0x100 | 0x200);
if (ctx == RuntimeContextType.RUNTIME) ctx = ((RuntimeScalar) registers[2]).getInt();
RuntimeScalar fh = (RuntimeScalar) registers[fhReg];
// Diamond operator <> passes a plain string scalar (not a glob/IO).
// Route to DiamondIO.readline which manages @ARGV / STDIN iteration.
// But blessed objects may have <> overload, so route those to Readline.
- if (fh.getRuntimeIO() == null) {
- if (RuntimeScalarType.blessedId(fh) < 0) {
- registers[rd] = Readline.readline(fh, ctx);
- } else {
- registers[rd] = DiamondIO.readline(fh, ctx);
- }
+ if (diamond || (RuntimeScalarType.blessedId(fh) < 0
+ && (fh.toString().isEmpty() || "<>".equals(fh.toString())))) {
+ registers[rd] = DiamondIO.readline(fh, ctx, doubleDiamond);
} else {
registers[rd] = Readline.readline(fh, ctx);
}
diff --git a/src/main/java/org/perlonjava/backend/jvm/EmitOperator.java b/src/main/java/org/perlonjava/backend/jvm/EmitOperator.java
index 1d4476b8e..631282b54 100644
--- a/src/main/java/org/perlonjava/backend/jvm/EmitOperator.java
+++ b/src/main/java/org/perlonjava/backend/jvm/EmitOperator.java
@@ -693,11 +693,13 @@ static void handleDiamondBuiltin(EmitterVisitor emitterVisitor, OperatorNode nod
// Handle null filehandle: <> <<>>
node.operand.accept(emitterVisitor.with(RuntimeContextType.SCALAR));
emitterVisitor.pushCallContext();
+ mv.visitInsn(Boolean.TRUE.equals(node.getAnnotation("doubleDiamond"))
+ ? Opcodes.ICONST_1 : Opcodes.ICONST_0);
// Invoke the static method for reading lines.
mv.visitMethodInsn(Opcodes.INVOKESTATIC,
"org/perlonjava/runtime/runtimetypes/DiamondIO",
"readline",
- "(Lorg/perlonjava/runtime/runtimetypes/RuntimeScalar;I)Lorg/perlonjava/runtime/runtimetypes/RuntimeBase;", false);
+ "(Lorg/perlonjava/runtime/runtimetypes/RuntimeScalar;IZ)Lorg/perlonjava/runtime/runtimetypes/RuntimeBase;", false);
// Handle context
if (emitterVisitor.ctx.contextType == RuntimeContextType.VOID) {
diff --git a/src/main/java/org/perlonjava/frontend/parser/OperatorParser.java b/src/main/java/org/perlonjava/frontend/parser/OperatorParser.java
index 9867836b8..e8a4659ee 100644
--- a/src/main/java/org/perlonjava/frontend/parser/OperatorParser.java
+++ b/src/main/java/org/perlonjava/frontend/parser/OperatorParser.java
@@ -326,6 +326,15 @@ static BinaryOperatorNode parsePrint(Parser parser, LexerToken token, int curren
handle = operand.handle;
operand.handle = null;
+ // `$.' immediately followed by concatenation, as in
+ // `print $..$ARGV.$_`, is a print argument rather than a filehandle.
+ // The lexer represents the second dot as the following expression,
+ // which otherwise makes the filehandle probe consume `$.' and discard
+ // the leading output value.
+ if (isInputLineNumber(handle)) {
+ operand.elements.addFirst(handle);
+ handle = null;
+ }
if (handle == null) {
// `print` without arguments means `print to last selected filehandle`
handle = new OperatorNode("select", new ListNode(currentIndex), currentIndex);
@@ -339,6 +348,13 @@ static BinaryOperatorNode parsePrint(Parser parser, LexerToken token, int curren
return new BinaryOperatorNode(token.text, handle, operand, currentIndex);
}
+ private static boolean isInputLineNumber(Node node) {
+ return node instanceof OperatorNode sigil
+ && "$".equals(sigil.operator)
+ && sigil.operand instanceof IdentifierNode identifier
+ && ".".equals(identifier.name);
+ }
+
/** True for {@code print(foo(...), ...)}, but not {@code print(FH (...))}. */
private static boolean isParenthesizedBarewordCall(Parser parser) {
if (!peek(parser).text.equals("(")) {
diff --git a/src/main/java/org/perlonjava/frontend/parser/StringParser.java b/src/main/java/org/perlonjava/frontend/parser/StringParser.java
index cf7d91372..eb3d7136e 100644
--- a/src/main/java/org/perlonjava/frontend/parser/StringParser.java
+++ b/src/main/java/org/perlonjava/frontend/parser/StringParser.java
@@ -1097,7 +1097,14 @@ public static Node parseRawString(Parser parser, String operator) {
}
ListNode diamondList = new ListNode(rawStr.index);
diamondList.elements.add(interpolated);
- return new OperatorNode("<>", diamondList, rawStr.index);
+ OperatorNode diamond = new OperatorNode("<>", diamondList, rawStr.index);
+ // <> interpolates to an empty string, while <<>> preserves a
+ // literal "<>" marker. Keep that syntactic distinction after
+ // the operand later resolves to the ARGV glob.
+ if (interpolated instanceof StringNode stringNode && "<>".equals(stringNode.value)) {
+ diamond.setAnnotation("doubleDiamond", true);
+ }
+ return diamond;
}
}
diff --git a/src/main/java/org/perlonjava/runtime/io/CustomFileChannel.java b/src/main/java/org/perlonjava/runtime/io/CustomFileChannel.java
index 668f86f50..c9aa187fb 100644
--- a/src/main/java/org/perlonjava/runtime/io/CustomFileChannel.java
+++ b/src/main/java/org/perlonjava/runtime/io/CustomFileChannel.java
@@ -441,7 +441,18 @@ public RuntimeScalar close() {
*/
@Override
public RuntimeScalar eof() {
- return new RuntimeScalar(isEOF);
+ if (isEOF) {
+ return new RuntimeScalar(true);
+ }
+ // Perl's eof() probes a regular file even before readline has tried
+ // to consume it. In particular, a freshly opened /dev/null is EOF;
+ // waiting for a read first makes argumentless eof() disagree with
+ // standard Perl after STDIN is reopened.
+ try {
+ return new RuntimeScalar(fileChannel.position() >= fileChannel.size());
+ } catch (IOException e) {
+ return new RuntimeScalar(false);
+ }
}
/**
diff --git a/src/main/java/org/perlonjava/runtime/operators/IOOperator.java b/src/main/java/org/perlonjava/runtime/operators/IOOperator.java
index cedf73690..6a4f17028 100644
--- a/src/main/java/org/perlonjava/runtime/operators/IOOperator.java
+++ b/src/main/java/org/perlonjava/runtime/operators/IOOperator.java
@@ -544,6 +544,10 @@ public static RuntimeScalar tell(RuntimeScalar fileHandle) {
boolean argless = !fileHandle.getDefinedBoolean();
RuntimeIO fh = fileHandle.getRuntimeIO();
+ if (argless && DiamondIO.hasActiveTraversal()) {
+ return DiamondIO.eof();
+ }
+
// If no explicit filehandle was provided (tell with no args),
// fall back to the last accessed handle like Perl does.
if (fh == null) {
@@ -1159,6 +1163,10 @@ public static RuntimeScalar eof(RuntimeScalar fileHandle) {
boolean argless = !fileHandle.getDefinedBoolean();
RuntimeIO fh = fileHandle.getRuntimeIO();
+ if (argless && DiamondIO.hasActiveTraversal()) {
+ return DiamondIO.eof();
+ }
+
// Handle undefined or invalid filehandle
if (fh == null) {
if (argless) {
diff --git a/src/main/java/org/perlonjava/runtime/operators/WarnDie.java b/src/main/java/org/perlonjava/runtime/operators/WarnDie.java
index 3b9918d8f..05c102b82 100644
--- a/src/main/java/org/perlonjava/runtime/operators/WarnDie.java
+++ b/src/main/java/org/perlonjava/runtime/operators/WarnDie.java
@@ -801,6 +801,9 @@ public static String getFilehandleContext() {
* @return String with the bare handle name (e.g., "DATA", "STDIN"), or null if not found
*/
private static String findFilehandleName(RuntimeIO handle) {
+ if (DiamondIO.isDiamondReader(handle)) {
+ return ""; // caller adds angle brackets: <> line N
+ }
if (handle.globName != null && !handle.globName.isEmpty()) {
// Strip package prefix (e.g., "main::DATA" -> "DATA")
String name = handle.globName;
diff --git a/src/main/java/org/perlonjava/runtime/runtimetypes/DiamondIO.java b/src/main/java/org/perlonjava/runtime/runtimetypes/DiamondIO.java
index 26a16b05b..b1febadef 100644
--- a/src/main/java/org/perlonjava/runtime/runtimetypes/DiamondIO.java
+++ b/src/main/java/org/perlonjava/runtime/runtimetypes/DiamondIO.java
@@ -3,6 +3,7 @@
import org.perlonjava.app.cli.CompilerOptions;
import org.perlonjava.runtime.io.ClosedIOHandle;
import org.perlonjava.runtime.operators.Readline;
+import org.perlonjava.runtime.operators.WarnDie;
import java.io.IOException;
import java.nio.file.Files;
@@ -25,6 +26,9 @@ public static final class State {
boolean eofReached;
boolean readingStarted;
boolean argvWasInitiallyEmpty;
+ boolean doubleDiamond;
+ RuntimeIO stdinReader;
+ RuntimeIO lastDiamondReader;
int accumulatedLineNumber;
String inPlaceExtension;
boolean inPlaceEdit;
@@ -37,6 +41,9 @@ void clear() {
eofReached = false;
readingStarted = false;
argvWasInitiallyEmpty = false;
+ doubleDiamond = false;
+ stdinReader = null;
+ lastDiamondReader = null;
accumulatedLineNumber = 0;
inPlaceExtension = null;
inPlaceEdit = false;
@@ -76,35 +83,52 @@ public static void reset() {
* undefined scalar if EOF is reached for all files.
*/
public static RuntimeBase readline(RuntimeScalar arg, int ctx) {
+ return readline(arg, ctx, arg != null && "<>".equals(arg.toString()));
+ }
+
+ /** Reads diamond input while preserving whether the source used <<>>. */
+ public static RuntimeBase readline(RuntimeScalar arg, int ctx, boolean doubleDiamond) {
State state = state();
if (ctx == RuntimeContextType.LIST) {
// Handle LIST context
RuntimeList lines = new RuntimeList();
RuntimeScalar line;
- while ((line = (RuntimeScalar) readline(arg, RuntimeContextType.SCALAR)).type != RuntimeScalarType.UNDEF) {
+ while ((line = (RuntimeScalar) readline(arg, RuntimeContextType.SCALAR, doubleDiamond)).type != RuntimeScalarType.UNDEF) {
lines.elements.add(line);
}
return lines;
} else {
+ RuntimeArray argv = getGlobalArray("main::ARGV");
+ // A completed diamond loop is reusable after Perl code refills
+ // @ARGV. In particular, `@ARGV = (...)` between separate `<>`
+ // loops must begin a new traversal rather than retain EOF forever.
+ RuntimeIO stdin = getGlobalIO("main::STDIN").getRuntimeIO();
+ // A new non-empty @ARGV always starts a new traversal. An empty
+ // @ARGV can also be intentional: reopening STDIN between diamond
+ // loops must make the later loop consume the replacement handle.
+ if (state.eofReached && (!argv.isEmpty()
+ || (stdin != null && stdin != state.stdinReader))) {
+ resetTraversalState(state);
+ }
// Handle SCALAR context
// Initialize the reading process if it hasn't started yet
if (!state.readingStarted) {
state.readingStarted = true;
+ state.doubleDiamond = doubleDiamond;
// Check if @ARGV was initially empty to determine STDIN fallback behavior
- state.argvWasInitiallyEmpty = getGlobalArray("main::ARGV").isEmpty();
+ state.argvWasInitiallyEmpty = argv.isEmpty();
- RuntimeIO argv = getGlobalIO("main::ARGV").getRuntimeIO();
+ RuntimeIO argvHandle = getGlobalIO("main::ARGV").getRuntimeIO();
// Only use ARGV filehandle directly if @ARGV is empty (handles aliased filehandles like *ARGV = *DATA)
- if (argv != null && !(argv.ioHandle instanceof ClosedIOHandle) && getGlobalArray("main::ARGV").isEmpty()) {
- state.currentReader = argv;
+ if (argvHandle != null && !(argvHandle.ioHandle instanceof ClosedIOHandle) && getGlobalArray("main::ARGV").isEmpty()) {
+ state.currentReader = argvHandle;
} else if (state.argvWasInitiallyEmpty) {
- RuntimeIO stdin = getGlobalIO("main::STDIN").getRuntimeIO();
if (stdin == null || stdin.ioHandle instanceof ClosedIOHandle) {
state.eofReached = true;
return scalarUndef;
}
// Only use STDIN if @ARGV was initially empty, not if it became empty after processing files
- RuntimeArray.push(getGlobalArray("main::ARGV"), new RuntimeScalar("-"));
+ RuntimeArray.push(argv, new RuntimeScalar("-"));
}
}
@@ -124,11 +148,18 @@ public static RuntimeBase readline(RuntimeScalar arg, int ctx) {
RuntimeScalar line = Readline.readline(state.currentReader);
if (line.type != RuntimeScalarType.UNDEF) {
state.accumulatedLineNumber = state.currentReader.currentLineNumber;
+ state.lastDiamondReader = state.currentReader;
return line;
}
- // EOF for current file — save accumulated line count before discarding reader
+ // EOF for current file — close it before discarding it. The
+ // ARGV glob otherwise retains an exhausted but still-open
+ // handle, which incorrectly wins over a later reopened STDIN
+ // when a new empty-@ARGV diamond loop begins.
state.accumulatedLineNumber = state.currentReader.currentLineNumber;
+ state.lastDiamondReader = state.currentReader;
+ state.currentReader.close();
+ state.lastDiamondReader.currentLineNumber = state.accumulatedLineNumber;
state.currentReader = null;
}
}
@@ -165,6 +196,33 @@ private static boolean openNextFile() {
String originalFileName = fileName.toString();
String backupFileName = null;
+ // Unlike the zero-argument diamond fallback, an explicit empty @ARGV
+ // entry is a filename and must fail immediately. Passing it to the
+ // generic path opener resolves it as a directory on some platforms,
+ // which turns `while (<>)` into a non-terminating read attempt.
+ if (originalFileName.isEmpty()) {
+ GlobalVariable.getGlobalVariable("main::!").set("No such file or directory");
+ return dieCannotOpen(originalFileName);
+ }
+
+ if (!state.doubleDiamond && isForkLikeOpen(originalFileName)) {
+ WarnDie.warn(new RuntimeScalar("Forked open '" + originalFileName
+ + "' not meaningful in <>"), new RuntimeScalar("\n"));
+ return false;
+ }
+
+ // $ARGV shares its spelling with the ARGV typeglob, but it must remain
+ // a scalar slot. Some script entry paths leave the scalar map pointing
+ // at that glob; assigning through it performs a typeglob assignment
+ // and corrupts subsequent diamond reads. Replace that placeholder with
+ // the actual scalar value instead.
+ RuntimeScalar argvName = GlobalVariable.getGlobalVariable("main::ARGV");
+ if (argvName instanceof RuntimeGlob) {
+ GlobalVariable.globalVariables.put("main::ARGV", new RuntimeScalar(originalFileName));
+ } else {
+ argvName.set(originalFileName);
+ }
+
// Check if in-place editing is enabled (either via -i switch or $^I variable)
boolean isInPlaceEnabled = state.inPlaceEdit;
String extension = state.inPlaceExtension;
@@ -252,12 +310,40 @@ private static boolean openNextFile() {
// Open the renamed file for reading
String readerPath = state.tempFilePath != null ? state.tempFilePath.toString() : (backupFileName != null ? backupFileName : originalFileName);
- state.currentReader = RuntimeIO.open(readerPath);
+ if ("-".equals(readerPath) && (!state.doubleDiamond || state.argvWasInitiallyEmpty)) {
+ // A diamond '-' is the current Perl STDIN handle, which can have
+ // been reopened by the program; it is not necessarily System.in.
+ // With <<>>, a synthetic '-' created solely because @ARGV was
+ // empty retains that fallback; an explicit '-' argument remains a
+ // literal filename as Perl requires.
+ state.currentReader = getGlobalIO("main::STDIN").getRuntimeIO();
+ state.stdinReader = state.currentReader;
+ } else {
+ // The explicit two-argument form intentionally does not apply
+ // RuntimeIO.open(String)'s special empty-string and "-" handling:
+ // double diamond treats both as ordinary filenames.
+ state.currentReader = RuntimeIO.open(readerPath, "<");
+ }
+ if (state.currentReader == null) {
+ return dieCannotOpen(originalFileName);
+ }
getGlobalIO("main::ARGV").set(state.currentReader);
return state.currentReader != null;
}
+ private static boolean dieCannotOpen(String fileName) {
+ RuntimeScalar error = GlobalVariable.getGlobalVariable("main::!");
+ WarnDie.die(new RuntimeScalar("Can't open " + fileName + ": " + error),
+ new RuntimeScalar(WarnDie.getPerlLocationFromStack()));
+ return false; // unreachable; keeps the compiler's flow analysis explicit
+ }
+
+ private static boolean isForkLikeOpen(String fileName) {
+ String normalized = fileName.replaceAll("\\s+", "");
+ return "|-".equals(normalized) || "-|".equals(normalized);
+ }
+
/** Restore the handle selected before diamond in-place editing began. */
private static void finishInPlaceEditing() {
State state = state();
@@ -266,4 +352,72 @@ private static void finishInPlaceEditing() {
state.selectedHandleBeforeInPlace = null;
}
}
+
+ /** Reset only per-traversal state while retaining command-line -i settings. */
+ private static void resetTraversalState(State state) {
+ if (state.currentReader != null) {
+ state.currentReader.close();
+ }
+ if (state.currentWriter != null) {
+ state.currentWriter.close();
+ }
+ state.currentReader = null;
+ state.currentWriter = null;
+ state.eofReached = false;
+ state.readingStarted = false;
+ state.argvWasInitiallyEmpty = false;
+ state.stdinReader = null;
+ state.lastDiamondReader = null;
+ state.accumulatedLineNumber = 0;
+ state.tempFilePath = null;
+ finishInPlaceEditing();
+ }
+
+ /** True when {@code handle} is the active diamond reader. */
+ public static boolean isCurrentReader(RuntimeIO handle) {
+ State state = state();
+ return state.readingStarted && handle != null && handle == state.currentReader;
+ }
+
+ /** True once <> or <<>> has established its per-runtime traversal state. */
+ public static boolean hasActiveTraversal() {
+ return state().readingStarted;
+ }
+
+ /** True for a reader used by <> or <<>>, including its final EOF reader. */
+ public static boolean isDiamondReader(RuntimeIO handle) {
+ State state = state();
+ return handle != null && (handle == state.currentReader || handle == state.lastDiamondReader);
+ }
+
+ /**
+ * Implements argumentless eof() for the diamond handle. Perl considers it
+ * true only when the current source is exhausted and no later @ARGV entry
+ * remains; individual file boundaries are not final EOF.
+ */
+ public static RuntimeScalar eof() {
+ State state = state();
+ RuntimeArray argv = getGlobalArray("main::ARGV");
+ RuntimeIO argvHandle = getGlobalIO("main::ARGV").getRuntimeIO();
+ // A closed ARGV handle with queued entries is an explicit `close ARGV`
+ // request. With an empty @ARGV it is commonly just the exhausted
+ // reader that diamond itself closed, so STDIN must take precedence.
+ if (!argv.isEmpty() && argvHandle != null && argvHandle.ioHandle instanceof ClosedIOHandle) {
+ return RuntimeScalarCache.scalarTrue;
+ }
+
+ // Pending @ARGV entries guarantee that argumentless eof() is false;
+ // this is what lets eof() look ahead across diamond file boundaries.
+ if (!argv.isEmpty()) {
+ return RuntimeScalarCache.scalarFalse;
+ }
+
+ // With an empty @ARGV, diamond is defined in terms of the current
+ // STDIN handle. It may have been reopened since a previous traversal.
+ RuntimeIO stdin = getGlobalIO("main::STDIN").getRuntimeIO();
+ if (stdin != null && !(stdin.ioHandle instanceof ClosedIOHandle)) {
+ return stdin.eof();
+ }
+ return state.eofReached ? RuntimeScalarCache.scalarTrue : RuntimeScalarCache.scalarFalse;
+ }
}
diff --git a/src/test/resources/unit/diamond_argv_lifecycle.t b/src/test/resources/unit/diamond_argv_lifecycle.t
new file mode 100644
index 000000000..e3bfe675e
--- /dev/null
+++ b/src/test/resources/unit/diamond_argv_lifecycle.t
@@ -0,0 +1,49 @@
+use strict;
+use warnings;
+use File::Temp qw(tempfile);
+use Test::More;
+
+sub fixture {
+ my ($content) = @_;
+ my ($fh, $path) = tempfile();
+ print {$fh} $content;
+ close $fh;
+ return $path;
+}
+
+my $first = fixture("first\n");
+my $second = fixture("second\n");
+
+@ARGV = ($first, $second);
+my $global_text = '';
+while (<>) {
+ $global_text .= $_;
+}
+is $global_text, "first\nsecond\n",
+ 'diamond reads files assigned to global @ARGV';
+
+{
+ local @ARGV = ($first, $second);
+ my ($text, @lines);
+ while (<>) {
+ $text .= "$ARGV:$_";
+ push @lines, $.;
+ }
+ is $text, "$first:first\n$second:second\n",
+ 'diamond publishes the current filename through $ARGV';
+ is_deeply \@lines, [1, 2], 'diamond preserves $. across @ARGV files';
+}
+
+{
+ local @ARGV = ($second);
+ is scalar(<>), "second\n", 'diamond restarts after a previous @ARGV traversal';
+}
+
+@ARGV = ($first);
+local $/ = undef;
+scalar <>;
+open STDIN, '<', '/dev/null' or die "cannot reopen STDIN: $!";
+@ARGV = ();
+ok eof(), 'argumentless eof uses reopened STDIN after an active diamond reader';
+
+done_testing;
From e2c13487ee80872fc78bf4ee972d0c80d7fda828 Mon Sep 17 00:00:00 2001
From: "Flavio S. Glock"
Date: Thu, 10 Sep 2026 14:11:38 +0200
Subject: [PATCH 7/8] fix(io): preserve files after aborted in-place edits
Treat a lone '*' in-place extension as extensionless editing and restore the
temporary source when an unhandled error aborts the implicit edit loop.
Generated with [Codex](https://openai.com/codex/)
Co-Authored-By: Codex
---
docs/about/changelog.md | 3 ++
.../java/org/perlonjava/app/cli/Main.java | 2 +
.../runtime/runtimetypes/DiamondIO.java | 40 ++++++++++++++++++-
src/test/resources/unit/inplace_edit_abort.t | 35 ++++++++++++++++
4 files changed, 78 insertions(+), 2 deletions(-)
create mode 100644 src/test/resources/unit/inplace_edit_abort.t
diff --git a/docs/about/changelog.md b/docs/about/changelog.md
index 2cf1e35cc..dac681a78 100644
--- a/docs/about/changelog.md
+++ b/docs/about/changelog.md
@@ -8,6 +8,9 @@ priorities and future plans.
- Identify PerlOnJava, its copyright, and its dual-license terms in
`jperl -v` output while retaining the standard Perl text.
+
+- Preserve source files when extensionless in-place editing aborts, and treat
+ a lone `'*'` in-place extension like Perl's extensionless form.
- Prevent eval-created named subs from treating lexical variables as
same-named constant calls, restoring `Types::Numbers` loading through
`Data::Float`.
diff --git a/src/main/java/org/perlonjava/app/cli/Main.java b/src/main/java/org/perlonjava/app/cli/Main.java
index 40aa64eba..abfe8b192 100644
--- a/src/main/java/org/perlonjava/app/cli/Main.java
+++ b/src/main/java/org/perlonjava/app/cli/Main.java
@@ -3,6 +3,7 @@
import org.perlonjava.app.scriptengine.PerlLanguageProvider;
import org.perlonjava.runtime.operators.WarnDie;
import org.perlonjava.runtime.runtimetypes.ErrorMessageUtil;
+import org.perlonjava.runtime.runtimetypes.DiamondIO;
import org.perlonjava.runtime.runtimetypes.GlobalVariable;
import org.perlonjava.runtime.runtimetypes.PerlExitException;
import org.perlonjava.runtime.runtimetypes.PerlRuntime;
@@ -155,6 +156,7 @@ private static void run(String[] args) {
System.exit(PerlRuntime.current().threadRegistry()
.requestedProcessExitOr(e.getExitCode()));
} catch (Throwable t) {
+ DiamondIO.abortInPlaceEditing();
if (parsedArgs.debugEnabled) {
// Print full JVM stack
t.printStackTrace();
diff --git a/src/main/java/org/perlonjava/runtime/runtimetypes/DiamondIO.java b/src/main/java/org/perlonjava/runtime/runtimetypes/DiamondIO.java
index b1febadef..4b0705548 100644
--- a/src/main/java/org/perlonjava/runtime/runtimetypes/DiamondIO.java
+++ b/src/main/java/org/perlonjava/runtime/runtimetypes/DiamondIO.java
@@ -33,6 +33,7 @@ public static final class State {
String inPlaceExtension;
boolean inPlaceEdit;
Path tempFilePath;
+ Path inPlaceOriginalPath;
RuntimeIO selectedHandleBeforeInPlace;
void clear() {
@@ -48,6 +49,7 @@ void clear() {
inPlaceExtension = null;
inPlaceEdit = false;
tempFilePath = null;
+ inPlaceOriginalPath = null;
selectedHandleBeforeInPlace = null;
}
}
@@ -244,8 +246,10 @@ private static boolean openNextFile() {
// Use RuntimeIO's existing path resolution methods for consistency
Path originalPath = RuntimeIO.resolvePath(originalFileName);
- if (extension == null || extension.isEmpty()) {
- // Create a temporary file for the original file
+ if (extension == null || extension.isEmpty() || "*".equals(extension)) {
+ // A lone '*' is Perl's extensionless form. It must use a
+ // temporary backup rather than substitute the source name
+ // into the backup path and move a file onto itself.
try {
state.tempFilePath = Files.createTempFile("temp_", null);
backupFileName = state.tempFilePath.toString();
@@ -294,6 +298,8 @@ private static boolean openNextFile() {
}
}
+ state.inPlaceOriginalPath = originalPath;
+
// Open the original file for writing (this is the ARGVOUT equivalent)
// Use the resolved path to ensure we write to the correct location
state.currentWriter = RuntimeIO.open(originalPath.toString(), ">");
@@ -353,6 +359,35 @@ private static void finishInPlaceEditing() {
}
}
+ /**
+ * Restore the current extensionless in-place source after an unhandled
+ * exception. Perl's -i uses a temporary backup in this form and leaves
+ * the source intact when the implicit loop aborts before completion.
+ */
+ public static void abortInPlaceEditing() {
+ State state = state();
+ if (state.currentReader != null) {
+ state.currentReader.close();
+ state.currentReader = null;
+ }
+ if (state.currentWriter != null) {
+ state.currentWriter.close();
+ state.currentWriter = null;
+ }
+ if (state.tempFilePath != null && state.inPlaceOriginalPath != null) {
+ try {
+ Files.move(state.tempFilePath, state.inPlaceOriginalPath,
+ StandardCopyOption.REPLACE_EXISTING);
+ } catch (IOException ignored) {
+ // Preserve the original Perl exception; failed restoration is
+ // reported by the already-active in-place edit diagnostic.
+ }
+ }
+ state.tempFilePath = null;
+ state.inPlaceOriginalPath = null;
+ finishInPlaceEditing();
+ }
+
/** Reset only per-traversal state while retaining command-line -i settings. */
private static void resetTraversalState(State state) {
if (state.currentReader != null) {
@@ -370,6 +405,7 @@ private static void resetTraversalState(State state) {
state.lastDiamondReader = null;
state.accumulatedLineNumber = 0;
state.tempFilePath = null;
+ state.inPlaceOriginalPath = null;
finishInPlaceEditing();
}
diff --git a/src/test/resources/unit/inplace_edit_abort.t b/src/test/resources/unit/inplace_edit_abort.t
new file mode 100644
index 000000000..2b3d74a3b
--- /dev/null
+++ b/src/test/resources/unit/inplace_edit_abort.t
@@ -0,0 +1,35 @@
+use strict;
+use warnings;
+use File::Temp qw(tempfile);
+use Test::More tests => 3;
+
+my ($handle, $path) = tempfile();
+print {$handle} "bar\n";
+close $handle;
+
+{
+ local $^I = '*';
+ local @ARGV = ($path);
+ while (<>) {
+ print "foo$_";
+ }
+}
+
+open $handle, '<', $path or die "Cannot read $path: $!";
+is(do { local $/; <$handle> }, "foobar\n",
+ q{$^I = '*' performs in-place editing without replacing the input by an empty backup});
+close $handle;
+
+open $handle, '>', $path or die "Cannot rewrite $path: $!";
+print {$handle} "bar\n";
+close $handle;
+
+my $runner = $^X eq 'jperl' ? './jperl' : $^X;
+my $status = system($runner, '-i', '-n', '-e', 'die', $path);
+ok($status != 0, 'an in-place program that dies exits unsuccessfully');
+
+open $handle, '<', $path or die "Cannot read aborted edit $path: $!";
+is(do { local $/; <$handle> }, "bar\n",
+ 'an aborted extensionless in-place edit preserves the original file');
+close $handle;
+unlink $path;
From 9c7955157cdd25460c6085b99b675ff9ae1ae151 Mon Sep 17 00:00:00 2001
From: "Flavio S. Glock"
Date: Thu, 10 Sep 2026 15:04:54 +0200
Subject: [PATCH 8/8] test(io): make diamond lifecycle regression portable
Use File::Spec's platform null device so the ARGV lifecycle regression runs on Windows.
Generated with [Codex](https://openai.com/codex/)
Co-Authored-By: Codex
---
src/test/resources/unit/diamond_argv_lifecycle.t | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/src/test/resources/unit/diamond_argv_lifecycle.t b/src/test/resources/unit/diamond_argv_lifecycle.t
index e3bfe675e..f854588fa 100644
--- a/src/test/resources/unit/diamond_argv_lifecycle.t
+++ b/src/test/resources/unit/diamond_argv_lifecycle.t
@@ -1,5 +1,6 @@
use strict;
use warnings;
+use File::Spec;
use File::Temp qw(tempfile);
use Test::More;
@@ -42,7 +43,7 @@ is $global_text, "first\nsecond\n",
@ARGV = ($first);
local $/ = undef;
scalar <>;
-open STDIN, '<', '/dev/null' or die "cannot reopen STDIN: $!";
+open STDIN, '<', File::Spec->devnull or die "cannot reopen STDIN: $!";
@ARGV = ();
ok eof(), 'argumentless eof uses reopened STDIN after an active diamond reader';