Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
83 changes: 83 additions & 0 deletions .agents/skills/graalpython-rota/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
---
name: graalpython-rota
description: Run GraalPy ROTA maintenance workflows for (1) import update pull requests and (2) triage of recent periodic job failures in Jira. Use when asked to perform or guide recurring ROTA tasks from `docs/contributor/ROTA.md`, including branch setup, `mx` update commands, PR creation with reviewers/gates via `ol-cli bitbucket`, and date-bounded periodic-failure issue triage via `ol-cli jira`.
---

# GraalPy ROTA

## Overview
Execute recurring GraalPy ROTA tasks with exact commands and strict output structure. Prefer the procedures in this skill, and use `docs/contributor/ROTA.md` as the detailed source text.

## Source Of Truth
- Primary: `docs/contributor/ROTA.md`
- If this skill workflow differs from the primary source, follow `docs/contributor/ROTA.md`.

## Choose Workflow
- Use `Import update` when asked to refresh imports and open the standard PR.
- Use `Recent periodic issues` when asked to triage periodic job failures in Jira.

## Import Update Workflow
1. Create a branch from latest `master`:
```bash
git checkout master
git pull --ff-only
git checkout -b "update/GR-21590/$(date +%d%m%y)"
```
2. Update graal import:
```bash
mx python-update-import
```
3. Update CPython unittest whitelist and inspect diff for plausibility. Expect mostly additions, not removals:
```bash
mx --dy /graalpython-enterprise python-update-unittest-tags
```
4. Create PR with description `[GR-21590] Import update`.
5. Use `ol-cli bitbucket` to create PR, start gates, and set reviewers:
- `tim.felgentreff@oracle.com`
- `michael.simacek@oracle.com`
- `stepan.sindelar@oracle.com`
6. Fix gate failures and push updates until gates pass.

## Recent Periodic Issues Workflow
1. Verify creator identity mapping:
- Treat `ol-automation_ww` as Jira username `olauto`.
- If query returns zero results, test both identities, then keep `creator = olauto` once verified.

2. Filter to recent periodic job failures, excluding in progress or closed.
- Default to the last 14 days unless user specifies otherwise.
- Always state concrete start/end calendar dates in the response.
```bash
ol-cli jira search --json --max 100 \
-f key,summary,creator,created,status,labels,components,assignee \
-jql "project = GR AND component = Python AND creator = olauto AND labels = periodic-job-failures AND created >= -14d AND status != Closed AND status != 'In Progress' ORDER BY created DESC"
```

3. Fetch shortlisted issue details with `get-issue`:
```bash
ol-cli jira get-issue --json -id GR-XXXX \
| jq '{key, summary:.fields.summary, status:.fields.status.name, created:.fields.created, labels:.fields.labels, assignee:(.fields.assignee.name // null), description:.fields.description, comments:(.fields.comment.comments | map({author:.author.name, created, body}))}'
```

7. Convert findings into an implementation-ready plan per issue:
- Extract failing job name, error signature, and log clue.
- Map probable source area in repo.
- Propose first verification command.
- Define exit criteria to close ticket.
- Prepare temporary git worktree per issue with branch naming based on Jira key plus very short hyphenated description.

## Output Contract For Periodic Triage
Return exactly:
1. Query scope used (component, creator, time window, status filter).
2. Count summary (total recent automation issues vs periodic failures).
3. Issue list with key, created date, summary, status.
4. Per-issue plan with:
- Hypothesis
- First code locations to inspect
- First reproducibility command
- Exit criteria for closing ticket
5. Recommended implementation order.

## Guardrails
- State concrete dates for recency windows.
- Prefer `--json` and explicit `-f` fields in searches.
- Use `get-issue` only for shortlisted issues to keep output small.
59 changes: 59 additions & 0 deletions .agents/skills/pr-gate-check/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
---
name: pr-gate-check
description: Check gate status for a Bitbucket PR by resolving the PR head commit, finding the gate merge commit, inspecting builds on that merge commit, and summarizing root-cause failures with actionable next steps.
---

# PR Gate Check

## Overview
Use this workflow when asked for gate status of a PR. Usually the builds are tied to a merge commit generated on Bitbucket, so this skill goes through finding the remote merge commit.

## Workflow
1. Get PR commits and identify PR head commit (first commit in `ol-cli bitbucket commits` output):
```bash
ol-cli bitbucket commits --project=G --repo=graalpython --pullrequest=<PR_ID> --all --json
```

2. Fetch refs and locate merge commit whose parent includes PR head:
```bash
git ls-remote origin 'refs/pull-requests/<PR_ID>/*'
git fetch --no-tags origin '+refs/heads/*:refs/remotes/origin/*' --prune
git rev-list --all --parents | rg ' <PR_HEAD_SHA>( |$)'
```
Pick the merge commit where one parent is `<PR_HEAD_SHA>` and the other is the target branch tip at merge time.

3. Check builds on that merge commit:
```bash
ol-cli bitbucket get-builds --commit=<MERGE_SHA> --all --format=key,state,url
```

4. Separate root failures from fan-out failures:
- `FAILED` + `/builders/.../builds/...` URL: executed failed build (root failure candidate).
- `FAILED` + `build_request?brid=` URL: usually not-run/downstream due to earlier failure.

5. Inspect root failed build logs and extract exact failing test/error:
- Open build URL and `Run executor` stdio log.
- Capture failing test id, traceback/assertion, and command context.

6. Report back:
- PR head SHA
- merge SHA + parents (target parent + PR parent)
- build summary counts
- root cause failure(s)
- fix options and next action question

## Output Template
1. `PR head:` `<sha>`
2. `Gate merge commit:` `<sha>` (`parent1=<target_sha>`, `parent2=<pr_sha>`)
3. `Builds:` `<total>` total, `<success>` successful, `<failed>` failed
4. `Root failure(s):`
- `<build key>`: `<error summary>`
- `<failing test/path>`
- `<build url>`
5. `Proposed fixes:` short list
6. Ask user what to do next.

## Guardrails
- Do not conclude from PR commit statuses alone; always resolve and inspect merge-commit builds.
- If many builds are failed but only one executed failure exists, treat that one as primary cause.
- Keep proposed fixes minimal and scoped to observed failure.
2 changes: 1 addition & 1 deletion ci/graal/common.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
"Jsonnet files should not include this file directly but use ci/common.jsonnet instead."
],

"mx_version": "7.69.0",
"mx_version": "7.71.0",

"COMMENT.jdks": "When adding or removing JDKs keep in sync with JDKs in ci/common.jsonnet",
"jdks": {
Expand Down
12 changes: 8 additions & 4 deletions graalpython/com.oracle.graal.python.test/src/tests/test_fcntl.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (c) 2020, 2024, Oracle and/or its affiliates. All rights reserved.
# Copyright (c) 2020, 2026, Oracle and/or its affiliates. All rights reserved.
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
#
# The Universal Permissive License (UPL), Version 1.0
Expand Down Expand Up @@ -65,6 +65,7 @@ def log(msg):
def python_flock_blocks_sh_flock(python_flock_type, sh_flock_type):
os.close(os.open(TEST_FILENAME_FULL_PATH, os.O_WRONLY | os.O_CREAT))
file = os.open(TEST_FILENAME_FULL_PATH, os.O_WRONLY)
p = None
try:
fcntl.flock(file, python_flock_type)
p = subprocess.Popen("flock -%s %s -c 'exit 42'" % (sh_flock_type, TEST_FILENAME_FULL_PATH), shell=True)
Expand All @@ -74,10 +75,13 @@ def python_flock_blocks_sh_flock(python_flock_type, sh_flock_type):
log("unlocking the file...")
fcntl.flock(file, fcntl.LOCK_UN) # release the lock
log("checking the retcode...")
time.sleep(0.25)
assert p.poll() == 42
log(f"{p.returncode=}")
retcode = p.wait(timeout=5)
assert retcode == 42
log(f"{retcode=}")
finally:
if p is not None and p.poll() is None:
p.terminate()
p.wait(timeout=2)
fcntl.flock(file, fcntl.LOCK_UN)
os.close(file)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -59,9 +59,11 @@ def test_map_timeout(self):
# GraalPy change: submit some dummy work first, so the next map call doesn't time out in the worker start up
list(self.executor.map(time.sleep, [0]))
try:
# GraalPy change: larger timeout for CI flakiness
# for i in self.executor.map(time.sleep, [0, 0, 6], timeout=5):
for i in self.executor.map(time.sleep,
[0, 0, 6],
timeout=5):
[0, 0, 8],
timeout=3):
results.append(i)
except futures.TimeoutError:
pass
Expand Down
11 changes: 9 additions & 2 deletions mx.graalpython/mx_graalpython.py
Original file line number Diff line number Diff line change
Expand Up @@ -2347,7 +2347,7 @@ def python_coverage(args):
'--strict-mode',
'--tags', args.tags,
] + jacoco_args, env=env)
run_mx([
jacoco_report_cmd = [
'--strict-compliance',
'--kill-with-sigquit',
'jacocoreport',
Expand All @@ -2356,7 +2356,14 @@ def python_coverage(args):
'coverage',
'--generic-paths',
'--exclude-src-gen',
], env=env)
]
# CI can occasionally leave transiently truncated execution data; retry once to reduce flakiness.
try:
run_mx(jacoco_report_cmd, env=env)
except Exception: # pylint: disable=broad-except
mx.warn("jacocoreport failed, retrying once after a short delay")
time.sleep(5)
run_mx(jacoco_report_cmd, env=env)

if args.mode == 'truffle':
executable = graalpy_standalone_jvm()
Expand Down
11 changes: 7 additions & 4 deletions mx.graalpython/mx_graalpython_python_benchmarks.py
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,8 @@
"bench_core.CountNonzero.time_count_nonzero_multi_axis(2, 1000000, <class 'str'>)", # Times out
"bench_core.CountNonzero.time_count_nonzero_multi_axis(3, 1000000, <class 'str'>)", # Times out
"bench_linalg.LinalgSmallArrays.time_det_small_array", # TODO fails with numpy.linalg.LinAlgError
"bench_indexing.IndexingSeparate.time_mmap_fancy_indexing", # Hangs in periodic job GR-73912
"bench_indexing.IndexingStructured0D.time_array_slice", # Hangs in periodic job GR-73912
]

DEFAULT_PANDAS_BENCHMARKS = [
Expand All @@ -98,6 +100,8 @@
"reshape.Explode.time_explode", # Transient failure GR-61245, exit code -11
]

SETUPTOOLS_PIN = "77.0.1"

DEFAULT_PYPERFORMANCE_BENCHMARKS = [
# "2to3",
# "chameleon",
Expand Down Expand Up @@ -564,7 +568,7 @@ class NumPySuite(PySuite):

BENCHMARK_REQ = [
"asv==0.5.1",
"setuptools==70.3.0",
f"setuptools=={SETUPTOOLS_PIN}",
"distlib==0.3.6",
"filelock==3.8.0",
"platformdirs==2.5.2",
Expand Down Expand Up @@ -671,7 +675,7 @@ class PandasSuite(PySuite):

BENCHMARK_REQ = [
"asv==0.5.1",
"setuptools==70.3.0",
f"setuptools=={SETUPTOOLS_PIN}",
"distlib==0.3.6",
"filelock==3.8.0",
"platformdirs==2.5.2",
Expand Down Expand Up @@ -762,8 +766,7 @@ def _vmRun(self, vm, workdir, command, benchmarks, bmSuiteArgs):
vm.run(workdir, ["-m", "venv", join(workdir, vm_venv)])
pip = join(workdir, vm_venv, "bin", "pip")
with tempfile.NamedTemporaryFile('w') as constraints:
# Constrain the version of setuptools used to build pandas
constraints.write('setuptools==70.3.0\n')
constraints.write(f"setuptools=={SETUPTOOLS_PIN}\n")
constraints.flush()
env = os.environ.copy()
env['PIP_CONSTRAINT'] = constraints.name
Expand Down
4 changes: 2 additions & 2 deletions mx.graalpython/suite.py
Original file line number Diff line number Diff line change
Expand Up @@ -53,15 +53,15 @@
},
{
"name": "tools",
"version": "bb17fd7e8ec441c087b63300c2d75e06828b8dde",
"version": "fcbc5553eb65a0d6c88495efc24e998606dd5fa6",
"subdir": True,
"urls": [
{"url": "https://github.com/oracle/graal", "kind": "git"},
],
},
{
"name": "regex",
"version": "bb17fd7e8ec441c087b63300c2d75e06828b8dde",
"version": "fcbc5553eb65a0d6c88495efc24e998606dd5fa6",
"subdir": True,
"urls": [
{"url": "https://github.com/oracle/graal", "kind": "git"},
Expand Down
Loading