Update PG16 prior to 1.7.0 part 1#2358
Merged
MuhammadTahaNaveed merged 20 commits intoapache:PG16from Mar 26, 2026
Merged
Conversation
…pache#2217) Fixed 2 issues with CALL/YIELD - 1) If a user defined function was in search_path, the transform_FuncCall logic would only find it, if it were part of an extension. 2) If a function were qualified, the transform_cypher_call_subquery logic would mistakenly extract the schema name instead of the function name. NOTE: transform_FuncCall should be reviewed for possible refactor. Added regression tests. modified: src/backend/parser/cypher_clause.c modified: src/backend/parser/cypher_expr.c modified: regress/expected/cypher_call.out modified: regress/sql/cypher_call.sql
…2212) Bumps gopkg.in/yaml.v3 from 3.0.0 to 3.0.1. --- updated-dependencies: - dependency-name: gopkg.in/yaml.v3 dependency-version: 3.0.1 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Added fast functions for checking edge uniqueness. This will help
improve performance for MATCH queries with paths longer than 3 but
less than 11. The normal edge uniqueness function will deal with
any path 11 and over.
modified: age--1.6.0--y.y.y.sql
modified: sql/agtype_graphid.sql
modified: src/backend/parser/cypher_clause.c
modified: src/backend/utils/adt/age_vle.c
Fixed issue 2243 - Regression in string concatenation using the + operator.
The issue was in versions 1.5.0 and 1.6.0, at least. It was due to using
Int8GetDatum instead of Int64GetDatum for the agtype integer field in the
following functions -
get_numeric_datum_from_agtype_value
get_string_from_agtype_value
This impacted more than what the original issue covered, but those additional
cases were resolved too.
Added regression tests.
modified: regress/expected/agtype.out
modified: regress/sql/agtype.sql
modified: src/backend/utils/adt/agtype_ops.c
…raph (apache#2248) Fixed issue 2245 - Creating more than 41 vlabels causes drop_grapth to fail with "label (relation) cache corrupted" and crashing out on the following command. This was due to corruption of the label_relation_cache during the HASH_DELETE process. As the issue was with a cache flush routine, it was necessary to fix them all. Here is the list of the flush functions that were fixed - static void flush_graph_name_cache(void) static void flush_graph_namespace_cache(void) static void flush_label_name_graph_cache(void) static void flush_label_graph_oid_cache(void) static void flush_label_relation_cache(void) static void flush_label_seq_name_graph_cache(void) Added regression tests. modified: regress/expected/catalog.out modified: regress/sql/catalog.sql modified: src/backend/utils/cache/ag_cache.c
…ache#2259) Fixed issue 2256: A segmentation fault occurs when calling the coalesce function in PostgreSQL version 17. This likely predates 17 and includes other similar types of "functions". And other versions of PostgreSQL. See issues 1124 (PR 1125) and 1303 (PR 1317) for more details. This issue is due to coalesce() being processed differently from other functions. Additionally, greatest() was found to exhibit the same behavior. They were added to the list of types to ignore during the cypher analyze phase. A few others were added: CaseExpr, XmlExpr, ArrayExpr, & RowExpr. Although, I wasn't able to find cases where these caused crashes. Added regression tests. modified: regress/expected/cypher.out modified: regress/sql/cypher.sql modified: src/backend/parser/cypher_analyze.c Conflicts: src/backend/parser/cypher_analyze.c
Adjusted the following type of error message. It was mentioned in
issue 2263 as being incorrect, which it isn't. However, it did need
some clarification added -
ERROR: could not find rte for <column name>
Added a HINT for additional clarity -
HINT: variable <column name> does not exist within scope of usage
For example:
CREATE p0=(n0), (n1{k:EXISTS{WITH p0}}) RETURN 1
ERROR: could not find rte for p0
LINE 3: CREATE p0=(n0), (n1{k:EXISTS{WITH p0}})
^
HINT: variable p0 does not exist within scope of usage
Additionally, added pstate->p_expr_kind == EXPR_KIND_INSERT_TARGET to
transform_cypher_clause_as_subquery.
Updated existing regression tests.
Added regression tests from issue.
modified: regress/expected/cypher_call.out
modified: regress/expected/cypher_subquery.out
modified: regress/expected/cypher_union.out
modified: regress/expected/cypher_with.out
modified: regress/expected/expr.out
modified: regress/expected/list_comprehension.out
modified: regress/expected/scan.out
modified: src/backend/parser/cypher_clause.c
modified: src/backend/parser/cypher_expr.c
Resolved Conflicts:
regress/expected/cypher_subquery.out
NOTE: This PR was partially created with AI tools and reviewed by a human.
ORDER BY clauses failed when referencing column aliases from RETURN:
MATCH (p:Person) RETURN p.age AS age ORDER BY age DESC
ERROR: could not find rte for age
Added SQL-99 compliant alias matching to find_target_list_entry() that
checks if ORDER BY identifier matches a target list alias before
attempting expression transformation. This enables standard SQL behavior
for sorting by aliased columns with DESC/DESCENDING/ASC/ASCENDING.
Updated regression tests.
Added regression tests.
modified: regress/expected/cypher_match.out
modified: regress/expected/expr.out
modified: regress/sql/expr.sql
modified: src/backend/parser/cypher_clause.c
Consolidated duplicate code, added helper functions, and reviewed
the grammar file for issues.
NOTE: I used an AI tool to review and cleanup the grammar file. I
have reviewed all of the work it did.
Improvements:
1. Added KEYWORD_STRDUP macro to eliminate hardcoded string lengths
2. Consolidated EXPLAIN statement handling into make_explain_stmt helper
3. Extracted WITH clause validation into validate_return_item_aliases helper
4. Created make_default_return_node helper for subquery return-less logic
Benefits:
- Reduced code duplication by ~150 lines
- Improved maintainability with helper functions
- Eliminated manual string length calculations (error-prone)
All 29 existing regression tests pass
modified: src/backend/parser/cypher_gram.y
apache#2267) - Changed '\s' to r'\s'
- Add pyproject.toml with package configuration - Simplify setup.py to minimal backward-compatible wrapper. - Updated CI workflow and .gitignore. - Resolves warning about using setup.py directly.
The file cypher_gram.c generates cypher_gram_def.h, which is directly
necessary for cypher_parser.o and cypher_keywords.o and their respective
.bc files.
But that direct dependency is not reflected in the Makefile, which only
had the indirect dependency of .o on .c. So on high parallel builds, the
.h may not have been generated by bison yet.
Additionally, the .bc files should have the same dependencies as the .o
files, but those are lacking.
Here is an example output where the .bc file fails to build, as it was
running concurrently with the bison instance that was about to finalize
cypher_gram_def.h:
In file included from src/backend/parser/cypher_parser.c:24:
clang-17 -Wno-ignored-attributes -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-unused-command-line-argument -Wno-compound-token-split-by-macro -O2 -I.//src/include -I.//src/include/parser -I. -I./ -I/usr/pgsql-17/include/server -I/usr/pgsql-17/include/internal -D_GNU_SOURCE -I/usr/include -I/usr/include/libxml2 -flto=thin -emit-llvm -c -o src/backend/parser/cypher_parser.bc src/backend/parser/cypher_parser.c
.//src/include/parser/cypher_gram.h:65:10: fatal error: 'parser/cypher_gram_def.h' file not found
65 | #include "parser/cypher_gram_def.h"
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
make: *** [/usr/pgsql-17/lib/pgxs/src/makefiles/../../src/Makefile.global:1085: src/backend/parser/cypher_parser.bc] Error 1
make: *** Waiting for unfinished jobs....
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wshadow=compatible-local -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -O2 -g -fmessage-length=0 -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -fPIC -fvisibility=hidden -I.//src/include -I.//src/include/parser -I. -I./ -I/usr/pgsql-17/include/server -I/usr/pgsql-17/include/internal -D_GNU_SOURCE -I/usr/include -I/usr/include/libxml2 -c -o src/backend/catalog/ag_label.o src/backend/catalog/ag_label.c
/usr/bin/bison -Wno-deprecated --defines=src/include/parser/cypher_gram_def.h -o src/backend/parser/cypher_gram.c src/backend/parser/cypher_gram.y
Previously, cypher_parser.o was missing the dependency, so it could
start before cypher_gram_def.h was available:
Considering target file 'src/backend/parser/cypher_parser.o'.
File 'src/backend/parser/cypher_parser.o' does not exist.
Considering target file 'src/backend/parser/cypher_parser.c'.
File 'src/backend/parser/cypher_parser.c' was considered already.
Considering target file 'src/backend/parser/cypher_gram.c'.
File 'src/backend/parser/cypher_gram.c' was considered already.
Finished prerequisites of target file 'src/backend/parser/cypher_parser.o'.
Must remake target 'src/backend/parser/cypher_parser.o'.
As well as cypher_parser.bc, missing the dependency on
cypher_gram_def.h:
Considering target file 'src/backend/parser/cypher_parser.bc'.
File 'src/backend/parser/cypher_parser.bc' does not exist.
Considering target file 'src/backend/parser/cypher_parser.c'.
File 'src/backend/parser/cypher_parser.c' was considered already.
Finished prerequisites of target file 'src/backend/parser/cypher_parser.bc'.
Must remake target 'src/backend/parser/cypher_parser.bc'.
Now cypher_parser.o correctly depends on cypher_gram_def.h:
Considering target file 'src/backend/parser/cypher_parser.o'.
File 'src/backend/parser/cypher_parser.o' does not exist.
Considering target file 'src/backend/parser/cypher_parser.c'.
File 'src/backend/parser/cypher_parser.c' was considered already.
Considering target file 'src/backend/parser/cypher_gram.c'.
File 'src/backend/parser/cypher_gram.c' was considered already.
Considering target file 'src/include/parser/cypher_gram_def.h'.
File 'src/include/parser/cypher_gram_def.h' was considered already.
Finished prerequisites of target file 'src/backend/parser/cypher_parser.o'.
Must remake target 'src/backend/parser/cypher_parser.o'.
And cypher_parser.bc correctly depends on cypher_gram_def.h as well:
Considering target file 'src/backend/parser/cypher_parser.bc'.
File 'src/backend/parser/cypher_parser.bc' does not exist.
Considering target file 'src/backend/parser/cypher_parser.c'.
File 'src/backend/parser/cypher_parser.c' was considered already.
Considering target file 'src/backend/parser/cypher_gram.c'.
File 'src/backend/parser/cypher_gram.c' was considered already.
Considering target file 'src/include/parser/cypher_gram_def.h'.
File 'src/include/parser/cypher_gram_def.h' was considered already.
Finished prerequisites of target file 'src/backend/parser/cypher_parser.bc'.
Must remake target 'src/backend/parser/cypher_parser.bc'.
Updated README to from psycopg2 to psycopg3 (psycopg) Resolved Conflicts: drivers/python/README.md
NOTE: This PR was created with AI tools and a human. When evaluating 'x IN []' with an empty list, the transform_AEXPR_IN function would return NULL because no expressions were processed. This caused a 'cache lookup failed for type 0' error downstream. This fix adds an early check for the empty list case: - 'x IN []' returns false (nothing can be in an empty list) Additional NOTE: Cypher does not have 'NOT IN' syntax. To check if a value is NOT in a list, use 'NOT (x IN list)'. The NOT operator will invert the false from an empty list to true as expected. The fix returns a boolean constant directly, avoiding the NULL result that caused the type lookup failure. Added regression tests. modified: regress/expected/expr.out modified: regress/sql/expr.sql modified: src/backend/parser/cypher_expr.c
* feat: Add 32-bit platform support for graphid type This enables AGE to work on 32-bit platforms including WebAssembly (WASM). Problem: - graphid is int64 (8 bytes) with PASSEDBYVALUE - On 32-bit systems, Datum is only 4 bytes - PostgreSQL rejects pass-by-value types larger than Datum Solution: - Makefile-only change (no C code modifications) - When SIZEOF_DATUM=4 is passed to make, strip PASSEDBYVALUE from the generated SQL - If not specified, normal 64-bit behavior is preserved (PASSEDBYVALUE kept) This change is backward compatible: - 64-bit systems continue using pass-by-value - 32-bit systems now work with pass-by-reference Motivation: PGlite (PostgreSQL compiled to WebAssembly) uses 32-bit pointers and requires this patch to run AGE. Tested on: - 64-bit Linux (regression tests pass) - 32-bit WebAssembly via PGlite (all operations work) Co-authored-by: abbuehlj <jean-paul.abbuehl@roche.com>
…2302) NOTE: This PR was created using AI tools and a human. Leverage deterministic key ordering from uniqueify_agtype_object() to access vertex/edge fields in O(1) instead of O(log n) binary search. Fields are sorted by key length, giving fixed positions: - Vertex: id(0), label(1), properties(2) - Edge: id(0), label(1), end_id(2), start_id(3), properties(4) Changes: - Add field index constants and accessor macros to agtype.h - Update age_id(), age_start_id(), age_end_id(), age_label(), age_properties() to use direct field access - Add fill_agtype_value_no_copy() for read-only scalar extraction without memory allocation - Add compare_agtype_scalar_containers() fast path for scalar comparison - Update hash_agtype_value(), equals_agtype_scalar_value(), and compare_agtype_scalar_values() to use direct field access macros - Add fast path in get_one_agtype_from_variadic_args() bypassing extract_variadic_args() for single argument case - Add comprehensive regression test (30 tests) Performance impact: Improves ORDER BY, hash joins, aggregations, and Cypher functions (id, start_id, end_id, label, properties) on vertices and edges. All previous regression tests were not impacted. Additional regression test added to enhance coverage. modified: Makefile new file: regress/expected/direct_field_access.out new file: regress/sql/direct_field_access.sql modified: src/backend/utils/adt/agtype.c modified: src/backend/utils/adt/agtype_util.c modified: src/include/utils/agtype.h
Note: This PR was created with AI tools and a human.
The pg-connection-string module (dependency of pg) now uses the node:
protocol prefix for built-in modules (e.g., require('node:process')).
Jest 26 does not support this syntax, causing test failures.
Changes:
- Upgrade jest from ^26.6.3 to ^29.7.0
- Upgrade ts-jest from ^26.5.1 to ^29.4.6
- Upgrade @types/jest from ^26.0.20 to ^29.5.14
- Update typescript to ^4.9.5
This also resolves 19 npm audit vulnerabilities (17 moderate, 2 high)
that existed in the older Jest 26 dependency tree.
modified: drivers/nodejs/package.json
Fix Issue 1884: Ambiguous column reference and invalid AGT header
errors.
Note: This PR was created with AI tools and a human, or 2.
This commit addresses two related bugs that occur when using SET to store
graph elements (vertices, edges, paths) as property values:
Issue 1884 - "column reference is ambiguous" error:
When a Cypher query uses the same variable in both the SET expression RHS
and the RETURN clause (e.g., SET n.prop = n RETURN n), PostgreSQL would
report "column reference is ambiguous" because the variable appeared in
multiple subqueries without proper qualification.
Solution: The fix for this issue was already in place through the target
entry naming scheme that qualifies column references.
"Invalid AGT header value" offset error:
When deserializing nested VERTEX, EDGE, or PATH values stored in properties,
the system would fail with errors like "Invalid AGT header value: 0x00000041".
This occurred because ag_serialize_extended_type() did not include alignment
padding (padlen) in the agtentry length calculation for these types, while
fill_agtype_value() uses INTALIGN() when reading, causing offset mismatch.
Solution: Modified ag_serialize_extended_type() in agtype_ext.c to include
padlen in the agtentry length for VERTEX, EDGE, and PATH cases, matching
the existing pattern used for INTEGER, FLOAT, and NUMERIC types:
*agtentry = AGTENTRY_IS_AGTYPE | (padlen + (AGTENTRY_OFFLENMASK & ...));
This ensures the serialized length accounts for alignment padding, allowing
correct deserialization of nested graph elements.
Appropriate regression tests were added to verify the fixes.
Co-authored by: Zainab Saad <105385638+Zainab-Saad@users.noreply.github.com>
modified: regress/expected/cypher_set.out
modified: regress/sql/cypher_set.sql
modified: src/backend/parser/cypher_clause.c
modified: src/backend/utils/adt/agtype_ext.c
- Previously, age only set ACL_SELECT and ACL_INSERT in RTEPermissionInfo, bypassing pg's privilege checking for DELETE and UPDATE operations. - Additionally, RLS policies were not enforced because AGE uses CMD_SELECT for all Cypher queries, causing the rewriter to skip RLS policy application. Permission fixes: - Add ACL_DELETE permission flag for DELETE clause operations - Add ACL_UPDATE permission flag for SET/REMOVE clause operations - Recursively search RTEs including subqueries for permission info RLS support: - Implemented at executor level because age transforms all cypher queries to CMD_SELECT, so pg's rewriter never adds RLS policies for INSERT/UPDATE/DELETE operations. There isnt an appropriate rewriter hook to modify this behavior, so we do it in executor instead. - Add setup_wcos() to apply WITH CHECK policies at execution time for CREATE, SET, and MERGE operations - Add setup_security_quals() and check_security_quals() to apply USING policies for UPDATE and DELETE operations - USING policies silently filter rows (matching pg behavior) - WITH CHECK policies raise errors on violation - DETACH DELETE raises error if edge RLS blocks deletion to prevent dangling edges - Add permission checks and rls in startnode/endnode functions - Add regression tests Assisted-by AI
- Added index creation for existing labels Assisted-by AI
There was a problem hiding this comment.
Pull request overview
This PR cherry-picks a set of upstream changes onto the PG16 branch in preparation for the 1.7.0 line, focusing on correctness fixes (parser/executor), RLS + permission enforcement, and performance improvements in agtype handling, along with driver/build/test updates.
Changes:
- Add Row-Level Security (RLS) support and strengthen permission checks across Cypher DML execution and select-style helper functions.
- Optimize agtype performance via scalar-compare fast paths and direct vertex/edge field indexing; add supporting regression coverage.
- Update build/test infrastructure and drivers (Makefile dependency fixes, 32-bit graphid handling, Python pyproject migration, Node Jest upgrade, Go dependency bump).
Reviewed changes
Copilot reviewed 49 out of 51 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| src/include/utils/agtype.h | Adds direct index-based access macros for vertex/edge fields |
| src/include/executor/cypher_utils.h | Declares RLS helper APIs + cache entry struct |
| src/backend/utils/cache/ag_cache.c | Reworks cache flush to destroy/recreate hash tables |
| src/backend/utils/adt/agtype_util.c | Adds scalar fast-path comparison and no-copy value extraction; switches to direct vertex/edge id access in comparisons/hashing |
| src/backend/utils/adt/agtype_ops.c | Fixes int64 datum conversion for string/numeric paths |
| src/backend/utils/adt/agtype_ext.c | Fixes extended-type offset sizing; improves invalid header error detail |
| src/backend/utils/adt/agtype.c | Uses direct field access in accessor functions; adds ACL + RLS checks for tuple retrieval |
| src/backend/utils/adt/age_vle.c | Adds fast uniqueness functions for small edge sets |
| src/backend/parser/cypher_gram.y | Refactors grammar actions (helpers), keyword duplication macro, minor formatting cleanups |
| src/backend/parser/cypher_expr.c | Improves undefined-variable errors with hints; handles empty-list IN; adjusts function call handling |
| src/backend/parser/cypher_clause.c | Adds permission propagation to RTE perminfos; fixes CALL/YIELD name handling; ORDER BY alias matching; edge uniqueness function selection tweaks |
| src/backend/parser/cypher_analyze.c | Expands list of node types ignored during cypher analyze to prevent crashes |
| src/backend/executor/cypher_utils.c | Implements RLS policy discovery + qual/WCO setup and evaluation helpers; enforces WCO checks on insert |
| src/backend/executor/cypher_set.c | Enforces RLS WCO + USING checks for SET updates with per-label caching |
| src/backend/executor/cypher_merge.c | Sets up RLS WCO checks for MERGE insert path |
| src/backend/executor/cypher_delete.c | Adds RLS USING checks for DELETE and DETACH DELETE behavior; introduces per-label caching in delete path |
| src/backend/executor/cypher_create.c | Sets up RLS WCO checks for CREATE insert path |
| sql/agtype_graphid.sql | Registers new fast uniqueness C functions |
| regress/sql/expr.sql | Adds regression tests for empty-list IN and ORDER BY aliasing |
| regress/sql/direct_field_access.sql | Adds new regression suite for direct-field and scalar-compare optimizations |
| regress/sql/cypher_set.sql | Adds regression coverage for Issue 1884 (SET ambiguity + nested extended-type serialization) |
| regress/sql/cypher_call.sql | Adds regression coverage for CALL/YIELD resolution fixes |
| regress/sql/cypher.sql | Adds regression coverage for coalesce/greatest crash fix |
| regress/sql/catalog.sql | Adds regression coverage for drop_graph cache corruption fix |
| regress/sql/agtype.sql | Adds regression coverage for int64 concatenation/arithmetic regression |
| regress/expected/scan.out | Updates expected outputs for improved undefined-variable hints |
| regress/expected/list_comprehension.out | Updates expected outputs for improved undefined-variable hints |
| regress/expected/expr.out | Updates expected outputs for empty-list IN and ORDER BY alias tests + hint text |
| regress/expected/direct_field_access.out | Adds expected output for new direct-field-access regression suite |
| regress/expected/cypher_with.out | Updates expected outputs for improved undefined-variable hints |
| regress/expected/cypher_union.out | Updates expected outputs for improved undefined-variable hints |
| regress/expected/cypher_subquery.out | Updates expected outputs for improved undefined-variable hints |
| regress/expected/cypher_set.out | Adds expected output for Issue 1884 regression block |
| regress/expected/cypher_match.out | Adjusts expected ordering in optional match output |
| regress/expected/cypher_call.out | Adds expected output for CALL/YIELD regression additions + hints |
| regress/expected/cypher.out | Adds expected output for coalesce/greatest regression additions |
| regress/expected/catalog.out | Adds expected output for Issue 2245 regression block |
| regress/expected/agtype.out | Adds expected output for Issue 2243 regression block |
| drivers/python/setup.py | Simplifies to a backward-compatible shim around pyproject.toml |
| drivers/python/pyproject.toml | Introduces modern Python packaging config |
| drivers/python/age/age.py | Fixes invalid escape sequence by using a raw regex string |
| drivers/python/README.md | Updates docs to psycopg3 and modern install instructions |
| drivers/nodejs/package.json | Upgrades Jest/typescript toolchain versions |
| drivers/golang/go.sum | Updates dependency sums (yaml v3.0.1, etc.) |
| drivers/golang/go.mod | Bumps gopkg.in/yaml.v3 to 3.0.1 |
| age--1.6.0--y.y.y.sql | Adds new C function declarations and id-index migration logic in upgrade path |
| Makefile | Fixes bison-generated header deps; adds regress targets; adds 32-bit graphid SQL rewrite hook |
| .gitignore | Ignores python venv + egg-info artifacts |
| .github/workflows/python-driver.yaml | Updates python version and installation method (pip install .) |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
MuhammadTahaNaveed
approved these changes
Mar 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR to cherry-pick PRs from the master to PG16
Indented PRs require additional work and will be in part 2