feat: add binary type support for host functions#40
Open
simongdavies wants to merge 3 commits intomainfrom
Open
feat: add binary type support for host functions#40simongdavies wants to merge 3 commits intomainfrom
simongdavies wants to merge 3 commits intomainfrom
Conversation
simongdavies
force-pushed
the
allow-binary-types-for-host-funcs
branch
2 times, most recently
from
e63255b to
88cd33b
Compare
Adds Uint8Array/Buffer support for host function arguments and returns. Architecture: - New hyperlight-js-common crate: shared wire-format constants, binary sidecar encode/decode, FnReturn enum, DecodeError type - Guest extracts Uint8Array from QuickJS VM into binary sidecar - Host dispatches via register() (typed serde) or register_js() (JS bridge) - NAPI layer creates native Node.js Buffers via C API (no base64) - Tagged return format (0x00=JSON, 0x01=binary) for return path Key changes: - Single CallHostJsFunction entry point (removed legacy JSON-only path) - Native Buffer marshalling in NAPI (JsArg/JsReturn types) - Depth limits on all recursive JSON tree traversals (MAX_JSON_DEPTH=64) - Trailing data rejection in sidecar decoder - Typed register() rejects binary args with clear error message - Comprehensive test coverage (Rust unit + integration + JS vitest) - Updated README with Binary Data section and wire protocol docs - Updated CI publish order for new common crate Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com>
simongdavies
force-pushed
the
allow-binary-types-for-host-funcs
branch
from
88cd33b to
db7d6ed
Compare
QuickJS stores JSON-parsed numbers as doubles internally, so value_to_json_with_binaries was emitting 42.0 instead of 42 for whole numbers. This caused serde deserialization failures on the host side when typed host functions expected i32/i64 args. Fix: when a float has no fractional part and fits in i64, emit it as an integer to match JSON.stringify behaviour. Also adds 7 numeric type tests covering i32, i64, f64, bool, mixed types, negative integers, and zero — all passing through the binary host function path with event data (JSON-parsed → float → host).
simongdavies
force-pushed
the
allow-binary-types-for-host-funcs
branch
from
db7d6ed to
4ba03cd
Compare
simongdavies
added a commit
to simongdavies/hyperlight-js
that referenced
this pull request
Mar 9, 2026
…n path QuickJS stores JSON-parsed numbers as doubles internally, so value_to_json_with_binaries was emitting 42.0 instead of 42 for whole numbers. This caused serde deserialization failures on the host side when typed host functions expected i32/i64 args. Fix: when a float has no fractional part and fits in i64, emit it as an integer to match JSON.stringify behaviour. Also adapts user_module_can_import_host_function test to use the new typed register() API (register_raw was removed in PR hyperlight-dev#40).
simongdavies
force-pushed
the
allow-binary-types-for-host-funcs
branch
from
4ba03cd to
9e70be3
Compare
napi_get_buffer_info returns data=null with len=0 for empty buffers. slice::from_raw_parts requires a non-null pointer even for zero-length slices, which caused a panic when returning empty buffers from host functions. - Handle len=0 case specially by returning Vec::new() directly - Add explicit null check with error for data=null with len > 0 - Add vitest for host returning Buffer.alloc(0) to guest Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com>
simongdavies
force-pushed
the
allow-binary-types-for-host-funcs
branch
from
9e70be3 to
bfc51b5
Compare
dblnz
reviewed
Mar 11, 2026
Contributor
There was a problem hiding this comment.
This looks good to me, although I am not very familiar to the js-host-api side of things.
One thought I have is, it seems to me that this serialization/deserialization to/from guest would benefit from something similar to what we do in hyperlight-wasm with the WIT world bindings generation, right?
Practically this boils down to having shared types definitions between host and guest downstream.
| This release contains the benchmark results and the source code for the release along with automatically generated release notes. | ||
|
|
||
| In addition the hyperlight-js crates will be published to crates.io. You can verify this by going to the [hyperlight-js page on crates.io](https://crates.io/crates/hyperlight-js) and checking that the new version is listed. | ||
| In addition the hyperlight-js crates will be published to crates.io in dependency order (`hyperlight-js-common` → `hyperlight-js-runtime` → `hyperlight-js`). You can verify this by going to the [hyperlight-js page on crates.io](https://crates.io/crates/hyperlight-js) and checking that the new version is listed. |
Contributor
There was a problem hiding this comment.
Nit:
Suggested change
| In addition the hyperlight-js crates will be published to crates.io in dependency order (`hyperlight-js-common` → `hyperlight-js-runtime` → `hyperlight-js`). You can verify this by going to the [hyperlight-js page on crates.io](https://crates.io/crates/hyperlight-js) and checking that the new version is listed. | |
| In addition, the hyperlight-js crates will be published to crates.io in dependency order (`hyperlight-js-common` → `hyperlight-js-runtime` → `hyperlight-js`). You can verify this by going to the [hyperlight-js page on crates.io](https://crates.io/crates/hyperlight-js) and checking that the new version is listed. |
| ); | ||
|
|
||
| // Calculate total size: 4 bytes for count + (4 bytes length + data) per blob | ||
| let total_size = 4 + blobs.iter().map(|b| 4 + b.as_ref().len()).sum::<usize>(); |
Contributor
There was a problem hiding this comment.
Since already using assert!, maybe use checked_add for total_size and assert for the result to be Some.
| } | ||
|
|
||
| blobs.push(data[offset..offset + len].to_vec()); | ||
| offset += len; |
Contributor
There was a problem hiding this comment.
I'm not sure if this can happen, but maybe use checked_add to avoid overflows?
|
|
||
| /// Maximum recursion depth for JSON tree traversal in the guest runtime. | ||
| /// Matches the host-side limit in `hyperlight-js-common::MAX_JSON_DEPTH`. | ||
| const MAX_GUEST_JSON_DEPTH: usize = 64; |
Contributor
There was a problem hiding this comment.
Why not use the const in hyperlight-js-common directly?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds Uint8Array/Buffer support for host function arguments and returns.
Architecture:
Key changes:
Closes #38