Failing to round-trip deserialize with Bincoded formula

[vlovich]

It looks like if I have a member that contains Bincoded, either deserialization fails or I'm doing something wrong. For some reason the byte array ends up pointing to the whole input buffer being deserialized instead of correctly round-tripping.

  left: `[4, 4, 5, 6, 7, 1, 2, 3, 4]`,
 right: `[1, 2, 3, 4]`: Full serialized [4, 4, 5, 6, 7, 1, 2, 3, 4, 5, 0, 0, 0, 0, 0, 0, 0, 5, 0, 0, 0, 0, 0, 0, 0]', src/main.rs:55:5

Left is the buffer we actually retrieved from deserialized, right is the expected deserialized buffer and "full serialized" is the input buffer provided to alkahest::deserialize. It's interesting that the buffer returned ends at the right spot & just has the wrong starting point.

Here's the example code snippet: https://gist.github.com/vlovich/9e2645ed83e546777422766e825cb463

Cargo.toml:

[package]
name = "alkahest-repro"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
alkahest = { path = "../../alkahest", default-features = false, features = [
    "bincoded",
    "fixed64",
    "derive"
]}
bincode = { version = "2.0.0-rc", features = ["derive"] }
serde = "1.0.189"

Everything seems OK if I avoid Bincoded (https://gist.github.com/vlovich/2b352bcfe26ad5d40958561f166d0aa7).

[vlovich]

Also it seems adding a dummy field to the end of the formula works around the issue.

[zakarumych]

Thank you for the report. I will investigate it next week

[zakarumych]

Sorry for late reply.
You use deserialize which won't work here because it has following requirements:

• The value must occupy the whole input slice.
• The value must be either sized or heap-less.

Code ensures to satisfy first one by truncating the buffer.
But the value is not heapless since it uses Bincoded and not sized since it has unsized field Bytes.
In this case you have to use deserialize_with_size to provide stack size of the serialized buffer.
It is the second value returned from serialize

[zakarumych]

For more straightforward usage consider using packet module which will encode stack size into the buffer, so it won't be necessary to carry it around.
It also allows to unpack from buffer with any other bytes after actual packet.
This way you can read values from data stream and unpack them from first to last, or read only next packet length and skip it.

[zakarumych]

I'm adding assert to the deserialize. Its requirement can and must be checked by looking at associated constants.