Added BTHome advertising field "packet id" for deduplication

[seime]

See https://bthome.io/format/

While most home automation system have some deduplication, not all do.

[rbaron]

Thanks @seime.

Have you tested this change? I'm asking because I think it would go over the maximum possible payload of 23 bytes (full packet defined here). We currently (before this PR) have 19 bytes of service data plus the "prst" device name in the advertisement packet. I have a feeling this change will bump the device name out of the packet. Can you try to verify?

If that's the case, we would have to either:

1. Drop the idea of using the counter (leave as it is now). This seems to work mostly fine, unless I'm missing something
2. Conditionally drop some data (e.g.: drop batt voltage, leave the percentage); add some configs to Kconfig
3. Make multiple advertisement packets. This is by far my least favorite one. It adds complexity and, more importantly, cut the battery life significantly at little benefit

Let me know your thoughts. I'm interested in learning about what issues you faced without the counter.

[seime]

It works fine - at least for me :) I've implemented the BTHome binding for openHAB, and with short enough update interval I can see the counter increase from 0 and upwards. Sometimes it skips a number, but that just means I've lost a packet. I'm using a set of ESP32s running ESPHome to sniff traffic, and I think there are more than 200 BLE devices in range. So no wonder why packets are lost ;)

Are you sure about the 23 byte limit? And that it isn't 27?

I'm not gonna say I'm right on this one, I've not studied the format recently. But I'm farily certain that in a previous project we pushed 27 bytes (and we did your option 3 - splitting and merging multiple advertisements into one larger - quite a few things to think about).

[rbaron]

I dug a little deeper and, as far as I understand, here's what's happening:

We have a maximum of 31 bytes for the whole advertising packet (assuming no extended adv for max compatibility), split into the following AD components:

3 bytes: flags (length: 2 + BT_DATA_FLAGS + flags)
6 bytes: name (length: 3 + BT_DATA_NAME_COMPLETE + "prst")

It leaves us with 22 bytes for advertising data.

For the BTHome service, we must have:

• 1 byte: length
• 1 byte: BT_DATA_SVC_DATA16 (signals that the following bytes are service data)

This leaves us with a 20 bytes hard limit for the actual service data.

To validate these assumptions, I modified bleak's detection_callback.py -- my version here:

% python3 detection_callback.py --macos-use-bdaddr
2024-06-26 08:45:03,484 __main__ INFO: Name: prs
2024-06-26 08:45:03,485 __main__ INFO: Service 0000fcd2-0000-1000-8000-00805f9b34fb: 4002ee092e4c051027000cbd0b2f0001640000
2024-06-26 08:45:03,788 __main__ INFO: Name: prs
2024-06-26 08:45:03,788 __main__ INFO: Service 0000fcd2-0000-1000-8000-00805f9b34fb: 4002ee092e4c051027000cbd0b2f0001640000
2024-06-26 08:45:06,412 __main__ INFO: Name: prs
2024-06-26 08:45:06,412 __main__ INFO: Service 0000fcd2-0000-1000-8000-00805f9b34fb: 4002bf092e49051027000cc10b2f0001640001
2024-06-26 08:45:06,441 __main__ INFO: Name: prs
2024-06-26 08:45:06,441 __main__ INFO: Service 0000fcd2-0000-1000-8000-00805f9b34fb: 4002bf092e49051027000cc10b2f0001640001
2024-06-26 08:45:06,477 __main__ INFO: Name: prs
2024-06-26 08:45:06,478 __main__ INFO: Service 0000fcd2-0000-1000-8000-00805f9b34fb: 4002bf092e49051027000cc10b2f0001640001
2024-06-26 08:45:06,826 __main__ INFO: Name: prs
2024-06-26 08:45:06,826 __main__ INFO: Service 0000fcd2-0000-1000-8000-00805f9b34fb: 4002bf092e49051027000cc10b2f0001640001
2024-06-26 08:45:07,018 __main__ INFO: Name: prs
2024-06-26 08:45:07,018 __main__ INFO: Service 0000fcd2-0000-1000-8000-00805f9b34fb: 4002bf092e49051027000cc10b2f0001640001
2024-06-26 08:45:07,267 __main__ INFO: Name: prs
2024-06-26 08:45:07,267 __main__ INFO: Service 0000fcd2-0000-1000-8000-00805f9b34fb: 4002bf092e49051027000cc10b2f0001640001
2024-06-26 08:45:09,333 __main__ INFO: Name: prs
2024-06-26 08:45:09,333 __main__ INFO: Service 0000fcd2-0000-1000-8000-00805f9b34fb: 4002aa092e45057837000cbd0b2f0001640002
2024-06-26 08:45:10,138 __main__ INFO: Name: prs
2024-06-26 08:45:10,138 __main__ INFO: Service 0000fcd2-0000-1000-8000-00805f9b34fb: 4002aa092e45057837000cbd0b2f0001640002
2024-06-26 08:45:12,546 __main__ INFO: Name: prs
2024-06-26 08:45:12,546 __main__ INFO: Service 0000fcd2-0000-1000-8000-00805f9b34fb: 40029f092e42057030000cc10b2f0001640003
2024-06-26 08:45:15,754 __main__ INFO: Name: prs
2024-06-26 08:45:15,754 __main__ INFO: Service 0000fcd2-0000-1000-8000-00805f9b34fb: 400296092e40051027000cbd0b2f0001640004
2024-06-26 08:45:16,321 __main__ INFO: Name: prs
2024-06-26 08:45:16,321 __main__ INFO: Service 0000fcd2-0000-1000-8000-00805f9b34fb: 400296092e40051027000cbd0b2f0001640004

It's nice to see the counter increments on the last byte of the adv data, but unfortunately as expected the local name got chopped into "prs" instead of "prst". This would unfortunately break our Home Assistant integration (here's why).

[seime]

OK, I didn't realize the name got chopped as I tested it on an already registered device in my system.
It sounds like you prefer option 1 (leave as is), so I'll skip arguing for option 2 and close this issue :)

Thanks for the detailed analysis though, your support is highly appreciated!