Constrained device and payload length

[Minishlink]

Hi,

A user of web-push-libs/web-push-php#62 had got the error 104-413 "Payload too large", because the library automatically pad the payload to the max length specified in the draft (4096 bytes), for security intents. This is not a standard behavior (5th paragraph).
If there are indeed restrictions (is it really the case though?), I feel the service worker should at least receive an empty data push event, or a push event with truncated data.

[jrconlin]

Autopush uses "bridges" to connect over proprietary networks like GCM/FCM and APNS. These bridges are more battery and CPU friendly than setting up another, dedicated connection.

One of the problems is that in order to use these bridges, payloads would need to be split across multiple deliverables. As an example, a 4096 byte payload would require two GCM delivery packets. This would require not only that both packets be delivered, but that the client be able to properly re-assemble them in order to decrypt. GCM is not guaranteed delivery, just like WebPush is not guaranteed delivery. In addition, since it's being carried by a bridged system, there's no standards means available for packet reconstruction or ACK/NACK for delivery for all bridged systems.

Data truncation is not an option since the data is end-to-end encrypted, meaning that any truncation results in complete data loss. Delivering an empty push event may also lead to undesired or unwanted application behaviors by applications that are not coded to expect data free content.

For now, it was felt that a reduced data size would be a reasonable compromise. As always, the application is free to accept a reduced data packet, or a data free packet, and perform any additional actions they might desire. For example, the data could be a URL back to the Application Server to pull the full content of a message.

Padding a message out to the full buffer size is not recommended, particularly when dealing with mobile clients, because many are under strict data limitations.

[Minishlink]

Thanks for your answer.

[marco-c]

@jrconlin couldn't this be supported when the "bridge" is between Autopush and FCM?

[jrconlin]

@marco-c It's the same basic problem. FCM and GCM both have the same remote server interface, and upper message size limit (4096 bytes for payloads). We lose about 33% of that from encoding and overhead, thus the reduced message size we have available.

To be fair, if Google opens up webpush to apps and our client switches to it, obviously our bridge restriction falls away since you'd go straight though FCM or whatever. There are some other issues at play there, but it's a future option.

[froodian]

Thank you for the explanations of what's going on here @jrconlin, it's appreciated.

This is a kind of rough patch for push senders - the implementation complexity aside, having only some users with a max payload size of ~2/3 of the standard (and it is approximate, right? We don't have an exact number?) is kind of limiting. Yes, a "tickle-and-fetch" solution such as you describe can theoretically work, but it introduces latency and another failure point and complexity on the service worker side and complexity on the sender/fetch-endpoint side associating the available payloads for individual users - there's a reason payloads were added to the Web Push spec in the first place, and it seems like it would be good to have spec-conformity as a goal.

I recognize the difficulties described, in that sending two FCM pushes and stitching them together isn't guaranteed to work... it seems like maybe the tickle-and-fetch solution could be implemented internally by Android Firefox, sending a "wake-up" push with just a URL in order to fetch the full payload... but I guess at that point it wouldn't be an autopush issue, it'd be an Firefox for Android issue - has anybody seen a similar issue on that side around this limitation? I didn't turned anything up in my bugzilla search.

[jrconlin]

Well, FWIW, it's a practice that we've done internally for our own services. (See https://github.com/mozilla-services/pushbox/ for one of the ways we do it) We're not immune either and we've got a lot more control in theory. One of the bigger problems is that third party messaging systems are unreliable for the same reasons that our own Push system can be unreliable (device disconnects, network issues, evil gnomes and sunspots) and at some point, you wind up re-implementing TCP over whatever protocol you thought you were using. In this case, we'd have to implement TCP over TCP over (FCM/APNS/ADM/etc). And there lies madness.

All on top of the fact that Push is supposed to be for short alert type messages, and even has a protocol built in to deal with splitting messages across multiple push packets, but that requires protocol considerations that are not really an option. (HTTP2 connections using an altered form of await frames)

I've been kicking around a few ideas on how to solve this problem, but they require reworking the clients, basically using host messaging as a tickle, and then kicking in our own websocket/HTTP2 protocol for message delivery, but it has a pile of problems itself, not all of which are technical.

I still hold that push is a glorified doorbell. It should be used to indicate to the remote client app "Hey, pay attention!" and then the app starts up whatever high bandwidth, bidirectional communications process makes the most amount of sense for the app and it's server.