data delivery difference

[korda]

Could you please outline difference in "smart device" data delivery methods "(mcast)sendDataModePkt" vs "(rpc)BeginSendDataModePkt" and possible advantages/disadvantages.
details like number of hops, how many times the same data can be delivered (max), timing/speed of delivery (considering RPC retries and mcast hops) end etc, based on your intuitive understanding of mash background comm layer.
thank you.
PS' would be great to see all methods comparison table in the developer's manual, based of performance, reliability, speed, etc.

[kbanks]

What we have here is a "multi-axis" question. When sending data between SNAP Nodes, you have to choose between DATA MODE versus RPC based, and between UNICAST versus MULTICAST.

So, even without changing any configuration parameters, you have four possible combinations to choose from.

Let's talk about DATA MODE versus RPC based first:

DATA MODE

Pros:

Has the highest potential bandwidth. Once configured, data bytes are transferred between the UART and the RADIO with no involvement from the SNAPpy interpreter. In other words, raw machine code is moving your data.

This method also results in the most "bytes transferred per radio packet". Data Mode packets are a dedicated packet type, with 0 overhead due to RPC name, etc.

Cons:

Since the SNAPpy interpreter is not involved, there is no opportunity to make decisions based on the actual data. This precludes making any sort of routing decisions based on the data.

RPC Based

Pros:

Because all data gets "seen" by your SNAPpy script, you can make decisions (including routing choices) based on the actual data.

Cons:

Because the SNAPpy interpreter is involved, the total achievable throughput wil be less than what DATA MODE can do (interpreted code versus compiled code).

Because the RPC packet includes the RPC name, various "data type" fields and other overhead, fewer data bytes get sent per packet.

General Guideline:

Use DATA MODE when possible, RPC based transfer when you have to.

Continued next post...

[kbanks]

Now lets talk about MULTICAST versus UNICAST:

When you configure DATA MODE, you can do so using either mcastSerial() or ucastSerial(). When you make RPC calls, you can use either rpc() or mcastRpc(). In both cases, you are choosing whether to use multicast or unicast methods.

MULTICAST

The originating node only sends the packet once (does no retries), and does not wait for any sort of acknowledgement or confirmation.

The packet still could be sent more than once. All multicast packets include a Time To Live (TTL) field, which specifies the maximum number of times they should be re-broadcast by nodes that hear them.

Pros:

Bypasses Mesh Routing completely. No Route Discovery step is needed before the data packet can be sent. This can result in faster data transfer.

Has the potential to travel more hops than a unicast packet can. The Time To Live (TTL) field is a byte, so 255 hops are possible.

Can be received by multiple nodes - by all nodes configured to belong to the specified group(s).

Can result in faster data transfer, IF the data makes it through.

Cons:

Data is more likely to be dropped (remember - no retries, no acknowledgements).

The "automatic re-broadcast" feature (based on TTL) can actually result in a sort of "jamming of the airwaves", if you have too many nodes, too close together, all configured to forward these packets.

Only 16 "groups" are possible (and one of them is the "broadcast" group), so the number of distinct destinations is limited.

UNICAST

The message is sent to a specific address, using Mesh Routing. Retries and acknowledges are used to help the data get through, but delivery is still not guaranteed.

Pros:

Much higher probability of data making it through. By default, up to 8 retries will be used for every packet.

Cons:

Can result in less throughput, due to the need for periodic Route Discovery.

Can travel fewer hops total. Mesh Routing can only reach something like 30 hops (the exact number can be found elsewhere on the forum).

General Guideline:

Use unicast messaging when possible.

Continued next post...

[kbanks]

Could you please outline difference in "smart device" data delivery methods "(mcast)sendDataModePkt" vs "(rpc)BeginSendDataModePkt" and possible advantages/disadvantages.details like number of hops,

Covered in the previous two posts.

how many times the same data can be delivered (max),

For unicast the data will only be delivered once (or not at all).
For multicast, the number of times the same data can be delivered depends on the number of nodes that are rebroadcasting it, and the overall amount of traffic in the network.

Each SNAP node tracks the "most recently received sequence number" from the other nodes, but each node can only track this for a few nodes at a time. With sufficient network traffic, a give node will have "forgotten" that it has already heard some data, and could re-process it.

timing/speed of delivery (considering RPC retries and mcast hops) end etc, based on your intuitive understanding of mash background comm layer.

This would vary from one network to another, based on the number of nodes, their placement, and their configuration.

It is the sort of thing best measured rather than guestimated.

We have measured some of the pieces... the following is from the SNAP Reference Manual.


Unicast Propagation Delay

Due to the collision avoidance mechanism used by SNAP, the average propagation delay is 5 milliseconds.

Multicast Propagation Delay

With a TTL = 1, the pin-to-pin propagation delay was measured at 4 milliseconds.

With a TTL > 1, the RPC message incurs a >20 millisecond delay for each hop, plus local retransmit delay.

[korda]

very good, this is classic. thank you. I think this is very helpful article.

can you add a similar and brief description for "embedded" commands, such as global ping, and its returns for example.

Can I summarize your previous post by saying that if someone needs:
Quantity, distance and speed - use mcast.
Short range, reliability of delivery, less traffic - use rpc.
is this close?

[kbanks]

can you add a similar and brief description for "embedded" commands, such as global ping, and its returns for example.

The "High Level" View

The main thing to understand is that most of the functions you see in Portal (the tool bar buttons for example) are implemented using (unicast) RPC and multicast RPC calls behind the scenes. For example, you hit the button labeled "Broadcast PING" and Portal makes a multicast RPC call to the built-in function vmStat().

The "Low Level" View

For this I have to refer everyone to the SNAP Reference Manual.

Can I summarize your previous post by saying that if someone needs:
Quantity, distance and speed - use mcast.
Short range, reliability of delivery, less traffic - use rpc.
is this close?

I think it is misleading to try and summarize the four combinations into two.

When you say "use mcast", do you mean multicast DATA MODE or multicast RPC?

When you say "use rpc", do you mean multicast RPC or unicast RPC?

[mgenti]

Also, remember that range does not always directly correlate to bad link quality or dropped packets.

[korda]

The "High Level" View

I think it is misleading to try and summarize the four combinations into two.

When you say "use mcast", do you mean multicast DATA MODE or multicast RPC?

When you say "use rpc", do you mean multicast RPC or unicast RPC?

This is where the confusion is - you have built-in reliable delivery mechanism, based on unicast and RPC (or Data mode) combination, which in my understanding gets a receipt confirmation, hops count and route. (Is this correct?)
But in case when you combine multicast and RPC (or Datamode), I am mot sure of this logic. Is this confirmed delivery and using route or just broadcast based on number of hops?

That is why I differentiated only two methods in my summary, but not sure if this is valid. Please correct me if I am wrong.
Please explain what exactly defines data delivery method and how many of them you offer?

[mgenti]

The only packet types that are currently used in SNAP are data mode (multi-cast and point-to-point), RPC (multi-cast and point-to-point), mesh (multi-cast and point-to-point), and traceroute (just point-to-point). Any other "features" we have done, such as broadcast PING", are done using these packet types. Portal does not have any special packets that it uses to communicate with SNAP nodes.

[korda]

In your list I counted 7 data delivery combination:

data mode multi-cast
data mode point-to-point
RPC multi-cast
RPC point-to-point
mesh multi-cast
mesh point-to-point
traceroute (just point-to-point)

Can you please put check marks for each method, if it uses delivery confirmation (re-try count), message route, hops count? Thanks.

[mgenti]

All point-to-point and multi-cast packets work the same for any packet type. So the pros and cons that Kevin mentioned in the previous posts apply for all packet types.

[korda]

Kevin, please explain your statement about UNICAST:
"Cons: Can travel fever hops total."

Also, if RPS and MULTICAST are combined, does that mean the function can be called several times? Can it be called again, before it had a chance to finish?

Please explain RPC UNICAST confirmation mechanics. For example it is not clear if originating node spits out all retries one after another and waits for confirmation or does it different way.

And finally, please explain the process of RPC confirmation in MULTICAST vs UNICAST.

[kbanks]

Please explain your statement about UNICAST:
"Cons: Can travel fever hops total."

Multicast - TTL can be as high as 255 (NOTE! - this could result in a LOT of collisions over the air).

Unicast - at three bytes each, the hop-to-hop addresses used during the Route Discovery eventually fill up all of the available packet space. This means there is a limit of 30-something hops (exact number is elsewhere on the forum).

Also, if RPS and MULTICAST are combined, does that mean the function can be called several times? Can it be called again, before it had a chance to finish?

As explained in previous posts, the "duplicate sequence number filtering" can be overwhelmed if enough nodes are sending at the same time. For best performance, any functions invoked using multicast should be what is called "idempotent". They should do the same thing no matter how many times they are invoked. Examples:

LightOn - OK
LightOff - OK

ToggleLight - not a good idea

Please explain RPC UNICAST confirmation mechanics. For example it is not clear if originating node spits out all retries one after another and waits for confirmation or does it different way.

It's textbook timeout/retry stuff. A unicast packet is sent, and a timeout started. If an ACK is heard before the timeout expires, then DONE. If the timeout expires before the ACK is received, retry.

And finally, please explain the process of RPC confirmation in MULTICAST vs UNICAST.

At the SNAP level, unicast has node-to-node (NOT end-to-end) acknowledgement. Multicast has no acknowledgement at all.

If you want end-to-end acknowledgement with unicast or multicast, you must implement that at the application layer (in other words, in your script).

[korda]

In case of UNICAST+RPC or UNICAST+DATA, how does the msg originating Node handles other outgoing messages of the same type in its buffer?
In other words, does it wait for 1st message confirmation or timeout (try out) until sending next message our, or it uses a time "gap" and fires next message?
Our routes are 5-8 hops long, with 4 re-try attempts programmed in - large network feels sluggish and operator loses his patience very easy.

There is rule: if button click does NOT produce reaction in 2 sec, then operator feels that something is not working right. With the setup above we are often exceeding the 2 sec rule. So he is sending another unicast+rpc and another one... Client software has no way of knowing what is going on behind the usb connection to bridge, so it just spits out all requests in 1 sec interval. That is why I am asking all mesh msg organization questions.

[kbanks]

In other words, does it wait for 1st message confirmation or timeout (try out) until sending next message our, or it uses a time "gap" and fires next message?

It sends the message, and waits for a reply. It only goes on to the next message when the first one has used up all of it's timeouts and retries.

Reminder - you can configure the number of retries

[korda]

1) can you estimate time needed by originating node for a single RPC-Unicast message with configuration 8 hops and 4 re-tries, in case when message can not be delivered? Routes can be in question too.
Is this time out can be controlled by user (or parameters)?

2) How would you organize your client, if you had hundreds of nodes out there and trying to get their status update asap? Right now we use DATA+Unicast and RPC+Unicast with very sluggish performance.

[mgenti]

The timeout before the node tries to resend the RPC packet is 20ms. As mentioned earlier in this thread, there is not end to end acknowledgement so the originating node only receives an ACK from the next hop node, not the final destination node. So in your example it would take at least (4retires*20ms)=80ms before discarding the RPC packet and trying to send the next one. No, this timeout is not user configurable.

[korda]

I your RPC timeout calc formula I did not see any influence from the hop numbers, which you explained well. But this generates new question - is there any mesh timing penalty for programming in big number of hops for RPC/DATA unicast?
I see only penalty for number of re-tries.

[mgenti]

is there any mesh timing penalty for programming in big number of hops for RPC/DATA unicast?
I see only penalty for number of re-tries.

You can not program the number of hops a unicast packet can take. The only time "penalty" that mesh takes for any number of hops is how long it takes to form the route. Once the nodes know how to reach each other there is no more mesh messages happening until one of the configured timeouts is reached.

[korda]

what about NV_MESH_INITIAL_HOPLIMIT_ID, NV_MESH_MAX_HOPLIMIT_ID. I was under impression these control RPC-Unicast shipping "distance". If not, what do they control then? I am trying to improve mesh sluggish performance and reliability...

I understood Multicast uses TTL for similar reason, which is buried in the message itself, rather then in the node's firmware.

[admin]

what about NV_MESH_INITIAL_HOPLIMIT_ID, NV_MESH_MAX_HOPLIMIT_ID. I was under impression these control RPC-Unicast shipping "distance". If not, what do they control then?

No, they basically control the TTL value for the initial route request message and then the subsequent route request message. The only way they relate to unicast messages is that if you can't find the route because the node is more hops away than the maximum limit you will never be able to form a route to send him a unicast message.


I understood Multicast uses TTL for similar reason, which is buried in the message itself, rather then in the node's firmware.

TTL is used to limit the number of hops a multicast message will propagate.