How can we buffer more that 64 bytes?

[Don Tyszko]

78We have some applications that require us to buffer up to 260 bytes being received on the uart. How can we overcome the 64 byte limit on the temporary buffers? We cannot write to nvmemory as there are several hundred transactions per day. The code we are using is attached.

[kbanks]

Side note - please post your questions in the Support Questions section of the forum.

The SNAPpy Script Exchange is meant for scripts (or script snippets) that are doing what you want, not for ones that aren't.

(We will move this thread and your other (CRC-16) thread to the Support Questions section at a later date).

Getting back to your question: the cleanest thing you can do with 2.1 nodes is use external memory. There are devices outside the "NV" realm that might work for you.

One candidate part (I have not used it yet, but like the specs) is the FM24L256 Serial F-RAM from RAMTRON International Corporation. It might be fast enough, and has no limit on the number of times it can be written.

In the upcoming SNAP 2.2, there will be more than one string available to your scripts, but the individual strings will still be only 60-something bytes each. There still won't be a convenient place to buffer up a lot of data.

Usually when customers need more memory and/or more CPU horsepower, they "pair" the RF Engine with some other microprocessor.

[wbschindler]

I have a similar need. The existing wire does support "packets" longer than 64 bytes (not uncommon I suggest). But, I have thought about doing something like ???

A function like "def initLongPacket():" which initializes "myLongPacket" to a NULL string [should not be necessary but helpful nonetheless], then

A function like "def partLongPacket(buff):" which concatenates the packet to the end of "myLongPacket", and finally

A function like "def dosomethingLongPacket():" which terminates the sequence and does something with "myLongPacket" including resetting it to NULL.

These functions could be called in the remote end via RPC, in the usual way but the processing is deferred until the sender terminates the sequence of partLongPacket() calls with dosomethingLongPacket().

Would this play? Presuming, of course that there is enough RAM for the "myLongPacket".

This would certainly be a desirable solution for me, I suggest, also for others.

[kbanks]

Presuming, of course that there is enough RAM for the "myLongPacket".

...and there lies the catch. In 2.0-2.1 there were only a couple of (short) dynamic strings (one for concatenation results and one for "slice" results).

In 2.2 there will be more than two such "string buffers", but they will still be short (something like 63 characters max, depending on the platform). You still won't be able to build up a single string with ALL of your characters, although I suppose you could work with multiple strings (msgPart1, msgPart2, msgPart3).

Sometimes you can use transparent mode, and not buffer the data into strings at all. BTW, the maximum "payload" for a unicast transparent mode packet is 108 bytes (you have to adjust an NV parameter to do this).

Sometimes you can "parse" the data as it comes in, and only hang on to the pieces you need (the GPS examples script comes to mind).

Sometimes the messages are small, and the available string space is sufficient.

Sometimes you have to use external storage (see earlier in this thread).

Sometimes you have to augment the RF Engine with a second processor.

[wbschindler]

For 99% of our messages the 63 byte size works well, however we also talk to devices like printers which can indeed have much longer strings.

The multiple buffers approach is interesting, but a bit cumbersome. Is there any remote possibility that you could implement (perhaps enabled by an NV parm) a single long buffer of say 254 bytes? I understand you may be constrained by the Freescale engine and I have not looked hard at it.

Parsing messages is not a desirable option either. We have over a hundred different devices and parsing device specific messages would be very hard and would clearly result in unique code in the Synapse module for every device. A maintennance nightmare, not to mention a development headache.

For the few devices that have the long message requirement, perhaps indeed using an external storage device is the answer. I have not looked at any serial ram devices but I will look at your suggested one in the forum.

Clearly, I would like to have every remote Synapse module have the exact same code and I can easily have the module carrier board sense the presence of the serial ram when present. This makes life pretty much easier. But, space in the Freescale engine permitting it, I'd much rather opt for a long buffer there.

[mgenti]

Is there any remote possibility that you could implement (perhaps enabled by an NV parm) a single long buffer of say 254 bytes?
Unfortunately the IEEE 802.15.4 specification limits packet sizes to 127 bytes.

[wbschindler]

Were there to be a long buffer available, I could indeed accumulate pieces into the buffer, perform any checksumming and then use print buff to output on the STDIO UART.

So, I would be prepared to fragment the large message into the MTU size and then reassemble in the target for delivery.

[kbanks]

Longer strings could possibly be supported "internally only". You would have to be careful with variables that could not be passed via RPC though...

Maybe the (hypothetical) NV option reserves a raw chunk of RAM, and the script peeks and pokes at it.

Maybe the NV option could instead enable a single global array variable (SNAPpy currently has NO arrays). Again, this would only be usable locally.

(Something like "trade 4 dynamic strings for 1 array").

Using something different than a string might make it clearer that it required special treatment/had special restrictions on it's usage.

FYI - the 9S08 we are using only has 4K RAM (other variants in the same family have less). Within that, we have to hold packet buffers, a processor stack, the core code variables, plus an entire virtual machine.

Dave will probably chime in on this topic when he gets a chance (he is out of the office at present).

[wbschindler]

That sounds like a good solution. I understand that I could not necessarily use the buffer in a script (e.g., in an RPC), but that is okay.

Clearly I can create a couple of functions to handle the peek/poke and make this interface pretty simple. I can build a string to use in the RPC calls or for the print statement to STDIO.

This solution has a great deal of appeal. Assuming that space exists in the on board ram to allocate a buffer of say 256 bytes, this would work!

[joecool]

Sometimes you can use transparent mode, and not buffer the data into strings at all. BTW, the maximum "payload" for a unicast transparent mode packet is 108 bytes (you have to adjust an NV parameter to do this).


Would this be NV ID location 14, Buffering Threshold? My 2.1 Reference Manual says the max bytes value is 123. Is your 108 bytes value what is left after accounting for the packet headers, or am I looking at the wrong NV ID?

[kbanks]

Would this be NV ID location 14, Buffering Threshold? My 2.1 Reference Manual says the max bytes value is 123.

Yes

Is your 108 bytes value what is left after accounting for the packet headers

Yes. The unicast header is 15 bytes of overhead, 108+15 = 123.