Codebase64 wiki

User Tools

Site Tools

Trace:
base:streaming_1_2_4_8-bit_numbers_without_spanning_bytes

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision		Previous revision
base:streaming_1_2_4_8-bit_numbers_without_spanning_bytes [2019-10-06 12:56] – created white_flamebase:streaming_1_2_4_8-bit_numbers_without_spanning_bytes [2019-10-06 13:09] (current) – white_flame
Line 5: Line 5:
 As long as you're outputting 1, 2, 4, and 8 bit tokens (not 3 or 5-7 bit), you can keep all the bitfields aligned so they do not span byte boundaries, making the reading simpler.  No length tokens need to be added to the stream. As long as you're outputting 1, 2, 4, and 8 bit tokens (not 3 or 5-7 bit), you can keep all the bitfields aligned so they do not span byte boundaries, making the reading simpler.  No length tokens need to be added to the stream.
  
-The reader will hold a byte buffer each for reading 1, 2, or 4-bit values.  If that bit buffer is empty upon reading it, pull the next byte from the input stream and seed that particular buffer with that byte value. Reading 8-bit values comes directly from the input byte stream.+The reader will hold a byte-sized buffer each for reading 1, 2, or 4-bit values.  If that buffer is empty upon reading it, pull the next byte from the input stream to refill that particular buffer. Reading 8-bit values comes directly from the input byte stream.
  
-The writer packs only tokens of the same bit length into each byte, creating a new output byte at the end of the stream whenever a new byte buffer for that bit length is required. The writer remembers where in the stream the various buffer bytes for each bit length resides, adding to it in place until it's full.+The writer packs only tokens of the same bit length into each byte, creating a new output byte at the end of the stream whenever a new buffer for that bit length is required. The writer remembers where in the stream the various buffer bytes for each bit length resides, adding to it in place until it's full.
  
 Example of writing this arbitrary token stream: Example of writing this arbitrary token stream:
Line 32: Line 32:
 11: Write into the 2-bit buffer byte: 11: Write into the 2-bit buffer byte:
 <code> ....0010 11000011 ..111110 10101010</code> <code> ....0010 11000011 ..111110 10101010</code>
 +11: Write into the 2-bit buffer byte:
 +<code> ....0010 11000011 11111110 10101010</code>
 0101: Since the 4-bit buffer byte is full, create a new byte at the end of the stream for it: 0101: Since the 4-bit buffer byte is full, create a new byte at the end of the stream for it:
-<code> ....0010 11000011 ..111110 10101010 ....0101</code>+<code> ....0010 11000011 11111110 10101010 ....0101</code>
  
 When the specifically 1/2/4/8-bit reads are done in the same order as the writes were, all ordering is preserved. When the specifically 1/2/4/8-bit reads are done in the same order as the writes were, all ordering is preserved.
base/streaming_1_2_4_8-bit_numbers_without_spanning_bytes.txt · Last modified: 2019-10-06 13:09 by white_flame