**Introduction**

What do you think of the following code snippet?

```
NaturalNumbers()
.filter(function (n) { return n % 2 === 0; })
.pick(100)
.sum();
```

Isn’t it beautifully succinct and neat? It reads just like English! That’s the power of streams.

Streams are just like lists but offer more capabilities because they simultaneously abstract data and computation.

**Streams vs Lists/Arrays?**

Let’s take a scenario from Mathematics, how would you model the infinite set of natural numbers? A list? An Array? Or a Stream?

Even with infinite storage and time, lists and arrays do not work well enough for this scenario. Why? Assuming the largest possible integer an array can hold is *x*, then you’ve obviously missed out on *x + 1*. Lists, although not constrained by initialization, need to have every value defined before insertion.

Don’t get me wrong, lists and arrays are valid for a whole slew of scenarios. However, in this situation, their abstraction model comes up short. And when abstractions do not perfectly match problem models, flaws emerge.

Once again, the constraints of this problem:

- The size of the problem set might be infinite and is not defined at initialization time (eliminates arrays).
- Elements of the set might not be defined at insertion time (eliminates lists).

Streams, which combine data and computation, provide a better abstraction for such infinite problem sets. Their ability to model infinite lists stems from lazy evaluation – values are only evaluated when they are needed. This can lead to significant memory and performance boosts.

The set of natural numbers starts from 1 and every subsequent number adds 1 to its predecessor (sounds recursive eh? ). So a stream that stores the current value and keeps adding one to it can model this set.

**Note: **As might have become obvious: extra data structures might be needed to store previously generated stream values. Streams typically only hold a current value and a generator for calculating the next value.

**What is a Stream?**

I published stream-js, a very small (4.1kb minified) library that provides stream processing capabilities. Grab it or read the source as the post builds on it.

Oh, do contribute to the repo too!

### How do I create a stream?

The Stream constructor expects an initial value and a generator function, these two values form the stream *head* and* tail *respectively.

An empty stream has null head and tail values. In infinite streams, the *tail *generator will endlessly generate successive values.

var emptyStream = new Stream(null, null); var streamOf1 = new Stream(1, null); var streamOf2 = new Stream(1, function () { return new Stream(2, null); }); var streamOf3 = Stream.create(1,2,3); var streamFromArray = Stream.fromArray([1,2,3]);

**Note:** The *fromArray* method uses the apply pattern to partially apply the input array to the arguments function above.

**Show me the code!**

Now that you know how to create Streams, how about a very basic example showing operations on Streams vs Arrays in JS?

**With Arrays**

var arr = [1,2,3]; var sum = arr.reduce(function(a, b) { return a + b; }); console.log(sum); //6

**With Streams**

var s = Stream.create(1,2,3); var sum = s.reduce(function(a, b) { return a + b; }); console.log(sum); //6

### The power of streams

The power of streams lies in their ability to hold model infinite sequences with well-defined repetition patterns.

The *tail* generator will always return a new stream with a *head *value set to the next value in the sequence and a *tail *generator that calculates the next value in the progression.

**Finite Streams**

The *Stream.create *offers an easy way to create streams but what if this was to be done manually? It’ll look like this:

var streamOf3 = new Stream (1, function() { return new Stream(2, function() { return new Stream(3, function () { return new Stream(null, null); }); }); });

**Infinite Streams**

**Infinite Ones**

Let’s take a dummy scenario again – generating an infinite series of ones (can be 2s too or even 2352s). How can Streams help? First the *head* should definitely be 1, so we have:

var ones = new Stream(1, ...);

Next, what should *tail* do? Since it’s a never-ending sequence of ones, we know that *tail* should generate functions that look like the one below:

var ones = new Stream(1, function() { return new Stream (1, function() { ... }; });

Have you noticed that the inner Stream definition looks like the *Ones* function itself? How about having *Ones *use itself as the *tail *generator? Afterall *head *would always be one and *tail *would also continue the scheme.

var Ones = function () { return new Stream(1, /* HEAD */ Ones /* REST GENERATOR */); };

**Natural Numbers**

Let’s take this one step further. If we can generate infinite ones, can’t we generate the set of Natural numbers too? The recurring pattern for natural numbers is that elements are larger than their preceding siblings by just 1.

Let’s define the problem constraints and add checkboxes whenever a stream can be used.

- Set is infinite ☑
- Set has a well-defined recurring pattern ☑
- Definition needs an infinite set of ones ☑

So can streams be used to represent natural numbers? Yes, stream capabilities match the problem requirements. How do we go about it?

The set of natural numbers can be described as the union of the set {1} and the set of all numbers obtained by adding ones to elements of the set of natural numbers. Yeah, that sounds absurd but let’s walk through it.

Starting from {1}, 1 + 1 = 2 and {1} ∪ {2} = {1,2}. Now, repeating the recursion gives rise to {1, 2} ∪ {2, 3} = {1,2,3}. Can you see that this repeats indefinitely? Converting to code:

function NaturalNumbers() { return new Stream(1, function () { return Stream.add( Stream.NaturalNumbers(), Stream.Ones() ); }); };

**Execution walkthrough**

The first call to *NaturalNumbers.head()* returns 1. The *tail* function is given below:

function () { return Stream.add( Stream.NaturalNumbers(), Stream.Ones() ); }

*Stream.NaturalNumbers*is now a stream that has a head of 1 and a tail generator that points to itself. Think of the sets {1} and Natural numbers.*Stream.Ones*is a stream with a head of one and a tail generator of ones.

Once invoked, this will give a new stream with a *head *of 1 + 1 and a new *tail *function that will generate the next number.

**Building upon natural numbers**

Generating the sets of even and odd numbers is a cinch – just filter the set of natural numbers!

var evenNaturals = NaturalNumbers().filter(function(val) { return val % 2 === 0; }); var oddNaturals = NaturalNumbers().filter(function(val) { return val % 2 === 1; });

Pretty simple right?

**Who needs infinite sets?**

Computers are constrained by storage and time limits so it’s not possible to ‘have’ infinite lists in memory. Typically only sections are needed at any time.

*stream-js* allows you to do that

*Stream.pick*: allows you to pick*n*elements of a stream.*toArray*: converts a stream to an array

A typical workflow with *stream-js* would involve converting an input array to a stream, processing and then converting back to an array.

For example, here is the array of the first 100 odd numbers; you need a 1000? Just *pick* them* *(pun intended).

var first100odds = oddNaturals.pick(100).toArray();

**Note: **Stream operations can be chained since most stream operations return new streams (i.e. are closed operations). Here is *‘odo‘*,* *v0.5.0 of stream-js. *Odo* means river in Yoruba, the language of my tribe.

And that’s about it! I hope you enjoyed this, now read how to write a promise/A+ compatible library next.

Any chance, you’ll be releasing an ES6 version? It’d be nice to see a version that uses iterators and generators.

BTW, I submitted a PR.

Great job!

LikeLike

Aha! Thanks a lot!! :)

Yeah, I should look at that – would you mind opening an issue? I can create one too.

Thanks a lot!

LikeLike

Hi When I execute below code I am getting “Stream.create is not a function”

var Stream = require(‘stream-js’);

var s = Stream.create(1,2,3);

var sum = s.reduce(function(a, b) {

return a + b;

});

console.log(sum);

node example.js

Can you help me in fixing this error?

LikeLike

Thanks bvenkatr!

I have a fix for this and should publish it soon. However to unblock you; you can add the following snipped to the end of stream.js in the bower_components file and you should be able to require it via node.

//Ensure it can run on node

(function (stream) {

if (module != null && module.exports ) {

module.exports = stream;

} else {

window.Stream = stream;

}

})(Stream);

Sorry for any inconvenience this might have caused and thanks for raising it too!

LikeLike

var Stream = require(‘stream-js’);

var s = Stream.create(1,2,3);

var sum = s.reduce(function(a, b) {

return a + b;

});

console.log(sum);

The above code is giving error as “Stream.create is not a function”, Can you help me in fixing this?

LikeLike

[…] Want to learn more about streams? Check out my stream library and accompanying blog post. […]

LikeLike