Boston, Massachusetts. One of my  favorite cities to visit.

20161012_073745.jpg

I’m returning now, but I’ve spent the last two days learning about and working with nD. Short for n-Dimensional, nD is an analytics & data platform Black & Veatch is partnering with to provide solutions. It’s an in-progress platform, and I’m one of the few existing users. As with any new and developing technology, there are kinks and quirks, and so I’ll use this blog to collect my experiences, thoughts, and lessons learned while using the nD platform.

My first impression is that nD is a very capable math engine that can perform powerful operations on streams of data. But it also combines some interesting concepts for application/calculation development. On-the-fly math editing, instant visualization, and tacit variable connections are all part of the architecture. What that means is that you can create your math in a playground where you can see your changes live on large datasets, then plop that math into any other project that has similar data/variable names and it works out of the box.

In my first two days, I was able to make a histogram widget that took a data set, partitioned it into N bins, counted the # in each bin, and plotted the result. Here are a couple of examples with different N bins.

100 bins

100-bin Histogram.JPG

1,000 bins

1000-bin Histogram.JPG

10,000 bins

10000-bin Histogram.JPG

The data set I was working with was United States Geology Service (USGS) daily river inflow data (about 30k points – about 80 years), and this histogram is pretty typical of natural incidence distributions. As N approaches the length of the data, the histogram starts to get really jagged and less informative.

One of the cool things about nD is that I wrote a short, 10-line script that can be dropped into any project with a variable named $dataFrame.data, and I get the histogram and chart automatically.

More to come…

Advertisements