Elixir needs at least 4 JSON parsers

elixir json

Elixir has, at the time of this post, 3 JSON parsers hosted on expm.

Some time back I went looking for one and started exploring elixir-json, including the github repo.  I loved how the encoder used protocols to implement the encoding of different types.  It’s beautiful.  Kudos to @carloslage.

Then I looked at the decoder.  It’s full of elixir-isms… tuples, binary pattern matching, etc.  Over the last few nights I decided to implement my own JSON parser learning from Carlos were I ran aground.

I don’t like the use of HashDict for objects, but spaces rule out most options like Keyword lists.  I’m also not a huge fan of the nested case for dealing with the what’s next? of key/value pairs.

def parse_object( acc, << rest :: binary >> ) do
  { key, rest } = parse_content rest
  { value, rest } = lstrip(rest) |> parse_object_value 

  acc = [ { key, value } | acc ] 

  case lstrip(rest) do
    << ?}, rest :: binary >> -> { HashDict.new(acc), rest }
    << ?,, rest :: binary >> -> parse_object acc, lstrip(rest)
  end
end

Overall it feels very cohesive.  You can see clearly how the pieces compose after being dispatched from the parse_content.

def parse_content( << m, rest :: binary >> ) when m in ?0..?9, do: parse_number << m, rest :: binary >>
def parse_content( << ?", rest :: binary >> ), do: rest |> parse_string
def parse_content( << ?{, rest :: binary >> ), do: lstrip(rest) |> parse_object
def parse_content( << ?[, rest :: binary >> ), do: lstrip(rest) |> parse_array

I also do nothing to support invalid JSON content as with elixir-json.  Oh well, it was a learning tool.

Using HashDict.update for Keyed Reductions (aka group by) in Elixir

I wanted to start playing with Elixir’s Map and Reduce functions to get a better feel for collection transformations in the language.  For this I grabbed some movie data here and planned on grabbing some perspectives.

First problem, we need to turn the data into a list of tuples.  The pseudo-transformation we want to apply:

file -> lines
lines -> parts
parts -> tuples

The results ended up looking like this

For our data perspectives, lets start small.  The number of movies per year.  This is still a transformation, but it’s not going to be a one-for-one.  We’re instead going to reduce the results after mapping.  Why would we map?  Turns out the only thing you need to know is a full list of the movie years… with dups.  With that we can do an “Add or Update” to a hash for each year.

What we’re doing is providing an entry point for count_unique which takes a collection.  This creates a new HashDict which seeds our {year, count} and then recursively calls down into a variant of HashDict.update.  This variant will insert a new key if not found with the 3rd parameter being the seed value.  If the key is found HashDict.update will call our anonymous function to increment the value already found.

This pattern seems to work well to get the sum also.  Here we map our collection to pull the year and rating.  I adjusted my original to_movies to use String.to_float so that I have a numeric rating.  From there I use the same HashDict pattern with the rating being our seed and accumulator.

Next we’ll look at doing something a little more interesting by calculating an average and distributing the effort across nodes using our previous parallel map.

Triangle Kata in Elixir using Erlang processes for Parallel Maps

elixir-logo

Last time I described my setup for learning Elixir, the language built on top of Erlang with a Ruby bent.  Today I wanted to get a hang of processes and mailboxes which are so important to both languages.  To do this I chose the Triangle Kata as my background story.  Take a look:

Update: You might wonder what that slow function is all about. I added that in there to clarify the parallelism. You would expect the slowest classifications to appears at the end of the resultant list.

My Learning Elixir (erlang) Setup

The last Dayton Clean Coders meeting spiked my squirrel sense as Chris McCord and Matt Sears of our awesome host Little Lines talked about their excitement for Elixir. Of course, I’ve hung my several dozen half baked projects up for the moment as I scratch that itch.

Elixir is a Ruby inspired language that compiles down to Erlang bitcode. You can fall back to any Erlang library and many of the syntaxes, but the language is crazy easy to pick up if you’ve tried any of the other Ruby inspired languages like Coffeescript.

Here’s my setup for walking through the Elixir Getting Started:

  1. vim with my vim-config
  2. iex, interactive elixir will load compiled elixir modules found in the same directory. This means I can put bigger functions, modules, protocols, etc. into *.ex files and then explore and play with them using iex.  You need to run
    r(ModuleName)

    to reload after compiles.

  3. Guard running this Guardfile for automatically compiling the *.ex files.
vim + iex + guard = flatting the curve
vim + iex + guard = flatting the curve

Creating an Offline Web Platform Installer for Service Bus 1.0

One of my clients is evaluating a number of Service Bus tools.  One of those includes the Azure Service Bus which is now available for on-premise deployment.

Service Bus 1.0 is available only through Web Platform Installer at this time and it’s not readily apparent how you might use WPI for a network isolated environment.  Hopefully this will help:

WebpiCmd.exe to find the Service Bus AppId

Web Platform Installer comes with a command line tool called WebpiCmd.exe.  With it you can list and install packages.  You can also create an offline WPI feed which is what we’ll do.

To list available products at the default Microsoft feed:

webpicmd /List /ListOption:Available

Somewhere in that mess is Service Bus 1.0 and it’s AppId which we’ll need to create a private, offline cache of the product and it’s dependencies.  I’m using Console2 with a PowerShell tab so I can do this:

webpicmd /List /ListOption:Available | ? { $_.Contains(“Service Bus”) }

image

Since WebpiCmd is just writing dumb lines of text, I can’t interrogate the Product attributes, so I’m using the where-object, aliased to ?, to see which line contains the string “Service Bus”.  WebPI really should be a better PS citizen.

Creating an Offline Feed

WPI works against feeds, so we’re going to create our own feed just for this product.  You might maintain a growing list of products in this feed, but that’s beyond this post.  To create your offline feed we’ll use the (drum roll) /Offline option:

WebpiCmd.exe /offline /Products:ServiceBusBeta /Path:C:\wpi-offline

NOTE: The Service Bus 1.0 entry in WPI seems like it has a problem with the App Fabric dependency.  I’m told that’s fixed now.

Installing from the Offline Feed

Once everything is resolved and downloaded, you can use it with WPI GUI or WebpiCmd.exe. 

You can not automate-all-the-things with:

webpicmd /install /Products:ServiceBus /xml:c:\wpi-offline\feeds\latest\webproductlist.xml

 

Or do the GUI thing by checking out WPI options:

image