Your Software is Wrong…but that’s okay

Here’s the thing – software is wrong. We create software to manage change: bring about efficiencies, enable easier regulatory compliance, make the difficult easy and the impossible doable. It’s all change. As software developers we are all about change – it is the essence of our industry and it pervades everything we do. Any static view of a software system in that world must be wrong.

We can choose how we frame this ceaseless change; we can either see it as a huge risk to be mitigated through contracts and gated responsibility or we can choose to see it as a source of endless opportunity to collaborate and seek mutual achievement. The former view is not, of itself, bad (issues around motivation, engagement, stress and productivity notwithstanding) but it does lead to a great deal of effort before we produce and deliver our software. Effort costs money and what is the one thing we know about the software all this expensive effort has produced? Yep – it’s wrong.

Over the past couple of decades we in the software industry have learned a great deal from the world of manufacturing – especially from the pioneers of efficiencies in Japan and The States. But the production of software is not analogous to a manufacturing process and it is a mistake to see it as such. There are two core differences: firstly, with software, we are targeting a moving target (the aforementioned change); secondly we have a much lower cost of change with software. The first demands flexibility and the second enables that flexibility. When we enshroud our software production with heavy process we make change expensive, lessen our flexibility and damage our capacity to respond to change.

Here’s a quick story from an experience at work the other day where a colleague used a very effective method to demonstrate the importance of detailed analysis. Firstly he produced a small black ball and asked me to describe it as though doing so for a technical requirement. I’m a fail fast kinda guy so went in with “black sphere”. Now my colleague produced a second black ball and asked if it met my given requirement. I in turn answered “yes,” after all it was a black sphere. Without wishing to give too much away and spoil a worthwhile demonstration, my colleague now proved that the two black balls were in fact very different. The point, according to my colleague, is that without getting the analysis correct in the first place we produce something that is wrong and it is expensive to correct those mistakes.

So, what’s the point here? Well my starting position is that my answer was always going to be wrong: inevitably incomplete. But, by being wrong as quickly as sensible, it is cheap to revise my position and arrive at a more correct position. Crucially by iterating in this way I have established a pattern that need never end; a pattern that will keep going beyond the customer’s original bounded concept of what they wanted into unforeseen realms of systems improvement.

This is the essence of agile development and software craftsmanship: we’re going to be wrong anyway so let’s get there as efficiently as possible with code that we can change as easily as possible. Let’s take the customer with us on a journey that will go beyond what they currently think they want and lead them to an evolving relationship of continuously handled change.

Advertisement

Crafty Coders Lifts Off

Thursday saw the start of a new user group in the West Midlands. Crafty Coders hit the ground running with a great run through of Dependency Injection from Ian Russell. I am biased but I think the evening was a thorough success.

Before Ian started his session we held a discussion about what we wanted out of the user group. We’ve always taken the line that we want the group to be owned by the members and attendees and we’ll take care of the facilitation. This post captures that incredibly important discussion.

Two questions capture the general nature of the discussion. What does everyone want to get out of the group? And what activities can support that?

The What

Here’s what we want to get out of the group:

  • Discuss the principles that can promote success in software projects
  • Provide a forum to share experiences and seek support
  • Create an atmosphere of open learning and discussion

Critical to this effort is an openness to other opinions in seeking pragmatic solutions. This all falls under the umbrella of Software Craftsmanship. From the name we chose for the group, its clear we had an eye on that fact but we were concerned that the term “Craftsmanship” can be misconstrued or appear elitist. However it was clear from Thursday’s meeting that we need not be concerned and can happily embrace Craftsmanship.

One of the motivating factors in starting Crafty Coders was to establish a group that could host and take part in code retreats. We’ll look to organise these from time to time to support our monthly activities.

The How

So what sort activities enable these outcomes and desires? A fantastic number of suggestions were made about what we might do with our monthly sessions. Here’s a summary:

Talk and Discuss

Much like our opening night; we’ll arrange for a speaker to come and talk to the group but ensure more of a discussion atmosphere with continual questions and anecdotes.

Learning Stories and Practice

In March, Ryan will be presenting a short talk on his experiences learning the asynchronous support in C# 5. After the talk we’ll break into pairs for some hands-on practise with async exercises.

I Want To See…

A short brief is posted up on the screen and the attendees break into pairs to tackle the brief. For example, this might be to implement a particular pattern, or produce a small app demonstrating particular patterns. Every 15 minutes or so we stop and discuss progress.

Last Week I Did…

Short sharing sessions that might form a regular part of every meeting where attendees can spend some time talking to the group about something they’ve implemented or put in place recently. The advantages this has bought, the challenges faced and the method adopted, etc.

Screen Casts

Pluralsight and Clean Coders offer user groups the opportunity to screen their content.

Book Chapter Discussions

Read something inspiring, thought provoking or infuriating lately? Bring it to the group and lets mull it over.

Tool Standoff

Ian was at pains during his talk to steer well clear of showing any specific IoC container…but they exist and we should probably use one in the fight to deliver decoupled code. But which one to choose? And everyone knows the best way to decide something like this is a fight careful discussion of the pros and cons of each. These sessions allow us to explore the options available for solving a particular problem.

Book Share

We’ve all got favourite books and books on the “to read” list. Lets pool our resources and share the knowledge.

We’re really keen for group members to lead these sessions too – so if there’s something you want to bring to the group, please please get in touch.

To support these efforts we’ll add a blog engine to the Crafty Coders website and we’ve started a Google+ Community. There’s doubtless plenty more we can do to make this group work so let us know and we’ll get right on it.

Personally

One of the things I’d like to see us achieve is becoming a resource, not necessarily of wisdom, but of consideration in all things related to the principles of successful software implementation. I’d love to see the attendees blog about the meetings, carrying on the discussions through twitter and playing active roles in the Google+ group.

Staying in Touch

You can email all of the organisers using all@craftycoders.net and tweet us @CraftyCodersUG

Finally

To reiterate, Crafty Coders user group belongs to those who attend the meetings. So keep the ideas and discussions going! If I’ve missed something, please post a comment or better yet – post a blog about your ideas.

A Quick Update

We’re now using Meetup to manage our get togethers – come join us!

HTTP Archive (HAR) to CSV Command Line Tool

Recently I’ve been doing some work to monitor the performance of a web application we develop. There are several places in the pipeline of the web request that are useful to monitor – from the times taken to download the individual assets of a web request, through instrumenting the application code to profiling database requests.

One of the nice features of Chrome’s developer tools is its ability to capture the network activity and export the capture in HTTP Archive (HAR) format. This enables the traffic to be reloaded in tools like Fiddler to be further analysed. What I really need for this current analysis, though, is to represent these traffic captures in a nice tabular format for inclusion in the final report. The easiest way to achieve this, I figure, is using a simple CSV (Comma Separated Values) file.

There are various tools for working with HAR files but of the options available only the commercial HttpWatch seems to export to other formats but the price tag is a bit steep when I need just that feature. So, without further explanation, justification or blithering, I present the command line tool: har2csv

har2csv can be found on github with a couple of downloads.

I hope it proves useful.

Does Intellisense Damage Productivity?

I have been thinking about productivity lately and have come to wonder about the unintended consequences of intellisense in modern IDEs.

Productivity is efficiently undertaking what you are primarily interested in achieving. The motivation for performing this task is not necessarily relevant. Feeling unproductive leads to frustration that causes positive feedback to being unproductive. It is important, therefore, that we feel we are making progress in our day to day work – identifying tasks, seeing their value and appreciating their completeness are topics for another discussion.

Tools such as intellisense help us achieve small things very quickly and chaining together lots of small things can be productive…or at least feel like its productive. Genuine productivity has two key requirements: experience and knowledge. It is all too easy, unfortunately, for tools to build a straw bridge over any gaps in these two.

Let me be be clear, tools such as intellisense can have a profoundly positive impact on individual productivity. But there is a very real danger that they are relied upon too heavily too early.

When we are writing code we are generally doing one of three things: interacting with out domain model, interacting with libraries and frameworks or producing infrastructure. In each instance experience and knowledge are required so that we know what we are going to do and why we are going to do it. For the purposes of this post I’m going to ignore infrastructure coding as essentially a specific instance of framework interaction.

When we are interacting with our domain model we should be doing so using a DSL distilled from our domain’s ubiquitous language. The ubiquitous language itself is a product of distinct and specific team effort. It should be our lingua franca as we talk about the domain. Some developers will be fluent in the language by virtue of having been involved in the project from the start, others will have been instrumental in establishing the language but new-comers to the project must make a specific effort to learn the domain and it’s language.

Intellisense, combined with the pressures on becoming productive as soon as possible, betrays us here. The constrained list of options displayed following the dot (.) takes away from the developer their responsibility and opportunity to learn the domain and its language in any meaningful sense. With an incomplete knowledge of the domain we are at risk of making changes that are inconsistent with the existing model or repeat aspects already implemented. The net result is that pressure for an individual to become productive leads to a reduction in productivity for the team as a whole.

For framework interaction the problem is different. Intellisense presents us with an arbitrary set of operations against the current object. But which one of those should we choose? Do I want Where().First()? Is this any different from First() with a predicate? Should I loop over Add() or just stuff everything in AddRange()? Will I even think to try AddRange()?

When we are coding the framework it is important that we know what we are doing and what the consequences of our choices are. The seductive power of the next method call following a dot leads to Train Wreck code (cf. Robert Martin’s Clean Code p.98) that violates the Law of Demeter.

In both these cases the problem is exacerbated by pressures on a team and especially its new members to become productive. Tools like intellisense are provided as productivity tools. It seems a no-brainer. But in order to become truly productive we need more knowledge at the edge of our brain than at the end of our fingers. Possibly then we may discover we don’t need quite as many bells and whistles in our IDE as we think we do.

7 Languages in 7 Weeks. Week 2: Io Day 1

Io> Ferris live

So week 2 brings me (a week late) to a language I’ve never heard of before. As Ruby was Mary Poppins, so Io is, apparently, Ferris Bueller. I have a poor familiarity with Ferris Beuller’s Day Off so here ends all references to Ferris for this week’s posts.

Language Overview

Io is an object based prototype language. Everything is an object that receives messages and how the object responds to the message is dictated by its prototype. If a given object cannot respond to a message based on its prototype then the its type’s prototype is consulted (and so on). This will become clearer in a moment. This is the analogy of inheritance in a prototype language.

Like Ruby, Io is an interpreted language. After installing the interpreter and supporting libraries, statements can be entered directly into the interpreter. To run programs from a file the interpreter can either be invoked on the command line or (if the host operating system supports this idiom) referenced at the start of the program file.

Running statements in the Io Interpreter

[dan@home ~]$ io
Io 20090105
Io> "hello world" println
hello world
==> hello world
Io> 

Running an Io Program by Invoking the Interpreter

[dan@home ~]$ vi testing.io
[dan@home ~]$ io testing.io 
hello world

Running an Io Program by Specifying Interpreter in Source

[dan@home ~]$ echo '#!'$(which io) > testing2.io
[dan@home ~]$ cat testing.io  >> testing2.io 
[dan@home ~]$ chmod +x ./testing2.io 
[dan@home ~]$ ./testing2.io
hello world

Installing Io

Grab the sources to Io from the Io Github repo and unzip them into a build directory. There are a number of libraries required in order to build Io and the packaged components. I’m running Fedora 15 and needed to install the following packages to get a build that issued no warnings.

  • libffi-devel
  • freetype-devel
  • zlib-devel
  • ncurses-devel
  • libpng-devel
  • libtiff-devel
  • libjpeg-devel
  • gmp-devel
  • libxml2-devel
  • pcre-devel
  • openssl-devel
  • libglfw-devel
  • python-devel
  • rpm_build

Use the build.sh script in the extracted contents of the download to build and install Io:

[dan@home ~]$ ./build.sh 
[dan@home ~]$ sudo ./build.sh install
[dan@home ~]$ sudo ./build.sh LinkInstall

Finally you will need to update the locations used for finding dynamically loaded libraries. On my Fedora system this requires adding a new file referencing /usr/local/lib into the /etc/ld.so.conf.d/ directory and re-running ldconfig:

[dan@home ~]$ sudo echo "/usr/local/lib" > /etc/ld.so.conf.d/usrlocal.conf
[dan@home ~]$ sudo ldconfig

Io Resources

Given it’s youth, I found it surprising how much material is available on Io. A google for “io language” reveals quite a wealth of results.

Primary amongst these resources is Io’s home at http://www.iolanguage.com/. There are links from there to reference material and a useful wiki. From the wiki there are links to a useful page of samples and a style guide.

And as a sign that the language has surely “arrived”, Stackoverflow has an iolanguage question tag.

Exercises

Typing

Io is a strong but dynamically typed language. By adding slots we can provide implementations of expected methods. We can see this with a simple example:

[dan@home ~]$ io
Io 20090105
Io> 1 + 1
==> 2
Io> 1 + "two"

  Exception: argument 0 to method '+' must be a Number, not a 'Sequence'
  ---------
  message '+' in 'Command Line' on line 1

Io> "one" + "two"

  Exception: Io Assertion 'operation not valid on non-number encodings'
  ---------
  message '+' in 'Command Line' on line 1

Io> "one" type
==> Sequence
Io> Sequence + := method(next, self .. next)
==> method(next, 
    self .. next
)
Io> "one" + "two"
==> onetwo
Io> "one" + 1
==> one1
Io> 1 + "one"

  Exception: argument 0 to method '+' must be a Number, not a 'Sequence'
  ---------
  message '+' in 'Command Line' on line 1

Io> 

Here we send the + message to two Number objects and we get the expected result. When we try this between a Number and Sequence we get an exception and ditto when we try it with two Sequences. Once we define a + against Sequence our tests work. Sort of. I included that final erroring case as it highlights that the message is going to a Sequence and not a Number. Adding a + to Number such that it would accept a Sequence proved a little beyond me at this stage.

Truth

Apart from nil and false it seems pretty much everything in Io evaluates to true.

Io> true and true
==> true
Io> true and false
==> false
Io> true and nil
==> false
Io> true and 1
==> true
Io> true and 0
==> true
Io> true and ""
==> true
Io> true and "true"
==> true
Io> true and "false"
==> true
Io> true not
==> false
Io> 

Slots

The slotNames message enables us to see what slots are available on a given object:

Io> Object slotNames
==> list(newSlot, ownsSlots, isError, foreachSlot, currentCoro, <, removeAllSlots, list, for, doString, uniqueHexId, clone, become, write, evalArgAndReturnNil, serializedSlots, isNil, method, block, pause, isActivatable, deprecatedWarning, isLaunchScript, coroWith, evalArg, uniqueId, ?, actorProcessQueue, do, in, setProto, super, writeln, setSlot, !=, inlineMethod, doRelativeFile, removeAllProtos, coroDo, asyncSend, continue, stopStatus, ancestorWithSlot, print, protos, evalArgAndReturnSelf, actorRun, not, type, and, return, break, slotSummary, >, message, ==, serialized, slotNames, ifNonNilEval, asSimpleString, hasLocalSlot, while, updateSlot, switch, perform, returnIfError, asString, hasSlot, try, returnIfNonNil, hasProto, prependProto, getSlot, wait, justSerialized, hasDirtySlot, thisContext, removeProto, appendProto, println, lazySlot, loop, slotDescriptionMap, launchFile, .., relativeDoFile, serializedSlotsWithNames, compare, , yield, setSlotWithType, init, resend, isTrue, lexicalDo, or, doFile, argIsActivationRecord, raiseIfError, ancestors, isIdenticalTo, ifNil, ifNilEval, performWithArgList, cloneWithoutInit, contextWithSlot, thisLocalContext, >=, if, isKindOf, memorySize, <=, ifNonNil, coroFor, thisMessage, apropos, @, getLocalSlot, ifError, markClean, coroDoLater, slotValues, -, doMessage, proto, setIsActivatable, futureSend, removeSlot, shallowCopy, handleActorException, @@, setProtos, argIsCall)
Io> Person := Object clone
==>  Person_0x82f9b60:
  type             = "Person"

Io> Person slotNames
==> list(type)

The proto will list all the slots available to an object via its prototype.

A Note on Assignment Operators

Io has 3 operators that play a similar role:

  • =
  • :=
  • ::=

The first is assignment to an existing slot. An exception is raised if the slot is not present. This stackoverflow answer deals nicely with the difference between the second two.

7 Languages in 7 Weeks. Week 1: Ruby Day 3

Mary, Meta

Today Mary Ruby goes Meta. We explored some techniques for using Ruby to create, er, more Ruby.

My previous posts on the 7 Languages in 7 Weeks have been of an unsustainable length so I am going to aim for more succinct offerings for the rest of the exercise.

Metaprogramming

I confess I’m still waiting for the penny to drop with metaprogramming. To me it still just looks like any other language feature. And yet there are great claims made about it’s utility in re-use and constructing DSLs. The former is clear but the path to the latter is still occluded to me. More examples and applications of it in the wild will hopefully bring me enlightenment.

Metaprogramming is presented as a way of using Ruby to write Ruby. After a mere three days learning the language I find this description unhelpful. Surely we don’t want to get into eval’ing strings of Ruby? And, indeed, none of the samples and exercises we explored went down this road. So what’s up with this description?

What we can do with metaprogramming and Ruby’s open classes is package up code wholly independent of it’s eventual “host object” allowing us to focus completely on the module we are implementing.

The Exercises

We are provided with a simple CSV parsing application and our task is to modify it such that each row is now encapsulated in a new CsvRow object that allows access to each field via method_missing. This will allow us to write the following:

csvParser = RubyCsv.New
csvParser.each do |row|
   puts "Field1 value: #{row.Field1}"
end

This is my solution to this problem. The latest version can be found on github. The file it uses as input is also available on github.

#!/usr/bin/ruby

module ActsAsCsv
  
  def self.included(base)
    base.extend ClassMethods
  end
  
  module ClassMethods
    def acts_as_csv
      include InstanceMethods
    end
  end
  
  module InstanceMethods
    
    def prepare
      @filename = self.class.to_s.downcase + '.txt'
      File.open(@filename) do |csv_file|
         @headers = csv_file.gets.chomp.split(', ')
      end
    end

    def _read_headers
    end

    def each
       File.open(@filename) do |csv_file|
          csv_file.gets #skip headers
          csv_file.each do |row|
             yield CsvRow.new(@headers, row.chomp.split(', '))
          end
       end
    end
    
    attr_accessor :headers
    attr :filename
    
    def initialize
      prepare
    end

  end

end

class CsvRow

   attr :row_data

   def initialize(headers, row)
      @row_data = {}
      
      i = 0
      headers.each do |header|
         @row_data[header] = row[i]
         i += 1
      end
   end

   def method_missing name, *args
      @row_data[name.to_s]
   end
end

class RubyCsv # no inheritance! You can mix it in
  include ActsAsCsv
  acts_as_csv
end

m = RubyCsv.new
puts m.headers.inspect
m.each do |csvrow|
   puts "one: #{csvrow.one}"
   puts "two: #{csvrow.two}"
   puts "three: #{csvrow.three}"
end

Teh Codez

As ever the github area for day 3.

7 Languages in 7 Weeks. Week 1: Ruby Day 2

There’s Something About Mary

Something of an Awakening

I can be a bit of a language snob. This is usually an unfair knee-jerk reaction. Except when it comes to VB, of course :) And I have a bad habit of treating languages as second class citizens once I find myself writing “#!/blah” at the top of a file. Today’s learning has rescued Ruby from my unwarranted judgement and raised into the realms of A Proper Language.

Controversially, no doubt, I blame Larry Wall for some of this. The ubiquity of Perl and The Camel Book in the previous decade spread a laudable message of “make easy things easy and hard things possible.” But this spawned the impression that Perl was a language that would bend in any direction to achieve whatever was required; a language that didn’t really ‘do’ types and that the freedom you had with its types was achieved more by luck than judgement; a floozy of a language. As I say these are my impressions and not a technical treatment of Perl the language. But to this day it tarnishes my first impression of anything beginning “#!/blah.”

I shall work on this and hopefully the 7 Languages in 7 Weeks process will eradicate this thinking.

The Learning

In day 2 we covered arrays, hashes, code blocks, mixins, modules and sets…oh my!

Arrays

This being Ruby, there’s more to arrays than we may expect from our experience of arrays from other languages.

First off, and consistent with duck typing, we can put anything we want inside an array:

irb(main):001:0> a = []
=> []
irb(main):002:0> a.push 'fred'
=> ["fred"]
irb(main):003:0> a.push 34
=> ["fred", 34]
irb(main):004:0> a.push 23.6534
=> ["fred", 34, 23.6534]
irb(main):005:0> a.push ['fred', 'jack', 12.23, 2325]
=> ["fred", 34, 23.6534, ["fred", "jack", 12.23, 2325]]
irb(main):006:0> a
=> ["fred", 34, 23.6534, ["fred", "jack", 12.23, 2325]]
irb(main):007:0> a.each { |element| puts element.class }
String
Fixnum
Float
Array

Ruby’s API on arrays is very complete. And perhaps this is surprising if we assume an array is just an ordered collection of elements allowing random, indexed access. For example, with push, pop and shift methods we can treat our arrays like stacks and queues.

Hashes

Hashes are Ruby’s associative arrays. We can store anything against any object.

irb(main):033:0* h = {}
=> {}
irb(main):034:0> h[:one] = 1
=> 1
irb(main):035:0> h[1] = "one"
=> "one"
irb(main):036:0> h["one"] = :one
=> :one
irb(main):037:0> h
=> {1=>"one", "one"=>:one, :one=>1}

We can iterate over a hash’s keys, values and key-value pairs:

irb(main):038:0> h.keys
=> [1, "one", :one]
irb(main):039:0> h.values
=> ["one", :one, 1]
irb(main):040:0> h.each { |kvp| p kvp }
[1, "one"]
["one", :one]
[:one, 1]

That last example, using each leads us on nicely to blocks.

Blocks

Ruby supports anonymous functions through Code Blocks. We can pass these blocks to methods either as first-class parameters or via Ruby’s yield keyword.

Here’s a simple example using yield:

def one_to_ten
   i = 1
   while i <= 10
      yield i
      i += 1
   end
end

one_to_ten {|x| puts x}

y = 1
one_to_ten do |x|
  y *= x
end
puts y

The code block is defined using either curly braces ({ and }) or by do and end. The convention being that {} is used for single line blocks. In both cases the argument list to the block is given between the two pipe symbols ‘|’.

To pass a block as a parameter we decorate the parameter name with an ampersand (&):

def do_something_with_this(&block)
   block.call
end

Modules and Mixins (Magic)

On the face of it Modules in Ruby appear quite innocuous. They define some methods. Good stuff. Then we look a little deeper and see that they are brought into other classes using the include keyword. And then another penny drops. Oh my.

Let me attempt to recreate this story with some simple code.

First we define our Module:

module DrinkSomething
   def drink_coffee
      puts "Hmm, roasted bean goodness"
   end

   def drink_tea
      puts "Ahhh, infused perfection"
   end
end

So far so so. Next up we make use of it:

class OverworkedPerson
   include DrinkSomething
end

mw = OverworkedPerson.new
mw.drink_tea

Ok, well that’s a bit wordy. I mean why not just define drink_tea on OverworkedPerson and be done? Well, okay but what if we now bring Ruby’s duck typing into the game. Lets ask something to “quack” inside our module:

module DrinkSomething
   def drink_coffee
      puts "Hmm, roasted bean goodness"
   end

   def drink_tea
      puts "Ahhh, infused perfection"
   end

   def go_have_a_drink
      if likes_coffee?
         drink_coffee
      else
         drink_tea
      end
   end
end

See that call to likes_coffee? Where’s that coming from?

class OverworkedPerson
   include DrinkSomething

   attr :coffee_drinker

   def initialize(likes_coffee)
      @coffee_drinker = likes_coffee
   end

   def likes_coffee?
      @coffee_drinker
   end
end

coffe_worker = OverworkedPerson.new(true)
coffe_worker.go_have_a_drink

tea_worker = OverworkedPerson.new(false)
tea_worker.go_have_a_drink

We’ve updated our OverworkedPerson to expose a drinking preference. And the module uses that through the power of duck typing. With languages such as C# or Java such a contract would need to be made explicit through an interface. Now our OverworkedPerson knows about it’s preference of beverage but stays blissfully unaware of the mechanics of hot drink making. Instead our module acts as a mixin to provide this service based on the implied contract of the likes_coffee? method.

This is how mixins work. For example the enumerable Ruby standard mixin relies on the including class implementing the each method. Powerful stuff.

There’s one more little thing here too. Ruby classes are open. If we want to enhance a standard class with some of our own goodness then we can do so by adding our own include:

class File
   include Grep
end

Here we are adding our Grep module to the standard File class.

The Homework

Day 2’s work began in the none-too-shabby setting of Yu-foria, Covent Garden. There I met with Aimee to tackle the first of these problems. Coffee, yoghurt, wifi, code and an apparent lack of appreciation for the market value of fruit made for a good hour.

Printing an Array’s Contents

Aimee and my efforts on this can be found on Aimee’s github repo

Following on from that I had a play with modules and mixins to solve the problem. The results of which can be found in array_each.rb

I don’t think the result is any cleaner than that achieved by the elegance of the each_slice solution shown in Aimee’s repo but it was an instructive play.

Update the Tree’s API

This was an interesting challenge. We were provided a simple tree implementation and our task was to transform it’s API such that it could take, as its constructor parameter, a hash of hashes that represented the tree.

Instinctively I wanted to use what we had learnt about mixins to make the Hash the tree. In retrospect I can see some problems with this: it relies on a specific structure of the hash such that every value is itself a hash yet it applies to all hashes. The code is on github but reproduced here too:

module HashTree
   def node_name
      keys[0]
   end

   def children
      children_list = []
      values[0].each_pair do |k,v|
         children_list.push({k => v})
      end
      children_list
   end

   def visit(&block)
      block.call self
   end

   def visit_all(&block)
      visit &block
      children.each { |child| child.visit_all &block }
   end
end

class Hash
   include HashTree
end

class Tree
   attr :node_hash

   def initialize(seed_hash = {})
      @node_hash = seed_hash
   end

   def visit(&block)
      node_hash.visit &block
   end

   def visit_all(&block)
      node_hash.visit_all &block
   end
end

my_tree = Tree.new({'grandpa' => {'dad' => {'child_1' => {}, 'child_2' => {}}, 'uncle' => {'child_3' => {}, 'child_4' => {}}}})
my_tree.visit { |node| puts node.node_name }

puts
puts 'Visiting all'
my_tree.visit_all { |node| puts node.node_name }

A Simple Grep

The challenge was to recreate a simple grep tool. The code below provides the core of a solution but does not tie in with the cli to give a user interface. Using a module the solution below will bestow grep-ability onto anything implements an each method.

module Grep
   def grep(match)
      line_count = 1
      each do |line|
         yield({:line_no => line_count, :text => line}) if line =~ Regexp.new(match)
         line_count += 1
      end
   end
end

class File
   include Grep
end

def grep_file(filename, match)
   File.open(filename) do |file|
      file.grep(match) { |match| puts "#{match[:line_no]} #{match[:text]}" }
   end
end

A colleague of mine pointed out something interesting. This code does not necessarily look very testable. After all we have an immediate dependency on a concrete type (File) – how can we stub that out in our tests? Now I have now idea what testing idioms exist in Ruby but I suspect duck typing plays a big part in stubbing out dependencies. So now I’m inclined to think that actually this is testable as the bit we care about testing is nicely wrapped up in our module which we can happily include in any object we want for testing purposes.

Quick Summary

I felt today’s work really brought Ruby to life for me. It separated itself from my previous experiences and became a free-standing language. The duck typing proved enormously powerful – especially when combined with the flexibility of modules and mixins.

One thing that came to mind whilst working with all these blocks is that lurking nasty of working with closures and deferred execution. I was just going to link to another post showing this behaviour but a quick google did not bring one up so here’s some code:

def bad_closure_list
   my_number_multiplier_list = []
   i = 0
   while i < 10
      my_number_multiplier_list.push lambda {|x| i * x}
      i += 1
   end
   return my_number_multiplier_list 
end

def do_something_with_bad_closures(closure_list)
  closure_list.each do |closure|
     num = closure.call(1)
     puts num
  end
end

do_something_with_bad_closures(bad_closure_list)

Note the problem happens at line 5 where we capture the loop variable in our closure. We are only ever capturing the same variable for every closure we create. Therefore when we later come to execute out code blocks, each will be referring to the same, single, variable.

Here’s a rather detailed examination of Ruby code blocks.

Teh Codez

Github repo for Ruby day 2

7 Languages in 7 Weeks. Week 1: Ruby. Day 1

Mary, Mary Quite Contrary

First up in our tour of 7 languages is Ruby. A language Bruce Tate characterises as Mary Poppins (It’s okay, I did the Dick van Dyke impression too). Day 1 of Ruby sees us take a high level view on Ruby’s programming model, control structures and typing system.

Before we can get our teeth into this content though we need to prepare our machines to play with Mary, er, Ruby. I currently only run Linux outside of the office so these instructions are only concerned with getting Running on a Fedora machine. Given the trivial nature of the install, you’re not missing much if you run other operating systems.

The packages ruby and ruby-irb provide the ruby interpreter and the interactive shell respectively. Install them using yum:

sudo yum install ruby ruby-irb

With the interactive shell installed we can now start writing some code and start playing with what we cover in the book. Start the shell using the irb command.

Useful Ruby Linky

Language Overview

Programming Model

Ruby has no huge surprises for us here. At least not yet. Ruby is a pure object-oriented language: everything is an object. The type of object we’re interrogating can let us know its class by sending it the class message

irb(main):001:0> 1234.class
=> Fixnum
irb(main):002:0> 

Control Structures

Here Ruby starts to show its colours a little more. Like Perl, we can postfix our expressions with our control statements. Allowing us to do things like this:

i = i * 2 until i == 100

Ruby has while and if statements but also less common versions of these: until and unless. These give us great freedom about how we structure our code and express our intent. Code that reads closer to a spoken language is easier to understand, bugfix and maintain.

A minor surprise perhaps lurks for C programmers. 0 (nought, the number) is true.

irb(main):003:0> puts "It's True" if 0
It's True
=> nil
irb(main):004:0>

But, of course, you wouldn’t be using numbers as conditional guards anyway, right?

Typing System

Ah. And here Ruby really starts to separate itself from the pack. Not least because of the term that’s been coined to describe the type system employed. It is a duck typed language. If it quacks, waddles and looks like a duck, its a duck.

If the object in question supports a message sent to it at a given time then that object will be sent the message and will behave itself. To support this feature Ruby must be a dynamically typed language. This means that the compiler will make no check of the validity of messages sent to the object, instead the run time will throw an error if the object does not support the message in question. This gives the final characteristic of Ruby’s type system: strongly typed.

This final point can be a bit subtle as its tempting to think a language that does not enforce type safety at compile time cannot be strongly typed. But the presence of these checks at run time means that we are preserved from the potentially disastrous scenario of an unsupported message being sent to an object that then attempts to handle it. This can only ever be undefined behaviour (ignoring language specific mechanisms for intercepting unsupported messages).

Daily Challenges

So on to the homework. There are text files in the github repo linked to below for these but interesting content is duplicated here.

String Substitution

Remember that everything is an object? So we know that a string, defined using either double quotes (“”) or single quotes (”) will be an object that supports methods. Its a fair bet then that one of those methods will be do a substitution or replace operation.

When we look at the core API docs we find the sub and sub! methods. As a note of minor interest, it seems Rubyists don’t go in for using brackets after method names when none are required. The first thing to note here is that there are two methods with apparently the same name excepting the trailing exclamation mark on the second. What gives? Here’s the text from the documentation:

Performs the substitutions of String#sub in place, returning str, or nil if no substitutions were performed. 

Without the trailing “!” the string will be treated as immutable and we will be handed back a new object. The trailing “!” will allow us to mutate the original string.

I’m reproducing the examples given in the core docs here, followed by my own notes.

irb(main):001:0> "hello".sub(/[aeiou]/, '*')
=> "h*llo"

This is a simple substitution of a character found within a character class. Note that Ruby is only applying the substitution to the first match found: the terminating ‘o’ is not altered.

irb(main):002:0> "hello".sub(/([aeiou])/, '<\1>')
=> "h<e>llo"

As before we are matching on a character within a character class but this time, by wrapping a part of the regexp in brackets, we are storing any match into a back-reference. In the replacement string we refer to this back-reference using its numerical ordinal (1) thus: ‘\1’.

irb(main):003:0> "hello".sub(/./) {|s| s[0].to_s + ' ' }
=> "104 ello"

The special character ‘.’ for matching any character is used this time. The replacement is provided as a block and the ‘current’ match is passed in as a parameter. The v1.8.7 version of the code uses the fact that a string can be accessed as an array to get back an object of type Fixnum with a value of the character’s ordinal value. The to_s call is important as without it the ‘+’ operator will attempt to coerce the space string (‘ ‘) to a number and fail.

Regular Expressions

As a one-time Perl hacker programmer I have a fairly strong if rusty grasp on regular expressions and my look into the string substitution methods above seem to suggest a good correlation between Perl’s regexps and Ruby’s. So, I confess, I skipped this bit of homework. Let’s assume the dog ate it.

However for completeness here are some links for Ruby regexp goodness. As Ruby Newbie (hah – there’s an expression!) I don’t have much feel for differences between versions but it seems v1.9 has added some extra features to the regexp side of things.

Ranges

A ruby range is a sequence containing all values from, and including, a start value up to, and optionally including, an end value. That makes it sound harder than it is:

irb(main):003:0> (1..4).each do |v|
irb(main):004:1* puts v
irb(main):005:1> end
1
2
3
4
=> 1..4

The range is constructed using a literal expression and we can see from the output that both the start and end are included in the range.

irb(main):006:0> Range.new(1,4).each do |v|
irb(main):007:1* puts v
irb(main):008:1> end
1
2
3
4
=> 1..4

The range is constructed using the Range.new method. We see the same results as last time

irb(main):009:0> Range.new(1,4,false).each do |v|
irb(main):010:1* puts v
irb(main):011:1> end
1
2
3
4
=> 1..4

Passing in the third boolean parameter we can see that the default for this parameter is false, as the end value is still included in the range.

irb(main):012:0> Range.new(1,4,true).each do |v|
irb(main):013:1* puts v
irb(main):014:1> end
1
2
3
=> 1...4

Setting the optional third constructor parameter to true has excluded the end value from the range.

Running Scripts In Files

As a first attempt at this I went with standard unix practice:

Script started on Mon 13 Jun 2011 06:27:44 PM BST
[dan@home day1]$ which ruby
/usr/bin/ruby
[dan@home day1]$ echo '#!/usr/bin/ruby' > test1.rb
[dan@home day1]$ echo 'puts "hello world"' >> test1.rb
[dan@home day1]$ chmod +x ./test1.rb
[dan@home day1]$ ./test1.rb
hello world
Script done on Mon 13 Jun 2011 06:28:52 PM BST

And a quick Google confirms that this is the way to go. Perverse use of echo notwithstanding.

Bonus Challenge – What’s My Number?

There are a number of minor tasks that exercise what we’ve learnt so far. The code for these can be found on the github repo. I won’t go into them here.

The bonus exercise is to write a Ruby program that will ask the user to guess a number. I ummed and ahhh’d on this one a bit but eventually settled on a brute force solution using nothing more than I’d already learnt today. It was tempting to look into class and method definitions to see how we could create better encapsulations. And it was very difficult not to look into ways of testing the Ruby code. So what we have is a very flat and procedural solution with nested loops and clunky great variables. It is, in truth, essentially C written in Ruby’s syntax.

As I said, this is not entirely without reason. I am curious to know where this journey of learning will take me and therefore I will keep as open a mind as possible, not second guessing too much. It’s also important to set a maintainable pace and heavy research into language features will be an unsustainable time drain at the moment.

Here is version 1.0 of the number guessing solution:

#!/usr/bin/ruby
#
#Dan Kendall 06/2011
#
#guess_the_random.rb
#

mySecret = rand(50)
guessCount = 0
puts "Guess the number (between 0 and 50) I'm thinking off!"

guess = "blah"
correctionMessageRoot = 'Bad luck. You were wrong, please try again. Hint: go '
goHigherHint = 'higher'
goLowerHint = 'lower'

guessIsNotCorrect = true
while(guessIsNotCorrect)

   guessIsNotANumber = true
   while(guessIsNotANumber)
      guess = gets
      guessIsNotANumber = !(guess =~ /^[0-9]+$/)
      if(guessIsNotANumber)
         puts "Please guess with numbers only"
      end
   end

   guessCount = guessCount + 1
   guessToTest = guess.to_i
   
   if(guessToTest == mySecret)
      guessIsNotCorrect = false
   else
      if(guessToTest < mySecret)
         puts correctionMessageRoot + goHigherHint
      else
         puts correctionMessageRoot + goLowerHint
      end
   end

end

puts "Congratulations you guessed correctly that my secret was #{mySecret} in #{guessCount} attempts"

Gentle but forceful comments have been made to suggest that this solution is not especially sympathetic to the way of The Rubyist. I am in complete agreement with this and look forward to learning more so that my code may better exercise common Ruby idioms and become more naturally of the language.

First Impressions

I like. Some of this I intend to cover in another post I have in draft form, but I felt really happy to be so close to what I was doing. Just me, vi and a console emulator: what more does a boy need to write code? Okay, so that’s not exactly Ruby specific but it was still a pleasant experience.

I confess some ill ease at the duck typing. But I can think of no specific problem so I’m putting that down to an unreasonable fear of change that will no doubt be abated as time goes on.

In many respects its difficult to give an impression at this stage. At this basic level, rightly or (and probably) wrongly I found Ruby behaved as expected based on my previous Perl experience. I have no doubt this will change, especially as we start to really explore the strong object oriented and duck typed nature of the language.

Teh Codez

Can be found in my github repo.

Settling Temporal Dependencies and Providing a Testing Seam with a Tuple

Having recently written a wrapper around WCF’s ChannelFactory class I ended up with an interface that looked like this:

public interface MySlightlyAwesomeChannelFactoryInterface<TContract>
{
    ChannelFactory<TContract> GetChannelFactory();
}

Ok, so I’ve got something I can mock up, right? After all: it’s an interface. But what happens when the SUT consumes this interface? In that case my mock will only return a concrete ChannelFactory type. That’s not especially useful as I can’t mock that in a satisfactory fashion.

Well thankfully it turns out that my consuming classes only care about the client proxy implementing TContract and the ICommunicationObject interface that ChannelFactory implements. So now I’ve got bit more of an awesome interface that’s testable:

public interface MyNearlyAwesomeChannelFactoryInterface<TContract>
{
    TContract GetClientProxy();
    ICommunicationObject GetServiceChannel();
}

So this bad boy allows us to mock up all dependencies in our SUT. Great, right? Almost. Let’s think about how we’re going to implement this.

We could have something like this:

public class MyNearlyAwesomeChannelFactory<TContract>: MyNearlyAwesomeChannelFactoryInterface<TContract>
{
    private readonly ChannelFactory<TContract> _channelFactory;

    public MyNearlyAwesomeChannelFactory()
    {
       _channelFactory = new ChannelFactory<TContract>();
    }

    public TContract GetClientProxy()
    {
       return _channelFactory.CreateChannel();
    }

    public ICommunicationObject GetServiceChannel()
    {
       return _channelFactory;
    }
}

However this has problems as exhibited by the fact that the consuming class must call GetClientProxy() before calling any methods on the ICommunicationObject. This is what I think of as a temporal dependency: a consumer must, in a non contractually enforceable way, call methodA() before calling methodB(). I suspect there’s a posh term for this.

We could create both the ChannelFactory and the TContract proxy in the constructor (or first call to either of the methods) but we would have a similar problem as the consuming class could easily Abort() or Close() the channel’s connection and yet sensibly expect a subsequent call to GetClientProxy() to return a working proxy.

To get around this I employed a Tuple. This allows us to create and return the channel and communication object in the one place and the consumer has the concrete hint that the two instances are linked. The error caused by calling Close() on the Tuple's communication object and then performing an operation against the proxy should not come as a surprise.

Our new interface looks like this:

public interface MyDangerouslyCloseToAwesomeChannelFactoryInterface<TContract>
{
    Tuple<ICommunicationObject,TContract> CreateServiceProxyChannel();
}

So, temporal dependency solved and testable mocking seam provided as we can cheerfully mock out our TContract, ICommunicationObject and MyDangerouslyCloseToAwesomeChannelFactoryInterface interfaces without a problem.

There’s one final piece of the puzzle here though. Tuple has property names of Item2 and Item2 (and Item3, etc.). Ewww. A consuming type has to guess at the intent of something called Itemx. So to finish this off nicely we create ourselves a Tuple wrapper with immutable properties, thus:

public ServiceProxyChannel<TContract>: Tuple<ICommunicationObject,TContract>
{
    public ICommunicationObject Channel { get { return Item1; } }
    public TContract ServiceProxy { get { return Item2; } }

    public ServiceProxyChannel(ICommunicationObject channel, TContract proxy)
        :base(channel, proxy) {}
}

Our ChannelFactory wrapper interface is finally awesome:

public interface MyAwesomeChannelFactoryInterface<TContract>
{
    ServiceProxyChannel<TContract> CreateServiceProxyChannel();
}

And we can implement it in a way that won’t surprise or disturb our consumers:

public ServiceProxyChannel<TContract> CreateServiceProxyChannel()
{
    var channelFactory = new ChannelFactory<TContract>();
    var serviceProxy = channelFactory.CreateChannel();
    return new ServiceProxyChannel<TContract>(channelFactory, serviceProxy);
}

It should be noted that the production version of this code has marginally better names for things (naming stuff is hard) and does more than simply provide a convoluted interface around ChannelFactory without providing any extra functionality.

Simple XCF to JPEG Batch Conversion Script

Introduction

A little while ago I wrote this very simple Gimp script to do a batch conversion of all .xcf files in a directory into .jpg files of a moderate quality. To install just copy the below code into a suitable in your gimp profile “scripts” directory, e.g.

~/.gimp-2.6/scripts/xcf2jpeg_batch.scm

The script will install itself on the menu under Mine/Batch/2JPEG as shown below.

The script installed The Gimp menu

Feel free to use and abuse as you see fit. As ever with stuff you get from the internet – if this script should sprout teeth and eat your hard drive, you’re on your own. But perhaps a quick comment would allow others to benefit. The code is a gist on Github

Thanks to Jerome for providing a fix that means the image is flattened before conversion! See the gist revisions for the changes.

The Script

(define (script-fu-xcf2jpg-batch xcfDirectory inQuality)
(let* ((xcfList (cadr (file-glob (string-append xcfDirectory "/*.xcf") 1))))
(while (not (null? xcfList) )
(let* ((xcfFilename (car xcfList))
(jpgFilename (string-append (substring xcfFilename 0 (- (string-length xcfFilename) 4) ) ".jpg"))
(xcfImage (car (gimp-file-load RUN-NONINTERACTIVE xcfFilename xcfFilename)))
(xcfDrawable (car (gimp-image-flatten xcfImage))) )
(file-jpeg-save RUN-NONINTERACTIVE xcfImage xcfDrawable jpgFilename jpgFilename
inQuality 0.0 0 0 "" 0 1 0 2)
)
(set! xcfList (cdr xcfList))
)
)
)
(script-fu-register
"script-fu-xcf2jpg-batch"
"Batch XCF 2 JPEG"
"Converts all xcf files from a directory to JPEGS"
"Dan Kendall & Jérôme Combaz"
""
"Apr 20 2013"
""
SF-DIRNAME "Gimp Image Directory" "/tmp"
SF-ADJUSTMENT "Quality" '(0.85 0.00 1.00 0.01 0.1 2 0)
)
(script-fu-menu-register "script-fu-xcf2jpg-batch"
"<Toolbox>/Mine/Batch/2JPEG")
view raw gistfile1.lisp hosted with ❤ by GitHub