OpenFIGI: Currencies and Spot FX Pairs

Anyone who knows me knows that I have an unhealthy interest in open symbologies.  Until we have a common reference for identifying things, it is difficult to build higher-order open connectivity (such as reusable APIs).

In the financial instrument space, the best symbology I am aware of is OpenFIGI.  Whilst it is maintained by Bloomberg, it is open and free to use.

I recently had a question and want to share the answer in case anyone else has the same query.  Incidentally, OpenFIGI support were super speedy and helpful when answering – kudos!

My question was regarding representing base currencies – USD or SGD.  If one searches on OpenFIGI for the Currency SGD, one gets the following:

From this result alone, it is unclear whether this represents a pure ‘SGD’ currency, or (as the name suggests) a USD/SGD spot FX pair.

The response from support was:

The FIGI BBG0013HFJF2 does represent the USDSGD Spot Exchange Rate – Price of 1 USD in SGD. All currency FIGI represent pairs. A FIGI is not assigned to a single representation of a currency alone. No FIGI exists for just USD or CAD or SGD, only in pairs of currency.

This clears up the confusion quite nicely.  Personally I would have chosen the ticker ‘USDSGD’ to represent the pair, but so be it.

Open symbology for the win!

virtualenvwrapper installation on MacOS in 2018

This post is to try and save someone the 15 minutes I stupidly wasted!

If you installed Python via Homebrew, it now names the python executable python2 (and similarly pip becomes pip2).

When you then try to install virtualenvwrapper you get the error:

Links such as https://stackoverflow.com/questions/11507186/python-virtualenv-no-module-named-virtualenvwrapper-hook-loader are absolutely correct – they were just written before Brew changed the naming convention to python2.

The fix is super simple – set your .bash_profile/.bashrc as follows:

 

Cheese grater cost management.

Ned’s new term of the day: Cheese grater cost management.

I’ve been thinking a lot recently about how to build great culture at Argomi, and as part of that I was thinking about where I’ve seen team morale suffer in the past. I’ve decided that in difficult times, big companies suffer from cheese grater cost management.

Imagine you have a big block of delicious cheese. Whether it’s Wensleydale or Crumbly Lancashire is totally up to you, I’m not judging. Now imagine that one corner of the cheese has gone mouldy. Oh no!

A smart person would cut off that corner, and enjoy the creamy goodness that remained. But a misguided soul might run a cheese grater across the whole block, hoping that by shaving off a small amount from everywhere, the mould will be gone.

Surprise surprise, that fiendish fungus remains. So the grater is dragged across once more. And again. And again. Until nothing is left.

And that, my friends, is cheese grater cost management.

Product/Market fit at large Financial Institutions

I had an interesting realisation today regarding prioritisation at a startup vs a large Financial Institution.

As any startup mentor will advise, the key point to early growth is product/market fit. Without that, one is essentially producing a product that isn’t quite right for the market in which it is aimed…
In a startup, this is comparatively (?) easy to measure – one is either making sales or not. In a large FI, it is much harder to measure as the reasons for a project are less clear:
  1. A senior stakeholder may have mandated the project. Political power determines which projects are greenlit and which aren’t – not product/internal market fit.
  2. The “sunk cost fallacy” whereby a project continues because it has already started, and money has already been spent.  As Mr Magnus Magnusson would put it: “I’ve started, so I’ll finish”.
  3. Unclear or conflicting business priorities lead to a senior technologist making a call over what to do next – often based on which technology he or she is most excited about at that point in time.
  4. Perhaps most nefarious of all – crappy systems that would die if allowed, but are kept alive due to a single user/desk who don’t realise the cost it is incurring due to ineffective/inefficient/incorrect chargeback models.  For this point specifically, I am sure that if we could do front-office P&L net of fees AND net of (true) internal costs, the behaviour towards tech projects would change.
The first reason includes cases where the senior technology manager can allocate budget to projects that improve his or her political standing, but is not aligned with business objectives.  Conversely, one must be careful of not preventing “pure” tech projects to be approved, purely because there is no short-term business benefit.
Net result is that technologists are not necessarily working on a project or product that would be paid for in the real world, and users have a product they wouldn’t pay for forced down their throat. Obvious now I say it, but less obvious at the time when you’re 6 months into a project from hell!

Class Libraries With Microservices

Back in the olden days, everyone went nuts for Service Oriented Architecture.  Many consultant dollars were made.  Everyone temporarily forgot who Fred Brooks was (No Silver Bullet). Time moved on, the hype died down.  But just like a Jeffrey – IT COMES BACK.  Yes, I’m talking about micro-services.

Martin Fowler has a really nice piece on this topic: http://martinfowler.com/articles/microservices.html where he talks about the advantages of being at least familiar with the approach – especially when designing for cloud-based applications.

With that in mind, having a decentralised, distributed design was important to me as I map out the design for Argomi on the back of whatever receipts and envelopes I can get my hands on (I look forward to the day when tablets provide me the functionality of pencil and paper).  However, I have now stumbled across a question I have previously agonised over: the shared class library.

If we are splitting up a design into pieces, then it seems obvious to want to have a class library shared amongst all services with business logic encapsulated and lots of nice code-reuse. The issue comes from the implicit coupling.

One of the ‘points’ of microservices is that each service is independent of the others, and can have its own release cycle – or be completely replaced with an improved version, and the other services don’t necessarily need to even know.  This leads to articles like: http://www.simplicityitself.io/our%20team/2015/01/12/sharing-code-between-microservices.html which suggest not sharing anything – even class libraries – since changes may require all services to be upgraded at the same time. It also pretty much forces you to use the same technology for all services so that the class library can be shared.

My feeling right now is that avoiding a potential coupling is normally not worth the cost of reimplementing complicated business logic and that one should:

  • Strive to design fixed interfaces and classes so basic interactions with the class don’t change over time.
  • Make changes backwards compatible between versions (e.g. providing a default implementation for a new field) so that the library doesn’t have to be updated in all services exactly simultaneously, however;
  • Don’t allow multiple versions of the library to run for longer than absolutely necessary.  This is just a modern version of dll-hell, and leads to unnecessary mental load for all developers who need to remember which versions are out in the wild.
  • For the same reason, unless absolutely necessary never create two interface points to the same function, with one simply acting as a pointer to the other so that original callers don’t need to be updated to consume a new interface signature.

Does anyone have any alternative experience they would like to share?  Perhaps a shared class library that got out of control?

No discussion of microservices would be complete without linking to the fallacies of distributed computing.

 

Argomi

I quit my job a few months ago and returned to Asia to start on the next chapter of my life: Argomi.

Argomi is a personal finance manager for people comfortable with interacting with online services for money management.  We aim to give our users advice about their investments, easy access to financial services which are otherwise perhaps a little intimidating and – over time – reducing their costs through aggregating volume.

I am the CTO, and will be writing a little about my experiences.  It has already been a very exciting transition from technical manager at a huge financial institution to a hands-on startup founder, with exposure to all sorts of business activities that were previously handled by someone else.

I will also write about the decisions that I am making and why – both technical and organisational.  Hopefully they will be of value to people going down a similar path.

In the words of the great King Theoden:

AngularFire 2.x.x

I recently struggled a little to find good documentation on AngularFire 2.x.x – basically the version post when Google bought out FireBase.  If you search for Firebase and AngularJS, or even if you go to http://angularfire.com, it all goes to the older version.

However, there is some fairly good documentation on the GitHub repo (https://github.com/firebase/angularfire) – but best of all, one of the examples on the repo has been migrated to the latest version:

https://github.com/gordonmzhu/angular-course-demo-app-v2

I found this example to be fantastic, not only of AngularFire but also AngularJS in general.  If you are looking for easy to understand examples of great architecture, I think you could do a lot worse.

The architecture is modelled after this excellent style guide:

https://github.com/johnpapa/angular-styleguide/blob/master/a1/README.md

Dummy Files in Unit Tests

(Recovered from my old Blog).

Let’s say that you have an application which accepts a filepath as part of its interface. When unit testing, you may have some dummy files which you use to test the interface.

However, getting the full path of these test files can be a pain.

Assuming that the files are (as per Maven conventions) in /src/test/resources/ then they will be put on the classpath (by default /test-classes/ as part of the build process.

If you’re using Spring, then you can take advantage of that to do the following:

inputFilename now contains the full path to the file, and can be passed into your application as per normal.

Load a CSV file into an ArrayList

(Recovered from my old Blog).

It isn’t hard to write code which iterates through files, cuts them up and loads them into an ArrayList. And if the delimiting character isn’t a comma? No problem, it isn’t hard to add that functionality. And if you want to skip the first set of lines because you’re paginating the file? Easy! And if you want to write some changes back out again – no problem, I can add that feature in no time.

Except it’s already been done. So why waste the time? 🙂

OpenCSV is an easy to use library which you can embed into your applications (Apache License). Then you can simply do something like:

And you’re done. See the OpenCSV homepage for some of the other features.

Custom XStream Converter

(Recovered from my old Blog).

XStream is a great utility for serializing objects to XML – and then back again.

The default XML output is neat enough – albeit a little verbose (using fully qualified class names for example). There are plenty of tutorials available showing how to use XStream aliases to clean up the output (e.g. Manual Tweaking Output).

However, sometimes it is not enough to just tweak and alias – more serious manipulation of the output stream is required. For example, complicated Collection-based objects come out very cumbersome (but generic – which is the point). Luckily, XStream provides a powerful interface for custom generation (A Converter in XStream-speak).

To fully control how an object is serialised, create a new class which implements the Converter interface:

As can be seen above, the canConvert method tells XStream which objects should use this converter. The other two methods are called when converting objects to XML (marshal) or converting XML to objects (unmarshal).

Marshalling the object is achieved by starting and ending Nodes, while populating their attributes and values. So for example:

will iterate through a collection generating the following output:

These patterns can become arbitrarily complex.

Unmarshalling is a little more complicated. XStream will work out which (registered – see later) Converter to use by matching the node to the class (in my head these seems like an extremely difficult task… I’m not sure how XStream has implemented this – maybe I should look into this!) The node (and all children) will be passed to the unmarshal method.

It is up to us to define how the target object is populated from these nodes. Methods are provided to:

Check if the node has children (reader.hasMoreChildren())
Move down to the child level (reader.moveDown())
Move back up to the parent level (reader.moveUp())
The moveUp/Down methods can be called as many times as necessary to walk the tree.

I provide a full example as I couldn’t find many good ones on the net:

Out of completeness, here is the equivalent marshal implementation:

To register the Converter simply add the following line when you instantiate the XStream object: