dev – Mind the Emigrant Gap

Building a 360 degree view of the customer with data is a two-way process

Comments Off

Oct 262016

Today I crosslink to my article Building a 360 degree view of the customer with data is a two-way process as originally published to Finextra on 18 October 2016.

“The banking sector in the UK has become much more competitive – new challenger banks have increased savings rates to attract customers while new service offerings aimed at underserved markets are also coming to market. The banking sector has seen growth, mergers and splits over the past few years as well, creating new entities that have to attract customers in new and smarter ways.”

Simplifying Market Tick Data Management

Comments Off

Sep 052016

Today I crosslink to my article Simplifying Market Tick Data Management as originally published to Finextra on 12 August 2016.

“For my first post, I thought I’d look at some of the issues that exist around managing trading data. After all, market trading generates phenomenal amounts of data. The NYSE, for example, saw more than 3.17 billion trades on the 18th of December 2015, with a value of over $110 billion. The NASDAQ has more than 2,000 equities that are being actively traded. As each market trade takes place, banks and financial services institutions have to ingest and track these transactions over time. The volume of trades for each equity can reach up to a few million ticks per single day, and exponentially more for equity options; multiply this across multiple stocks and exchanges, and the volume of data created over time is vast.”

Software development in banking: Where MiFID II can lead to better applications

Comments Off

Aug 282016

I am pleased to relay that JAX Magazine published my article Software development in banking: Where MiFID II can lead to better applications in the August 2016 issue.

“The Markets in Financial Instruments Directive, or MiFID, was originally developed to regulate financial markets across the European Union, creating a single market for investment services and other activities. Following the launch of MiFID in 2007, a new version has been discussed and is due to be brought into force in January 2018.”

Searchable XML Documents with DataStax Enterprise

Comments Off

Feb 072016

Recently a customer asked for advice on storing a large number of XML documents in DataStax Enterprise. Though each XML document will be just a few kilobytes in size, there will be many billions of them. Multiple fields inside each XML document must be indexed and searchable in real time.

I have come across this and similar use cases a few times as a Solutions Engineer for DataStax, and indeed we are an excellent database platform for building a large, distributed, searchable document store. So in turn I created the DataStax XML Demo to show one way to store, index, and search XML documents with DataStax Enterprise. Please check out the GitHub repository and write me with your thoughts and questions.

DataStax Summit Europe 2016
To strike a related but tangential note, please come heckle me and my colleagues at DataStax Summit Europe 2016 in April. Open to all, free to register. You have no excuse, and you just might learn something.

Apple and Adobe: It’s the Censorship, Stupid

Comments Off

May 172010

The hearts of trendy technology fanboys the world ’round have been aflutter over the Apple versus Adobe showdown. All this agitation on the intellectual sea has produced some foam, as agitation will, but I worry when emotions run high and hyperbole runs rampant. A private technology mailing list to which I am privy began dissecting Kevin Lynch’s interview with Brady Forrest at Web 2.0 Expo SF. One gentleman, targeting Mr Lynch’s dark depiction of “walled gardens” (read: the iPad), went so far as to call Mr Lynch an “evil, evil man” for his demands that Apple open its precious platform to Adobe’s products.

Kevin Lynch a twice-evil man, is that not a tick harsh? An unabashed puller of the corporate line, certainly…though as the CTO, he helps write the corporate line. A egotistical geek, sure…again, CTO. Evil? The legal-but-immoral thoughts Steve Jobs wakes up with on a given Tuesday would make Mr Lynch curl up in a ball and beg for his mother, yet still I would hesitate to call Mr Jobs evil. (What is Steve, then? It rhymes with brass soul.)

To consumers, Apple sells delicious little bundles of hardware and software: iPhones, iPods, iPads. Apple also sells, to developers, access to that army of captive consumers, taking its famous 30% cut on App Store purchases. Apple created a platform and is her gatekeeper, censoring content and now dictating development methods. Do we as a society want Apple, or any corporation, to be the lone censor for a widespread and important consumer platform? I submit that society does not. The printing press manufacturer may not censor the daily newspaper. The descendants of Johannes Gutenberg may not challenge J. K. Rowling, no matter how derivative her imagination or childish her writing.

Our entire body of work here at Lab49 depends on tools and platforms built by others, and specifically, much of my own craft involves running Adobe tools on Apple platforms (excepting when bank clients force me onto Secure Build Windows, itself an oxymoron if ever there were one). On one hand I want Apple and Adobe to kiss, make up, and get the band back together for an overpriced reunion tour. On the other hand I enjoy watching two promoters of closed systems accuse each other of promoting closed systems. Hypocrisy may be a scoundrel’s refuge, but it is my entertainment.

Flash Builder Fail: Flex Run Time Resources

5 Responses »

Apr 292010

Flash Builder is, speaking the King’s for a tick, pure rubbish. Adobe took its sweet time releasing this purportedly major upgrade, and I held a fool’s hope that Flex developers the world over might no longer gaze longingly at the highly functional and pleasantly integrated Silverlight tools across the aisle. Indeed, Adobe incremented the version (to 4 from 3) and switched the label (to Flash Builder from Flex Builder), but still the product is rubbish. I know it was you, Adobe. You broke my heart.

I am not alone in my sentiments. My associate Scott Selikoff focuses on Flex 4’s code verbosity and duplicate component libraries, but his verdict is the same. Rubbish. Unfortunately, Flash Builder is my primary development environment.

My present Flex frustration (Flexstration!) is resource modules, by which a developer may externalize resources into SWFs that may be loaded at run time. It sounds wonderful, but sadly the process is quite manual, requiring a human to take a number of heavy steps:

Determine the resource bundles required, by running the compiler with the -resource-bundle-list option
Gather the contents of the output file generated in step 1
Run mxmlc from the command line — Flash Builder offers zero support here — specifying no MXML file but supplying the -locale, -source-path and -include-resource-bundles options

The poor developer must repeat this process for each resource module in each locale. As an example, let us assume the application has four resource modules and must support merely four locales: American English (en_US), British English (en_GB), Mexican Spanish (es_MX), and Spanish Spanish (es_ES). Even for these decidedly unambitious numbers, the now-heartbroken developer — or his helper monkey — must repeat the manual operations listed above 4 x 4 = 16 times. Pray for Mojo.

Of course, a clever developer could easily write a build script, perhaps leveraging Ant and Adobe’s Ant tasks for Flex. Most Flex developers, myself included, have done just that. My point is that this sort of automation of common tasks is exactly what a £400 enterprise development tool should do. Epic fail, Adobe.

[Update 21-July-2011] Corrected a typo in the “heavy steps” list.

Achieving Nirvana with Flex

1 Response »

Apr 132010

In my previous article, Achieving Nirvana on Mac OS X, I set up a Nirvana server on port 8887 and imported a realm for the TradeSpace demo. The TradeSpace demo instructions had impudently asked me to run two .BAT scripts, setup.bat and startall.bat, and when we left off, I performed manually the equivalent operations for the former. Today I will tackle the latter which, according to the instructions, will create and populate the demonstration news and rates channels in our Nirvana realm.

The script contains only two simple lines to run two other scripts, startnews.bat and startrates.bat, and thankfully these secondary scripts are short and sweet. Inspecting them, .BAT-style rem comments redacted:

#startnews.bat set CLASSPATH=%CLASSPATH%;..\classes java -cp "%CLASSPATH%;..\classes" -DRNAME=nhps://showcase.my-channels.com:443 -DRNAME2=nsp://localhost:9000 -DCHANNAME=/showcase/consolidatedrss com.pcbsys.demos.feedsimulators.NewsFeeder

#startrates.bat set CLASSPATH=%CLASSPATH%;..\classes java -cp "%CLASSPATH%;..\classes" -DRNAME=nsp://localhost:9000 -DRNAME2=nsp://localhost:9000 -DCHANNAME=/showcase/stockindices com.pcbsys.demos.feedsimulators.Stockindices

It is clear I must run two Java programs. Unfortunately, my herculean efforts to run them from the terminal end as assorted failures, from Java classpath issues to brutal Mac OS X crashes (see Figure 1).

Instead, I launch Eclipse, create a Java project, and add the Java source from the TradeSpace demo package (in my case, ~/Sites/nirvana/demo/src/java/tradespace). I also add three entries to the build path:

External Class Folder – /Applications/Nirvana_5.1.6710/classes
External JAR – {NIRVANA_HOME}/lib/nClient.jar
External JAR – {NIRVANA_HOME}/lib/swing-layout-1.0.jar

Now I create two launch configurations:

Main Class: com.pcbsys.demos.feedsimulators.NewsFeeder

VM Arguments:
-DRNAME=nhps://showcase.my-channels.com:443 -DRNAME2=nsp://localhost:9000 -DCHANNAME=/showcase/consolidatedrss

Main Class: com.pcbsys.demos.feedsimulators.Stockindices

VM Arguments:
-DRNAME=nsp://localhost:9000 -DRNAME2=nsp://localhost:9000 -DCHANNAME=/showcase/stockindices

I launch both configurations in debug mode. Inspecting the Nirvana Enterprise Manager, I can see activity in the Event History chart and the Event Status table. I navigate to http://localhost:8887/ in a browser once again and click on the Flex link under the Web Client Demos section. The Flex client starts up and loads the news and rates data (see Figure 2). Voilà.

Figure 1: Mac OS X Epic Fail

Figure 2: TradeSpace Web Client Demo in Flex

Achieving Nirvana on Mac OS X

5 Responses »

Apr 062010

"Giant Buddha" on Lantau Island, Hong Kong

Here at Lab49 we develop right on technology’s bleeding edge. Innovation requires investigation, and so in its wisdom the Lab encourages a culture of research. Our own Matt Davey championed an internal venture, now involving a small team of 49ers, to demonstrate a new risk management architecture. While the project at its core focuses on back-end technologies such as data stores, for this article I will discuss mostly middleware.

Early on (and prior to my involvement) the team planned to create a straightforward Microsoft Silverlight front-end that connected to Nirvana middleware. We have since added a twist: the team will build two front-ends in parallel, one in Silverlight and one in Adobe Flex. Given the political intesity surrounding RIA technologies, I have no doubt this will be an interesting experiment. With luck it will also be informative beyond the usual Flex versus Silverlight religiosity, as we will be able to judge how easily each platform integrates with a proven middleware product in Nirvana. Of course, to begin I must install Nirvana Enterprise on my trusty MacBook Pro.

The shop behind Nirvana, my-Channels, provides a free developer edition. Generally I work on a Mac, so I grabbed the Mac OS X package. They also provide official installation instructions, but unfortunately the curtains do not match the carpet, as it were. There are instructions for Win32, Linux/Solaris and Generic Unix, but there is not a word about OS X. A reasonable man might assume Generic Unix is the appropriate replacement, but he would be wrong. The good news is that when I download and unzip nirvana_osx_novm.zip, out pops a proper OS X application install.app. I launch the installer and go with the defaults most of the way. I do choose a custom data directory, ~/Documents/nirvana_data. Installation proceeds without trouble, using the default /Applications/Nirvana_5.1.6710, henceforth called {NIRVANA_HOME}, for the Nirvana application folder.

Moving on, I download the TradeSpace demo package tradespace_demos.zip, which is free on the my-Channels downloads page. The demo instructions recommend unzipping to the Nirvana application folder, but instead I choose ~/Sites/nirvana/demo. Following the Environment Configuration section of the demo instructions, I update ~/Sites/nirvana/demo/cfg/demo.xml.

Update value of property BasePath to ~/Sites/nirvana/demo/www.
Update value of property port to 8887. This number is somewhat arbitrary, though I recommend avoiding the default 80.

Now I am able to launch the Nirvana server. In the Finder, I navigate to {NIRVANA_HOME}/server/nirvana and double-click the Start Nirvana Realm Server alias. A Terminal window opens and launches the Nirvana server, which in turn prints some server and environment details to the console. The server lives. The next section of the instructions, Interface, Plugin and Channel Setup, suggests I open an item from my Windows Start menu and then execute a pair of .BAT scripts. This is clearly impossible on OS X, and the instructions offer no alternative. I am on my own.

I open the first script, setup.bat, in a text editor and see a single command, nimportrealm ..\cfg\demo.xml -all. This implies the next step is to import the demo realm, and fortunately I can do this by GUI. I navigate to {NIRVANA_HOME}/Administration/nirvana/ and double-click the Nirvana Enterprise Manager. I find my default realm nirvana under the Realms node and right-click to bring up the context menu, selecting Import Realm from XML. I find the demo.xml file from earlier and click Import All. After I click OK, a directory showcase with four channels appears beneath my realm nirvana. To confirm the server is operational, I head to http://localhost:8887/ in a browser, and sure enough, a welcome page congratulates me.

For my encore, I will take care of the second .BAT script, startall.bat, and at last connect a Flex client to Nirvana.

[Update 13-April-2010] Continue with the follow up article, Achieving Nirvana with Flex.