Let's take an example from the previous route handler which has already been refactored.

import fastify from 'fastify';

// for testing
export async function start(options: { port: number }) {
  const server = fastify({ logger: true });

  // basic health response can be anything
  server.get('/health', async () => 'OK');
  const { port } = options;

  await server.listen(port);

  return server;
}

// so we can node ./build/index.js to start the server
(async () => start({ port: Number(process.env.PORT) }))();

The minimum required to pass the test is

(async () => { 
  const server = fastify();

  await server.get('/health', async () => 'OK')
  await server.listen(4000);

  return server;
 })();

But the issue with this version of the code is this error:

 FAIL  src/index.test.ts (7.241 s)
  Given start
    ✕ Server is listening (5002 ms)

  ● Given start › Server is listening

    connect ECONNREFUSED 127.0.0.1:4000



  ● Given start › Server is listening

    thrown: "Exceeded timeout of 5000 ms for a test.
    Use jest.setTimeout(newTimeout) to increase the timeout value, if this is a long-running test."

By using a fixed port number the server under test has trouble being started and stopped.

The solution to this would be to randomise the port number. The issue is the test runner would need to know where the server is listening. We solve this by using Inversion of Control and giving the server under test the port to listen on. 👂🏽

export async function start(options: { port: number }) {
  const server = fastify();
  const { port } = options;

  await server.get('/health', async () => 'OK')
  await server.listen(port);


  return server;
}

It's a good habit to refactor tests and the code under tests separately. By writing our tests first, we can be more confident when we make changes.

Martin Fowler goes into a lot more detail in his book on Refactoring. 🔧

The 2020 Summer Olympic Games were held in Tokyo between 23 July and 8 August. The enduring motto of the Olympics is “Faster, Higher, Stronger”.

From 1908 to 2003 the British cycling team won only 1 gold medal at the Olympics. This changed when Sir Dave Brailsford became the head of British Cycling in 2002. During the ten-year span from 2007 to 2017, British cyclists won 178 world championships and 66 Olympic or Paralympic gold medals and captured 5 Tour de France victories in what is widely regarded as the most successful run in cycling history.

Sir Dave Brailsford credits this remarkable achievement to 3 pillars: 🏛

1. Strategy - what it takes to win a specific event by analysing the best performers.
2. Human performance - how to get the best out of each athlete.
3. Continuous Improvement in all areas where cumulative improvements are as small as 1% aka. "marginal gains" can make a big difference in total.

Tech debt 💸

One organization defined it as the negative impact of technology on the business, particularly as manifested in rising operational and technology costs, slower time to market, and reduced flexibility

-- Mckinsey - Tech debt: Reclaiming tech equity (October 2020. Accessed June, 2021)

I find the idea of marginal gains a compelling way to understand tech debt. Marginal gains describe a solution for how we can make future successes more achievable by taking small steps today. And addressing tech debt is the small steps we can take to improve our software.

We end up with tech with good intentions because we prioritise tasks directly related to customer needs. Those tasks are easy to prioritise because we don’t want to disappoint our customers or colleagues. By only prioritising those tasks, we neglect to make improvements that make the next task easier. The end result is that doing work in the future takes more time and effort. We break our budgets eating out 🍕 because we don't have time to cook at home. 🍲

Mckinsey in Tech debt: Reclaiming tech equity outline 7 steps to address tech debt:

1. Start with a shared definition of tech debt.
2. Treat tech debt as a business issue, not a technology problem.
3. Create transparency to value the debt position.
4. Formalize the decision-making process.
5. Dedicate resources to tackling tech debt.
6. Avoid a big-bang approach to writing down all debt.
7. Determine which areas are “bankrupt” and explore shifting them to a greenfield stack.

This will sound familiar if you already know about Continuous Improvement a.k.a. Kaizen. And it's interesting to see how a small idea made such a huge impact in competitive sports. It shows how small gains like improved build times, code refactoring, updated documentation and other small improvements add up.

The problem was, that when a request was made by the browser, the server handling the request made a request to another API and that request returned an error. When the browser received the error message it reported the issue as a CORS issue.

CORS only applies to requests made by the browser, if the request is made by a server then it cannot be CORS. If the requests made by cURL, Postman or non-browser client are working then it’s not CORS.

Troubleshooting

• Is the domain the browser is communicating from allowed by the server’s CORS settings?
• Are the API servers able to communicate with each other?
• Are there any API limits that could be causing the error?
• Are there network configurations that should be checked? e.g. Cloudflare, proxies, DNS, etc.

References

• Cross-Origin Resource Sharing - MDN
• How to debug CORS errors

Florida and Minnesota each developed their Statewide Automated Child Welfare Information System (SACWIS). In Florida, system development started in 1990 and was estimated to take 8 years and cost $32 million. As Johnson spoke in 2002, Florida had spent $170 million and the system was estimated to be completed in 2005 at the cost of $230 million. Meanwhile, Minnesota began developing essentially the same system in 1999 and completed it in early 2000 at the cost of $1.1 million. That’s a productivity difference of over 200:1. Johnson credited Minnesota’s success to a standardized infrastructure, minimized requirements, and a team of eight capable people. - (Poppendieck, 2003)

When I started as a web developer I was obsessed with productivity. When I read the Toyota Way by Jefferey Liker it changed my understanding of productivity. Toyota was founded in 1933 by a family-owned, textile manufacturing company that transformed itself into a world-leading automaker. The collection of principles and practices that led to their success is known as the Toyota Production System or Lean and is so effective that they have been copied by its competitors.

Software developers and automakers share a similar problem - when we develop products for our customers we make complex design decisions that balance variation, costs and time.

Mary and Tom Poppendieck translate Lean principles to agile practices in their excellent book - Lean Software Development.

This is a summary of their ideas.

1. Stop waste

Waste is anything that does not add value to a product, where value is perceived by the customer. There are two questions here - who is the customer and what is valuable? This is the most important and the hardest part of lean because an organization or team has to agree on the answer.

For example, the customer of the Infrastructure team is the Development team, the customer for marketing could be the sales team etc.

Hot tip: A frequent, large waste in software development is waiting for things to happen.

1. Amplifying learning

Our customers come to us with unique problems which we often know little about. We have to explore different solutions, measure the impact and learn from them. This way, we iterate to more optimal solutions. But does it scale? Scaling is another set of challenges and the same mindset should apply - plan, do, learn, repeat.

Hot tip: Write code in a way that is easy to change. For example, abstractions, DRY principles, feature flags, configurable behaviour etc.

1. Decide as late as possible

Plans are worthless but planning is everything

-- D. Eisenhower

This is the hardest lesson for me - I'm constantly reading crystal balls, they're so shiny! 🔮👀

When project managers, sales teams and clients want costs and time estimates to fill up their timesheets, it's hard to say, "It will be OK, I haven't made up my mind yet about that". 😬

The idea here is that change is inevitable so delaying commitments allows us to make decisions when we have the most information. For example, how much RAM will this API need? Any estimate will change based on actual usage metrics.

Hot tip: The best ideas often occur towards the end of a project because that's when we know the most.

1. Deliver as fast as possible

As an industry, this is where most of our productivity dollars are spent. Every new framework has to have a benchmark with it at the top to have clout. 📈 But we should consider softer issues like faster decision making, access to information etc. Toyota is famously slow to adopt new technologies because they want to avoid optimising some production steps at the cost of the whole system. Don't take risks to speed ahead, just to get stuck at a traffic light. 🚦

Hot tip: Speed is measured by the time it takes for the customer to receive the value.

1. Empower the team

There is an amusing story about a pig and chicken wanting to open a Bed and Breakfast restaurant. They argued over the menu design when the chicken suggested "Bacon and eggs". Who should decide? The pig should decide since the decision would impact them more.

Hot tip: Treat work colleagues like volunteers. Trust and respect are vital.

1. Build integrity in

As software developers, we have to make trade-offs between competing goals. Business realities, competing standards, and deadlines leave their mark on the product. These compromises can leak into the customer experience in the form of performance issues (slow network requests, API quotas, large file sizes etc.), uninformed designs (missing localisation, clunky legally required information etc.) and other questionable choices.

It would be much better if we can build products with a single vision. Domain-driven design can help us get on the same page as our customers.

Hot tip: Get close to the customer and focus on what matters to them.

1. See the whole

This is why I became a full-stack developer. My personal preference is to be able to understand problems from start to finish and use that information when writing software. Our industry has chosen particular knowledge boundaries like "Front-end", "Backend", "Site Reliability" and others.

Optimisations in one step of a product can make it harder to be productive in others. For example, front-end frameworks can hurt user experience through large file sizes and heavy workloads. Serverless solutions have to contend with cold starts. Database choice can limit the type of application features like region distribution, data governance etc.

Hot tip: Be careful how we measure success and set rewards. We win when we all win.

That's it - only 7 things 😂. Please let me know what you think.

And some parting words that keep me humble -

If today’s problems come from yesterday’s solutions, then tomorrow’s problems will come from today’s solutions. Avoid creating a pendulum that swings from high ceremony to low ceremony and back; look for the balance point of the lean principles.

I want to begin with the end in mind. When our API is deployed in a cloud environment, the automated, cloud deployment will need a way to know that our API has started correctly. That is the purpose of the GET /health route.

I have created a workspace on Codesandbox.io with the completed code. And I'll walk you through the code.

If you want to skip ahead, you can take a look at the "Getting Started" section in the README.md.

Acceptance criteria ✅

Well written tests are like documentation that is always in sync with operational code. So I like to use a style called Acceptance criteria from Behaviour-Driven Design (BDD) when describing my tests. This takes the format of -

• Given: the initial context at the beginning of the scenario, in one or more clauses;
• When: the event that triggers the scenario;
• Then: the expected outcome, in one or more clauses.

Red 🔴

The entry point of our application is index.ts. A typical NodeJs server is started with a command like node ./build/index.js. I follow the pattern of placing test files next to the code they test. This makes them easier to find and shows which files are missing tests.

index.test.ts contains -

// helper method to make a HTTP request
const asyncGet = (
  options: http.RequestOptions
): Promise<http.IncomingMessage> =>
  new Promise((resolve, reject) => {
    http.get(options, (res) => resolve(res));
  });

describe(`Given start`, () => {
  let server: FastifyInstance;
  let port: number;

  // SETUP
  beforeAll(async () => {
    // generate a random port number for the server
    port = await getPort({ port: getPort.makeRange(3000, 3100) });
    // configure the server from outside using params
    // this makes it much easier to test
    server = await start({ port });
  });

  // CLEAN-UP
  afterAll(async () => {
    await server.close();
  });

  // ACT
  test(`Server is listening`, async () => {
    // make a request to the server port
    // does it work?
    const res = await asyncGet({
      hostname: 'localhost',
      port,
      path: '/health',
    });

    // check the response code
    expect(res.statusCode).toEqual(200);
  });
});

Green 🟢

The minimum viable code to get the tests to pass is a simple route to respond to GET '/health'

import fastify from 'fastify';

// for testing
export async function start(options: { port: number }) {
  const server = fastify({ logger: true });

  // basic health response can be anything
  server.get('/health', async () => 'OK');
  const { port } = options;

  await server.listen(port);

  return server;
}

// so we can node ./build/index.js to start the server
(async () => start({ port: Number(process.env.PORT) }))();

Refactor

There isn't anything to do here since the code is so simple. But in the next post, 📮 we'll add more routes that will require some refactoring.

I have opened a Github pull request that shows all the code changes which I'll explain below.

Formatting 💅🏾

I prefer to use Visual Studio Code as my IDE and I recommend installing the Prettier extension to format code. This keeps our code tidy and easier to read.

• prettierignore
• prettierrc

Typescript 🦾

Typescript supercharges JavaScript with type checking. Type checks limit bugs by looking over our shoulders for mistakes like strings instead of numbers. More advanced uses of types can even help document our code. If you've ever wanted to phone a friend for some help while coding in JavaScript then give Typescript a try.

• tsconfig.json

Jest 🤡

Jesting is another way of saying joking. And another word for joking is mocking. I guess that's why this testing and mocking library is called Jesting.

No place like 127.0.0.1 🏡

Jest doesn't yet work with Typescript so we'll need a plugin to help translate our Typescript code for Jest. I've used the ts-jest plugin for this.

• jest.config.js

To get started I need to install the packages and run the tests. So in a terminal window at the root of the project, I have run -

• npm install
• npm test

Jest reports "No tests found, exiting with code 1". Don't panic. In Test-Driven Development we have to get used to our tests not working at first. When we add the correct code and the tests pass, we prove that the code works. Don't trust tests that can't fail.

Next ➡️

That's all the setup steps. The next step is a code walkthrough.

If I've missed anything that you would like explained, please let me know in the comments

Introduction

Fastify, first of its name, is a low overhead (aka. fast) server-side JavaScript framework. Express is currently the most popular NodeJs framework and Fastify is compatible with Express. What makes Fastify a beauty 🌸 is that it's faster, has more features and makes testing our APIs so easy.

What we are going to build together is a simple Calculator API. When we POST an array of numbers e.g. [2, 4] our API will add them together and return 6. It's simple so we keep our attention on testing.

Test-Driven 🚘

Test-Driven Development is a programming practice where:

• Tests are written before operational code
• Operational Code is updated to make the tests pass
• Code is refactored to improve its design

This is commonly known as Red, Green and Refactor. It reminds me of traffic lights 🚥.

Plan

The plan is that I will create a blank repo and open PR's so you can see how we can evolve an API using tests.

If you would like to skip ahead, take a look at this Github repository.

See you soon.

✌️

Let’s consider a recipe to make a tasty salsa dip for our Doritos. We can make a tasty, tomato salsa sauce with a recipe that includes tomatoes, red onion, garlic, white wine vinegar, lime juice and coriander. We peel and chop the tomatoes, peel and dice the onions, peel and crush the garlic and slice the coriander. We add and stir all of these ingredients in a bowl with vinegar and lime juice. Now we have a salsa dip for our Doritos - yum! Notice how it’s not possible to peel the white wine vinegar? Or chop lime juice? Those operations are not allowed on those ingredients.

Computers process data to create results like in a recipe. We call these recipes algorithms and prefer them to take less time and memory. Like in our recipe, there is some preparation required before an algorithm can work. We prepare the data points by grouping them together in a meaningful way. Data points relate to each other like which one comes first or last. Or in what shapes like a list or a tree. We can also add, remove or move around data points in a group.

One more thing, so that the computer can keep track of all of the data, the data is saved as Binary Objects With a fixed Length (B.O.W.L.). Let’s picture this to let it sink in – Doritos dipped in a bowl of tasty, tomato salsa: algorithms and data structures have never tasted so good.

Putting this together – we create data structures by grouping data points into a structure (data structure) that defines the relationships and operations between each of them. Computers use algorithms to operate on data structures and this takes time and space (memory). We describe the time and space complexity of an algorithm using the Big O notation.

We conceive of memory and time, in a Big O state of mind.

Does the CPU have the juice? The clue is in the queue if you don’t want to lose out to complexity in a while do { }. Be in a Big O state of mind.

What more could you ask for? When I, operate in constant time? If you complain about how big the data is, this algo loves it though. It goes like a pro, "Go go gadget!", to the beat of the algo rhythm.

When I'm recursing I bless the call stacks and tail call optimise them. Sprinkle some holy water, cross the compiler, O(log)N is my bridge over disordered waters. Double the input and it only needs one more step over.

Linear takes one pass, but X squared is to be feared. Cubic? Exponential? Deadlock is what they deserve.

Yeah, yeah I'm checking the space-time.

And I'm gonna move the perf line.

Yea, yeah, I’m in a Big O state of mind.

Inspired by Song lyrics generator

As I learn Elixir, what I'm coming across in the literature doesn't fill me with the same excitement. I've read about Elixir structs, maps, lists and pattern matching too many times. I want to get to a working knowledge of the language and at this point, I don't really care if strings are UTF-8, how large a float is or the Erlang representation behind them.

When I'm on holiday in Spain, what I care about is how to say "Good morning, table for two please" not "Spanish is a Romance language that originated in the Iberian Peninsula and today has over 450 million native speakers in Spain and the Americas. It is a global language and the world's second-most spoken native language, after Mandarin Chinese".

So this is an ode to Atomic scala.

Methods aka. Function aka. Do something

A method groups programming logic that can be executed by name or anonymously. Simple functions are used to build more complex logic structures. You don't need a return statement.

Terms

• Arity - the number of parameters to a method e.g List.first/1 means one parameter required for the function named first in the List module.
• Typedef - i.e. @spec - explicit type information used to find errors. Learn more
• Pattern matching - the method signature is determined by the type of arguments
• Guard - a conditional that is checked before executing a function. Learn more
• Default value - i.e. argument \\ value

Explanation

Named functions exist in modules and have the structure:

defmodule Module_name do
 # note the comma before 'do'
  def function_name (arg1), do
  ...
 end

 @spec function_name(type()) :: return_type()
 def function_name (arg1, arg2) when arg1 > 0 do
  ...
 end
end

Private functions start with defp instead of def.

Anonymous functions have two structures with fn and &. E.g.

function_name = fn (arg1, arg2) -> ... end
capture_function = &(&1 ... &2)

A header function is a function with the same name as a named function and is used to set the default value for the named function's argument(s). E.g.

defmodule Module_name do
  def function_name(arg1 \\ default_value)
 def function_name(arg1), do
    ...
  end
end

Examples

@spec multiply(number()) :: number()
defmodule Calculator do
  def multiply(arg1, arg2) when arg1 > 0 do
    arg1 * arg2
  end
end

Calculator.multiply(0, 2) # -> (FunctionClauseError) no function clause matching in Calculator.named_multiply/2
Calculator.multiply(2, 3) # -> 6

anon_multiply = fn (arg1, arg2) -> arg1 * arg2 end
capture_multiply = &(&1 * &2)

anon_multiply.(2, 4) # -> 8
capture_multiply.(5, 4) # -> 20

defmodule Greeter do
  def hello(name \\ "world")
  def hello(names) when is_list(names) do
    names
    |> Enum.join(", ")
    |> hello
  end

  def hello(name) when is_binary(name) do
    phrase() <> name
  end

  def phrase, do: "Hello, "
end

Greeter.hello() # -> "Hello, world"
Greeter.hello("Mark") # -> "Hello, Mark"
Greeter.hello(["Mark", "Luke"]) # -> "Hello, Mark, Luke"

suite('getBlogFiles', () => {
    let server;

    beforeEach(() => {
        server = Hapi.Server();
        const preResponse = function(request, h) {
            const { response } = request;
            if (!response.isBoom) {
                return h.continue;
            }

            return console.warn(response);
        };

        server.ext('onPreResponse', preResponse);
    });

    test('get posts route gets markdown files', async () => {
        const files = ['that.md', 'this.md'];
        const getBlogFiles = sinon.stub().resolves(files);
        const stubMethods = [{ name: 'blog.getBlogFiles', method: getBlogFiles }];

        await server.register({
            plugin,
            options: { methods: stubMethods },
        });

        const { method, url } = routes.v1.get_blog_posts();
        // errors disappearing between here...
        const { result } = await server.inject({
            method,
            url,
        });
        // ... and here

        expect(result).to.equal(files);
    });
});

So I got on the Hapi hour slack channel to ask this question and a in a few minutes Eran told me

@iampeterbanjo it's coming soon, but for now just throw server.events.on('request', console.log) to see what's up.

Brilliant! It feels great to be using a framework from such a friendly and helpful community.

Another challenge is when there is already some code in place that was not written with tests in mind. The code is written like a black box, giving out no secrets, taking no prisoners or arguments. That's why tests should be written first.

Sometimes, I convince myself that it's quicker to do manual tests by refreshing the browser or repeating the commands in the terminal. Then I realise that I'm making slower progress, because automated tests are much faster and less error prone than manual tests.

Do I want to spend my time writing log statements and stepping through breakpoints? No, this sucks, but the alternative is that I'll have to set up Jest, Mocha or Cypress.

Setting up tests is a one-off toll to get on the highway.

Eventually, I find my way. Either the path becomes clear and I know what I'm aiming for, or I'm tired of repeating the same actions over and over - code, run, check, code, run, check. And I commit to writing the first test.

Code, check, repeat.

Features

• ReactJS (I know this one)
• Webpack (I usually avoid this when I can)
• Graph QL (Oooo... looks nice)
• Service workers (Yikes)

But the combination of all these together feels like riding a bicycle down hill with faulty brakes. I keep running into problems with no way out.

Problems

• What happened to my static pages? With Gatsby, although it looks like your generated "static" site has regular URLs it is TOTALLY dependent on client-side JavaScript. I'm from the school of Progressive Enhancement and Progressive web apps don't have to fall apart without JavaScript. The good news is that Google seems to be able to read my site fine. But there should be an "eject to HTML" button somewhere.
• Cryptic webpack build errors. Lord deliver me from the evil of Webpack build errors! What does this error output mean?? 🤯

⢀ Building static HTML for pages/korin/profiles/Khalid/My-Bad

error Building static HTML failed for path "/korin/profiles/The-Black-Keys/LOHI"

See our docs page on debugging HTML builds for help https://gatsby.dev/debug-html

   8 |  else
   9 |          root["lib"] = factory(root["@reach/router"], root["core-js/modules/es6.array.iterator"], root["core-js/modules/es6.array.sort"], root["core-js/modules/es6.function.name"], root["core-js/modules/es6.map"], root["core-js/modules/es6.object.assign"], root["core-js/modules/es6.object.to-string"], root["core-js/modules/es6.regexp.constructor"], root["core-js/modules/es6.regexp.replace"], root["core-js/modules/es6.regexp.split"], root["core-js/modules/es6.regexp.to-string"], root["core-js/modules/es6.string.ends-with"], root["core-js/modules/es6.string.fixed"], root["core-js/modules/es6.string.iterator"], root["core-js/modules/web.dom.iterable"], root["crypto"], root["fs"], root["lodash"], root["path"], root["react"], root["react-dom/server"], root["react-helmet"]);
> 10 | })(this, function(__WEBPACK_EXTERNAL_MODULE__reach_router__, __WEBPACK_EXTERNAL_MODULE_core_js_modules_es6_array_iterator__, __WEBPACK_EXTERNAL_MODULE_core_js_modules_es6_array_sort__, __WEBPACK_EXTERNAL_MODULE_core_js_modules_es6_function_name__, __WEBPACK_EXTERNAL_MODULE_core_js_modules_es6_map__, __WEBPACK_EXTERNAL_MODULE_core_js_modules_es6_object_assign__, __WEBPACK_EXTERNAL_MODULE_core_js_modules_es6_object_to_string__, __WEBPACK_EXTERNAL_MODULE_core_js_modules_es6_regexp_constructor__, __WEBPACK_EXTERNAL_MODULE_core_js_modules_es6_regexp_replace__, __WEBPACK_EXTERNAL_MODULE_core_js_modules_es6_regexp_split__, __WEBPACK_EXTERNAL_MODULE_core_js_modules_es6_regexp_to_string__, __WEBPACK_EXTERNAL_MODULE_core_js_modules_es6_string_ends_with__, __WEBPACK_EXTERNAL_MODULE_core_js_modules_es6_string_fixed__, __WEBPACK_EXTERNAL_MODULE_core_js_modules_es6_string_iterator__, __WEBPACK_EXTERNAL_MODULE_core_js_modules_web_dom_iterable__, __WEBPACK_EXTERNAL_MODULE_crypto__, __WEBPACK_EXTERNAL_MODULE_fs__, __WEBPACK_EXTERNAL_MODULE_lodash__, __WEBPACK_EXTERNAL_MODULE_path__, __WEBPACK_EXTERNAL_MODULE_react__, __WEBPACK_EXTERNAL_MODULE_react_dom_server__, __WEBPACK_EXTERNAL_MODULE_react_helmet__) {
     |  ^
  11 | return


  WebpackError: Invariant Violation: Minified React error #130; visit https://reactjs.org/docs/error-decoder.html?invariant=130&args[]=object&args[]= for the full message or   use the non-minified dev environment for full errors and additional helpful warnings.

  - universalModuleDefinition:10 ba
    lib/webpack/universalModuleDefinition:10:2

// truncated for brevity

In this case the error is at the url buried in the terminal.

Element type is invalid: expected a string (for built-in components) or a class/function (for composite components) but got: object.

Usually with create-react-app you get error details nicely laid out in the terminal. So this Webpack format is not a step in the right direction.

• Service worker weirdness. This feature is meant to cache your site so that it works great with less than ideal internet connections - even offline. In practice it means that I have to keep refreshing my site to see what's on it. In development, my end-to-end tests sometimes fail on the first pass. Or second and third pass until I delete browser data or the Cypress binary... and re-install. Below is a screenshot of the same page on first load and then after refresh.

Fig 1: On first load

Fig 2: After refresh

Create react app has disabled their service workers as a default setting for similar reasons.

So what's a developer to do? Keep it simple - Gatbsy was fun to test drive but I see too many maintenance issues ahead for my side project(s).

I've tried using the cloud services of Open Shift, Amazon Web Services (AWS), Microsoft Azure, IBM Watson and GCP and I found that getting the service working with the right options is a major source of confusion. I found AWS to be the hardest - their fine granularity of rules and privileges felt tedious.

Qwik labs include a user login with the right permissions to complete the lab step-by-step. This means that I could realistically commit an hour to learning the basics of Machine learning.

Qwik labs have pay-as-you-go or subscription pricing but I managed to get some free credits for attending the Google Devfest on air 2018. And I'm glad I did.

#1 Ask for help

I recruited my team by posting messages and getting in touch with other coders on the #hackathon channel. I ended up with a team of 6 and had to turn down a few others as we were too close to the start of the hackathon to bring them up to speed. I also had to get in touch with the Koding team to clarify some of the competition rules and they responded quickly.

#2 Adapt

“No plan survives contact with the enemy” – Helmet von Multke.

We had 48 hours to complete the hackathon and after the first day we were still struggling to get data out of Gmail and the agreed mode of working was failing. We tried to keep track of commits and issues on Github but we couldn’t close them fast enough. In addition, two team members were not able to fully participate due to personal and other issues.

#3 Start with the deal breakers

Interestingly I had read this about the Google[X]’s approach to projects last week:

Ask cheerfully, “How are we going to try to kill our project today!”

But I had not yet internalised this lesson.

Our app idea involved connecting with IBM Watson Personality Insights and Gmail but we only prototyped the IBM Watson service and no one in the team had experience with getting data out of Gmail. We assumed that the Google+ login API would give us a token that would work for all Google services as long as we enabled the service in the Google Developer Console. In retrospect this was a critical assumption which should have been verified first and if false, we should have changed our plans before the hackathon started.

#4 Communication overheads slow down team velocity

We had a team of 6, including myself, and communicated mostly on Slack and once on Skype. Our main concern was agreeing on the app idea and not creating code conflicts. As new members joined it took longer to reach a consensus because our team was distributed (London, Switzerland, India and Romania) and it took more time to explain and go through everyone’s contributions. The ideal number might be less than 4. This reminds me of analogy of the tower of Babel in the Mythical Man Month by Brooks.

#5 Use what you have

By Sunday night (GMT) it was clear that we had to move away from Gmail as a data source because we hadn’t figured it out well enough. So with only about 10 hours left I stood at my desk at about 22:30hrs and started coding until 05:30hrs non-stop. I imagined that the entertainment industry would be an interesting subject but it was difficult to find accessible APIs on such short notice. Last.fm’s API worked and so did Lyrics N Music so I combined the two to create artist profiles based on their lyrics.

#6 Be gracious

There was a little voice in my head wanting to give myself all the credit for the final app since I contributed most of the working code in the final submission. Maybe I could split credit according to the Github repo commits. Primarily our team was participating in the hackathon as a learning exercise and we all contributed time, ideas, suggestions and feedback – if not code. And if code is the expression of ideas, where do those ideas come from? As a developer I value working code highly but everything and everyone starts from somewhere. I would also like to give credit to my wife for her encouragement during my emotional roller coaster with doubt, ambition and sleep deprivation.

Check out our app – Song Intelligence

* British past tense of learn although often confused with “learned” by my American cousins.