Featured image of post Automating order fulfilment pipeline for Mayfair based Shopify ecommerce

Automating order fulfilment pipeline for Mayfair based Shopify ecommerce

In brief; it’s not just using zapier and software integration is only the start. The scaling issue is the source of truth for the data and business logic.

For complex businesses and services, dedicated servers can be beneficial in handle data inputs, outputs and performing business logic to scale and differentiate from competitors.

Outcome

Before After
1x orders 3x orders throughput (thousands of orders per week)
Orders going missing All orders and logistics tracked, full pipeline visibility
3am day start, high stress, working weekends 7am day start, zen, weekends off
Spreadsheet required manual checking before start of day Order fulfilment automated

Commercial scenario

Looking back: it’s 2021, the middle of the pandemic in London and there’s a lot of confusion. Small businesses are struggling, some are holding ground, others scramble to adapt and pivot. But at the time new shops are popping up all over to problem solve and fulfil demand that is rocketing for products and services: this was one of them, and it needed to move fast.

The service required multiple 3rd party resources working in a coordinated sequence. To wire processes together the dilemma is whether to use third party software integration tools like zapier gluing services like monday.com to hubspot or just jump in and code a tool from scratch.

Without the ability to manipulate the data going in and out it’s hard to quickly adapt to evolving competition and the product will become stale, fast.

The business pressure

Explosive demand for COVID testing outstripped ability to supply and management sleep deprivation was setting in, orders were being turned down and customers were on the phones ringing constantly as it all cranked into action. Unless processes are automated the businesses can’t scale while reducing management pressures.

Is Zapier the right tool?

Software integration tools like zapier are tempting to use to get going quickly - and they’re cheap to begin with but they ultimately limit the business logic and place a glass ceiling on scale and feature scope. The costs also scale.

If the data format input is roughly the data format output, then zapier is fine. If the business logic and product differentiation doesnt rely on data manipulation and back end calculations then third party is the way to go.

The business

Rapid response COVID testing kits, dropped door to door within London perimeter and a laboratory grade result within hours and fit-to-fly certificate: for business people needing last minute certificates to fly and for those that wanted a bespoke couriered service: based in Mayfair.

It started as a simple Shopify storefront coupled with a bunch of SERP keywords and a buy now button.

Quickly it evolved to an army of drivers dropping off testing kits all over London to hundreds of customers a week couriering kits to multiple labs as the pandemic regulations evolved. The process had to work with a zero failure rate: customer’s flights were on the line if they didn’t receive their certificates on time.

The fulfilment pipeline, before

Before: disconnected systems

Commercial stack

  • Shopify order page
  • Excel spreadsheet (for order details)
  • Customer service team
  • Friends dropping off kits
  • One laboratory

The conversation

Decision required

In the office decisions centered around using monday.com as a cloud based spreadsheet to house the order fulfilment data. It allows the customer service team to all read from the same source of information and can receive data inputs from other sources.

Shopify alone wasn’t suitable for fulfilment and logistics tracking as the pipeline involved multiple pick up and drop off locations and a changing logistics flow. At any one time the business needs to know where a customer’s test is, whether its arrived at the lab, what the results are, and whether the customer has been notified and sent their certificate.

Onfleet was chosen to onboard driver couriers and manage where they are, send notifications of deliveries, drop offs and locations etc.

Monday.com was advertising aggressively at the time and was a natural next step for a dashboard.

Pros and Cons of Monday.com

Pros: quick to start and get going with dynamic data flows. Very good developer documentation.
Cons: costs scale quickly, investment in time and resource required to understand their ecosystem, data integrity and source of truth conflictions, can constrain product.

It’s a spreadsheet in the cloud with add ons. It’s WYSIWYG features enable no code teams to integrate data sources and dynamically update rows and columns. For example, once a driver arrives at a customers house, it can trigger a spreadsheet cell to turn green - to indicate successfully arrived.

The Decision

Monday.com, Onfleet and a VPS were in, and zapier was out. With the quality of API documentation it would be straightforward to set up a VPS server to integrate the software systems. Some code was lying around that was easy to repurpose so ExpressJS was initially used, it’s lightweight and quick to mobilise but a fraction slow on speed. The main criteria was to deploy the updates to the commercial system, fast.

The fulfilment pipeline, after

After: integrated systems

Commercial stack

  • Shopify custom storefront
  • Excel Monday.com
  • Friends in cars Onfleet.com
  • VPS server and database
  • One Multiple laboratories
  • Thousands of orders per week

Zapier not suitable

For this application zapier wasn’t suitable. It was given a trial on the free then paid tier over a 48 hour period: as ever about 6 hours wasted grokking data APIs reading zapier’s proprietary documentation and being forced to get to grips with their annoying parlance and terminology.

Without the ability to manipulate data as it transitions from one service to another or perform simple calculations there wasn’t a meaningful service upgrade or differentiation to the customers. For example: calculating driver delivery times before notifying labs or automating PDF certificate creation depending on a variety of lab criteria.

Added to which the commercial landscape was rapidly changing as well as the updating fit-to-fly certificate regulations were evolving weekly. Reacting fast was the key to staying ahead of the competition.

What is Onfleet.com?

A last mile delivery platform.

Upside: good UI, easy driver onboarding, extensive documentation.
Downside: costs scale quickly.

VPS: Strong decision, paid off

 Invest in a good engine

At the heart of the service, COVID tests needed to be taken to dedicated laboratories for testing. Week on week labs were popping up and others shutting down. Using a VPS as the engine at the heart of the business meant adaptability was baked in, and the ability to scale out the box.

Almost immediately the laboratories set up APIs to allow certificate sellers to integrate with their systems retrieving test results automatically. It was impractical for laboratories to set up their external facing API end points with third party software integrators (in the time required) - they were only able to interface with their clients programmatically (via APIs and servers) or via email and over the phone.

VPS (virtual private server)

A simple server hosted in the cloud. Providers like Amazon AWS, Google Cloud provide VPS services. Back in the 90s they were metal boxes in the corner of the office plugged into the phone line, before cloud systems arrived.

Benefits of integrating software services with a dedicated server

Cloud based servers

  1. Future proofing the product. With good engineers its easy to adapt the data pipelines, and business logic to update the product and stay ahead of the competition.
  2. Access to a huge pool of engineering talent. Developing software from scratch using open source methods leans on a vast talent resource. Using third party tools, you’re restricted to finding gurus who’s focus is just on learning that products documentation.
  3. Keeps costs in control. Code once, and costs scale at the server level on the metal, not at the third party application layer. These scale less steep with fewer surprises.

The future with AI agents

Coming soon

If done again, what would be different.

Coming soon

Need help with software and IT?

Reach out via email: use153east@gmail.com

Keywords: monday.com, shopify, order fulfilment, logistics, software, software consultancy, software engineering, zapier, software integration, software integration tools, event-driven architecture.