Making store caching a little smarter

Table of contents

1. Set the scene
2. Application state
3. Getter "caching"
4. Getting values with specificity
5. Sparks of inspiration
6. RecordManager
7. Designing a developer experience
8. Initialization of a RecordManager
9. The inner workings
10. Real world performance
11. Closing thoughts

Vue’s official store libraries have consistently been touted for their intuitive developer experience and deep considerations for runtime performance.

I’ve seen firsthand the level of complexity a front-end application can snowball into. A huge set of data might be housed across multiple store modules and getters in each module might interact with each other or depend on data across modules in ways that quickly result in redundant calculations.

It’s not uncommon for a single action to trigger a cascade of reactive updates across a large subset of the application state. This in-turn can easily result in exponential growth in runtime resource consumption.

We need our front-end to be snappy, so let’s dive into how values are cached in the existing solutions and why I built a RecordManager utility to make it a little smarter.

NOTE: To demonstrate the concepts here, I’ve built and integrated a demo application which you can interact with and included real-time statistics as part of this article. The first few sections are dedicated to explaining the problem and what I set out to solve.

🧑‍🎨 Set the scene

Consider a hypothetical e-commerce electronics dealer where a user has purchased a number of items and those items have been optimally distributed across multiple shipments. This site supports viewing prices in multiple currencies, let’s keep things simple and say USD and CAD.

A very basic type definition for a Product in our system might look like this:

export interface Product {
  id: string;
  name: string;
  price: {
    USD: number;
    CAD: number;
  };
}

We have records of all orders and shipments that our customer has made and have already provided that data to the front-end.

Let’s give our customer a way to view their order history and see the shipping status of each order!

Our hypothetical e-commerce service is optimized by mixing and matching order items between shipments. This means shipments shouldn’t be tied to a single order, and an order can have a set of items placed across multiple shipments.

We’ll need some type definitions for the order history and the shipping status of each order. We can house a list of order item references in each order, and the shipment reference with each order item.

export interface OrderItem {
  id: string;
  orderID: string;
  productID: string;
  quantity: number;
  discount?: {
    [Currency in CurrencyCode]: number;
  };
  shipmentID: string;
}

export interface Order {
  id: string;
  createdAt: Date;
  // IDs stored by reference
  orderItems: string[];
}

export interface Shipment {
  id: string;
  createdAt: Date;
  // IDs stored by reference
  orderItems: string[];
  // The date the shipment was delivered
  date?: Date;
  estimatedDeliveryDuration: number; // in days
  status: 'pending' | 'shipped' | 'delivered';
}

🍍 Application state

Now that we’re familiar with our type definitions, let’s see what our store module for our application state might look like. Pinia’s official documentation recommends defining a state object and returning that as part of the setup function. Each entity is organized by type in a Record<string, T> and we can easily store and access any of our entities by their id values.

import { defineStore } from 'pinia';

interface State {
  orders: Record<string, Order>;
  orderItems: Record<string, OrderItem>;
  products: Record<string, Product>;
  shipments: Record<string, Shipment>;
  selectedCurrency: CurrencyCode;
}

export const useOrderHistoryStore = defineStore('orderHistory', () => {
  const state = reactive<State>({
    orders: {},
    orderItems: {},
    products: {},
    shipments: {},
    selectedCurrency: CurrencyCode.USD,
  });

  const actions = {
    // Fetch actions, mutation helpers, etc...
  };

  return {
    state,
    actions,
    // TODO: Add getters
  };
});

Let’s see what our front-end components look like to know what getters we’ll define for our store module to collect the data we want to display.

There are five functions in our order history example that we’ll need to define:

• totalNumberOfOrders Total number of orders
• orderItem/getPrice The final price of an order after discounts
• order/getTotalPrice Sum of all item prices in an order
• order/getShipments Aggregate list of shipments on an order
• shipment/getEstDeliveryDate Estimated shipment delivery date

🧳 How does getter "caching" even work?

Let’s take a look at how some of these might be defined to understand the store caching mechanics.

In Vuex, Vue’s legacy store library, the only way to define getters was as an object of function definitions. But with the introduction of Pinia, getters can be defined the same way, and also can be defined as computed values.

In Vue, computed values are reactive values which by way of magic, listen to any reactive dependencies that are present in the function you provide. Only when any of those dependencies are updated, the computed value is re-evaluated. If you’re a React developer, this is similar to the useMemo hook but dependencies are instead automatically inferred.

Under Pinia's hood, the computed values are treated identically to the functional getters of Vuex. We’ll use these examples to highlight how getter values are cached.

For now, let’s focus on the first getter totalNumberOfOrders.

  const getters = {
    ...
    totalNumberOfOrders: computed(() => Object.keys(state.orders).length),
    ...
  }

Here, state.orders is the only reactive dependency for the totalNumberOfOrders computed value. This means, any time the state.orders object is updated, the totalNumberOfOrders getter function will be re-evaluated.

In the footer of the demonstration window, a "Remount" button simulates a component remount. You could imagine in the context of a real-world SPA, the user is navigating between pages and components, and the getter is being accessed again.

I also set up a monitoring utility to compare the number of times the JS event loop explicitly enters our getter functions in real-time. The graph below displays the number of times totalNumberOfOrders is re-evaluated.

Notably, no matter how many times we remount the order history list, the getter function defined as a computed instance only runs once!

So effectively, this value is "cached" in the sense that the getter function does not need to re-run when the data is requested again because the dependencies haven’t changed.

🫥 Getting values with specificity

You might notice that this getter doesn’t depend on any explicit values within any of the records of objects.

In our application, the total price of the order is not stored as a single value on the order object. So the getter order/getTotalPrice is a good example of when an argument would need to be passed to the getter function.

From the component context, we know the ID of the order we want to get the price of. But a computed function doesn’t accept arguments. Vue’s official documentation for both Vuex and Pinia recommends setting up a function instead of a computed value to handle this.

  const getters = {
    ...
    'order/getTotalPrice': (orderID: string) => {
      const order = state.orders[orderID];
      /** 
       * ...
       * Sum prices of all items in the order and
       * use `orderItem/getPrice` to get the price of each item
       * ...
       */
      return {
        price,
        discountedPrice,
      };
    },
    ...
  }

// Within the setup of OrderCard.vue
const { orderID } = defineProps<{ orderID: string }>();
const price = computed(() =>
  store.getters['order/getTotalPrice'](orderID),
);

What does this mean at runtime? Here’s another bar chart to visualize the number of times both the totalNumberOfOrders and order/getTotalPrice getters are re-evaluated.

Upon the first access, the order/getTotalPrice getter is evaluated three times total, once for each component that accesses it. That’s not so bad, but triggering a remount of the component will cause the getter to run again. Go ahead and try it out!

In reality, this getter isn’t providing a value back! It’s sending back a function that the component calls with its respective orderID to get the total price of that particular order. So, every time the component is remounted, the store hands back a new function instance rather than re-using the value calculated previously.

With order/getTotalPrice, this is being called three times on every remount. But we set this getter up to depend on another getter: orderItem/getPrice.

  const getters = {
    ...
    'order/getTotalPrice': (orderID: string) => {
      const order = state.orders[orderID];
      return order.orderItems.reduce((acc, item) => {
        return acc + store.getters['orderItem/getPrice'](item.id), 0);
      }, 0);
    },
    ...
  }

Which means, every time order/getTotalPrice is called, it will call orderItem/getPrice once for each item in the order, uniquely calculating the price of each item. On top of that, our OrderCard houses two subcomponents: OrderItemCard and ShipmentCard, each of which also call orderItem/getPrice!

On EACH remount, orderItem/getPrice is being called 25 times uniquely! You can see how this can quickly get out of hand...

We’re talking about a simple example, and you might be thinking that it’s no big deal because it’s just 25 function calls and computers are fast? But these considerations need to be made when your application requires interactions with a large number of records, maybe changing often, and you might be given limited processing resources (i.e. mobile), all while providing a snappy user experience. You’d be surprised how many people are still rocking iPhone 6’s.

⚡ Sparks of inspiration

Let’s take a second to think about what we know so far:

• A computed getter is only re-evaluated when its dependencies change
• A function getter is re-evaluated every time it is accessed, even when separate invocations input the same argument.
• It’s very common that argument is an ID value for some record in our application state.

Thinking about this inspired me to think of a way to put it all together. What if we procedurally generate a set of computed values for each of our values in our records? This would be possible if every time we add some value, we also initialize a subscription for each getter and treat the ID of the record as a constant value! All of it’s dependencies would be pinned to that computed value so the re-evaluation would always be appropriately invoked.

📚 RecordManager

From here, I had a conceptual understanding of how to solve this and a clear strategy for what this utility should look like. I named it RecordManager since it will manage records in our application state and manage the subscriptions for each of our value-unique getter functions!

My goal was to make this as simple to use as possible and cleverly provide a way to interact with dynamic computed values for each record. All in the name of DX.

🫴 Designing a developer experience

As we earlier saw, what do our record specific getters look like right now?

const getters = {
  ...
  'order/getTotalPrice': (orderID: string) => {
    /** ... something something ... */
    return total;
  },
  ...
};

I wanted to make working with this record manager utility with a few goals in mind for improving developer experience.

• When accessing one of the computed values, I don’t want to even have to think about whether it’s a function! Interacting with these values should be almost as simple as accessing an actual property.
• The set of computed values' types should be inferred as a result of what getters I define! That way I don’t have to worry about manually and separately maintaining the types for the getter functions.
• I want to be able to interact with the record manager in a mutable way to allow setting and mutating records from any context.

🐣 Initialization of a RecordManager

The first two goals were the most restrictive on designing the initialization experience for the record manager.

For example, here’s how we might define a RecordManager for our orders record.

const useOrderHistoryStore = defineStore('orderHistory', () => {
  /** recordManager definition */
  const orders = recordManager<Order>({
    /** ... getters ... */
  });

  /** ... */

  return {
    ...
    orders,
    ...
  };
});

Interacting with the RecordManager should feel like interacting with a normal reactive object.

const orderHistoryStore = useOrderHistoryStore();

// Typescript correctly infers the type of `price` to `number`
const price = orderHistoryStore.orders.get('orderID').computed.totalPrice;

To achieve this, the RecordManager needs to know the type of the getters provided.

I’m adamant that the initialization of the RecordManager should avoid explicitly defining the type expectations of the getter functions. Though unfortunately, TypeScript can’t infer the types of an object when another type argument has been provided.

So to hack around this, the recordManager initialization function returns a proxy function which then can infer the type of the object of getter definitions. Meaning, you’d need to call the resulting function itself.

/** RM initialization */
const orders = recordManager<Order>()({
  /** ... getters ... */
  totalPrice: (orderID: string) => { /** ... */ },
});

/** Under the hood looks something like this */
export const recordManager = <T extends { id: string }>() => {
  return <C extends object>(computedGetters: C) => {
    /** ... */
  };
};

It’s a bit of a strange interaction but acceptable for the benefit of spinning up new getters without noise around type definitions, an interaction which tends to slow down my development process.

🫀 The inner workings

Now that I knew what both RecordManager's initialization and interaction will look like, the next step was to built out a system to dynamically create and manage records and the subscriptions for each of the getter functions.

Every time a record is set on the RecordManager utility, we need to make sure to create a subscription for each of the getter functions. Instead of an actual computed value, we can use a watch under the hood to re-evaluate the getter function and update the reactive value. It might look something like this:

/** Inside `recordManager` */

/** Maintain an object of the records joined with their computed values */
const recordWithComputed = reactive<Record<string, WithComputed<T, C>>>({});

/** Maintain a map of the watchers for each record and watch instances */
const watches = new Map<string, Partial<Record<keyof C, typeof watch>>>();

const watchComputedGetters = (id: string) => {
  /** Create a new watch for each computed getter */
  Object.entries(computedGetters).map(([k, fn]) => {
    /** Grab the computed getter function */
    const getter = computedGetters[k];
    
    /** Create a new watcher against the computed getter with the ID provided */
    watch(() => getter(id), (newVal) => {
      const oldVal = recordWithComputed[id]?.computed?.[k];

      /** If the new value is the same as the old value, skip updating */
      if (isEqual(oldVal, newVal)) return;

      /** Update the computed value for the record */
      recordWithComputeds[id].computed[k] = newVal;
    }, { deep: true, immediate: true });
  });

In one reactive object recordWithComputed we maintain a list of all the records joined with their computed values. The function watchComputedGetters is responsible for creating a new watch instance for each of the computed getter functions for a particular record.

The watch instance would need to be invoked when a new value is set in the record manager instance.

  return {
    get: (id: string) => recordWithComputed[id],
    /** Set a new record on the RecordManager */
    set: (record: T) => {
      const { id } = record;
      recordWithComputed[id] = record;
      watchComputedGetters(id);
    },
    /** Other helper functions */
  };
};

This is high level pseudocode to show how the RecordManager was implemented, and I’ll add a link to the full implementation in the footer of this article.

In the final implementation, I added a few more bells and whistles like local storage persistence (which should be reserved for data of limited size) and a other interface utilities like getChildOf or setChildOf which is for interfacing with child values of some particular record.

📉 Real world performance

Let’s revisit our demo application and see how the performance of the getter functions is affected by the introduction of the RecordManager utility.

To see the getter invocation count for this optimization strategy, make sure to enable the "Optimized store" switch at the bottom of the window.

The following graph shows the number of times each getter is evaluated for both the original and optimized store implementations.

As you can see, the most dramatic improvement of this optimization is in a getter like orderItem/getPrice which dropped from 25 evaluations on every remount, to only 5 (one for each unique order item).

This is a significant improvement that an application of cross-dependent values at scale can easily take advantage of.

Another key benefit is in the remounting of the component itself. With our strategy, since the getters in a RecordManager listen to the changes of their related record and dependencies, this means the getters will not be re-evaluated if the record itself or any other dependencies are not updated. This is a significant improvement for getters with many external dependencies or relatively expensive functional logic.

It is important though to be mindful of when those records are no longer needed and to properly clean up the watchers to avoid another form of unnecessary computation.

🧑‍🎨 Closing thoughts

I built this utility with the intent to optimize the performance of a real-world application. In a previous blog post I illustrated a high level overview of the project this was used for, as a result of real-time data impacting the application state in largely spanning ways. This was quite successful and brought the number of evaluations for the entire context down to a much more reasonable number, significantly reducing the peak memory usage of the application at runtime.

Upon further evaluation, I’ll proably further optimize getters at runtime to only initialize the watcher for getters when the callee explicitly accesses the getter for a reactive purpose. I’d ensure that the watch instance is properly cleaned up when there are no longer any callee’s with reactive dependencies. This would at least make sure that all getters aren’t necessarily being watched if they aren’t needed.

The developer experience benefits are great, though. One additional feature I built into my own implementation of this system was an optional database synchronization system which updated the database with all modifications to the records immediately so I could rely on a consistent and type-safe interface for saving changes to the database. Building on top of the RecordManager in ways like this was a breeze and I had the ability to interact with store modules in a way that felt much more natural.

Thank you for reading!

- Pumposh