Good and bad APIs: IndexedDB

I ventured on a quest (aka fight) with IndexedDB: bending its weird API to conform to the power of the effect model 👊

Sandro Maglione

@SandroMaglione

·Follow

Okay, something works here with IndexedDb and @EffectTS_ The IndexedDb layer creates the database, IndexedDbQuery provides a type-safe API based on IndexedDbVersion Keep working on it 🔬

3:18 PM · Apr 6, 2025

IndexedDB is madness

Let me give you a TLDR of why IndexedDB is so bad:

No Promise, all callback (errors included)
Versioning and migrations restricted inside a callback (onupgradeneeded)
Severely untyped

const database = yield* Effect.async(() => {
  const request = window.indexedDB.open(identifier, version);

  request.onblocked = (event) => {
    // ...
  };

  request.onerror = (event) => {
    // ...
  };

  // If `onupgradeneeded` exits successfully, `onsuccess` will then be triggered
  request.onupgradeneeded = (event) => {
    const idbRequest = event.target as IDBRequest<IDBDatabase>;
    const database = idbRequest.result;
    const transaction = idbRequest.transaction;
    const oldVersion = event.oldVersion;

    if (transaction !== null) {
      transaction.onabort = (event) => {
        // ...
      };

      transaction.onerror = (event) => {
        // ...
      };
    }
  };

  request.onsuccess = (event) => {
    const idbRequest = event.target as IDBRequest<IDBDatabase>;
    const database = idbRequest.result;
    // ...
  };
});

Forget about modern TypeScript. You mostly cannot even rely on try/catch. The database on the web of today is all callbacks-hell and jumping between scopes 🤯

DexieJs to the rescue (partially)

The madness of IndexedDB made solutions like Dexie a must-have.

You just create a new Dexie instance, add your keys and schema, and then interact with a sane API (get, add, delete, all Promise-based):

import Dexie, { type EntityTable } from 'dexie';

interface Friend {
  id: number;
  name: string;
  age: number;
}

const db = new Dexie('FriendsDatabase') as Dexie & {
  friends: EntityTable<
    Friend,
    'id'
  >;
};

db.version(1).stores({
  friends: '++id, name, age'
});

export type { Friend };
export { db };

I used Dexie a lot in many projects (DexieJs: Reactive local state in React) 👈

But, I am still not satisfied. The model is still too weak and not strongly-typed enough.

I want a clean IndexedDB effect module: composition, layers, full type-safety, better migrations, and error handling.

That's what I started to implement 🫡

Sandro Maglione

@SandroMaglione

·Follow

I did some experiments with IndexedDB in @EffectTS_ IndexedDB: Weird API, mostly untyped, full of callbacks, errors behind every corner Then you get into migrations, queries, and reactivity and it all breaks down I am (nearly) convinced that is better to bet on SQLite+File

9:50 AM · Apr 3, 2025

Read 2 replies

API design for full type-safety

My goals:

Full type safety (typed errors, schema, migrations, queries)
Schema and data migrations
Composition with services and layers
Completely abstract away the underlying IndexedDB API

As of right now (still work in progress), this is what I have:

IndexedDbTable: Model for a table with Schema
IndexedDbVersion: Single database version (collection of IndexedDbTable)
IndexedDbMigration: Schema/data migrations (sequence of IndexedDbVersion)
IndexedDb: Service to open database and apply migrations
IndexedDbQuery: Service to perform type-safe queries

This requires some tricky magic with TypeScript types 🪄

export const make = <
  Tables extends readonly IndexedDbTable.IndexedDbTable.Any[]
>(
  // Not empty list of generic tables, each unique based on a shared model, all inferred 🙌
  ...tables: Tables & { 0: IndexedDbTable.IndexedDbTable.Any }
): IndexedDbVersion<Tables[number]> => {
  return makeProto({
    tables: HashMap.fromIterable(
      tables.map((table) => [table.tableName, table])
    ),
  });
};

The end result for the user is a minimal API that just works, that also accommodates all use cases without manual typing:

// 1️⃣ Define tables with schema and options
const Table1 = IndexedDbTable.make(
  "table1",
  Schema.Struct({
    id: Schema.UUID,
    value: Schema.Number,
  }),
  { keyPath: "id" }
);

const Table2_1 = IndexedDbTable.make(
  "table2",
  Schema.Struct({
    name: Schema.String,
    age: Schema.Number,
  }),
  { keyPath: "name" }
);

const Table2_2 = IndexedDbTable.make(
  "table2",
  Schema.Struct({
    nameAndAge: Schema.String,
  }),
  { keyPath: "nameAndAge" }
);

// 2️⃣ Define database version (collection of tables)
const CurrentDb = IndexedDbVersion.make(Table1, Table2_2);

// 3️⃣ Define migration (initial database creation and first migration)
const Migration1 = IndexedDbMigration.make({
  fromVersion: IndexedDbVersion.makeEmpty,
  toVersion: CurrentDb,
  execute: (_, toQuery) =>
    Effect.gen(function* () {
      yield* toQuery.createObjectStore("table1");
      yield* toQuery.createObjectStore("table2");
    }),
});

const Migration2 = IndexedDbMigration.make({
  fromVersion: CurrentDb,
  toVersion: CurrentDb,
  execute: (_, toQuery) =>
    Effect.gen(function* () {
      yield* toQuery.deleteObjectStore("table2");
    }),
});

// 4️⃣ Compose layers with database name and migration sequence
const layer = IndexedDbQuery.layer.pipe(
  Layer.provide(IndexedDb.layer("db", Migration1, Migration2))
);

// 5️⃣ Use type-safe API to query database
export const main = Effect.gen(function* () {
  const { makeApi } = yield* IndexedDbQuery.IndexedDbApi;
  const api = makeApi(CurrentDb);
  const key = yield* api.insert("table1", {
    id: "2",
    value: 2,
  });
  yield* Effect.log(key);
}).pipe(Effect.provide(layer));

This is the result of multiple iterations, and questions on "But why?" and "Is this even possible?" 🫠

I aim to make this functional enough to land inside @effect/platform-browser. I plan to open a PR once I am confident this works as it should 🔜

Sandro Maglione

@SandroMaglione

·Follow

Okay, something works here with IndexedDb and @EffectTS_ The IndexedDb layer creates the database, IndexedDbQuery provides a type-safe API based on IndexedDbVersion Keep working on it 🔬

3:18 PM · Apr 6, 2025

IndexedDB is madness

DexieJs to the rescue (partially)

API design for full type-safety

Start here.