[RFC]: Add the .test method to CSF factories

[kasperpeulen]

RFC: Add the .test method to CSF factories

In modern frontend development, Storybook has become the go-to tool for building and documenting UI components. It allows developers to create "stories" that highlight various component states, aiding both development and documentation. Yet, integrating comprehensive testing within Storybook can be challenging. This post examines the current testing limitations in Storybook and proposes a new syntax for seamlessly incorporating tests with a jasmine-style syntax on top of static stories.

Are stories tests?

Storybook provides testing features through hooks like play and beforeEach and leverages Vitest for a fast, browser-based test runner. However, using Storybook for testing means learning a new testing syntax and best practices built on top of the Component Story Format (CSF).

Every story, even those without a play hook, functions as a render test. Rendering a component in a known configuration and verifying that it doesn’t break is a basic yet valuable form of validation.

Even though those stories are run automatically in CI with our Vitest integration, traditional tests usually have clear assertions and specifications.

The Limitations of the play Function

The play function in Storybook allows users to add those assertions on top of a story. However, its limitation of supporting only one play function per story complicates the process of writing multiple tests against the same component state.

A common workaround is to create separate "test stories" with their own play functions. This approach often involves duplicating base stories and adding different play functions to test various behaviors:

export const meta  = {
  component: AddLabelForm,
} satisfies Meta<typeof AddLabelForm>

type Story = StoryObj<typeof meta>;

export default meta;

export const ManyLabels: Story = {
  args: { labels: [{...}, {...}] }}
};

export const SubmittingInvalidLabelShowsInlineError: Story = {
  ...ManyLabels,
  async play() { 
    await userEvent.type(canvas.getByText('Label:'), 'invalid label');
    await userEvent.click(canvas.getByRole('button'));
  
    await expect(...).toHave(...);
  }
})

export const AutocompletionSuggestionsAreShownWhenTyping: Story = {
  ...ManyLabels,
  async play() {
    await userEvent.type(canvas.getByText('Search label'), 'a label');
    await userEvent.click(canvas.getByRole('button'));
  
    await expect(...).toHave(...);
  }
})

export const NoLabels: Story = {
  args: { labels: [] }}
};

export const NoAutocompletionSuggestionsAreShownWhenTyping: Story = {
  ...NoLabels,
  async play() {
    await userEvent.type(canvas.getByText('Search label'), 'a label');
    await userEvent.click(canvas.getByRole('button'));
  
    await expect(...).toHave(...);
  }
})

This best practice is not obvious from an API perspective, but even if you discover this, it's not ideal. These test stories clutter the sidebar, which is problematic if Storybook is used primarily for documentation. And when you click on it, you might not expect the story to take 15 seconds to finish. Additionally, using PascalCase for lengthy spec names reduces readability.

We're essentially treating AddLabelForm and SubmittingInvalidLabelShowsInlineError as both a story and a test at the same time. But are they truly the same? Arguably not—they serve different purposes.

Embrace “Jasmine” test syntax on top of static stories

In the CSF factories proposal, we introduced this new factory syntax for writing stories:#30112

import { config } from '#.storybook/preview';
import { AddLabelForm } from './AddLabelForm';

const meta = config.meta({ component: AddLabelForm });

export const ManyLabels = meta.story({ 
  args: { labels: [{...}, {...}] } 
});

export const NoLabels = meta.story({ 
  args: { labels: [] } 
});

This proposal is about adding a .test method on the story object.

import { config } from '#.storybook/preview';
import { AddLabelForm } from './AddLabelForm';

const meta = config.meta({ component: AddLabelForm });

// the story can be seen as the arrange step in testing
export const ManyLabels = meta.story({ 
  args: { labels: [{...}, {...}] } 
});

// 👇 easy to write (and read!) full sentence specs!
ManyLabels.test('submitting invalid label shows inline error', async ({ canvas }) => {
  // act/assert
  await userEvent.type(canvas.getByText('Label:'), 'invalid label');
  await userEvent.click(canvas.getByRole('button'));
  
  await expect(...).toHave(...);
});

ManyLabels.test('autocompletion suggestions are shown when typing', async ({ canvas }) => {
  await userEvent.type(canvas.getByText('Search label'), 'a label');
  await userEvent.click(canvas.getByRole('button'));
  
  await expect(...).toHave(...);
});

export const NoLabels = meta.story({ 
  args: { labels: [] } 
});

NoLabels.test('no autocompletion suggestions are shown when typing', async ({ canvas }) => {
  await userEvent.type(canvas.getByText('Search label'), 'a label');
  await userEvent.click(canvas.getByRole('button'));
  
  await expect(...).toHave(...);
});

This model aligns with the natural thinking of stories as documented UI states and tests as actions performed on those states. Here, stories represent the "arrange" step of a test, while the test method handles the "act" and "assert" phases, mirroring familiar testing patterns.

The UI

To prevent tests from cluttering the sidebar, they only appear when you open a specific story.

You can also search for specific tests in the search bar.

Play gives you extra control over the “arrange” step

We created the args abstraction because it enables you to quickly define your key UI states, while also allowing you to adjust props in the controls panel interactively.

const meta = config.meta({ component: AddLabelForm });

export const ManyLabels = meta.story({ 
  args: { labels: [{...}, {...}] } 
});

However, sometimes you need more flexibility. What if you must set up mocks before rendering the component? What if you need to open a dropdown after the story has been rendered?

From a testing perspective, you can think of the play hook as providing full control over the arrange step in your tests:

export const LabelFormSettings = meta.story({ 
  async play({ mount, canvas }) {
    // create mock data
    const label = await db.labels.create({
      data: {
        title: 'Some label',
        createdBy: 'storybookjs',
      },
    });
    // pass it to the component
    await mount(<LabelForm id={label.id}/>);
    // open settings dropdown
    await userEvent.click(canvas.getByText('Label settings'));
  },
});

Sharing data

We can share data between the story and the test in a typesafe way:

export const LabelForm = meta.story({ 
  async play({ mount, canvas }) {
    const label = await db.labels.create({
      data: {
        title: 'Some label',
        createdBy: 'storybookjs',
      },
    });
    await mount(<LabelForm id={label.id}/>);
    return { label }
  },
});

AddLabelForm.test('submitting invalid label shows inline error', async ({ label }) => {
  await canvas.getByText(label.title);
});

Overwriting parameters for a test

We can support an API that overrides story props such as parameters and tags:

const meta = config.meta({ component: AddLabelForm });

// the story can be seen as the arrange step in testing
export const ManyLabels = meta.story({ 
  args: { labels: [{...}, {...}] } 
});

ManyLabels.test('submitting invalid label shows inline error', 
  {
    parameters: {
      // Disables Chromatic's snapshotting on a test level
      chromatic: { disableSnapshot: true },
    },
    tags: ['some-tag']
  }, 
  async ({ canvas }) => {
    await userEvent.type(canvas.getByText('Label:'), 'invalid label');
    await userEvent.click(canvas.getByRole('button'));
  
    await expect(...).toHave(...);
});

[DEfusion]

This looks great and is better than what I was imagining when commenting on #22361

I love that you've put tons of thought into this especially things like them not cluttering the sidebar until you open the parent story, just come up with an awesome icon for them 😄

[Sidnioulz]

A derivative of the AAA pattern consists of writing data suites for tests, and then running tests on each item of a data suite.

How could we do that with stories acting as test data? Could I bundle together a bunch of stories (e.g. different variants of a button) and check they all satisfy a behaviour (e.g. all become aria-disabled once clicked and until the click handler returns, or all contain an aria-label and role button)?

I don't know how this should be written so here are a bunch of proposals in no particular order and to which I have no particular attachment, just for the sake of triggering conversation.

// meta providing an each method?
meta
  .each([Primary, Secondary, Tertiary, Frogginary])
  .test('prevent double-submit by setting aria-disabled while the handler is running for variant %s', () => {
  // act
  // assert
})

// With a test global?
test
  .each([Primary, Secondary, Tertiary, Frogginary])
  ('prevent double-submit by setting aria-disabled while the handler is running for variant %s', () => {
  // implicit arrange?
  // act
  // assert
})

// If there is a test global, does it enable other syntax formats, e.g.
test('prevent double-submit by setting aria-disabled while the handler is running', () => {
  arrange(Primary); // or render(Primary), etc. naming here indicates purpose and is not an actual suggestion
  // act
  // assert
})

// Leading to an explicit handling of arrange in test.each?
test
  .each([Primary, Secondary, Tertiary, Frogginary])
  ('prevent double-submit by setting aria-disabled while the handler is running for variant %s', (story) => {
  arrange(story);
  // act
  // assert
})

[shilman]

On a related note we're considering bringing modes to Storybook

https://www.chromatic.com/docs/modes/

[IntusFacultas]

This would be a dream come true, I wholeheartedly approve of this proposal.

The tiniest bit of feedback would be perhaps we can try to visually distinguish the tests in the sidebar when expanded so that they're more obviously "this is a test rather than a traditional story", that would be great since we also use Storybook for non-technical folks to be able to explore the current state of the application development.

Perhaps something like replacing the bookmark with a small "T" or "Test" (space depending) in some kind of badge styling?

[domyen]

Do Storybook features like tags and parameters apply to these tests?

[shilman]

Yes. Under the hood a test would be very similar to a story and I believe we'll make it possible to configure tags/parameters/etc as an optional second argument to the test method.