I've always wanted to build a UX tester that uses accessibility tech. We get to test both at the same time, and it might make the accessibility tech saner and easier to use also.
SafeTest: A novel approach to front end testing
11–20 of 35 posts
Re: SafeTest: A novel approach to front end testing
#12> Conversely, using integration testing tools like Cypress or Playwright provides control over the page, but sacrifices the ability to instrument the bootstrapping code for the app. These tools operate by remotely controlling a browser to visit a URL and interact with the page.
I don't think the author has used Cypress or Playwright. Their real value is that they do not drive browsers from the outside like slow, flakey WebDriver. They also allow injecting test doubles. And to override app functionality (not having to wait for a "normal" 60 second timeout like the author hypothetically suggests).
This test framework seems to solve non-problems.
Is Netflix not quite so FAANG these days?
Re: SafeTest: A novel approach to front end testing
#13> Conversely, using integration testing tools like Cypress or Playwright provides control over the page, but sacrifices the ability to instrument the bootstrapping code for the app. These tools operate by remotely controlling a browser to visit a URL and interact with the page. I don't think the author has used Cypress or Playwright. Their real value is that they do not drive browsers from the outside like slow, flak…
Author here. Can you show an example of how Playwright would progress a timer on a page? For example, how would you make this component pass faster than 60 seconds?
export const Countdown = () => {
const [time, setTime] = React.useState(60);
React.useEffect(() => {
const interval = setInterval(() => {
setTime(t => t - 1);
if (t === 1) clearInterval(interval);
}, 1000);
return () => clearInterval(interval)
}, []);
return time > 0 ? Time left {time} : Done
};
I also found an issue where the only solution was to inject sinon into the page which I wouldn't consider a great option: https://github.com/microsoft/playwright/issues/10230
Re: SafeTest: A novel approach to front end testing
#14> Conversely, using integration testing tools like Cypress or Playwright provides control over the page, but sacrifices the ability to instrument the bootstrapping code for the app. These tools operate by remotely controlling a browser to visit a URL and interact with the page. I don't think the author has used Cypress or Playwright. Their real value is that they do not drive browsers from the outside like slow, flak…
Author here. Can you show an example of how Playwright would progress a timer on a page? For example, how would you make this component pass faster than 60 seconds? export const Countdown = () => { const [time, setTime] = React.useState(60); React.useEffect(() => { const interval = setInterval(() => { setTime(t => t - 1); if (t === 1) clearInterval(interval); }, 1000); return () => clearInterval(interval) }, []); ret…
If you want to do it with the whole page and talk only to the local code, then yes, I'd recommend Sinon. I think that's a much simpler solution than . . . creating an all new NIH framework!
I'd also recommend refactoring to a more mock-friendly way to do that countdown if you don't want to cover up all the internal logic.
If the timeout interval is loaded remotely from some API (and it probably is if you have reasonably configurable nag popups), then you can always mock that API call.
Re: SafeTest: A novel approach to front end testing
#15how many billions of dollars and they still host their blog on a nagware site SMH It's not like moving would be high drama since they already have a custom domain for it https://scribe.rip/introducing-safetest-a-novel-approach-to-...
From the Hacker News Guidelines: "Please don't complain about tangential annoyances—e.g. article or website formats, name collisions, or back-button breakage. They're too common to be interesting."
Re: SafeTest: A novel approach to front end testing
#16Earlier quoted context omitted.
Author here. Can you show an example of how Playwright would progress a timer on a page? For example, how would you make this component pass faster than 60 seconds? export const Countdown = () => { const [time, setTime] = React.useState(60); React.useEffect(() => { const interval = setInterval(() => { setTime(t => t - 1); if (t === 1) clearInterval(interval); }, 1000); return () => clearInterval(interval) }, []); ret…
If you want to do it with the whole page and talk only to the local code, then yes, I'd recommend Sinon. I think that's a much simpler solution than . . . creating an all new NIH framework! I'd also recommend refactoring to a more mock-friendly way to do that countdown if you don't want to cover up all the internal logic. If the timeout interval is loaded remotely from some API (and it probably is if you have reasona…
The point is that you shouldn't need to rewrite your countdown component to allow testing. Can you provide a snippet of that change and what the test would look like?
Not being toggle parts of the app is the root of the issue when creating e2e tests. For example overriding a feature flag, you could find the API call for the feature but what if it's a grpc call and part of your dev build pulls in any update, you can't easily change the binary response with confidence anymore.
The current solutions are good enough to do smoke tests, but nitty-gritty e2e tests don't scale well.
In the example above it's simple
export const CountdownTime = createOverride(60);
export const Countdown = () => {
const initialTime = CountdownTime.useValue();
const [time, setTime] = React.useState(initialTime);
React.useEffect(() => {
const interval = setInterval(() => {
setTime(t => t - 1);
if (t === 1) clearInterval(interval);
}, 1000);
return () => clearInterval(interval)
}, []);
return time > 0 ? Time left {time} : Done
};
it('tests faster', async () => {
const { page } = await render(
app => 5}>{app}
);
await expect(page.getByText('Done')).not.toBeVisible();
page.waitForTimeout(5000);
await expect(page.getByText('Done')).toBeVisible();
});
Re: SafeTest: A novel approach to front end testing
#17Earlier quoted context omitted.
If you want to do it with the whole page and talk only to the local code, then yes, I'd recommend Sinon. I think that's a much simpler solution than . . . creating an all new NIH framework! I'd also recommend refactoring to a more mock-friendly way to do that countdown if you don't want to cover up all the internal logic. If the timeout interval is loaded remotely from some API (and it probably is if you have reasona…
The point is that you shouldn't need to rewrite your countdown component to allow testing. Can you provide a snippet of that change and what the test would look like? Not being toggle parts of the app is the root of the issue when creating e2e tests. For example overriding a feature flag, you could find the API call for the feature but what if it's a grpc call and part of your dev build pulls in any update, you can't…
I like the library, but isn't createOverride / useValue basically modifying components to allow testing?
Re: SafeTest: A novel approach to front end testing
#18Earlier quoted context omitted.
The point is that you shouldn't need to rewrite your countdown component to allow testing. Can you provide a snippet of that change and what the test would look like? Not being toggle parts of the app is the root of the issue when creating e2e tests. For example overriding a feature flag, you could find the API call for the feature but what if it's a grpc call and part of your dev build pulls in any update, you can't…
I like the library, but isn't createOverride / useValue basically modifying components to allow testing?
Creating an override is basically just providing a placeholder for a value to be injected via React Context. I view this as a form of dependency injection. Contrast this with how this would be done in vanilla Playwright with reading it from a query param or exposing a global to call page.evaluate on which is more along the lines of forcing test code into a component.
Note that if you needed a specific reference in an override there isn't a good way to get that via Playwright, consider this silly example:
it('can test spies', async () => {
const spy = browserMock.fn();
const { page } = await render(
app =>
spy}>
{app}
);
await page.getByText('Log out').click();
expect(await spy).toHaveBeenCalled();
});Re: SafeTest: A novel approach to front end testing
#19Are there potential performance issues to consider with overrides - I'm supposing
const FetchPerson = createOverride(fetchPerson);
must be less performant by some amount than just using fetchPerson? How many over-rides do you find a reasonably complex component needs?
Re: SafeTest: A novel approach to front end testing
#20Are there potential performance issues to consider with overrides - I'm supposing const FetchPerson = createOverride(fetchPerson); must be less performant by some amount than just using fetchPerson? How many over-rides do you find a reasonably complex component needs?
React is pretty performant when context isn't changing. We haven't done any benchmarking but I doubt there's any real world perf hit. For large applications the number of overrides tend to be under 20.
Overrides are opt-in so you can just expose any overridable value as a prop and run a isolated component test on it.