How to test a WebExtension popup in Puppeteer

Yes, it's possible! I've known about some workarounds for this for a while, but I recently discovered some new possibilities which made me want to write a blog post. There's still a lot of room to make WebExtension testing better, but we're closer than I realised, and I think that some of the things I've found are useful both for doing something today and for highlighting what it would be great to see in the future.

The Status Quo

If you search Google for this question (for any testing framework I've come across!), you'll probably find the same answer - determine the chrome-extension:// URL for your popup, open it in a new tab, and test as best as you can. This is a good workaround but has limitations. Most notably, the popup isn't running on top of another page, so any tab-specific context it might rely on is missing. You could work around that with test-specific code but it definitely falls more in to the workaround category.

What we need to properly test an extension popup

So what would we need to properly test this? Ultimately I think there are two key pieces:

• We'd need to be able to programatically open the popup.
• We'd need to be able to get a 'handle' on that popup and interact with it.

Programatically opening the popup

This is actually possible as I discussed in my last blog post. With the action.openPopup API, you can programatically invoke the popup by running code in the background page/service worker. Expanding on the Chrome Extensions docs from the Puppeteer website, it'd look similar to this:

const workerTarget = await browser.waitForTarget(
  target => target.type() === 'service_worker'
);
const worker = await workerTarget.worker();

await worker.evaluate("chrome.action.openPopup()");

Note that in Chrome, this API is limited to Manifest V3. While testing in Chromium, it seemed to work for MV2 extensions, but the behaviour was unreliable. I haven't looked in to this too closely but suspect it may be using a different implementation.

Interacting with the popup

This is the part which I really thought was impossible until recently. Without support for extension popups in the Chrome Dev Tools Protocol, which Puppeteer is built on, it felt like this was a non-starter. But we can actually do the following (where extensionId is the ID of your extension and path is the expected path of the popup):

const popup = await browser.waitForTarget(
  (target) =>
    target.type() === "other" &&
    target.url() === `chrome-extension://${extensionId}${path}`
);

Surprisingly, this works! My mental modal of waitForTarget is that it's an escape hatch allowing us to peek in to the internals of Puppeteer. While it doesn't officially have support for popups, that is really just a new web context behind the scenes, and therefore we can obtain it like this.

So, can we just call popup.page() and get access to a page that we can start interacting with? Unfortunately not. Puppeteer has an internal isPageTargetCallback function, which presumably exists as a safeguard to prevent sending page-like commands to a target other than a page. The fact that this returns false causes popup.page() to return null.

I've opened a Puppeteer issue to consider providing a way of overriding this. I could really seeing that going either way - I think as an experimental feature, this is fun to let people play around with, but it also feels very much unsupported and it may be desirable to keep things locked down until the Chrome Dev Tools protocol provides a better option.

Longer term, the best solution for this would be updating the Chrome Dev Tools protocol to provide a new "popup" type which has a well documented set of features and can be properly integrated in to Puppeteer. https://crbug.com/1223710 feels like the bug to watch.

Ok, but let me see something working!

Alright, I won't leave you empty handed! Over on GitHub, I've put together a proof of concept overriding this internal property to show that there is at least some underlying support today. That's what you're seeing in the video. To stress, this is very much undocumented and could break at any time. But I do think it's really cool!

Does it work in other browsers?

I've only done limited testing here, but at least in Firefox, it doesn't appear that extension targets are exposed. However, I expect that this may change if we're able to define a proper API! That's more reason to avoid using a hack and work together to properly define how WebExtensions are tested.

The future

While this makes for a very exciting demo, it's clear that proper support in the Chrome Dev Tools protocol and libraries like Puppeteer would be better for everyone. If you're reading this, I have a few requests:

1. Star the Chromium issue! This helps the Chrome team to see that there's interest in this capability and prioritise spending time on it.
2. Leave a reaction on this Puppeteer issue to show that there's interest in exposing a better way of doing something like this.

There has also been some discussion in the Web Extensions Community Group (see here). It hasn't gone anywhere yet but there were some clear next steps and if this could become a focus for the group, I can see a future where we have a standardised, well-documented way of testing extensions across browsers.

For now, I'll be crossing my fingers!

13 Nov 2022