Hello Giorgio,

I tested the stateless branch with the following results:

Firefox 115 ESR

• integration tests pass
• https://www.fit.vutbr.cz/~ipolcak/jsr/farbling/canvas.html works as intended

Chromium 120

• Integration tests do not work due to https://bugs.chromium.org/p/chromium/issues/detail?id=1498558, there is a way to bypass that bug but I did not go there yet, see next
• https://www.fit.vutbr.cz/~ipolcak/jsr/farbling/canvas.html does not work with the stateless branch. It works in JShelter from Chrome Store.

Giorgio, can you have a look at https://www.fit.vutbr.cz/~ipolcak/jsr/farbling/canvas.html in Chromium/Chrome/related browser? You should see some pixels farbled. The red shape should be the same for Canvas and WebGL farbling. If you refresh the page, the shape should be the same. The shape should be the same on https://www6.fit.vutbr.cz/~ipolcak/jsr/farbling/canvas.html provided you have IPv6. The shape should change when you restart the browser.

Tentatively, I think I will go on and release a new Firefox version with the code from that branch. I guess that the bug in Chromium-based browser lays in common/session_hash.js. I can see some exceptions in background console of JShelter suggesting that the problem is on line 37, looks like browser.storage.session is undefined.

Hi Libor!

The Chromium issue should be fixed by c337f2a.

I suggest to release both Firefox and Chromium (unless you find other issues) in order to have a larger audience testing these changes which are the foundation of our Manifest V3 transition strategy.

Thanks!

Thank you, Giorgio. It looks like the fix works well.

I wanted to release JShelter for Firefox ahead of dealing with the issues in Chrome. Of course that now that reason is gone, there is no need to delay Chrome release.

However, I looked more thoroughly at the changes.

Let's go through our background scripts:


"nscl/lib/browser-polyfill.js",
"nscl/service/TabCache.js",
"nscl/service/NavCache.js",
"nscl/service/DocStartInjection.js",
"nscl/common/log.js",
"nscl/common/uuid.js",
"nscl/common/SyncMessage.js",
"nscl/common/tld.js",


I suppose that NSCL is stateless so I will not examine these further.


lib/sha256.js
helpers.js
update.js
url.js
levels_browser.js
settings_tweaks.js
levels.js
fp_levels.js


These are stateless or already read the state from the storage. If unloaded there will just be some additional workload. But that is expected from MV3.

However, most (e.g. helpers, url, levels_browser) provide functions or variables used inside other scritps (e.g. level.js). I hope that if the background scripts are reloaded, they are initiated again in the same order as specified in the Manifest and that the browser run all scripts. Am I correct?


fp_detect_background.js


stores the number of API calls and related information. Specifically fpDb, latestEvals, availableTabs, and fpd_track_callers_tab look like they need to be saved into some storage to prevent "random" loss of content. If JShelter loses the content of these variables, FPD might miss detecting fingerprinting behaviour, the number of calls listed in the popup will not show correct numbers and a re-opened popup window might display a lower count of API calls. The fingerprint report will not report all calls if the content of the variables is lost.


background.js


tab_status and tab_urls hold a state. Speciffically, if lost, I guess, the number of API groups (badge colour, API group count) might be lost (although I am not sure about the colour).


level_cache.js


I think that this one should be fine but it would be great if you can confirm.


http_shield_firefox.js
http_shield_common.js


These should be stateless.

http_shield_chrome.js holds state in dnsCache and blockedHosts but we will need to remove NBS anyway.

Observations (in Firefox)

1. Load extension in about:debugging (temporary)
2. Visit a web page in a different tab, open popup, click on "Detail tweaks ..." to see the number of calls.
3. "Terminate background script" in about:debugging for the temporary JShelter
4. Go to the tab from step 2, open the popup, click on "Detail tweaks ..." -> nothing happens (this is due to the deleted fpDB and possibly other variables mentioned above.
5. Open a new tab and a new page there, clicking on "Detail tweaks ..." works in its popup.
6. Going back to the tab from step 2 and 4. Clicking on "Detail tweaks ..." shows no calls.

Badge icon:

In the above scenario, badge color and text is kept by Firefox even if the background script is stopped. However, once JShelter detects a new call in a tab, the stats are reset, so the number of API groups goes down which might result in color change (downgrade) and even dropping the ban to initiate calls.

1. Turn on blocking FPD.
2. Go to the JShelter test page https://polcak.github.io/jsrestrictor/test/test.html, FPD will detect fingerprinting and block further communication.
3. Open console, run 'fetch("https://polcak.github.io/jsrestrictor/test/test.html")'; the console shows TypeError: NetworkError and Network tab shows Blocked by JShelter for that resource.
4. The badge icon shows 6 and is red. FPD shows hight number of APIs misusable for fingerprinting in the popup, and fingerptint report can show all the calls.
5. "Terminate background script" in about:debugging for the temporary JShelter (do not close the testing page in the mean time).
6. Go back to the tab with the test page. The badge color is green. I saw the number in badge go down some time. FPD report cannot be opened.
7. Open console, run 'fetch("https://polcak.github.io/jsrestrictor/test/test.html").then((x) => console.log(x))'; the console shows the response and Network tab shows that the resource was not blocked.

Suggested next steps:

• We should not rush a new release until we have the above fixed.
• We should store variables identified above (more or less) to the local storage as done in session_hash.js.
• However, that means that work with fpDb will be delayed as the access to the local storage is async and may be interleaved with run of page scripts. That means more race conditions during the detection of the scripts. See below:

Possible optimization:

• Keep fpDB in memory as is.
• Write changes asynchronously to the local storage.
• During initialization check if the local storage holds some data, and load them.

Possible enhancement of NSCL

We will need a similar code for each variable stored in local storage like


const DEFAULTS = {sessionHash: null, visitedDomains: {}};
({sessionHash, visitedDomains} =
(await browser.storage.session?.get(DEFAULTS)) || DEFAULTS
);
this.sessionHash = sessionHash ??= gen_random64().toString();
this.visitedDomains = visitedDomains;


and


await browser.storage.session?.set({
sessionHash,
visitedDomains,
});


I think that we should add a code either to NSCL or to helpers that wrap that code into functions like init(varName, defaultValue), store(varName, value).

Thanks for the investigation, it's exactly what I hoped for.
I agree with your suggested next steps.

One clarification: session storage is different than local storage. The former (session) is not written permanently on disk, but it's kept in RAM instead until the browser is restarted, which is what we want to replace long-lived variables/properties.
It being asynchronous as well, nonetheless, is very unfortunate, but I guess that's the price to pay for multi-process WebExtension browser infrastructure.

Abstracting away this best effort code variable<->cached session storage dance into NSCL is a good idea, indeed.

I managed to run integration tests in Chromium 120 and the tests work as expected.

Ad: session storage vs. local storage. Sorry, if I mixed them up somewhere in my text. All the variables that keep the state across background script restarts should go to RAM, so to the session storage. They were in RAM and need to stay in RAM.

Abstracted the stateless initialization / save dance in nscl/common/CachedStorage.js and adopted it with b51634a.

Next step will be using this to persist the volatile globals in fp_detect_background.js and backround.js, therefore fixing the issues reported in this comment.

Persist the volatile globals in fp_detect_background.js and backround.js, therefore fixing the issues reported in this comment.

A bit more involved than supposed above, but still relatively straightforward (thanks to the CachedStorage abstraction): done in e6bef48 :wine_glass:

Hello Giorgio,

going back to https://pagure.io/JShelter/webextension/issue/127#comment-890848, the first problem seems to be fixed.

However, I identified two problems:

Problem 1

Badge text background is always gray.

Problem 2

Expected behaviour:

1. Load stateful extension like current main in about:debugging (temporary)
2. Turn on full blocking for FPD.
3. Go to https://polcak.github.io/jsrestrictor/test/test.html badge icon background is red and JShelter raises a notification about detected fingerprinting.
4. Open the pop up window by clicking on the JShelter icon, the window appear without any noticeable delay. You should see that the number of APIs misusable for fingerprinting is high. Close the popup.
5. Open console, run 'fetch("https://polcak.github.io/jsrestrictor/test/test.html")'; the console shows TypeError: NetworkError and Network tab shows Blocked by JShelter for that resource.

Stateless behaviour:

1. Remove JShleter added above, load stateless JShelter in about:debugging (temporary)
2. Make sure that FPD full blocking is on.
3. Go to https://polcak.github.io/jsrestrictor/test/test.html badge icon background is gray (BUG) and JShelter does not raise a notification about detected fingerprinting (BUG).
4. Open the pop up window by clicking on the JShelter icon, the window sometimes appear with a delay of several seconds (about 11 seconds in my case) - LOOKS LIKE A BUG. The last line that should report the number of APIs misusable for fingerprinting called by the page is completely missing (BUG). Close the popup. Repeated attempts to open the pop up window are fast until you reload the page or go to a different page. In that case, the pop up window appears after a very long delay, again.
5. Open console, run 'fetch("https://polcak.github.io/jsrestrictor/test/test.html")'; the request is not blocked (BUG).

Closing remarks

All problems are likely related. FPD does not work as expected. The question is who should try to fix the problem.

I am using Firefox 115.8 ESR.

The question is who should try to fix the problem.

I am investigating, thanks.

Commit f8095ef should fix all the problems from this comment, except the delay on popup opening, which I'm still investigating.

@gioma1: Thanks, nice catch. I tried the patched version and it looked like it worked until I clicked the icon to raise the pop up (on the test page). It seems that there is to much communication and the whole desktop environment froze.

It looks like I cannot move on until the pop up issue is fixed.

It seems that there is to much communication and the whole desktop environment froze.

Indeed that specific page triggers a lot of fpd-related IPC, and the issue was caused by the storage.session API getting clogged by the rapid-fire CachedStorage.save() calls due to the FPD information updates.

It looks like I cannot move on until the pop up issue is fixed.

Should be fixed by 91dd275

Hello Giorgio,

It works great. Both integration and FPD tests are OK. I can test that JShelter works. Firefox allows me to unload the background script. I cannot see a similar option in Chromium so I did not test that in Chromium.

What are the next steps? Merging to main but not releasing, yet?

Merged to main and released as 0.18 https://jshelter.org/first-mv3-step/, https://lists.nongnu.org/archive/html/js-shield/2024-04/msg00003.html.