Improve representation of behind-the-scene network requests in the logger

[foobar13373]

Describe the issue

With one of the latest uBO Versions (can't exactly tell which one, somewhen in the 1.15.* line) aome Atom/RSS feeds stopped working. Disabling uBO makes them work again.

An example feed is https://blog.fefe.de/rss.xml - I can still add it perfectly fine to Firefox, the browser asks me if I want to add this live bookmark and this works. But when I try to open it in the bookmakrs menu I only get the message that the live bookmark could not be loaded. Disabling uBO + Browser restart makes it work again.

Some feeds are working normally however, like https://www.heise.de/newsticker/heise-atom.xml.

One or more specific URLs where the issue occurs

Does not work: https://blog.fefe.de/rss.xml
Does work: https://www.heise.de/newsticker/heise-atom.xml

Steps for anyone to reproduce the issue

1. add live bookmark
2. try to let FF load the recent articles

Your settings

• OS/version: Windows 10
• Browser/version: FF 59 64 Bit
• uBlock Origin version: 1.15.18

uBO is set to "everything globally denied" (top left column red button for all resources), sites are whitelisted on a per domain basis (right column domain-sepcific grey buttons)

Your filter lists

All, except "uBlock filters – Experimental​​​​​​​​​", "Fanboy’s Anti-Thirdparty Social (see warning inside list)​​​​​​​​" and "JPN".

[gorhill]

Works fine here. Use the logger to investigate such filter issues.

[foobar13373]

Ah, I see.
* * * block | -- | xhr | https://blog.fefe.de/rss.xml
But there must be some change in behaviour recently, either in FF or in uBO. Because all feeds suddenly stopped working where the host was not explicitly allowed by me - but these feeds were working before, so uBO suddenly handles requests that it didn't handle before. Does uBO in recent versions also process requests that were not done in the active tab, but also requests done by FF in the background?

I never had blog.fefe.de allowed in uBO (because it does not need anything but plain HTML to work), the feed was still working however - but now it isn't since 1 or 2 weeks, either since FF 59 or a recent uBO update.

What is the intended behaviour here? Is it possible that now other "background requests" are blocked as well? OCSP, FF version checks and so on (whatever this browser does "blindly")?

[gorhill]

Provide exact repro steps please if you want me to answer meaningfully.

[gorhill]

Also, keep in mind that if you are using advanced user mode and on top of this a block rule such as * * * block, you are assumed to be able to unbreak whatever breakage this may cause -- this is not a uBO issue. You had the logger to disagnose this yourself.

[foobar13373]

The log tells me that uBO is blocking the request. But it did not block the request in an earlier version using exact same settings and filters. This issue is about this undocumented behaviour change and whether it was intended.

Steps to reproduce:

1. block all resources using uBO in FF59 (left column, top red button).
2. load an RSS/Atom feed as live bookmark, like https://blog.fefe.de/rss.xml
3. put the live bookmark in the boomark menu
4. right click the live bookmark and request a reload.
5. Feed is not reloaded, instead an error is displayed
6. go to the feed domain, allow it, trigger reload of the live bookmark, it loads correctly

[gorhill]

You forgot to tell whether behind-the-scene is whitelisted. It is by default until 1.15.18 -- which is the version you said you used.

[foobar13373]

behind-the-scene is in my whitelist, as it always has been, never touched this list.

[gorhill]

I can reproduce. This is a change in behavior from the fix to #3546. You are dealing with a filterable behind-the-scene network request. The solution is to manually add a rule to override your global block rule:

The request is categorized as a behind-the-scene one, because uBO is not provided the information from which tab it originates. However, uBO is provided the context in which the network request was made, as you can see from the logger (above): "from blog.fefe.de".

Solution, create a noop rule, however granular you wish it to be. Example:

Or you could manually add blog.fefe.de blog.fefe.de * noop in the "My rules" tab.

These filterable behind-the-scene network requests can be filtered because Firefox API provide the needed information to do so, and hence are bypassing the behind-the-scene whitelist directive.

Question: did you have blog.fefe.de as a whitelist directive?

[gorhill]

I am going to provide a different icon from the behind-the-scene one in the logger so that there is no confusion.

[foobar13373]

Thanks for explaining this. No, I did not have blog.fefe.de whitelisted in any way, as stated above already. That's why the issue came up - the feeds were working even though everything was blocked and suddenly they stopped working with some version update without me doing any settings or whitelist changes.

I explicitly added the feed domains to my rules now, they are working again.

Popup in logger needs fixing:

No mention about blog.fefe.de

---

For "Google Dictionary" Chrome extension, context is content.googleapis.com and destination apis.google.com

https://www.reddit.com/r/uBlockOrigin/comments/8lqvbn/chrome_extention_google_dictionary_not_working_if/

This may also happen with static filtering. I can reproduce issue from comment in above reddit post using ||apis.google.com^ (optionally with $domain=content.googleapis.com) or /googleapis.proxy.js (I cannot reproduce selecting all available uBO filters - must be something external or already whitelisted in other filter list). In this case noop may be not enough - static filtering exception or explicit dynamic/URL filtering allow is needed.