webNavigation.onCommitted doesn't 'see' initial frame creation / page rendering, though if the frame navigations someplace other than its initial page then the listener will catch it. If options were to include the key 'allFrames': true the problem is _somewhat_ resolved. Any frame in the static html page will have the script injected, although you then have origin/url matching problems. Further, if a frame is created using Javascript the script is not injected.

The easiest solution I could think of would be to replace webNavigation.onCommitted with webRequest.onResponseStarted with a filter of {'urls': ['<all_urls>'], 'types': ['main_frame', 'sub_frame']}.

I did some limited testing and no further changes were needed. I was able to execute a script within a frame and a frame created using Javascript.

@arantius any plans to merge the fix from @Sxderp ?

This is affecting scenarios where the iframes are cross-origin objects so it's impossible to do any kind of modification without running GM on those particular iframes.

Thanks !

Missed this, will look into it soon.

Just an observation during my testing of old scripts, don't know, if it helps...

Given a script, that is supposed to work on an iframe,
it looks as if it executes the changes on the page,
but then refreshes again to the unmodified page.

I can only see the flickering when I refresh the page very fast.

Just an observation during my testing of old scripts, don't know, if it helps...

Is this with my patch or with the released version?

Ugh, I think it was 4.0 release...
I WAS @4.1b3, but for the tests, I'm pretty sure, that I reinstalled the release version (until now!)

Same here with iframes like Eselce describes. In some way it is executed, but then stopped after the iframe or page is loaded.

If I inject this script, I only get 1 and "self !== top":

console.log('1');
if (self !== top) {
   console.log('self !== top');
   setTimeout(function() {
      console.log('Timeout');
   }, 2000);  
} else {
   console.log('self === top');
}

"Timeout" is not shown in the log, neither all functions and binds are.

I use 4.1b3.

I have the same issue running GM 4.0 on Quantum. I wrote a very simple dummy example with two pages: main.html and framed.html, and a GM script which gets loaded on every page and outputs the URL of the page it's loaded on.

Most of the time, I only get notifications about main.html, but in about 5% of cases, especially if I hold down F5, I might also get a notification about framed.html.

Is there any hack to reliably force GM 4.0 to execute inside iframes until a patch is out?

I just found out that userscripts are reliably executed in <embed src="..."> but not in <iframe src="..">
I wrote a small test script:
https://openuserjs.org/scripts/cuzi/iframe_embed_Test_Greasemonkey_4

Some more details: In some cases, my scripts are completely executed in the frame (but the view is overwritten later by page scripts and so).
Sometimes, the synchronous parts of my script are ended, but the asynchronous parts are suddenly interrupted by page activity...
Hope, that helps!

Has anyone more infos on this?

Just a short summary of this thread (issue):

• Forget most of the postings, they don't apply
• Probably, the scripts are executed always (but not till the end)
• Unfortunately, the page is refreshed later - layout is recalculated, execution is aborted
• This applies mostly (but not entirely) on async parts of the scripts

I'm not so much into these internals, but probably someone is...

As a temporary fix, I've been swapping out iframes for embeds (example script), which does work to get a script corresponding to the frame to trigger (credit to @cvzi for figuring out that <embed src="..."> does work).

It might be worth noting that Violentmonkey and Tampermonkey work just fine inside embedded frames. Since VM is open source, maybe see how they did it?

Unfortunately, Violentmonkey and Tampermonkey still use the old GM_ naming scheme for the special functions, so scripts aren't portable yet.

https://github.com/greasemonkey/gm4-polyfill

Tampermonkey => GM.* calls, if not provided
Violentmonkey => GM.* calls
Greasemonkey -3.17 / FF -56.0 => GM.* calls
Greasemonkey 4.0+ / FF 57.0+ => builtin GM.* calls

// @grant        GM.getValue
// @grant        GM.setValue
// @require      https://greasemonkey.github.io/gm4-polyfill/gm4-polyfill.js
// @grant        GM_getValue
// @grant        GM_setValue

Has @Sxderp 's fix been incorporated into the main branch yet? If not, how might I go about installing his fork?

Has @Sxderp 's fix been incorporated into the main branch yet? If not, how might I go about installing his fork?

1. No it hasn't.
2. Unfortunately that's one of my PRs that I haven't been keeping synced with master and therefore hasn't been rebased. So the branch itself is missing some of the current changes.
3. I also never implemented the changes suggested in the PR comments. Honestly, those changes _shouldn't_ be required but because Mozilla continually screws things up the changes are required.
4. If you still want to use it (not recommended) then you can follow the below steps.

1. git clone -b use-on-response-started-for-execute --single-branch https://github.com/Sxderp/greasemonkey.git[1]
2. Run ./package.sh, this will create an XPI file.
3. Go to about:config in Firefox and set xpinstall.signatures.required to false
4. Go to about:addons in Firefox, click the cog then select install from file.
5. Select the XPI that was created in step 2.

[1] If your version of git does not support the -b and / or --single-branch flags (older version of git) then you can do git clone https://github.com/Sxderp/greasemonkey.git and git checkout use-on-response-started-for-execute.

Hello,
Are you going to fix the issue? It's pretty old defect and impact all scripts based on iframes structure...

As a reminder, that we have March 13 tomorrow (see Firefox 59.0)...

I want to refer to @Sxderp:

webNavigation.onCommitted doesn't 'see' initial frame creation / page rendering, though if the frame navigations someplace other than its initial page then the listener will catch it. If options were to include the key 'allFrames': true the problem is somewhat resolved. Any frame in the static html page will have the script injected, although you then have origin/url matching problems. Further, if a frame is created using Javascript the script is not injected.

Actually, I have proof, that (on my system) the listener executeUserscriptOnNavigation is called reliably with chrome.webNavigation.onCommitted, so that chrome.tabs.executeScriptInFrame is called with the correct frameId. Why doesn't this solve all our problems with frames? We should not need chrome.webRequest.onResponseStarted to react on an iframe! (Or did you mean, it reacts on the event, but the frame is not visible?) It is definitely called...

Is there a known bug with chrome.tabs.executeScriptInFrame and frameId? There were problems years ago, but they are fixed now. all_frames is not set, thus frameId should be valid. Setting option matchAboutBlank to true seems to matter (otherwise executeScript returned an error <unavailable>), although I didn't fully understand that about:blank stuff (where is it?)...

Any ideas?

Features? This is base functionality missing from the start... I hope, I misinterpreted that.

@Eselce, this was a long time ago and I'm not sure I entirely remember what I was referring to but I'll give it a shot. Furthermore, this only applies if my understanding of the GM 3.x frame matching is correct. That is, each frame origin+path is individually matched to determine which scripts need to run. NOT just the parent document.

Now, onto the issue. When I did my initial testing I had a static page with a same origin frame and a remote frame. On initial page load I could only get the onCommitted callback to fire once. It fired for the main document but did not fire for any of the static frames in the document [1]. Thus no scripts would be injected into them.

However, after initial load, if I caused one of the frames to 'navigate' somewhere the onCommitted callback would be called and scripts were injected into the frame at the new location.

An attempt at using the all_frames option key was made in order to work around the above issue. While using the key caused the scripts to be injected into the frames on the page, there was the obvious shortcoming of not properly matching the frame's origin+path to what the script should or should not run on.

As an aside, I also mentioned that using the all_frames key would not inject scripts into frames created by Javascript.

[1] Whether this issue is still present I do not know. If I recall this issue wasn't present in FF 52 ESR but was present in 56 (57?) (therefore regression). Perhaps it has been fixed.

I agree with you in almost all points.

And you are correct in that each frame is matched seperately, as if it was a whole tab (with its own window and its own document, embedded in a frame).

Well, I've used almost always the same menu/frame structure, so maybe I should test a different one.

When you say "it fired", do you mean the pure call of the listener?

As I said, I've encountered some examples with executeScript producing an error condition, but the listener was still called.

all_frames can't work, because of the wrong location (different window, different document). BTW: The menu only handles mainframe urls of each tab - if the menu is wrong, that doesn't mean, that the script is not called...

When you say "it fired", do you mean the pure call of the listener?

In that particular message I meant 'The function passed into onCommitted.addListener was called.'.

Hello,

I read this post carefully but I do not understand how to use your alternative solution on my local script ".user.js". How can I apply your solution? Sorry, I'm new.

(Since updating Firefox, a generated popup-iframe is no longer recognized by the additional script, but if I open the same popup in a new window, the script is applied.)

Thank you in advance for your help

You've described exactly what the problem is: It's like @noframes is activated. The frame content's URL does not activate your script correctly. Hopefully this is fixed soon (opening frames in extra windows is annoying)...

Thanks Eselce.
And always since the update of Firefox, I am obliged in the header to declare all the scipts '.js' (with require) already used on the target site. (including jquerry)
And it's not the same.... It creates bugs or conflicts.
Do you also know this problem?

2945 contains another example for the observation, that scripts are started in frames, but abandoned after some milliseconds.

I'm hesitant to give the specific URLs I'm testing as they are outside of my control, but I'm finding some consistent but weird behavior to pass along.

I have a site with which I need to interact. It initially loads a frameset/frame with rows="100%,0" (one frame to fill the screen) on a different domain. This frame contains one frameset / 3 frames within the domain of the intermediate frameset/frame.

Some of the 1+3 child frames' GM scripts will "blip" into existence, then disappear after the initial cycle -- they never come back after any asynchronous operation. That fits some behavior described on this thread. Note that which ones "blip" and the described behavior below vary by browser / GM version, but it is not random; the patterns are weird but 100% repeatable for any given setup.

1. The first frameset/frame will NEVER blip into existence. I've tried detaching and rebuilding the frame tag, both through window.document and unsafeWindow.document, both in the initial cycle and after a delay, but nothing will cause that frame's GM script to ever report anything to console.log. (The @include is *, with no @exclude or other URL filtering.)
2. Some of the later behavior will differ between Firefox 52.8 / GM 4.1 and Firefox 60.0 / GM 4.3, but I can get frames' GM scripts in each case to "blip" into existence whether @noframes is set or not. This is with @includes *, no other URL filters. These should never blip with @noframes set. I have verified that window.top!==window, ie the browser knows (or should know) these these are in frames.
3. In Firefox 52.8 / GM 4.1, The next frameset / 3 frames will ALWAYS blip into existence. In Firefox 60.0 / GM 4.3, they do not "blip" on initial frame load.
4. In Firefox 60.0 / GM 4.3, clicking a link in one of the 3 frames that navigates another of the 3 frames (via a "target" attribute on the anchor link, not script) will "blip" -- not the new URL, but the old URL for the navigated frame. (This is one of the frames that blipped on initial load in item# 3.)
5. Here's the weirdest part. In both browser setups --- Following the steps, we've opened the initial page with 2 layers of framesets, 4 total frames, and clicked a link in one frame to navigate a different frame on the same page. To give names to that, our initially-displayed second-level frameset had frames "top.htm", "menu.htm", and "start,htm". We've clicked a link in the "menu.htm" frame to cause the frame holding "start.htm" to navigate to "content.htm", with similar but slightly different behavior per browser setup, noted above. Now, we click a link inside of the "content.htm" frame, to navigate inside the same frame, same domain.

At this point, the script for "content.htm" will not only "blip"... it will also remain alive after a GM.xmlHttpRequest has completed -- an asynchronous event. At this point, "content.htm" is nowhere on the browser display, but its script will continue to operate.

So it appears to me that the reason GM scripts on pages within frames are not working is because the script is being loaded on page unload instead of page load. Setting @run-at to document-start and putting the script into a DOMContentReady event of unsafeWindow.document had no improvement. (Setting it on window.document never fires the event.)

-Ryan

In Firefox 52.8 / GM 4.1, The next frameset / 3 frames will ALWAYS blip into existence. In Firefox 60.0 / GM 4.3, they do not "blip" on initial frame load.

In the transition to Firefox 57 Mozilla changed _something_. Whatever it was it changed (or broke) the way that Frames could be triggered. This was briefly talked about in some other issue. As a result the scripts would not run on initial frame load (57+).

The eventual switch to userScript or contentScript APIs should resolve all of this anyway.

So people keep saying, yet Violentmonkey continues to execute in frames. VM doesn't properly separate userscripts from web page content (CSPs block userscripts, web pages can rewrite global objects etc.), or I'd have chucked Greasemonkey long ago. Perhaps these are related problems, but I'm using userscript engines so I don't have to explore Firefox's chrome deep enough to find out on my own.

Hmm this is a difficult issue, some of my scripts no longer work because they can't work with iframes. So it seems it's not really possible to fix it, and we have to way for Mozilla to implement some API? Is there nothing we can do ourselves, like a work-around? I just need to do some stuff to a page in an iframe, often even from the same domain.

My own workaround for the moment is to run Greasemonkey in top frames and Violentmonkey in child frames. I can't use Tampermonkey because it has an ambiguous homemade license, but if that's not a problem for you, then it might or might not work any better.

Bear in mind that VM drops the script into the page's own context. It's like GM's unsafeWindow without the safe equivalent. One of the most memorable times I had was debugging a script where the page content had defined a 'toJSON()" method on Array.prototype, causing JSON.stringify() to spit out invalid JSON inside my script. I've had to defensively trap and fix these as I've found them.

Another huge concern with VM is that it respects the content page's content-security-policy, so any directive that limits script sources will cause the VM script to never execute. You can see this in the browser console (but not the web console). That's why I can't run VM entirely and just get rid of GM. I hven't yet encountered a child frme with a CSP set, but when I do, it will be completely unusable.

@RyanHanekamp Thanks for the tip! Perhaps I will use Violentmonkey for some scripts, then.

Does Violent also have something like synchronous GM_getValue? That's another problematic issue that breaks a ton of scripts in the new Greasemonkey. I still trust Greasemonkey the most, though, for various reasons, so I won't leave it.

As to iframes, I've been trying to replace them with object tags, which one can apparently access directly using Javascript, like so:

myObject= document.createElement('object');
myObject.setAttribute('id', 'myObject'); 
document.body.appendChild(myObject);
myObject.setAttribute('src', 'https://example.com');

Then, once the object is loaded:
document.querySelector('#myObject').contentDocument.defaultView.document.querySelectorAll('someElementInsideObjectPage')
At least this works for me in a script where the object is at the same host as the main page. You can also send messages from and to ( ... contentDocument.defaultView.postMessage('hello, object')) the object.

I'm not a VM expert, but I believe it does implement at least most of the original GM_* API. I would think it's better to adapt your scripts to asynchronous than to backtrack to a synchronous platform in the long run, though. Per my understanding, Greasemonkey did this as a performance boost within the new Quantum framework, which doesn't allow synchronous calls between background and content scripts.

As for the object solution, it won't solve my particular problem, but I'm glad others have found it useful. Aside from marshaling CSS / properties / etc and getting it to work with frames as well as iframes, I have to filter out objects in a capture process as potentially unsafe. There are ways around all of these problems, but VM was the easier interim until GM finally does what it says on the tin.

Also, if you have a same-origin frame/iframe, you can access the content ofthose directly as well. The harder part is cross-origin, which is why I need userscripts inside the frame. It sets up a window.postMessage() channel to talk back to the parent window.

@RyanHanekamp Good to know that Violent Monkey still has the old, simple GM_*. I really wish Greasemonkey had kept the old, synchronous GM_getValue for backward compatibility, in addition to the new version. I might try to implement the new asynchronous function in a new script, but I'm not a programmer and I'm not sure whether I could make it work. And I am certainly unable to refactor the use of old GM_getValue in an ancient script of 2000 lines that I found online...so many scripts are broken now.

I understand why Violent was the better option for you. Let's hope Anthony or Sxderp or someone else finds out how to do implement this eventually. I really wish I could contribute, but I'm a total layman.

Oh, you can directly access the content of a same-origin iframe (without postMessage etc.)? I remember spending a lot of time trying to find a way, but it didn't seem possible. That's why I switched to postMessage as well.

Frames and iframes have a contentWindow property that's equivalent to the window property. Both have a document property to access the DOM.

The hardest part of working with iframes (on the same origin) is detecting when its content is loaded, because you can't do squat until that happens. onload doesn't work how it looks. Firefox provides a DOMFrameContentLoaded event that fires for EVERY frame loaded, including grandchild / great-grandchild etc frames, which you can match to the original frame / iframe element with the event.target property. If you control the content of the frame/iframe, you can also have it talk back to the parent with either postMessage or calling a global method on the window.parent object.

Speaking of which... that's a potential workaround for this issue. If there is or could be a way to code a GM script to manually inject a userscript into a cross-domain window reference, it would take a lot more coding for the userscript creator, but it could get the job done. The pattern would be listen to DOMFrameContentLoaded, check if the event.target is first-generation, and manually inject the script if so. (Presuming that the first-generation frame's script can listen to DOMContentLoaded for second generation frames, and thus get a complete chain.) There would be no way to get @run-at dom-start behavior, and there might also be timing issues, but we could probably work around those for most use-cases.

I've personally given up on this issue ever being resolved, and have instead moved on to directly coding an extension. Which works fine in all frames!

The difference between Greasemonkey and Violentmonkey on this point seems to be that Violentmonkey is being triggered from content scripts with all_frames set to true, while Greasemonkey has no install-time content scripts and relies entirely on the dubious ability of the background script to sniff when a tab's frame has navigated. (And Violentmonkey fails on CSP pages because it temporarily injects a SCRIPT tag instead of using the far safer tabs.executeScript().)

Put in a static content script with all_frames, run_at start, matches everything to notify the background process for start / document.DOMContentLoaded / document.Idle to trigger userscripts for each run_at, and you're good to go. A non-trivial but manageable amount of work to make this problem go away. I'd fix it myself, but I have no interest in plodding through your dev dependencies and could only produce the output code.

@RyanHanekamp

and have instead moved on to directly coding an extension. Which works fine in all frames!

Would you be willing to share that extension code of yours?

My extension isn't general purpose. The point is that using a static content_script in the manifest with all_urls, all_frames will run the script whenever any frame is loaded or navigated, and can even eval / Function constructor code just fine regardless of the page's Content-Security-Policy.

I have not tested with programmatically constructed frames / windows, but I would imagine they would run at initial creation regardless of the run_at setting, because such frames are initially created blank and then populated -- the engine would probably only see the initial creation. I also didn't test data: urls, which might require explicit matching -- I'm not sure if all_urls covers them, or just http/https.

It might not have to be a static content_script reference, either, but it appears uncertain from the documentation if dynamically-invoked content scripts load automatically when the page is navigated. My impression is that they're only injected into currently matching tabs / frames per the supplied match pattern, and navigation doesn't trigger a re-injection. But I see no downside to using a static content_script for this purpose.

Greasemonkey obviously can't include userscripts as static resources in the manifest, and content scripts don't appear to have direct access to tabs.executeScript (though I'm hardly an expert on this as I'm only a few days in), but what a static content script CAN do is message the background process to let it know that the frameId has been navigated, and to what URL. This would be more reliable than how I perceive the attempts mentioned on this thread to hook the right event in webRequest or webNavigation. The signal from the static content_script becomes the event we're looking for to trigger Greasemonkey's userscript loader / injector.

There would possibly be a delay for userscripts that strictly MUST run_at document_start. The call to the background script is asynchronous, and the document will have progressed by the time the userscript is invoked. This is quite possibly why Violentmonkey uses a temporary script tag instead of tabs.executeScript, as the script tag injection can be done directly from the content_script, synchronously. I would find being forced to work around uncertainty of the document's state for run_at document_start to be troublesome, but preferable to the script not running at all.

Greasemonkey ... CAN [use a static content script to] message the background process to let it know that the frameId has been navigated, and to what URL ...

This is what we used to do for detecting .user.js navigations. Seems like an obvious and good solution: detect content by running a content script!

But it turns out even extension content scripts can be broken by CSP (#2631 and http://bugzil.la/1267027 and http://bugzil.la/1411641).

I can execute my own plugin, including a Function() constructor from directly within the content_script, on Firefox 60.0.1 and 52.8.1ESR with the following CSP set:

frame-src data:; object-src 'none'; script-src 'none'; style-src 'unsafe-inline' data:; connect-src 'none'; media-src 'none';

2631 has been closed, apparently because Firefox fixed its underlying bug. The first bugzilla is in reference to injecting script tags (the Violentmonkey method), not the content_script itself. The second is in reference to the sandbox attribute for CSP, not surprising because it forces the domain to never successfully complete a domain match even to itself. Kinda like NaN!==NaN.

When this was first filed several months ago we did things differently. Today we use webNavigation.onCommitted to detect navigation, and in the test I'm running right now, for some reason I'm not seeing it trigger on a(n i)frame).

Hello, is this solved now?

I couldn't get a script I created for Tampermonkey to be launched on an iframe with Greasemonkey.

The code for execution hasn't been changed on our side. So unless Mozilla has changed something on their end this is still broke. However #2663 should resolve it.

It might be worth noting that Violentmonkey and Tampermonkey work just fine inside embedded frames.

Tampermonkey has problems with iframes in Chrome, at least for me.

It might be worth noting that Violentmonkey and Tampermonkey work just fine inside embedded frames.

Tampermonkey has problems with iframes in Chrome, at least for me.

But it works for me in Firefox Violentmonkey. So I wonder how it can work there?

I've just noticed that a script using the old sync GM_GetValue still works fine in Violentmonkey as well. How is that possible? I thought Firefox had forced async GM.GetValue? I'm so confused now: presumably Violentmonkey had to sacrifice something else in order to still support sync and the other stuff?

@Cerberus-tm The change in Firefox meant that data can only be requested from extension storage or the background context asynchronously (the userscripts themselves are sent to the content script asynchronously). However, data could be provided to userscripts synchronously, if each GM4 content script prefetched and cached in the content script the data that's stored for each userscript being loaded in that page.

Such a cache would allow the content script to respond synchronously to the userscript's requests for data. Implementing a cache runs into issues with maintaining consistency across the various content scripts, but that's resolvable by having the content scripts listen for all changes to GM4's extension storage. In addition, if a userscript is storing a large amount of data, then caching it in every content script where the userscript is run also significantly increases the memory required for each content script.

TM and VM chose to do something similar to the above in order to not break compatibility with the original Greasemonkey APIs when those userscript managers were implemented for Chrome, which has the same restrictions wrt. asynchronous communication with extension storage, etc. Given that they are already doing it for Chrome, there was no reason for them to change when implemented for Firefox.

So, the FF57 cut-over to WebExtensions did force a rewrite of GM, but it did not force the adoption of the asynchronous APIs for GM.getValue, GM.setValue, etc. WebExtensions did make the use of async-based APIs easier to implement than synchronous would be, but it didn't make it required.

Personally, I feel that the choice, and other choices to break compatibility, were/are unfortunate. The lack of backwards compatibility with scripts which ran fine in GM3 and/or compatibility with TM result in many people choosing not to use GM4. My experience is that well more than 30 of the userscripts I use, all of which worked fine in GM3, don't work with GM4 (or at least didn't work prior to being rewritten to be compatible with GM4). There are still 28 userscripts I use daily which ran fine under GM3 which don't work with GM4.

I posted a workaround to this issue on Stack Overflow as an answer to how to Apply a Greasemonkey/‌Tampermonkey/‌userscript to an iframe. Basically, I'm waiting for the frame to load and then I'm operating on the window.frames array. I use a custom marker in each frame's body to denote that I've already seen the frame.

Perhaps Greasemonkey could implement a solution in a similar manner?

It'd be awesome if we also had a GM.waitFor(css_selector, action_function) like waitForKeyElements(), but that's an aside.