EWW in Emacs 31 would not use browse-url-generic as browse-url-secondary-browser-function

I recently built Emacs 31 from source to get the newest Emacs on Debian. Most things worked like they used to in Emacs 30 except the few things I knew about from reading up on Emacs 31, but for some strange reason, when browsing with eww inside Emacs, I could no longer hit & and get the site to open in LibreWolf. These are the relevant eww settings in my init.el:

(use-package eww                        ; Web browser
  :config
  (setopt browse-url-browser-function 'eww-browse-url
          browse-url-secondary-browser-function 'browse-url-generic
          browse-url-generic-program "librewolf"
  ; ...

I had a look in the manual to see if there was something new about how browse-url-secondary-browser-function should work, but it seems like it should work exactly like before. I then had a look in the Emacs News (C-h C-n), but it did not say anything about this. The documentation for the relevant variables and functions have not changed either.

Since browse-url-secondary-browser-function no longer accepts 'browse-url-generic, I tried giving it a lambda to start-process "librewolf" instead and that worked perfectly.

(use-package eww                        ; Web browser
  :config
  (setopt browse-url-browser-function 'eww-browse-url
          browse-url-secondary-browser-function (lambda () (start-process "librewolf"))
  ; ...

Another thing that does work is when I do this:

(use-package eww                        ; Web browser
  :config
  (setopt browse-url-browser-function 'eww-browse-url
          browse-url-secondary-browser-function (lambda () (browse-url-generic))
          browse-url-generic-program "librewolf"
  ; ...

When I look up what is bound to & in eww with C-h k, I get eww-browse-with-external-browser.

(defun eww-browse-with-external-browser (&optional url)
  "Browse the current URL with an external browser.
Use `browse-url-secondary-browser-function' if it is an external
browser, otherwise use `browse-url-with-browser-kind' to open an
external browser."
  (interactive nil eww-mode)
  (setq url (or url (plist-get eww-data :url)))
  (if (eq 'external (browse-url--browser-kind
                     browse-url-secondary-browser-function url))
      (let ((browse-url-browser-function browse-url-secondary-browser-function))
        (browse-url url))
    (browse-url-with-browser-kind 'external url)))

When I look into that function's source code, it calls browse-url–browser-kind to determine if the function used for browser-url-secondary-browser-function is an external browser or an internal one and if it is external, uses the function to browse the url by setting browse-url-browser-function temporarily and then use browse-url. If browse-url–browser-kind returns something else, it runs browse-url-with-browser-kind 'external on the url.

When I look at the source for browse-url–browser-kind, it determines if the function used is internal or external by looking at properties set with put for the function. When I looked up browse-url-firefox, after the function definition, there is a line that says (function-put 'browse-url-firefox 'browse-url-browser-kind 'external). If I look up browse-url-generic, I find (function-put 'browse-url-generic 'browse-url-browser-kind 'external) there as well.

When I call browse-url-generic interactively, it works perfectly. When I try to run (browse-url--browser-kind 'browse-url-generic "https://mostad.eu/") it returns external which I also get when running (browse-url--browser-kind browse-url-secondary-browser-function "https://mostad.eu/").

Going back to eww-browse-with-external-browser, when I execute

 (if (eq 'external (browse-url--browser-kind
                     browse-url-secondary-browser-function "https://mostad.eu/"))
      (let ((browse-url-browser-function browse-url-secondary-browser-function))
        (browse-url "https://mostad.eu/"))
    (browse-url-with-browser-kind 'external "https://mostad.eu/"))

it launches mostad.eu in eww, not in librewolf. (This is the part of the function that does the real work, but with url replaced with "https://mostad.eu/" so it works outside the function.) I did a C-x C-e after the if statement and it returned t as expected, so the let should be evaluated. I also tried modifying the else clause to message something if it kicked in and it did not happen.

When I tried the below to see if browse-url-browser-function somehow was something else, then the message buffer showed the even further below output, confirming that it was in fact browse-url-generic.

  (if (eq 'external (browse-url--browser-kind
                     browse-url-secondary-browser-function "https://mostad.eu/"))
      (let ((browse-url-browser-function
             browse-url-secondary-browser-function))
        (message "%s" browse-url-browser-function)
        (browse-url "https://mostad.eu/"))
    (message "neida"))

browse-url-generic
Contacting host: mostad.eu:443
#<buffer  *http mostad.eu:443*>

At this point I was running out of patience and concentration and needed to eat something. Before eating, I remembered when looking through the source code for browse-url-firefox, that it could also open firefox variants if browse-url-firefox-program was set to a firefox downstream, so I tried setting it to "librewolf" and had a go with & in an eww buffer to see if that would work when using & in eww. It did not work. It just reloaded the site I was in in eww.

(use-package eww                        ; Web browser
  :config
  (setopt browse-url-browser-function 'eww-browse-url
          browse-url-secondary-browser-function 'browse-url-firefox
          browse-url-firefox-program "librewolf"
  ; ...

I then reset the setting to browse-url-generic. When trying & in eww, it relaunches the same page in eww. If I set browse-url-secondary-browser-function to any function except the obvious ones like 'browse-url-firefox, 'browse-url-generic etc, for example if I set it to (lambda ()), when I use & in eww, it opens the site in LibreWolf. This seems slightly insane. I went down the rabbit hole and found browse-url-default-browser's source code and it seems to use browse-url-firefox which I previously set up to use librewolf, so maybe that is why this works?

Maybe there is logic saying that browse-url-browser-function and browse-url-secondary-browser-function cannot be the same that stops browse-url from opening the site in librewolf when eww-browse-with-external-browser temporarily sets browse-url-browser-function to browse-url-secondary-browser-function like it does in eww-browse-with-external-browser? I did not see anything like that in the functions I read if I understood them correctly, but I only read some functions and my understanding of Emacs Lisp is still at the beginner level.

I think for now, I will use

(use-package eww                        ; Web browser
  :config
  (setopt browse-url-browser-function 'eww-browse-url
          browse-url-secondary-browser-function (lambda ())
          browse-url-generic-program "librewolf"
  ; ...

which somehow works. I don't know why, but since it works, I'll leave it at that for now.

Is this problem reproducible at all for other people or is it just a problem on my machine? I am on Sway and have set $BROWSER="MOZ_ENABLE_WAYLAND=1 librewolf" in my .bash_profile. I configured Emacs 31 –with-native-compilation=aot –with-cairo –with-harfbuzz –with-gnutls –with-gsettings –with-tree-sitter –with-gif –with-png –with-jpeg –with-rsvg –with-tiff –with-imagemagick –with-pgtk –without-x –with-mailutils

Maybe someone more experienced with Emacs Lisp can look into it further if it is a reproducible problem and clear up why the obvious settings do not work, but other functions works. This could also be user error since I am still learning Emacs Lisp and Emacs in general. I know some things, but there are a lot more things yet for me to learn. Are my settings plain wrong? Have I misunderstood something here?