UC Mode stopped handling CF checkboxes

[JimKarvo]

I have an script that running 24/7, no problem till some hours ago.

I updated the SeleniumBase just in case, but seems that there is an update on Cloudflare that detects the UC.

I tryied the example code, with no luck.

"""SB Manager using UC Mode for evading bot-detection."""
from seleniumbase import SB

with SB(uc=True, test=True) as sb:
    url = "https://gitlab.com/users/sign_in"
    sb.driver.uc_open_with_reconnect(url, 3)
    if not sb.is_text_visible("Username", '[for="user_login"]'):
        sb.driver.uc_open_with_reconnect(url, 4)
    sb.assert_text("Username", '[for="user_login"]', timeout=3)
    sb.assert_element('label[for="user_login"]')
    sb.highlight('button:contains("Sign in")')
    sb.highlight('h1:contains("GitLab.com")')
    sb.post_message("SeleniumBase wasn't detected", duration=4)

My working code till some hours ago:

with SB(uc=True) as driver:
      while True:
            if driver.is_element_visible('iframe[src*="challenge"]'):
                loguru.logger.error("have the iframe element.")
                with driver.frame_switch('iframe[src*="challenge"]'):
                    try:
                        driver.click("span.mark")
                    except:
                        pass
                    driver.sleep(1.5)
            else:
                break
            _loop_cnt += 1
            driver.sleep(1.5)
            if _loop_cnt > 20:
                exit()

Maybe somehow I can help with the debugging?

[mdmintz]

This script is working for me at getting past the initial CAPTCHA screen:

from seleniumbase import SB

with SB(uc=True, test=True) as sb:
    url = "https://gitlab.com/users/sign_in"
    sb.driver.uc_open_with_reconnect(url, 3)
    if not sb.is_text_visible("Username", '[for="user_login"]'):
        sb.driver.uc_open_with_reconnect(url, 4)
    sb.assert_text("Username", '[for="user_login"]', timeout=3)
    sb.assert_element('label[for="user_login"]')
    sb.highlight('button:contains("Sign in")')
    sb.highlight('h1:contains("GitLab.com")')
    sb.post_message("SeleniumBase wasn't detected", duration=4)

Is it not working for you?

[oculunto]

I just came to comment the same, it is detecting the selenium/seleniumbase click and uc_click somehow, because disconnecting the driver and clicking manually works
EDIT: I mean the click on the .mark

[mdmintz]

Looks like Cloudflare made a change today.

It's good news that the page-load bypasses still work. That means there will be a way to make the clicks work again too.

Here's a Game of Thrones quote response:

currently_at_work.mp4

[JimKarvo]

It's good news that the page-load bypasses still work. That means there will be a way to make the clicks work again too.

Are there any example code for this?

[mdmintz]

@JimKarvo Of the page-load bypass? That's just: #2626 (comment)

[JimKarvo]

@JimKarvo Of the page-load bypass? That's just: #2626 (comment)

Also this is not working for me (code running on Ubuntu server, no GUI)

[mdmintz]

@JimKarvo Page-load bypasses are working for me on Ubuntu.
Show me some code where it doesn't work for you.

For headless Ubuntu, make sure that you're using:

from seleniumbase import SB

with SB(uc=True, headed=True, xvfb=True) as sb:
    sb.driver.uc_open_with_reconnect(URL, 4)
    # ...

[JimKarvo]

I made this code, for testing:

The first part (to win the CF without click) not worked.

I tryied also to click the box, with no luck (the box is clicked, but I get again and again the same page...)

"""SB Manager using UC Mode for evading bot-detection."""
from seleniumbase import SB

with SB(uc=True, test=True, headed=True, xvfb=True) as sb:
    url = "https://gitlab.com/users/sign_in"
    sb.driver.uc_open_with_reconnect(url, 4)

    #This part of code completelly fails!

    sb.driver.save_screenshot("test-pre.png")
    if not sb.is_text_visible("Username", '[for="user_login"]'):
        sb.driver.uc_open_with_reconnect(url, 4)
    sb.driver.save_screenshot("test-after.png")


    # Trying to click the span, using headed and xvfb
    _loop_cnt = 0
    while True:
        if sb.driver.is_element_visible('iframe[src*="challenge"]'):
            sb.driver.save_screenshot("test-middle.png")
            with sb.frame_switch('iframe[src*="challenge"]'):
                try:
                    print("trying to click...")
                    sb.driver.click("label.ctp-checkbox-label")
                    print("clicked.")
                except Exception as e:
                    print(e)
                    exit()
                sb.driver.sleep(1.5)
        else:
            if sb.driver.get_title() != "Just a moment...":
                print("Finnaly, we won the CF?")
                print(sb.driver.get_title())
                break
            else:
                sb.driver.save_screenshot("test-middle.png")
                print(sb.driver.get_title())
        _loop_cnt += 1
        sb.driver.sleep(5)
        if _loop_cnt > 20:
            exit()

    sb.driver.save_screenshot("test-middle.png")
    sb.post_message("SeleniumBase wasn't detected", duration=4)

[mdmintz]

I'm aware of the click getting detected after the latest CF changes.
But the part with just getting through without doing anything is working correctly for me.

[JimKarvo]

I'm aware of the click getting detected after the latest CF changes. But the part with just getting through without doing anything is working correctly for me.

Even at ubuntu server? (No monitor, no Desktop Enviroment)

[mdmintz]

@JimKarvo What's the output of sb.get_user_agent() for you? I think that may be your issue.

[JimKarvo]

@mdmintz I got this Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36

[mdmintz]

I think I'm close to a solution for macOS and Windows, but I don't have a headed Linux machine to test with. If the User Agent of a Desktop Linux Machine is different from headless Linux, then I think changing the UA could help. Someone will have to check for me.

[ertugrulcakici]

I'm using the same library, and I believe the issue is more than just clicking the checkbox. The problem also involves Cloudflare redirecting me back to the protection page, even when I click manually. I managed to bypass this by opening a new tab on current driver and waiting there for a few seconds. Afterwards, I click the checkbox, try again until it accepts and return to my original page. This method allowed me to successfully bypass on my original page. I hope this helps other bot creators