Skip to content

curl_multi_select did not complete or did timeout when calling curl functions 2 (or more) times on Guzzle or AWS-SDK. #4393

New issue
New issue
Closed
Closed
curl_multi_select did not complete or did timeout when calling curl functions 2 (or more) times on Guzzle or AWS-SDK.#4393
Labels
fixed
@m3m0r7

Description

@m3m0r7
m3m0r7
opened on Sep 7, 2021

Please answer these questions before submitting your issue. Thanks!

1. What did you do? If possible, provide a simple script for reproducing the error.

Pattern 1

I'm using Guzzle. (guzzlehttp/guzzle, v7.3.0)

The below script does timeout curl_multi_select in GuzzleHttp\Handler\CurlMultiHandler::tick at line 161.
And does not complete GuzzleHttp\Handler\CurlMultiHandler::execute at line 175.

<?php
require __DIR__ . '/vendor/autoload.php';

\Co\run(function () {

    \Swoole\Runtime::enableCoroutine(
        true,
        SWOOLE_HOOK_ALL,
    );

    $s = microtime(true);

    $guzzle = new \GuzzleHttp\Client();
    $promises = [
        $guzzle->getAsync('https://www.google.com'),
    ];

    foreach (\GuzzleHttp\Promise\Utils::settle($promises)->wait() as $k => $v) {
        var_dump($k);
    }

    var_dump('--------------------');

    $promises = [
        $guzzle->getAsync('https://www.google.com'),
    ];

    foreach (\GuzzleHttp\Promise\Utils::settle($promises)->wait() as $k => $v) {
        var_dump($k);
    }

    var_dump(microtime(true) - $s);

});

it was entered to infinity loop after output for divider line (did not output int(0) and float(0....) after divider line).

int(0)
string(20) "--------------------"

Pattern 2
I'm using AWS-SDK (aws/aws-sdk-php, 3.192.0).
The below script has a problem at the same above.
However, the script was completed to execute the GuzzleHttp\Handler\CurlMultiHandler::execute but it is very very slow.

<?php

use Aws\S3\S3Client;

require __DIR__ . '/vendor/autoload.php';

\Co\run(function () {

    \Swoole\Runtime::enableCoroutine(
        true,
        SWOOLE_HOOK_ALL,
    );


    $s = microtime(true);

    $s3 = new S3Client([
        'version' => 'latest',
        'region' => 'ap-northeast-1',
        'credentials' => new \Aws\Credentials\Credentials(
            'api-key-is-here',
            'api-secret-is-here'
        ),
    ]);
    var_dump(microtime(true) - $s);

    $s3->listBuckets();
    var_dump(microtime(true) - $s);

    $s3->listBuckets();
    var_dump(microtime(true) - $s);

    $s3->listBuckets();
    var_dump(microtime(true) - $s);
});

The script is very slow to complete when calling 2 or more times.

float(0.67984986305237)
float(1.099485874176)
float(4.3584179878235)
float(7.3696670532227)

2. What did you expect to see?

I want to complete to execute pattern 1 and pattern 2.
As for pattern 2, I hope it will be completed soon.

However, when using PHP 7.4.21 + Swoole 4.6.7, curl functions are executed successfully.
And if I use PHP 7.4.22 (or later), Swoole 4.6.7/4.7.0 (or later) will not work.
(Other than that, I tested that PHP 7.4.21 + Swoole 4.7.0 (or later) will work, but it is very slow to complete pattern 1 and pattern 2.)

I want to expect is here (The below output is using PHP 7.4.21 and Swoole v4.6.7):

Pattern1

int(0)
string(20) "--------------------"
int(0)
float(0.60461807250977)

Pattern2

float(0.76096701622009)
float(1.1838958263397)
float(1.2151129245758)
float(1.2453739643097)

3. What did you see instead?

Pattern 1

int(0)
string(20) "--------------------"

Pattern 2

float(0.67984986305237)
float(1.099485874176)
float(4.3584179878235)
float(7.3696670532227)

4. What version of Swoole are you using (show your php --ri swoole)?

5. What is your machine environment used (show your uname -a & php -v & gcc -v) ?

uname -a

Linux 4d4b79bc6725 5.4.39-linuxkit #1 SMP Fri May 8 23:03:06 UTC 2020 x86_64 GNU/Linux

gcc -v

root@4d4b79bc6725:/var/www/html# gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/10/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none:amdgcn-amdhsa:hsa
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Debian 10.2.1-6' --with-bugurl=file:///usr/share/doc/gcc-10/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-10 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-bootstrap --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-plugin --enable-default-pie --with-system-zlib --enable-libphobos-checking=release --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none=/build/gcc-10-Km9U7s/gcc-10-10.2.1/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-10-Km9U7s/gcc-10-10.2.1/debian/tmp-gcn/usr,hsa --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu --with-build-config=bootstrap-lto-lean --enable-link-mutex
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 10.2.1 20210110 (Debian 10.2.1-6)

The problem pattern

php -v

PHP 7.4.22 (cli) (built: Aug 18 2021 13:24:08) ( ZTS )
Copyright (c) The PHP Group
Zend Engine v3.4.0, Copyright (c) Zend Technologies
    with Zend OPcache v7.4.22, Copyright (c), by Zend Technologies

php --ri swoole

root@4d4b79bc6725:/var/www/html# php --ri swoole

swoole

Swoole => enabled
Author => Swoole Team <[email protected]>
Version => 4.7.0
Built => Sep  7 2021 04:21:32
coroutine => enabled with boost asm context
epoll => enabled
eventfd => enabled
signalfd => enabled
cpu_affinity => enabled
spinlock => enabled
rwlock => enabled
sockets => enabled
openssl => OpenSSL 1.1.1k  25 Mar 2021
dtls => enabled
http2 => enabled
json => enabled
curl-native => enabled
pcre => enabled
c-ares => 1.17.1
zlib => 1.2.11
brotli => E16777225/D16777225
mutex_timedlock => enabled
pthread_barrier => enabled
futex => enabled
async_redis => enabled

Directive => Local Value => Master Value
swoole.enable_coroutine => On => On
swoole.enable_library => On => On
swoole.enable_preemptive_scheduler => Off => Off
swoole.display_errors => On => On
swoole.use_shortname => On => On
swoole.unixsock_buffer_size => 8388608 => 8388608

The successful pattern

php -v

PHP 7.4.21 (cli) (built: Jul 22 2021 03:18:05) ( ZTS )
Copyright (c) The PHP Group
Zend Engine v3.4.0, Copyright (c) Zend Technologies
    with Zend OPcache v7.4.21, Copyright (c), by Zend Technologies

php --ri swoole

root@d42162056364:/var/www/html# php --ri swoole

swoole

Swoole => enabled
Author => Swoole Team <[email protected]>
Version => 4.6.7
Built => Sep  5 2021 10:03:10
coroutine => enabled with boost asm context
epoll => enabled
eventfd => enabled
signalfd => enabled
cpu_affinity => enabled
spinlock => enabled
rwlock => enabled
sockets => enabled
openssl => OpenSSL 1.1.1d  10 Sep 2019
dtls => enabled
http2 => enabled
json => enabled
curl-native => enabled
pcre => enabled
zlib => 1.2.11
mutex_timedlock => enabled
pthread_barrier => enabled
futex => enabled
async_redis => enabled

Directive => Local Value => Master Value
swoole.enable_coroutine => On => On
swoole.enable_library => On => On
swoole.enable_preemptive_scheduler => Off => Off
swoole.display_errors => On => On
swoole.use_shortname => On => On
swoole.unixsock_buffer_size => 8388608 => 8388608

Metadata

Metadata

Assignees

No one assigned

    Labels

    fixed

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions