Модераторы: Aliance, skyboy, MoLeX, ksnk
  

Поиск:

Ответ в темуСоздание новой темы Создание опроса
> Многопоточность: многократная запись, Не могу разобраться 
:(
    Опции темы
CyClon
Дата 6.10.2007, 15:22 (ссылка) | (нет голосов) Загрузка ... Загрузка ... Быстрая цитата Цитата


Опытный
**


Профиль
Группа: Участник
Сообщений: 838
Регистрация: 3.12.2005

Репутация: нет
Всего: 4
Решал написать функцию для получения страничек/файлов в несколько потоков. Получается:

Код
<?php

error_reporting(E_ALL);
ini_set('display_errors', '1');

function getFiles($url, $timeout = 30)
{
    foreach ($url as $id => $purl) {
        $purl = parse_url($purl);
        $purl['port'] = (isset($purl['port'])) ? $purl['port'] : '80';
        if (isset($purl['host']) && $stream = stream_socket_client($purl['host'] . ':' . $purl['port'], $errno, $errstr, $timeout, STREAM_CLIENT_ASYNC_CONNECT | STREAM_CLIENT_CONNECT)) {
            $streams[$id] = $stream;
            $info[$id] = $purl;
        }
    }
    while (sizeof($streams)) {
        $read = $write = $streams;
        if (stream_select($read, $write, $except = NULL, $timeout) > 0) {
            foreach ($write as $sid => $stream) {
                $id = array_search($stream, $streams);
                echo $stream . ' for write ready' . PHP_EOL;
                if (isset($info[$id]['path'])) {
                    $headers = "GET " . $info[$id]['path'] . " HTTP/1.0\r\n";
                } else {
                    $headers = "GET / HTTP/1.0\r\n";
                }
                $headers .= "Accept: */*\r\n";
                $headers .= "Accept-Language: ru\r\n";
                $headers .= "Host: " . $info[$id]['host'] . "\r\n";
                $headers .= "User-Agent: SH-Bot\r\n";
                $headers .= "Connection: Close\r\n\r\n";
                fputs($stream, $headers);
            }
            foreach ($read as $sid => $stream) {
                echo $stream . ' for read ready' . PHP_EOL;
                $id = array_search($stream, $streams);
                $result[$id] = '';
                while (!feof($stream)) {
                    $result[$id] .= fgets($stream);
                }
                fclose($stream);
                unset($streams[$id]);
            }
        } else {
            break;
        }
    }
    return (isset($result)) ? $result : FALSE;
}

if ($result = getFiles(array('http://ya.ru', 'http://mediazona.ru'))) {
    //print_r($result);
}

?>

Результат работы:
Код
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #2 for write ready
Resource id #2 for write ready
Resource id #2 for write ready
Resource id #2 for write ready
Resource id #2 for write ready
Resource id #2 for write ready
Resource id #2 for write ready
Resource id #2 for write ready
Resource id #2 for write ready
Resource id #2 for write ready
Resource id #2 for write ready
Resource id #2 for write ready
Resource id #2 for write ready
Resource id #2 for write ready
Resource id #2 for write ready
Resource id #2 for write ready
Resource id #2 for write ready
Resource id #2 for write ready
Resource id #2 for write ready
Resource id #2 for write ready
Resource id #2 for write ready
Resource id #2 for write ready
Resource id #2 for write ready
Resource id #2 for write ready
Resource id #2 for write ready
Resource id #2 for write ready
Resource id #2 for write ready
Resource id #2 for write ready
Resource id #2 for write ready
Resource id #2 for write ready
Resource id #2 for write ready
Resource id #2 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #2 for read ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for write ready
Resource id #3 for read ready


Но есть один косяк - цикл foreach $write проходится по несколько десятков раз, пока не считаются данные. То есть происходит повторная запись заголовков в поток, который еще не считался. Как предотвратить? Пытался в конце цикла foreach $write делать unset($write[$id]) но не помагает. В общем по разному извращался, пока что сделал так:

Код
<?php

error_reporting(E_ALL);
ini_set('display_errors', '1');

function getFiles($url, $timeout = 30)
{
    foreach ($url as $id => $purl) {
        $purl = parse_url($purl);
        $purl['port'] = (isset($purl['port'])) ? $purl['port'] : '80';
        if (isset($purl['host']) && $stream = stream_socket_client($purl['host'] . ':' . $purl['port'], $errno, $errstr, $timeout, STREAM_CLIENT_ASYNC_CONNECT | STREAM_CLIENT_CONNECT)) {
            $streams[$id] = $stream;
            $info[$id] = $purl;
        }
    }
    $saved = array();
    while (sizeof($streams)) {
        $read = $write = $streams;
        if (stream_select($read, $write, $except = NULL, $timeout) > 0) {
            foreach ($write as $sid => $stream) {
                $id = array_search($stream, $streams);
                if (in_array($id, $saved)) {
                    continue;
                }
                echo $stream . ' for write ready' . PHP_EOL;
                if (isset($info[$id]['path'])) {
                    $headers = "GET " . $info[$id]['path'] . " HTTP/1.0\r\n";
                } else {
                    $headers = "GET / HTTP/1.0\r\n";
                }
                $headers .= "Accept: */*\r\n";
                $headers .= "Accept-Language: ru\r\n";
                $headers .= "Host: " . $info[$id]['host'] . "\r\n";
                $headers .= "User-Agent: SH-Bot\r\n";
                $headers .= "Connection: Close\r\n\r\n";
                fputs($stream, $headers);
                $saved[] = $id;
            }
            foreach ($read as $sid => $stream) {
                echo $stream . ' for read ready' . PHP_EOL;
                $id = array_search($stream, $streams);
                $result[$id] = '';
                while (!feof($stream)) {
                    $result[$id] .= fgets($stream);
                }
                fclose($stream);
                unset($streams[$id]);
            }
        } else {
            break;
        }
    }
    return (isset($result)) ? $result : FALSE;
}

if ($result = getFiles(array('http://ya.ru', 'http://mediazona.ru'))) {
    //print_r($result);
}

?>

Результат работы:
Код
Resource id #3 for write ready
Resource id #2 for write ready
Resource id #2 for read ready
Resource id #3 for read ready


То есть если произошла запись заголовков в поток, то в массив saved добавляется id потока и потом при повторном проходе массива $write данный поток игнорирутеся. Но имхо это далеко не рациональное решение. Должен быть какой-то вариант типа unset($write[$id]), но как его организовать.

ЗЫ: Что делает функция stream_select? Я так понял, возвращает количество потоков доступных для записи или чтения? Но как имеено идет расчет? Ведь на входе - два массива $write и $read.

Это сообщение отредактировал(а) CyClon - 6.10.2007, 15:25


--------------------
user posted image
PM   Вверх
sTa1kEr
Дата 6.10.2007, 16:57 (ссылка) | (нет голосов) Загрузка ... Загрузка ... Быстрая цитата Цитата


9/10 программиста
***


Профиль
Группа: Завсегдатай
Сообщений: 1553
Регистрация: 21.2.2007

Репутация: нет
Всего: 146
CyClon, так вы не помещайте сразу потоки в массивы и на чтение и на запись. Требуется прочитать из потока помещайте в массив на чтение, требуется на запись, соответственно. Я сначала столкнулся с похожими проблемами. Почитайте этот топик http://forum.vingrad.ru/forum/topic-159049.html

Цитата(CyClon @  6.10.2007,  15:22 Найти цитируемый пост)
Что делает функция stream_select? Я так понял, возвращает количество потоков доступных для записи или чтения? Но как имеено идет расчет? Ведь на входе - два массива $write и $read.

В stream_select() вы передаете три массива с потоками. В первом те потоки в которых ожидается вы ожидаете читать данные, во втором те в которые вы хотите посылать данные и третия где ожидакется выброска исключений. Далее stream_select() оставляеет в этимх массивах только те, которые уже ожидают запись/чтение и возращает суммарное количество ожидающих массивов.
PM MAIL   Вверх
  
 Ответ в темуСоздание новой темы Создание опроса
0 Пользователей читают эту тему (0 Гостей и 0 Скрытых Пользователей)
0 Пользователей:
« Предыдущая тема | PHP: Сеть | Следующая тема »

Подписаться на тему | Подписка на этот форум | Скачать/Распечатать тему


 




[ Время генерации скрипта: 0.0481 ]   [ Использовано запросов: 22 ]   [ GZIP включён ]


Реклама на сайте     Информационное спонсорство

 
 По вопросам размещения рекламы пишите на vladimir(sobaka)vingrad.ru
Отказ от ответственности     Powered by Invision Power Board(R) 1.3 © 2003  IPS, Inc.