3
�P\� � @ s� d Z ddlZddlZddlmZ ddlmZ G dd� de�ZG d d
� d
e�Z G d d � d e �Z
G d
d� de �Z dd� Z dd dgZ
dS )zEModule implementing the Pool for :mod:``requests_toolbelt.threaded``.� N� )�thread� )�queuec @ sn e Zd ZdZdddejfdd�Zdd� Zedd� �Z edd d
��Z
d d � Z d
d� Z dd� Z
dd� Zdd� ZdS )�Poola> Pool that manages the threads containing sessions.
:param queue:
The queue you're expected to use to which you should add items.
:type queue: queue.Queue
:param initializer:
Function used to initialize an instance of ``session``.
:type initializer: collections.Callable
:param auth_generator:
Function used to generate new auth credentials for the session.
:type auth_generator: collections.Callable
:param int num_process:
Number of threads to create.
:param session:
:type session: requests.Session
Nc s| |d krt j� pd}|dk r$td��|� _tj� � _tj� � _|� _|pJt � _
|pTt � _ |� _ � fdd�t
� j�D �� _d S )Nr z)Number of processes should at least be 1.c s& g | ] }t j� j� � j� j� j��qS � )r Z
SessionThread� _new_session�
_job_queue�_response_queue�
_exc_queue)�.0�_)�selfr �/usr/lib/python3.6/pool.py�
<listcomp>+ s z!Pool.__init__.<locals>.<listcomp>)�multiprocessing� cpu_count�
ValueErrorr r �Queuer
r Z
_processes� _identity� _initializer�_auth�_session�range�_pool)r � job_queueZ initializerZauth_generatorZ
num_processesZsessionr )r r �__init__ s
z
Pool.__init__c C s | j | j| j� ��S )N)r r r )r r r r r 0 s zPool._new_sessionc K s4 t j� }x|D ]}|j|j� qW | f d|i|��S )a2 Create a :class:`~Pool` from an :class:`~ThreadException`\ s.
Provided an iterable that provides :class:`~ThreadException` objects,
this classmethod will generate a new pool to retry the requests that
caused the exceptions.
:param exceptions:
Iterable that returns :class:`~ThreadException`
:type exceptions: iterable
:param kwargs:
Keyword arguments passed to the :class:`~Pool` initializer.
:returns: An initialized :class:`~Pool` object.
:rtype: :class:`~Pool`
r )r r �put�request_kwargs)�cls�
exceptions�kwargsr �excr r r �from_exceptions3 s
zPool.from_exceptionsc K s^ ddi}|j |pi � tj� }x,|D ]$}|j� }|j d|i� |j|� q$W | f d|i|��S )a� Create a :class:`~Pool` from an iterable of URLs.
:param urls:
Iterable that returns URLs with which we create a pool.
:type urls: iterable
:param dict request_kwargs:
Dictionary of other keyword arguments to provide to the request
method.
:param kwargs:
Keyword arguments passed to the :class:`~Pool` initializer.
:returns: An initialized :class:`~Pool` object.
:rtype: :class:`~Pool`
�methodZGET�urlr )�updater r �copyr )r Zurlsr r! Z request_dictr r% Zjobr r r � from_urlsI s
zPool.from_urlsc c s" x | j � }|dkrP |V qW dS )zoIterate over all the exceptions in the pool.
:returns: Generator of :class:`~ThreadException`
N)�
get_exception)r r" r r r r b s
zPool.exceptionsc C s8 y| j j� \}}W n tjk
r( dS X t||�S dS )zSGet an exception from the pool.
:rtype: :class:`~ThreadException`
N)r �
get_nowaitr �Empty�ThreadException)r �requestr" r r r r) m s
zPool.get_exceptionc C s8 y| j j� \}}W n tjk
r( dS X t||�S dS )zPGet a response from the pool.
:rtype: :class:`~ThreadResponse`
N)r
r* r r+ �ThreadResponse)r r- �responser r r � get_responsey s
zPool.get_responsec c s" x | j � }|dkrP |V qW dS )zmIterate over all the responses in the pool.
:returns: Generator of :class:`~ThreadResponse`
N)r0 )r Zrespr r r � responses� s
zPool.responsesc C s x| j D ] }|j� qW dS )z*Join all the threads to the master thread.N)r �join)r Zsession_threadr r r �join_all� s z
Pool.join_all)N)�__name__�
__module__� __qualname__�__doc__�requestsZSessionr r � classmethodr# r( r r) r0 r1 r3 r r r r r s r c @ s e Zd ZdZdd� ZdS )� ThreadProxyNc C s4 t j}|| jkr&|| | j�}t||�S || |�S dS )z/Proxy attribute accesses to the proxied object.N)�object�__getattribute__�attrs� proxied_attr�getattr)r �attr�getr/ r r r � __getattr__� s
zThreadProxy.__getattr__)r4 r5 r6 r> rB r r r r r: � s r: c @ s( e Zd ZdZdZeddg�Zdd� ZdS )r. a> A wrapper around a requests Response object.
This will proxy most attribute access actions to the Response object. For
example, if you wanted the parsed JSON from the response, you might do:
.. code-block:: python
thread_response = pool.get_response()
json = thread_response.json()
r/ r c C s || _ || _d S )N)r r/ )r r r/ r r r r � s zThreadResponse.__init__N)r4 r5 r6 r7 r> � frozensetr= r r r r r r. � s r. c @ s( e Zd ZdZdZeddg�Zdd� ZdS )r, a= A wrapper around an exception raised during a request.
This will proxy most attribute access actions to the exception object. For
example, if you wanted the message from the exception, you might do:
.. code-block:: python
thread_exc = pool.get_exception()
msg = thread_exc.message
� exceptionr c C s || _ || _d S )N)r rD )r r rD r r r r � s zThreadException.__init__N)r4 r5 r6 r7 r> rC r= r r r r r r, � s r, c C s | S )Nr )Z session_objr r r r � s r )r7 r r8 � r Z_compatr r; r r: r. r, r �__all__r r r r �<module> s