File: //lib64/python3.6/site-packages/tornado/__pycache__/simple_httpclient.cpython-36.pyc
3
��f�k � @ s� d dl mZ d dlmZ d dlmZmZmZmZm Z m
Z
d dlm Z d dl m
Z
mZ d dlmZ d dlmZmZ d dlmZmZmZmZ d d lmZ d d
lm Z d d l Z d d l Z d d l Z d d l Z d d l!Z!d d l"Z"d d l#Z#d d l$Z$d d l%Z%d d l&m'Z' d d l(Z)d d
l*m+Z+m,Z,m-Z-m.Z.m/Z/m0Z0 d dl1m2Z2 d d l*Z*e*j3�rLd dl*m4Z4m5Z5m6Z6 G dd� de�Z7G dd� de�Z8G dd� de�Z9G dd� de j:�Z;e<dk�r�ej=e9� e� d S )� )�_unicode)�gen)� HTTPResponse� HTTPError�AsyncHTTPClient�main�
_RequestProxy� HTTPRequest)�httputil)�HTTP1Connection�HTTP1ConnectionParameters)�IOLoop)�StreamClosedError�IOStream)�Resolver�OverrideResolver�_client_ssl_defaults� is_valid_ip)�gen_log)� TCPClientN)�BytesIO)�Dict�Any�Callable�Optional�Type�Union)�
TracebackType)�Deque�Tuple�Listc s6 e Zd ZdZedd�� fdd� Zed�dd�Z� ZS ) �HTTPTimeoutErrorz�Error raised by SimpleAsyncHTTPClient on timeout.
For historical reasons, this is a subclass of `.HTTPClientError`
which simulates a response code of 599.
.. versionadded:: 5.1
N)�message�returnc s t t| �jd|d� d S )NiW )r" )�superr! �__init__)�selfr" )� __class__� �)/usr/lib64/python3.6/simple_httpclient.pyr% 5 s zHTTPTimeoutError.__init__)r# c C s
| j pdS )N�Timeout)r" )r& r( r( r) �__str__8 s zHTTPTimeoutError.__str__)�__name__�
__module__� __qualname__�__doc__�strr% r+ �
__classcell__r( r( )r' r) r! , s r! c s6 e Zd ZdZedd�� fdd� Zed�dd�Z� ZS ) �HTTPStreamClosedErroraZ Error raised by SimpleAsyncHTTPClient when the underlying stream is closed.
When a more specific exception is available (such as `ConnectionResetError`),
it may be raised instead of this one.
For historical reasons, this is a subclass of `.HTTPClientError`
which simulates a response code of 599.
.. versionadded:: 5.1
N)r" r# c s t t| �jd|d� d S )NiW )r" )r$ r2 r% )r& r" )r' r( r) r% H s z HTTPStreamClosedError.__init__)r# c C s
| j pdS )Nz
Stream closed)r" )r& r( r( r) r+ K s z HTTPStreamClosedError.__str__)r, r- r. r/ r0 r% r+ r1 r( r( )r' r) r2 < s
r2 c
s� e Zd ZdZd eeeef eeeeef eedd�� fdd�
Z dd�� fd d
� Z
e e e
gdf dd �d d
�Zdd�dd�Zed�dd�Ze e g df e e
gdf dd�dd�Zedd�dd�Zedd�dd�Zd eedd�dd �Z� ZS ) �SimpleAsyncHTTPClientai Non-blocking HTTP client with no external dependencies.
This class implements an HTTP 1.1 client on top of Tornado's IOStreams.
Some features found in the curl-based AsyncHTTPClient are not yet
supported. In particular, proxies are not supported, connections
are not reused, and callers cannot select the network interface to be
used.
�
N� @)� max_clients�hostname_mapping�max_buffer_size�resolver�defaults�max_header_size�
max_body_sizer# c s� t t| �j|d� || _tj� | _i | _i | _|| _ || _
|| _ |rR|| _ d| _
nt� | _ d| _
|dk rxt| j |d�| _ t| j d�| _dS )a Creates a AsyncHTTPClient.
Only a single AsyncHTTPClient instance exists per IOLoop
in order to provide limitations on the number of pending connections.
``force_instance=True`` may be used to suppress this behavior.
Note that because of this implicit reuse, unless ``force_instance``
is used, only the first call to the constructor actually uses
its arguments. It is recommended to use the ``configure`` method
instead of the constructor to ensure that arguments take effect.
``max_clients`` is the number of concurrent requests that can be
in progress; when this limit is reached additional requests will be
queued. Note that time spent waiting in this queue still counts
against the ``request_timeout``.
``hostname_mapping`` is a dictionary mapping hostnames to IP addresses.
It can be used to make local DNS changes when modifying system-wide
settings like ``/etc/hosts`` is not possible or desirable (e.g. in
unittests).
``max_buffer_size`` (default 100MB) is the number of bytes
that can be read into memory at once. ``max_body_size``
(defaults to ``max_buffer_size``) is the largest response body
that the client will accept. Without a
``streaming_callback``, the smaller of these two limits
applies; with a ``streaming_callback`` only ``max_body_size``
does.
.. versionchanged:: 4.2
Added the ``max_body_size`` argument.
)r: FTN)r9 �mapping)r9 )r$ r3 �
initializer6 � collections�deque�queue�active�waitingr8 r; r< r9 � own_resolverr r r �
tcp_client)r& r6 r7 r8 r9 r: r; r< )r' r( r) r> Y s" *
z SimpleAsyncHTTPClient.initialize)r# c s, t t| �j� | jr | jj� | jj� d S )N)r$ r3 �closerD r9 rE )r& )r' r( r) rF � s
zSimpleAsyncHTTPClient.close)�request�callbackr# c C s� t � }| jj|||f� t| j�| jk st|jd k s6t�|jd k sDt�| j j
| j j � t |j|j� t
j| j|d��}nd }|||f| j|< | j� | jr�tjdt| j�t| j�f � d S )Nzin request queuezImax_clients limit reached, request queued. %d active, %d queued requests.)�objectrA �append�lenrB r6 �connect_timeout�AssertionError�request_timeout�io_loop� add_timeout�time�min� functools�partial� _on_timeoutrC �_process_queuer �debug)r& rG rH �key�timeout_handler( r( r) �
fetch_impl� s z SimpleAsyncHTTPClient.fetch_implc C sp xj| j rjt| j�| jk rj| j j� \}}}|| jkr4q| j|� ||f| j|< tj| j |�}| j
|||� qW d S )N) rA rK rB r6 �popleftrC �_remove_timeoutrS rT �_release_fetch�_handle_request)r& rX rG rH �release_callbackr( r( r) rV � s
z$SimpleAsyncHTTPClient._process_queuec C s t S )N)�_HTTPConnection)r& r( r( r) �_connection_class� s z'SimpleAsyncHTTPClient._connection_class)rG r_ �final_callbackr# c C s& | j � | |||| j| j| j| j� d S )N)ra r8 rE r; r< )r& rG r_ rb r( r( r) r^ � s z%SimpleAsyncHTTPClient._handle_request)rX r# c C s | j |= | j� d S )N)rB rV )r& rX r( r( r) r] � s z$SimpleAsyncHTTPClient._release_fetchc C s: || j kr6| j | \}}}|d k r.| jj|� | j |= d S )N)rC rO �remove_timeout)r& rX rG rH rY r( r( r) r\ � s
z%SimpleAsyncHTTPClient._remove_timeout)rX �infor# c C sn | j | \}}}| jj|||f� |r0dj|�nd}t|dt|�| jj� |j d�}| jj ||� | j |= dS )z�Timeout callback of request.
Construct a timeout HTTPResponse when a timeout occurs.
:arg object key: A simple object to mark the request.
:info string key: More detailed timeout information.
z Timeout {0}r* iW )�error� request_timeN)
rC rA �remove�formatr r! rO rQ �
start_time� add_callback)r& rX rd rG rH rY �
error_messageZtimeout_responser( r( r) rU � s z!SimpleAsyncHTTPClient._on_timeout)r4 Nr5 NNNN)N)r, r- r. r/ �intr r0 r r r> rF r r r rZ rV �typera r^ rI r] r\ rU r1 r( r( )r' r) r3 O s* =
r3 c
@ sj e Zd Zedddddddg�Zee eeg df ee gdf e
e e
e
dd � d
d �Z dd �d
d�Z
eedeeef ejf d�dd�Zd4edd�dd�Zdd �dd�Zeed�dd�Zedd�dd �Zdd �d d �Z e dd �d d!�Z d"ee ee ed#�d$d%�Z dd �d&d'�Z!ee"j#e"j$f e"j%dd(�d)d*�Z&ed �d+d,�Z'dd �d-d.�Z(dd �d/d0�Z)e*dd1�d2d3�Z+dS )5r` �GETZHEAD�POST�PUTZDELETE�PATCHZOPTIONSN) �clientrG r_ rb r8 rE r; r< r# c C s� t j� | _| jj� | _tj� | _|| _|| _|| _|| _ || _
|| _ || _ || _
d | _d | _g | _d | _d | _d | _t j� jtj| j� �dd� � d S )Nc S s | j � S )N)�result)�fr( r( r) �<lambda> s z*_HTTPConnection.__init__.<locals>.<lambda>)r
ZcurrentrO rQ ri �start_wall_timerr rG r_ rb r8 rE r; r< �code�headers�chunksZ
_decompressor�_timeout� _sockaddrZ
add_futurer Zconvert_yielded�run) r& rr rG r_ rb r8 rE r; r< r( r( r) r% s&
z_HTTPConnection.__init__)r# c � s� �y�t jjt| jj��| _| jjd-kr6td| jj ��| jj }d|krV|j
d�\}}}t j |�\}}|d kr�| jjdkr|dnd}t
jd|�r�|dd.� }|| _| jjd kr�tj}ntj}| j| jj�}d }| jj� r�t| jj�r�| jj}ntd
| jjf ��t| jj| jj�} | �r�| jj| j | t j | j d ��| _ | j!j"||||| j#|d �I d H }
| j$d k�rl|
j%� d S |
| _&| j&j'| j(� | j)� | j$d k�r�d S | jj�r�| jj| j | jj t j | j d
��| _ | jj*| j+k�r�| jj, �r�t-d| jj* ��x*d/D ]"} t.| j| d ��r�t/d| ���q�W d| jj0k�r8d| jj0d< d| jj0k�r�d| jj k�rp| jj j
d�d0 | jj0d< n| jj | jj0d< d1\} }
| jj1d k �r�| jj1| jj2 } }
n$| jj3d k �r�| jj3} | jj4�p�d}
| d k �r$|
d k �s�t5�| jj6d2k�rtd| jj6��dtt7j8t j9| |
��� | jj0d < | jj:�r>| jj:| jj0d < | jj,�s�| jj*d3k}| jj;d k �pl| jj<d k }|�r|| �s�|�r�| �r�td!|�r�d"nd| jj*f ��| jj=�r�d#| jj0d$< | jj;d k �r�t>t?| jj;��| jj0d%< | jj*d k�r d&| jj0k�r d'| jj0d&< | jj@�r"d(| jj0d)< | jjA�p.d*| jjB�rDd+| jjB nd }| jC|
�| _Dt jE| jj*|d�}| jDjF|| jj0� | jj=�r�| jDjG| �I d H n| jHd,�I d H W n( tIk
�r� | jJtKjL� � �s Y nX d S )4N�http�httpszUnsupported url scheme: %s�@i� �P z^\[.*\]$� Fz?Unrecognized IPv4 or IPv6 address for network_interface, got %rzwhile connecting)�af� ssl_optionsr8 � source_ipzduring requestzunknown method %s�
proxy_host�
proxy_port�proxy_username�proxy_password�proxy_auth_modez%s not supportedZ
ConnectionrF �Host� �basiczunsupported auth_mode %szBasic Z
Authorizationz
User-Agentro rq rp zLBody must %sbe None for method %s (unless allow_nonstandard_methods is true)znot z 100-continueZExpectzContent-Lengthz Content-Typez!application/x-www-form-urlencodedZgzipzAccept-Encoding�/�?T)r} r~ ���)r� r� r� r� r� r� )NN)Nr� )ro rq rp )M�urllib�parseZurlsplitr rG �urlZparsed�scheme�
ValueError�netloc�
rpartitionr
Zsplit_host_and_port�re�matchZparsed_hostnameZ
allow_ipv6�socketZAF_INETZ AF_UNSPEC�_get_ssl_optionsZnetwork_interfacer rR rL rN rO rP ri rS rT rU rz rE Zconnectr8 rb rF �streamZset_close_callback�on_connection_closer\ �method�_SUPPORTED_METHODSZallow_nonstandard_methods�KeyError�getattr�NotImplementedErrorrx �username�passwordZ
auth_usernameZ
auth_passwordrM Z auth_mode�base64Z b64encodeZencode_username_passwordZ
user_agent�body�
body_producer�expect_100_continuer0 rK �decompress_response�pathZquery�_create_connection�
connection�RequestStartLineZ
write_headers�
read_response� _write_body� Exception�_handle_exception�sys�exc_info)r& r� Zuserpass�_�hostZportr� r� r� Ztimeoutr� rX r� r� Z
body_expectedZ body_presentZreq_pathZ
start_liner( r( r) r| ! s�
z_HTTPConnection.run)r� r# c C s� |dkr�| j jd k r | j jS | j jrL| j jd krL| j jd krL| j jd krLtS tjtj j
| j jd�}| j jsxd|_ tj |_
| j jd k r�|j| j j| j j� ttd�r�| jtjO _|S d S )Nr~ )ZcafileF�OP_NO_COMPRESSION)rG r� Z
validate_certZca_certsZ client_certZ
client_keyr �sslZcreate_default_contextZPurposeZ SERVER_AUTHZcheck_hostnameZ CERT_NONEZ verify_modeZload_cert_chain�hasattrZoptionsr� )r& r� Zssl_ctxr( r( r) r� � s(
z _HTTPConnection._get_ssl_options)rd r# c C s8 d| _ |rdj|�nd}| jdk r4| jtt|�d� dS )z�Timeout callback of _HTTPConnection instance.
Raise a `HTTPTimeoutError` when a timeout occurs.
:info string key: More detailed timeout information.
Nz Timeout {0}r* )rz rh rb r� r! )r& rd rk r( r( r) rU � s
z_HTTPConnection._on_timeoutc C s" | j d k r | jj| j � d | _ d S )N)rz rO rc )r& r( r( r) r\ � s
z _HTTPConnection._remove_timeout)r� r# c C s6 |j d� t|dtd| j| jt| jj�d�| j�}|S )NT)Z
no_keep_aliver; r< Z
decompress) Z set_nodelayr r r; r< �boolrG r� r{ )r& r� r� r( r( r) r� � s
z"_HTTPConnection._create_connection)�
start_readr# c � s� | j jd k r | jj| j j� n.| j jd k rL| j j| jj�}|d k rL|I d H | jj� |r�y| jj| �I d H W n$ tk
r� | jt j
� � s�� Y nX d S )N) rG r� r� �writer� �finishr� r r� r� r� )r& r� �futr( r( r) r� � s
z_HTTPConnection._write_bodyc C s | j d k r | j }d | _ |� d S )N)r_ )r& r_ r( r( r) �_release s
z_HTTPConnection._release)�responser# c C s0 | j � | jd k r,| j}d | _| jj||� d S )N)r� rb rO rj )r& r� rb r( r( r) �
_run_callback s
z _HTTPConnection._run_callbackz Optional[Type[BaseException]])�typ�value�tbr# c C s� | j rr| j� t|t�r2|jd kr,td�}n|j}| jt| jd|| j j
� | j | j d�� t
| d�rn| jj� dS t|t�S d S )Nz
Stream closediW )re rf ri r� T)rb r\ �
isinstancer Z
real_errorr2 r� r rG rO rQ ri rv r� r� rF )r& r� r� r� r( r( r) r� s"
z!_HTTPConnection._handle_exceptionc C sR | j d k rNd}| jjr | jj�y t|��W n" tk
rL | jtj� � Y nX d S )NzConnection closed)rb r� re r2 r� r� r� )r& r" r( r( r) r� < s
z#_HTTPConnection.on_connection_close)�
first_linerx r# c � s� t |tj�st�| jjr6|jdkr6| jd�I d H d S |j| _|j| _|| _ | j
� rXd S | jj d k r�| jj d| � x*| j j � D ] \}}| jj d||f � q�W | jj d� d S )N�d Fz
%s %s %s
z%s: %s
z
)
r� r
�ResponseStartLinerM rG r� rw r� �reasonrx �_should_follow_redirectZheader_callbackZget_all)r& r� rx �k�vr( r( r) �headers_receivedF s z _HTTPConnection.headers_receivedc C sL | j jrH| j jd k st�| jd koF| j jdkoF| jd k oF| jjd�d k S dS )
N�- �. �/ �3 �4 r �LocationF)r� r� r� r� r� )rG Zfollow_redirects�
max_redirectsrM rw rx �get)r& r( r( r) r� ] s
z'_HTTPConnection._should_follow_redirectc s� | j d k st�dj| j�}| j� t| jd| j�}| j� �r&t| jt �sLt�t
j
| jj�}t j j
| jj| jd �|_| jjd |_|jd= | j dkr�| jjd kr�d
|_d |_x0dD ](}y| jj|= W q� tk
r� Y q�X q�W ||_| j� d | _| j� | jj|dd�}|j� fdd�� | j� d S | jj�r8t � }nt |�}t || j t| dd �| j| j j � | j | j!|| jjd�}| j"|� | j� d S )N� �original_requestr� r� r� �- �. �/ ro rn �Content-Length� Content-Type�Content-Encoding�Transfer-EncodingF)Z raise_errorc s � | j � �S )N)rs )rt )rb r( r) ru � s z(_HTTPConnection.finish.<locals>.<lambda>r� )r� rx rf ri �bufferZ
effective_url)r� r� r� )r� r� r� r� )#rw rM �joinry r\ r� rG r� r� r �copyr� r� Zurljoinr� rx r� r� r� r� r� rb r� rr ZfetchZadd_done_callback�_on_end_request�streaming_callbackr r rO rQ ri rv r� )r& �datar� Z new_request�hr� r� r� r( )rb r) r� h sZ
z_HTTPConnection.finishc C s | j j� d S )N)r� rF )r& r( r( r) r� � s z _HTTPConnection._on_end_request)�chunkr# c C s6 | j � r d S | jjd k r&| jj|� n | jj|� d S )N)r� rG r� ry rJ )r& r� r( r( r) �
data_received� s
z _HTTPConnection.data_received)N),r, r- r. �setr� r r3 r r r rl r r% r| r0 r r r r� Z
SSLContextr� rU r\ r r r� r� r� r� r� �
BaseExceptionr r� r� r
r� r� Z HTTPHeadersr� r� r� r� �bytesr� r( r( r( r) r` � sB
" =r` �__main__)>Ztornado.escaper Ztornador Ztornado.httpclientr r r r r r r
Ztornado.http1connectionr r Ztornado.ioloopr
Ztornado.iostreamr r Ztornado.netutilr r r r Z tornado.logr Ztornado.tcpclientr r� r? r� rS r� r� r� r� rQ �ior Z urllib.parser� Ztypingr r r r r r �typesr Z
TYPE_CHECKINGr r r r! r2 r3 ZHTTPMessageDelegater` r, Z configurer( r( r( r) �<module> sH - :