3
�~�f� � @ s� d dl Z d dlZd dlZd dlZd dlZd dlZd dlZd dlmZ d dl m
Z
m Z d dl m
Z
mZmZmZmZ d dlmZmZmZmZmZ d dlmZ d dlmZ d dlm Z d d l m Z d d
l m Z m!Z! d d l"m#Z# d d l$m%Z% d d
l&m'Z' d dl(m)Z) dZ*dZ+dZ,dZ-e*e+e,gZ.dZ/dZ0dZ1dZ2dZ3dZ4dZ5dZ6ej7e8�Z9dd d � fd d d � fd d!d � fd"�Z:e G d#d$� d$e
��Z;G d%d&� d&e<�Z=G d'd(� d(e<�Z>G d)d*� d*e<�Z?d+f fd,d-�Z@e3fd.d/�ZAed0d1d2d3d4g�ZBed5d6d7g�ZCG d8d� de'e jDd9�ZEd:d;� ZFeeEeGf d<�d=d>�ZHd?d@� ZIdVeGeJdB�dCdD�ZKdEdF� ZLdWdHdI�ZMG dJdK� dKeN�ZOdLdM� ZPeEeGeJdN�dOdP�ZQeGeeE dQ�dRdS�ZReGd<�dTdU�ZSdS )X� N)�
namedtuple)�Enum�unique)�Any�Dict�List�Optional�Tuple)�
atomic_helper�dmi�importer�net�
type_utils)� user_data)�util)�
write_json)�Distro)�
EventScope� EventType)� launch_index)�Paths)�CloudInitPickleMixin)�eventsZdisabled�localr
�passZ
FILESYSTEM�NETWORK�
DataSourcez|EXPERIMENTAL: The structure and format of content scoped under the 'ds' key may change in subsequent releases of cloud-init.zredacted for non-root userz
cloud-nameZ_unset�unknownz aws-chinac C s | dkS )N�aws� )�cr r � /usr/lib/python3.6/__init__.py�<lambda>? s r" zaws-govc C s | dkS )Nr r )r r r r! r" @ s z azure-chinac C s | dkS )NZazurer )r r r r! r" A s )zcn-zus-gov-Zchinac @ s2 e Zd ZdZdZdZdZdZdZe d�dd �Z
d
S ) �NetworkConfigSourcezb
Represents the canonical list of network config sources that cloud-init
knows about.
�cmdline�dsZ
system_cfgZfallbackZ initramfs)�returnc C s | j S )N)�value)�selfr r r! �__str__R s zNetworkConfigSource.__str__N) �__name__�
__module__� __qualname__�__doc__�CMD_LINE�DS�
SYSTEM_CFGZFALLBACK� INITRAMFS�strr) r r r r! r# E s r# c @ s e Zd ZdZdS )� DatasourceUnpickleUserDataErrorzERaised when userdata is unable to be unpickled due to python upgradesN)r* r+ r, r- r r r r! r3 V s r3 c @ s e Zd ZdS )�DataSourceNotFoundExceptionN)r* r+ r, r r r r! r4 Z s r4 c @ s e Zd ZdZdS )�InvalidMetaDataExceptionz8Raised when metadata is broken, unavailable or disabled.N)r* r+ r, r- r r r r! r5 ^ s r5 � c
C s� t j| �}g }g }x�| j� D ]�\}}|r6|d | }n|}|j� |ksR|j� |kr\|j|� t|t�r�|jd�r�|j|� |jdd�||< t|t �r t
|||�} |j | j d�� |j | j d�� | ||< q W t
|�|d< t
|�|d< |S )z�Process all instance metadata cleaning it up for persisting as json.
Strip ci-b64 prefix and catalog any 'base64_encoded_keys' as a list
@return Dict copy of processed metadata.
�/zci-b64:r6 �base64_encoded_keys�sensitive_keys)�copy�deepcopy�items�lower�append�
isinstancer2 �
startswith�replace�dict�process_instance_metadata�extend�pop�sorted)
�metadata�key_pathr9 �md_copyr8 Z sens_keys�key�valZ sub_key_pathZ
return_valr r r! rC b s,
rC c C s� | j dg �s| S tj| �}xh| j d�D ]Z}|jd�}|}x6|D ].}||kr>t|| t�r>||d kr>|| }q>W ||kr&|||< q&W |S )z�Redact any sensitive keys from to provided metadata dictionary.
Replace any keys values listed in 'sensitive_keys' with redact_value.
r9 r7 � ���)�getr: r; �splitr? rB )rG Z redact_valuerI rH �
path_parts�obj�pathr r r! �redact_sensitive_keys� s
rS � URLParamsZmax_wait_secondsZtimeout_secondsZ num_retries�sec_between_retries�DataSourceHostname�hostname�
is_defaultc
@ s� e Zd ZU eZdZdZdZdZdZ dZ
e j e j
e je jfZee df d|ZdZdZdZejejejejej hiZ ejejhiZ de fd e fd
i fd}d~dd�d�d�f Z eee!e"f df dZ#d�Z$ee!df $dZ%dZ&d�e'e(d�dd �Z)e*dd �d d �Z+d d!� Z,e-d"�d#d$�Z.e-d"�d%d&�Z/d'd(� Z0d)d*� Z1f fd+d,�Z2e-d"�d-d.�Z3d�d0d1�Z4e-d"�d2d3�Z5d4d5� Z6d�d6d7�Z7d8d9� Z8d:d;� Z9e:d<d=� �Z;e:d>d?� �Z<e:d@dA� �Z=dBdC� Z>e:dDdE� �Z?dFdG� Z@e:dHdI� �ZAdJdK� ZBe:dLdM� �ZCdNdO� ZDdPdQ� ZEdRdS� ZFdTdU� ZGdVdW� ZHdXdY� ZIdZd[� ZJd\d]� ZKd^d_� ZLe:d`da� �ZMe:dbdc� �ZNddde� ZOd�dfdg�ZPdhdi� ZQeRe dj�dkdl�ZSeRe e-dm�dndo�ZTdpdq� ZUdrds� ZVeWd�dtdu��ZXe:dvdw� �ZYdxdy� ZZdzd{� Z[dS )�r z en_US.UTF-8Z_undefN.rL �
� � ec2_metadata� network_jsonrG �userdata� userdata_raw�
vendordata�vendordata_raw� vendordata2�vendordata2_rawF�combined_cloud_config�
merged_cfg�merged_system_cfg�security-credentials� user-datar � vendor-data�ds/vendor_data)�distro�pathsc C s~ || _ || _|| _d | _i | _d | _d | _d | _d | _d | _ t
j | j d| j fi �| _
| j
s`i | _
|sttj| j�| _n|| _d S )NZ
datasource)�sys_cfgrj rk r] rG r^ r_ ra r` rb r Zget_cfg_by_path�dsname�ds_cfg�udZUserDataProcessor�ud_proc)r( rl rj rk rp r r r! �__init__ s" zDataSource.__init__)�ci_pkl_versionr& c C s� t | d�sd| _t | d�s d| _t | d�s0d| _t | d�sJt| ddd� � t | d �r�| jdk r�yt| j� W n6 tk
r� } ztj d
|� t
� |�W Y dd}~X nX dS ) z(Perform deserialization fixes for Paths.ra Nrb �skip_hotplug_detectF� check_if_fallback_is_allowedc S s dS )NFr r r r r! r" = s z&DataSource._unpickle.<locals>.<lambda>r] z:Unable to unpickle datasource: %s. Ignoring current cache.) �hasattrra rb rs �setattrr] r2 �AttributeError�LOG�debugr3 )r( rr �er r r! � _unpickle3 s
zDataSource._unpicklec C s
t j| �S )N)r �obj_name)r( r r r! r) O s zDataSource.__str__)r& c C s dS )z#Check if running on this datasourceTr )r( r r r! � ds_detectR s zDataSource.ds_detectc C sN | j j� t� j� kr$tjd| � dS | jjdg �| j gkrJtjd| � dS dS )aI Override if either:
- only a single datasource defined (nothing to fall back to)
- commandline argument is used (ci.ds=OpenStack)
Note: get_cmdline() is required for the general case - when ds-identify
does not run, _something_ needs to detect the kernel command line
definition.
zOMachine is configured by the kernel commandline to run on single datasource %s.TZdatasource_listz5Machine is configured to run on single datasource %s.F)rm r= �
parse_cmdlinerx ry rl rN )r( r r r! �override_ds_detectV s z DataSource.override_ds_detectc C s@ | j � r| j� S | j� r,tjd| � | j� S tjd| � dS dS )z&Overrides runtime datasource detectionz8Detected platform: %s. Checking for active instance dataz#Datasource type %s is not detected.FN)r � _get_datar} rx ry )r( r r r! �_check_and_get_datam s z DataSource._check_and_get_datac C s� | j � j}| j� }| j}|d }ddg||t| j| j| j�| j| j|d d |d d |d d | j| j� |d |||d d |||d d
| j| j |d |d d
�iS )z2Return a dictionary of standardized metadata keys.�sys_info�v1� subplatformZdistr rL � �python�uname� �platform�variant)Z
_beta_keyszavailability-zone�availability_zone�cloud_idz
cloud-name�
cloud_namerj Zdistro_versionZdistro_releaser� Zpublic_ssh_keysZpython_versionz instance-id� instance_idZkernel_releasezlocal-hostname�local_hostname�machine�regionr� Zsystem_platformr� )
� get_hostnamerW �get_instance_idr� �canonical_cloud_idr� r� �
platform_type�get_public_ssh_keysr� )r( �
instance_datar� r� r� Zsysinfor r r! �_get_standardized_metadata{ s8
z%DataSource._get_standardized_metadatac C sP | j s
dS |r|}n| j}x&|D ] \}}t| |�r t| ||� q W |sLd| _ dS )z�Reset any cached metadata attributes to datasource defaults.
@param attr_defaults: Optional tuple of (attr, value) pairs to
set instead of cached_attr_defaults.
NF)� _dirty_cache�cached_attr_defaultsru rv )r( Z
attr_defaultsZ attr_valuesZ attributer' r r r! �clear_cached_attrs� s
z DataSource.clear_cached_attrsc C s" d| _ | j� }|s|S | j� |S )z�Datasources implement _get_data to setup metadata and userdata_raw.
Minimally, the datasource should return a boolean True on success.
T)r� r� �persist_instance_data)r( Z return_valuer r r! �get_data� s zDataSource.get_dataTc C st |r&t jj| jj�r&t| | jjd�� t| d�rbtj t
| d��}|j dd� |j dd� d|i}n^dd| j ii}t| d�r�t
| d�}|t
kr�||d d< t| d �r�t
| d �}|t
kr�||d d < t|d d
< tj | j�|d < d |d d
< tj |d �|d
< d|d
d
< tj� |d< |j| j|�� y"tj|�}ttj|�| jd�}W nb tk
�rv } ztj dt |�� dS d}~X n2 t k
�r� } ztj dt |�� dS d}~X nX | jj d�} |d j dd�}
t jj!| jj"d�} tj#| � d|
� �|
� d�� d} | � d|
� �}
t jj$| ��r t jj%| �} tj&|
| dd� | �rH| |
k�rHtj'| � t(| |d d � | jj d �}t(|t)|�� dS ) aP Process and write INSTANCE_JSON_FILE with all instance metadata.
Replace any hyphens with underscores in key names for use in template
processing.
:param write_cache: boolean set True to persist obj.pkl when
instance_link exists.
@return True on successful write, False otherwise.
Zobj_pklZ_crawled_metadataz user-dataNz vendor-datar% Z meta_datar\ r[ Z_docrd z<DEPRECATED: Use merged_system_cfg. Will be dropped from 24.1re zUMerged cloud-init system config from /etc/cloud/cloud.cfg and /etc/cloud/cloud.cfg.d/r� )r9 z'Error persisting instance-data.json: %sFZinstance_data_sensitiver� r� Znonezcloud-id�-�
T)�forcei� )�moder� )*�osrR �lexistsrk Z
instance_link� pkl_storeZ
get_ipath_curru r: r; �getattrrE rG �UNSET�EXPERIMENTAL_TEXTrl r Z system_info�updater� r
Z
json_dumpsrC �json�loads�sensitive_metadata_keys� TypeErrorrx �warningr2 �UnicodeDecodeErrorZ get_runpathrN �joinZrun_dir�
write_file�exists�realpathZsym_linkZdel_filer rS )r( Z write_cacheZcrawled_metadatar� r\ r[ ZcontentZprocessed_datarz Zjson_sensitive_filer� Z
cloud_id_fileZprev_cloud_id_fileZnew_cloud_id_fileZ json_filer r r! r� � sf
z DataSource.persist_instance_datac C s t d��dS )z@Walk metadata sources, process crawled data and save attributes.zlSubclasses of DataSource must implement _get_data which sets self.metadata, vendordata_raw and userdata_raw.N)�NotImplementedError)r( r r r! r� s zDataSource._get_datac % C sL | j }yt| jjd| j ��}W n, tk
rJ tjtd| jjd�|� Y nX | j}y t dt| jjd| j���}W n2 tk
r� | j}tjtd| jjd�|� Y nX | j
}yt| jjd| j
��}W n, t k
r� tjtd| jjd�|� Y nX | j }yt| jjd| j ��}W n. t k
�r< tjtd | jjd�|� Y nX t
||||�S )
z�Return the Datasource's preferred url_read parameters.
Subclasses may override url_max_wait, url_timeout, url_retries.
@return: A URLParams object with max_wait_seconds, timeout_seconds,
num_retries.
�max_waitz6Config max_wait '%s' is not an int, using default '%s'r �timeoutz5Config timeout '%s' is not an int, using default '%s'�retriesz5Config retries '%s' is not an int, using default '%s'rU zAConfig sec_between_retries '%s' is not an int, using default '%s')� url_max_wait�intrn rN �
ValueErrorr �logexcrx � url_timeout�max� url_retries� Exception�url_sec_between_retriesrT )r( r� r� r� rU r r r! �get_url_params sP
zDataSource.get_url_paramsc C s2 | j d kr | jj| j� �| _ |r,| j| j �S | j S )N)r] rp �process�get_userdata_raw�
_filter_xdata)r( Z apply_filterr r r! � get_userdataU s
zDataSource.get_userdatac C s" | j d kr | jj| j� �| _ | j S )N)r_ rp r� �get_vendordata_raw)r( r r r! �get_vendordata\ s
zDataSource.get_vendordatac C s" | j d kr | jj| j� �| _ | j S )N)ra rp r� �get_vendordata2_raw)r( r r r! �get_vendordata2a s
zDataSource.get_vendordata2c C s2 | j dkr,tj� | _ | j dkr,tjd| j� | j S )zADetermine the network interface used during local network config.Nz(Did not find a fallback interface on %s.)�_fallback_interfacer
Zfind_fallback_nicrx r� r� )r( r r r! �fallback_interfacef s
z DataSource.fallback_interfacec C s. t | d�s| jj� | _| js(| jj� | _| jS )N�_platform_type)ru rm r= r� )r( r r r! r� q s
zDataSource.platform_typec C s* t | d�s| j� | _| js$| j� | _| jS )a� Return a string representing subplatform details for the datasource.
This should be guidance for where the metadata is sourced.
Examples of this on different clouds:
ec2: metadata (http://169.254.169.254)
openstack: configdrive (/dev/path)
openstack: metadata (http://169.254.169.254)
nocloud: seed-dir (/seed/dir/path)
lxd: nocloud (/seed/dir/path)
� _subplatform)ru �_get_subplatformr� )r( r r r! r� z s
zDataSource.subplatformc C s t | d�rdt| d� S tS )z?Subclasses should implement to return a "slug (detail)" string.Zmetadata_addressz
metadata (%s))ru r� �METADATA_UNKNOWN)r( r r r! r� � s
zDataSource._get_subplatformc C sv | j r | j S | jrb| jjt�rb| jjt�}t|t�r@|j� | _ qp| j� j� | _ tj dtt
|�� n| j� j� | _ | j S )z�Return lowercase cloud name as determined by the datasource.
Datasource can determine or define its own cloud product name in
metadata.
z5Ignoring metadata provided key %s: non-string type %s) � _cloud_namerG rN �METADATA_CLOUD_NAME_KEYr? r2 r= �_get_cloud_namerx ry �type)r( r� r r r! r� � s
zDataSource.cloud_namec C s | j S )z�Return the datasource name as it frequently matches cloud name.
Should be overridden in subclasses which can run on multiple
cloud names, such as DatasourceEc2.
)rm )r( r r r! r� � s zDataSource._get_cloud_namec C s" | j s
d S d| j kr | j d S d S )Nz launch-index)rG )r( r r r! r � s
zDataSource.launch_indexc C s4 t jtj| j ��g}|}x|D ]}|j|�}q W |S )N)r �Filterr Zsafe_intZapply)r( Z processed_ud�filtersZnew_ud�fr r r! r� � s
zDataSource._filter_xdatac C s dS )NFr )r( r r r! �is_disconnected� s zDataSource.is_disconnectedc C s | j S )N)r^ )r( r r r! r� � s zDataSource.get_userdata_rawc C s | j S )N)r` )r( r r r! r� � s z DataSource.get_vendordata_rawc C s | j S )N)rb )r( r r r! r� � s z DataSource.get_vendordata2_rawc C s i S )Nr )r( r r r! �get_config_obj� s zDataSource.get_config_objc C s t | jjd��S )Nz public-keys)�normalize_pubkey_datarG rN )r( r r r! r� � s z DataSource.get_public_ssh_keysc C s dS )a5 Publish the public SSH host keys (found in /etc/ssh/*.pub).
@param hostkeys: List of host key tuples (key_type, key_value),
where key_type is the first field in the public key file
(e.g. 'ssh-rsa') and key_value is the key itself
(e.g. 'AAAAB3NzaC1y...').
Nr )r( Zhostkeysr r r! �publish_host_keys� s z DataSource.publish_host_keysc C sd ddi}xV|j � D ]J\}}|j|�s&qx4|D ],}d||t|�d � f }tjj|�r,|S q,W qW d S )NZsd�vd�xvd�vtbz /dev/%s%s)r� r� r� )r< r@ �lenr� rR r� )r( Z
short_nameZmappingsZnfromZtlistZntoZcandr r r! �
_remap_device� s
zDataSource._remap_devicec C s d S )Nr )r( �_namer r r! �device_name_to_device� s z DataSource.device_name_to_devicec C s. | j }y| jj� }W n tk
r( Y nX |S )z<Default locale is en_US.UTF-8, but allow distros to override)�default_localerj �
get_localer� )r( Zlocaler r r! r� � s zDataSource.get_localec C s2 | j jd| j jd��}|r |S | j jdi �jd�S )Nzavailability-zoner� Z placement)rG rN )r( Z top_level_azr r r! r� s
z DataSource.availability_zonec C s | j jd�S )Nr� )rG rN )r( r r r! r�
s zDataSource.regionc C s$ | j sd| j krdS t| j d �S )Nz instance-idziid-datasource)rG r2 )r( r r r! r� s zDataSource.get_instance_idc C sd d}d}|}d}| j s&| j jd� r�|r4td|�S g }tj� } | dkrLd}tj| �}
|
rx|
jd�dkrxt|
�jd�}n8| r�| jd�dkr�t| �jd�}n| r�| |g}n||g}nZ| j d } t j
| ��rg }|r�tj | �}|r�t|�jd�}nd | j dd
� g}n
| jd�}t
|�d k�r6|d } dj|d d� �}n|d } |�rZ||k�rZd | |f } t| |�S )
a� Get hostname or fqdn from the datasource. Look it up if desired.
@param fqdn: Boolean, set True to return hostname with domain.
@param resolve_ip: Boolean, set True to attempt to resolve an ipv4
address provided in local-hostname meta-data.
@param metadata_only: Boolean, set True to avoid looking up hostname
if meta-data doesn't have local-hostname present.
@return: a DataSourceHostname namedtuple
<hostname or qualified hostname>, <is_default> (str, bool).
is_default is a bool and
it's true only if hostname is localhost and was
returned by util.get_hostname() as a default.
This is used to differentiate with a user-defined
localhost hostname.
Optionally return (None, False) when
metadata_only is True and local-hostname data is not available.
Z localdomainZ localhostFzlocal-hostnameNT�.r zip-%sr� rL z%s.%s)rG rN rV r r� Zget_fqdn_from_hosts�findr2 rO r
Zis_ipv4_addressZ
gethostbyaddrrA r� r� ) r( ZfqdnZ
resolve_ipZ
metadata_onlyZ defdomainZdefhostZdomainrX ZtoksrW Z
hosts_fqdnZlhostr r r! r� sF
zDataSource.get_hostnamec C s | j j| d�S )N)Z data_source)rj �get_package_mirror_info)r( r r r! r� _ s z"DataSource.get_package_mirror_info)�source_event_typesc C sX i }xN|D ]F}x@| j j� D ]2\}}||kr|j|�s>t� ||< || j|� qW q
W |S )N)�supported_update_eventsr< rN �set�add)r( r� �supported_events�eventZ update_scopeZ
update_eventsr r r! �get_supported_eventsb s
z DataSource.get_supported_events)r� r& c C s� | j |�}xH|j� D ]<\}}tjd|jdjdd� |D ��� | jd| tff� qW |rp| j� | j� }|rpdS tjd| djdd� |D ��� d S )
a� Refresh cached metadata if the datasource supports this event.
The datasource has a list of supported_update_events which
trigger refreshing all cached metadata as well as refreshing the
network configuration.
@param source_event_types: List of EventTypes which may trigger a
metadata update.
@return True if the datasource did successfully update cached metadata
due to source_event_type.
z:Update datasource metadata and %s config due to events: %sz, c S s g | ]
}|j �qS r )r' )�.0r� r r r! �
<listcomp>� s z;DataSource.update_metadata_if_supported.<locals>.<listcomp>z
_%s_configTz(Datasource %s not updated for events: %sc S s g | ]
}|j �qS r )r' )r� r� r r r! r� � s F) r� r< rx ry r' r� r� r� r� )r( r� r� ZscopeZmatched_events�resultr r r! � update_metadata_if_supportedo s"
z'DataSource.update_metadata_if_supportedc C s dS )NFr )r( rl r r r! �check_instance_id� s z DataSource.check_instance_idc C s dS )a check_if_fallback_is_allowed()
Checks if a cached ds is allowed to be restored when no valid ds is
found in local mode by checking instance-id and searching valid data
through ds list.
@return True if a ds allows fallback, False otherwise.
Fr )r( r r r! rt � s z'DataSource.check_if_fallback_is_allowedc C sR |d kr t }|d krt}x4| D ],}|d kr,q ||kr8|S tjd||� |S q W |S )Nz%invalid dsmode '%s', using default=%s)�DSMODE_NETWORK�
VALID_DSMODESrx r� )Z
candidates�defaultZvalid� candidater r r! �_determine_dsmode� s
z DataSource._determine_dsmodec C s d S )Nr )r( r r r! �network_config� s zDataSource.network_configc C s dS )a( setup(is_new_instance)
This is called before user-data and vendor-data have been processed.
Unless the datasource has set mode to 'local', then networking
per 'fallback' or per 'network_config' will have been written and
brought up the OS at this point.
Nr )r( �is_new_instancer r r! �setup� s zDataSource.setupc C s dS )a� activate(cfg, is_new_instance)
This is called before the init_modules will be called but after
the user-data and vendor-data have been fully processed.
The cfg is fully up to date config, it contains a merged view of
system config, datasource config, user config, vendor config.
It should be used rather than the sys_cfg passed to __init__.
is_new_instance is a boolean indicating if this is a new instance.
Nr )r( Zcfgr� r r r! �activate� s zDataSource.activaterM )r] N)r^ N)r_ N)r` N)ra N)rb N)
rc rd re rf r] rg r r_ rh ri )N)T)F)FFF)NN)\r* r+ r, r� Zdsmoder� rm r� r� r� r� r# r. r1 r0 r/ Znetwork_config_sourcesr r� r� r� r� r r r �BOOT_NEW_INSTANCEZBOOTZ BOOT_LEGACYZHOTPLUGr� Zdefault_update_eventsr� r� r2 r r� r� rs Z_ci_pkl_versionr r rq r� r{ r) �boolr} r r� r� r� r� r� r� r� r� r� r� �propertyr� r� r� r� r� r� r r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r r� r� r� rt � staticmethodr� r� r� r� r r r r! r � s�
&
R<
H$
)� metaclassc C s� g }| s |S t | t�r | j� S t | ttf�r4t| �S t | t�r�xN| j� D ]B\}}t |t�r`|g}t |ttf�rHx|D ]}|rt|j|� qtW qHW |S )N)r? r2 �
splitlines�listr� rB r< r> )Z pubkey_data�keysZ_keynameZklistZpkeyr r r! r� � s
r� )r& c C s t |||�}dd� |D �}t|kr&dnd} tjd| |� x�t||�D ]�\}
} tjd|
jdd� d | |
f d
| |
f |d �} yV| �H tjd | � | | ||�}
|
jt j
g�r�d
| |
f | _ |
t j
| �fS W d Q R X W qD tk
r� tjtd| � Y qDX qDW ddj|� }t|��d S )Nc S s g | ]}t j|��qS r )r r| )r� r� r r r! r� � s z find_source.<locals>.<listcomp>Znetworkr z#Searching for %s data source in: %sz search-%sr r6 z searching for %s data from %szno %s data found from %s)�name� description�message�parentz%Seeing if we can get any data from %szfound %s data from %szGetting data from %s failedz4Did not find any data source, searched classes: (%s)z, )� list_sources� DEP_NETWORKrx ry �zipr ZReportEventStackrA r� r r� r r r| r� r r� r� r4 )rl rj rk Zds_deps�cfg_list�pkg_listZreporter�ds_listZds_namesr� r �clsZmyrep�s�msgr r r! � find_source� s.
r c C s� g }t jd| ||� xt| D ]l}tj|�}tj||dg�\}}|sLt jd|� x8|D ]0}tj|�} t| d�}
|
|�} | rR|j| � P qRW qW |S )z�Return a list of classes that have the same depends as 'depends'
iterate through cfg_list, loading "DataSource*" modules
and calling their "get_datasource_list".
Return an ordered list of classes that match (if any)
zLLooking for data source in: %s, via packages %s that matches dependencies %sZget_datasource_listzDCould not import %s. Does the DataSource exist and is it importable?) rx ry r Z"match_case_insensitive_module_name� find_module�error�
import_moduler� rD ) r
�dependsr Zsrc_listr% Zds_nameZm_locsZ _looked_locsZm_loc�modZlisterZmatchesr r r! r s,
r � system-uuid)�fieldr& c C s* | sdS t j|�}|sdS | j� |j� kS )NF)r Z
read_dmi_datar= )r� r Z dmi_valuer r r! � instance_id_matches_system_uuid9 s
r c C sl | st } |st }|t kr(| t kr$| S |S x2tj� D ]&\}}|\}}|j|�r2|| �r2|S q2W | t krh| S |S )z@Lookup the canonical cloud-id for a given cloud_name and region.)r� �CLOUD_ID_REGION_PREFIX_MAPr< r@ )r� r� r� �prefixZ
cloud_id_testr� Z valid_cloudr r r! r� G s r� Tc C sj | sdS t | t�r| S t | t�r*tj| �S t | t�rV|dkrNt| jd�dd�S td��tdt | � ��dS )aL data: a loaded object (strings, arrays, dicts).
return something suitable for cloudinit vendordata_raw.
if data is:
None: return None
string: return string
list: return data
the list is then processed in UserDataProcessor
dict: return convert_vendordata(data.get('cloud-init'))
NTz
cloud-initF)�recursez'vendordata['cloud-init'] cannot be dictz$Unknown data type for vendordata: %s)
r? r2 r r: r; rB �convert_vendordatarN r� r� )�datar r r r! r Z s
r c @ s e Zd ZdS )�BrokenMetadataN)r* r+ r, r r r r! r r s r c C s8 g }t | �}x&|D ] \}}|t |�kr|j|� qW |S )N)r� r> )r r Zret_listZdepsetr
Zdepsr r r! �list_from_depends} s r )rQ �fnamer& c C sp yt j| �}W n" tk
r0 tjtd| � dS X ytj||ddd� W n" tk
rj tjtd|� dS X dS )z[Use pickle to serialize Datasource to a file as a cache.
:return: True on success
z Failed pickling datasource %sF�wb� )Zomoder� z Failed pickling datasource to %sT)�pickle�dumpsr� r r� rx r� )rQ r Z pk_contentsr r r! r� � s r� )r r& c C s� d}yt j| dd�}W n< tk
rR } z tjj| �rBtjd| |� W Y dd}~X nX |s\dS y
tj |�S t
k
rz dS tk
r� t j td| � dS X dS )zBUse pickle to deserialize a instance Datasource from a cache file.NF)�decodez failed loading pickle in %s: %sz#Failed loading pickled blob from %s) r Z load_filer� r� rR �isfilerx r� r# r� r3 r� )r Zpickle_contentsrz r r r! �pkl_load� s
r' c C s� t j� } tjd| �}tjd| �}tjd| �}|p6|p6|}|p>|}|rv|jd�j� }t jd|� d|� �dd|� d �d
� |r�|jd�r�|jd�S d S ) z�Check if command line argument for this datasource was passed
Passing by command line overrides runtime datasource detection
z ds=([^\s;]+)zci\.ds=([^\s;]+)zci\.datasource=([^\s;]+)rL z7Defining the datasource on the commandline using ci.ds=z or ci.datasource=z23.2zUse ds=z instead)�
deprecatedZdeprecated_versionZ
extra_messager6 )r Z get_cmdline�re�search�group�stripZ deprecate)r$ Z
ds_parse_0Z
ds_parse_1Z
ds_parse_2r% r( rm r r r! r~ � s
r~ )r )T)T�abcr: r� Zloggingr� r# r) � collectionsr �enumr r Ztypingr r r r r Z cloudinitr
r r r
r r ro r Zcloudinit.atomic_helperr Zcloudinit.distrosr Zcloudinit.eventr r Zcloudinit.filtersr Zcloudinit.helpersr Zcloudinit.persistencer Zcloudinit.reportingr ZDSMODE_DISABLEDZ DSMODE_LOCALr� Z DSMODE_PASSr� ZDEP_FILESYSTEMr Z DS_PREFIXr� ZREDACT_SENSITIVE_VALUEr� r� r� Z getLoggerr* rx r r# r� r3 r4 r5 rC rS rT rV �ABCMetar r� r2 r r r� r r� r �IOErrorr r r� r' r~ r r r r! �<module> s�
#
+ %