3
\�mep� � @ s� U d Z ddlZddlZddlZddlmZ ddlmZmZ ddl m
Z
ddl m Z ddl
mZmZ ddlmZ dd lmZ ejd
�Zejd �Zejd �Zejd
�Zejd�Zejd�Zejd�ZddiZ ej e �Z dZ ddde egeed�gddgd�Z!e!ee!�Z e"e e
e#dd�d d �Z$d d � Z%d d!� Z&dTd"d#�Z'dUd%d&�Z(d'd(� Z)dVd)d*�Z*d+d,� Z+d-d.� Z,dWd0d1�Z-d2d3� Z.d4d5� Z/d6d7� Z0d8d9� Z1d:d;� Z2d<d=� Z3d>d?� Z4d@dA� Z5dBdC� Z6dDdE� Z7dFdG� Z8dHdI� Z9dJdK� Z:dLdM� Z;dNdO� Z<dPdQ� Z=dRdS� Z>dS )Xz1Disk Setup: Configure partitions and filesystems.� N)�dedent)�subp�util)�Cloud)�Config)�
MetaSchema� get_meta_doc)� ALL_DISTROS)� PER_INSTANCEZsfdiskZsgdiskZlsblkZblkidZblockdevZ partprobeZwipefsZLANG�Cai This module is able to configure simple partition tables and filesystems.
.. note::
for more detail about configuration options for disk setup, see the disk
setup example
.. note::
if a swap partition is being created via ``disk_setup`` then a ``fs_entry``
entry is also needed in order for mkswap to be run, otherwise when swap
activation is later attempted it will fail.
For convenience, aliases can be specified for disks using the
``device_aliases`` config key, which takes a dictionary of alias: path
mappings. There are automatic aliases for ``swap`` and ``ephemeral<X>``, where
``swap`` will always refer to the active swap partition and ``ephemeral<X>``
will refer to the block device of the ephemeral image.
Disk partitioning is done using the ``disk_setup`` directive. This config
directive accepts a dictionary where each key is either a path to a block
device or an alias specified in ``device_aliases``, and each value is the
configuration options for the device. File system configuration is done using
the ``fs_setup`` directive. This config directive accepts a list of
filesystem configs.
Z
cc_disk_setupz
Disk Setupz$Configure partitions and filesystemsan device_aliases:
my_alias: /dev/sdb
swap_disk: /dev/sdc
disk_setup:
my_alias:
table_type: gpt
layout: [50, 50]
overwrite: true
swap_disk:
table_type: gpt
layout: [[100, 82]]
overwrite: true
/dev/sdd:
table_type: mbr
layout: true
overwrite: true
fs_setup:
- label: fs1
filesystem: ext4
device: my_alias.1
cmd: mkfs -t %(filesystem)s -L %(label)s %(device)s
- label: fs2
device: my_alias.2
filesystem: ext4
- label: swap
device: swap_disk.1
filesystem: swap
- label: fs3
device: /dev/sdd1
filesystem: ext4
mounts:
- ["my_alias.1", "/mnt1"]
- ["my_alias.2", "/mnt2"]
- ["swap_disk.1", "none", "swap", "sw", "0", "0"]
- ["/dev/sdd1", "/mnt3"]
�
disk_setup�fs_setup)�id�name�title� descriptionZdistrosZ frequencyZexamplesZactivate_by_schema_keys)r �cfg�cloud�args�returnc s� |j di ��� �fdd�}|j d�}t|t�r�t||� tjdt|�� x�|j� D ]~\}}t|t�srtjd|� qRy*tjd� t j
tjd| t ||fd � W qR t k
r� } zt j
td
| � W Y d d }~X qRX qRW |j d �} t| t��r�tjd
t| �� t| |� x�| D ]�}t|t��s*tjd|� �q
y2tjd� |j d�}
t j
tjd|
t|fd � W n4 t k
�r� } zt j
td| � W Y d d }~X nX �q
W d S )z[
See doc/examples/cloud-config-disk-setup.txt for documentation on the
format.
�device_aliasesc s �j | �}� j|p| �p|S )N)�getZdevice_name_to_device)Zcandr )r r � �#/usr/lib/python3.6/cc_disk_setup.py�alias_to_devicey s
z handle.<locals>.alias_to_devicer zPartitioning disks: %sz Invalid disk definition for %sz!Creating new partition table/diskzCreating partition on %s)Zlogfunc�msg�funcr z Failed partitioning operation
%sNr
zsetting up filesystems: %sz"Invalid file system definition: %szCreating new filesystem.�devicezCreating fs for %sz%Failed during filesystem operation
%s)r �
isinstance�dict�update_disk_setup_devices�LOG�debug�str�items�warningr Zlog_time�mkpart� Exception�logexc�list�update_fs_setup_devices�mkfs) r r r r r r �disk�
definition�er
r r )r r r �handler sJ
&
r/ c C s� x�t | �D ]t}||�}|d ks
||kr(q
|| krDtjd||� | |= | | | |< t| | t�rj|| | d< | |= tjd||� q
W d S )Nz0Replacing %s in disk_setup for translation of %s� _orignamez,updated disk_setup device entry '%s' to '%s')r) r! �infor r r" )r �tformer�orignameZ transformedr r r r � s$ r c C s� x�| D ]�}t |t�s"tjd|� q|jd�}|d kr6qtj|�\}}||�}|d k rx|}tjd|||� ||d< ||d< |rd|kr�tjd|||� |d |d< ||d< qW d S )Nz"entry in disk_setup not a dict: %sr z %s is mapped to disk=%s part=%sr0 � partitionzKPartition '%s' from dotted device name '%s' overrides 'partition' key in %sZ
_partition)r r r! r% r r Zexpand_dotted_devnamer" )r r2 r- r3 Zdev�partZtformedr r r r* � s0
r* c c sD t j| �}|r||d� }x$dd� |D �D ]\}}||fV q*W dS )zd
Returns the key/value pairs of output sent as string
like: FOO='BAR' HOME='127.0.0.1'
Nc S s g | ]}|j d ��qS )�=)�split)�.0�xr r r �
<listcomp>� s z"value_splitter.<locals>.<listcomp>)�shlexr7 )�values�startZ_values�key�valuer r r �value_splitter� s
r@ Fc c s� t ddd| g}|r |jd� d}ytj|�\}}W n4 tk
rf } ztd| |f �|�W Y dd}~X nX dd� |j� j� D �}xB|D ]:}ddddd �}x t|�D ]\} }
|
|| j� < q�W |V q�W dS )
a�
Enumerate the elements of a child device.
Parameters:
device: the kernel device name
nodeps <BOOL>: don't enumerate children devices
Return a dict describing the disk:
type: the entry type, i.e disk or part
fstype: the filesystem type, if it exists
label: file system label, if it exists
name: the device name, i.e. sda
z--pairsz--outputzNAME,TYPE,FSTYPE,LABELz--nodepsNz"Failed during disk check for %s
%sc S s g | ]}t |j� �d kr|�qS )r )�lenr7 )r8 r9 r r r r: s z"enumerate_disk.<locals>.<listcomp>)r �type�fstype�label) � LSBLK_CMD�appendr r' � RuntimeError�strip�
splitlinesr@ �lower) r �nodepsZ lsblk_cmdr1 �_errr. �partsr5 �dr> r? r r r �enumerate_disk� s.
rO c C s. x(t | dd�D ]}d|kr|d j� S qW dS )z@
Return the device type of the device by calling lsblk.
T)rK rB N)rO rJ )r rN r r r � device_type& s rP c C sX d}y t | �}W n tk
r0 tjd| � dS X |rB|dkrBdS | rT|dkrTdS dS )z0
Check if the device is a valid device.
� z Query against device %s failedFr5 Tr, )rP r' r! r% )r r4 Zd_typer r r �is_device_valid1 s rR c
C s� d
\}}}}t dd| g}ytj|ddgd�\}}W n4 tk
rf } ztd| |f �|�W Y dd}~X nX |r�t|j� �dkr�xLt|dd �D ]<\}} |j� d
kr�| }q�|j� d kr�| }q�|j� d kr�| }q�W |||fS )z�
Check if the device has a filesystem on it
Output of blkid is generally something like:
/dev/sda: LABEL="Backup500G" UUID="..." TYPE="ext4"
Return values are device, label, type, uuid
Nz-cz /dev/nullr � )�rcsz"Failed during disk check for %s
%s� )r= rD rB �uuid)NNNN)� BLKID_CMDr r' rG rA rI r@ rJ )
r �outrD �fs_typerV Z blkid_cmdrL r. r> r? r r r �check_fsC s$ rZ c C s t | �\}}}|S )z7
Returns true if the device has a file system.
)rZ )r �_rY r r r �
is_filesystemc s r\ Tc C s� |dkr d}|sddg}d}x�t | �D ]�}|d |krN|dkrNd|d dfS |d |kr�|rj|d |ksp| r�d|d d
fS |d |kr&|d dks�|d r�d
}|d dkr�q&|d s&d|d dfS q&W |s�| dfS tjd � d
S )a
Find a device that is either matches the spec, or the first
The return is value is (<device>, <bool>) where the device is the
device to use and the bool is whether the device matches the
fs_type and label.
Note: This works with GPT partition tables!
NrQ r, r5 FrC z/dev/%sr rD TrB z5Failed to find device during available device search.)NF)rO r! r% )r rY rD Z
valid_targets� label_match�
replace_fsZraw_device_usedrN r r r �find_device_nodek s,
r_ c C s2 t tt| ���dkrdS t| �\}}}|r.dS dS )z�
Check if the device is currently used. Returns true if the device
has either a file system or a partition entry
is no filesystem found on the disk.
rU TF)rA r) rO rZ )r r[ � check_fstyper r r � is_disk_used� s ra c C sr y,t j td| g�\}}t j td| g�\}}W n4 tk
r` } ztd| |f �|�W Y d d }~X nX t|�t|� S )Nz --getsize64z--getsszFailed to get %s size
%s)r �
BLKDEV_CMDr' rG �int)r Z
size_in_bytesr[ Z sector_sizer. r r r � get_hdd_size� s $rd c C s t | � td| g}ytj|d| d�\}}W n4 tk
r` } ztd| |f �|�W Y dd}~X nX g }x�|j� D ]�}|j� }t|�dkr�qp| |d krp|d
j� dkr�qpd} x@t t
dt|��d
d �D ]&}
||
j � r�||
d kr�||
} P q�W |j | � qpW |S )z�
Returns true if the partition layout matches the one on the disk
Layout should be a list of values. At this time, this only
verifies that the number of partitions and their labels is correct.
z-lz%s
)�dataz(Error running partition command on %s
%sNr rU �extended�emptyT)�reverse�/���)rf rg )
� read_parttbl�
SFDISK_CMDr r' rG rI r7 rA rJ �sorted�range�isdigitrF ) r �layout�prt_cmdrX rL r. � found_layout�line�_lineZ
type_labelr9 r r r �check_partition_mbr_layout� s.
ru c C s� t d| g}ytj|td�\}}W n4 tk
rT } ztd| |f �|�W Y d d }~X nX t|j� �}x |D ]}|j� jd�rhP qhW dd� |D �}g } x8|D ]0}
t |
�dkr�|
j
d�r�|
d d
� }
| j |
� q�W | S ) Nz-p)Z
update_envz(Error running partition command on %s
%s�Numberc S s g | ]}|j � j� d �qS )� )rH r7 )r8 rs r r r r: s z.check_partition_gpt_layout.<locals>.<listcomp>� Z00r rS ) �
SGDISK_CMDr �
LANG_C_ENVr' rG �iterrI rH �
startswithrA �endswithrF ) r rp rq rX rL r. Z out_linesrs ZcodesZcleaned�coder r r �check_partition_gpt_layout� s$
r c C s� d| krt ||�}n d| kr(t||�}ntd��tjd| |||� t|t�rd|r`t|�dkr`dS dS t|�t|�kr�dd � |D �}tjd
||� x2t||�D ]$\}}|d k r�t |�t |�kr�dS q�W dS dS ) z�
See if the partition lay out matches.
This is future a future proofing function. In order
to add support for other disk layout schemes, add a
function called check_partition_%s_layout
�gpt�mbrz Unable to determine table typez6called check_partition_%s_layout(%s, %s), returned: %srU TFc S s* g | ]"}t |ttf�r"t|d �nd�qS )rU N)r �tupler) r# )r8 r9 r r r r: 2 s z*check_partition_layout.<locals>.<listcomp>z Layout types=%s. Found types=%sN)
r ru rG r! r" r �boolrA �zipr# )�
table_typer rp rr Z layout_typesZitypeZftyper r r �check_partition_layout s0
r� c
C s t |t� rt |t�rdS t|�dkr0t |t�s<t |t� rDtd��t|�}|dkr\td��g }d}x�|D ]�}d}|}|d7 }t |t�r�t|�dkr�td | ��|\}}tt| �t|�d
�}||kr�|jd | � qj|jd ||f � qjW d
j|�} t|�dk�rtd| ��| S )a@
Calculate the layout of the partition table. Partition sizes
are defined as percentage values or a tuple of percentage and
partition type.
For example:
[ 33, [66: 82] ]
Defines the first partition to be a size of 1/3 the disk,
while the remaining 2/3's will be of type Linux Swap.
z,,83r zPartition layout is invalidrx z$Only simply partitioning is allowed.�S rU rS z%Partition was incorrectly defined: %s�d z,,%sz,%s,%s�
z-Calculated partition definition is too big
%s) r r) r� rA rG rc �floatrF �join)
�sizerp Z
last_part_num�part_definitionZpart_numr5 Z part_type�percent� part_sizeZsfdisk_definitionr r r �get_partition_mbr_layout? s:
r� c C s� t |t�rd ddgfgS g }xp|D ]h}t |t�rRt|�dkrHtd| ��|\}}n|}d }tt| �t|�d �}|j|ddj|�gf� q"W d|d d d < |S )
Nr rS z%Partition was incorrectly defined: %sr� z+{}rU rj rj rj ) r r� r) rA rG rc r� rF �format)r� rp Zpartition_specsr4 r� �partition_typer� r r r �get_partition_gpt_layoutx s
r� c
C sb d}d}d}t | d��:}|j|| � |j| tj� |j|| � |j� W d Q R X t| � d S )N� i zrb+i i )�open�write�seek�os�SEEK_END�flushrk )r ZnullZ start_lenZend_len�fpr r r �purge_disk_ptable� s r� c C s� x�t | �D ]v}|d d
kr
tdd|d g}y tjd|d � tj|� W q
tk
r~ } ztd|d �|�W Y d d }~X q
X q
W t| � d S ) z'
Remove parition table entries
rB r, �cryptz--allz/dev/%sr z Purging filesystem on /dev/%szFailed FS purge of /dev/%sN)r, r� )rO �
WIPEFS_CMDr! r1 r r' rG r� )r rN Z
wipefs_cmdr. r r r �
purge_disk� s r� c C s0 d| krt ||�S d| kr$t||�S td��dS )z�
Call the appropriate function for creating the table
definition. Returns the table definition
This is a future proofing function. To add support for
other layouts, simply add a "get_partition_%s_layout"
function.
r� r� z Unable to determine table typeN)r� r� rG )r� r� rp r r r �get_partition_layout� s
r� c C sr t dk rt | g}n
td| g}tj� ytj|� W n2 tk
rd } ztjtd| � W Y dd}~X nX tj� dS )zq
`Partprobe` is preferred over `blkdev` since it is more reliably
able to probe the partition table.
Nz
--rereadptz%Failed reading the partition table %s)�
PARTPROBE_CMDrb r �udevadm_settler r' r( r! )r Z probe_cmdr. r r r rk � s
"rk c C sb t d| g}ytj|d| d� W n4 tk
rT } ztd| |f �|�W Y dd}~X nX t| � dS )zV
Break out of mbr partition to allow for future partition
types, i.e. gpt
z--forcez%s
)re z Failed to partition device %s
%sN)rl r r' rG rk )r rp rq r. r r r �exec_mkpart_mbr� s
r� c C s� y�t j td| g� xrt|�D ]f\}\}\}}|d7 }t j tddj|||�| g� |d k r t|�jdd�}t j tddj||�| g� q W W n" tk
r� tjd | � � Y nX t | � d S )
Nz-ZrU z-nz{}:{}:{}rx �0z-tz{}:{}z Failed to partition device %s)
r ry � enumerater� r# �ljustr' r! r% rk )r rp �indexr� r= �endZpinputr r r �exec_mkpart_gpt� s" r� c C s8 t jj| �s,tj� t jj| �s,td| ��tj� dS )z?Assert that device exists and settle so it is fully recognized.zBDevice %s did not exist and was not created with a udevadm settle.N)r� �path�existsr r� rG )r r r r �assert_and_settle_device s r� c C s� t | � tjj| �} tjd| � |jdd�}|jdd�}|jdd�}tjd� t|t�r^| sd| rrtjd� d S tjd
| � t | �s�t
d j | d ���t|t �r�|j
� d
kr�tjd� t| � d S tjd� t|| |�r�tjd� dS tjd� | �r t| ��s t| ��r tjd| � d S tjd| � t| �}tjd� t|||�}tjd|� tjd| � d|k�rtt| |� n d|k�r�t| |� nt
d��tjd| � d S )a�
Creates the partition table.
Parameters:
definition: dictionary describing how to create the partition.
The following are supported values in the dict:
overwrite: Should the partition table be created regardless
of any pre-exisiting data?
layout: the layout of the partition table
table_type: Which partition table to use, defaults to MBR
device: the device to work on.
z!Checking values for %s definition� overwriteFrp r� r� z Checking against default devicesz)Device is not to be partitioned, skippingNz'Checking if device %s is a valid devicez%Device {device} is not a disk device!)r �removez,Instructed to remove partition table entriesz!Checking if device layout matchesz"Device partitioning layout matchesTz'Checking if device is safe to partitionz-Skipping partitioning on configured device %sz Checking for device size of %sz Calculating partition layoutz Layout is: %sz Creating partition table on %sr� z Unable to determine table typez Partition table created for %s)r� r� r� �realpathr! r" r r r� rR rG r� r# rJ r� r� ra r\ rd r� r� r� )r r- r� rp r� Z device_sizer� r r r r& sL
r& c C sD dddddd�}d| j � kr d} | j � |kr4|| S tjd| � dS )z9
A force flag might be -F or -F, this look it up
z-Fz-f)�extZbtrfsZxfsZreiserfsZswapr� z Force flag for %s is unknown.rQ )rJ r! r% )Zfs�flagsr r r �lookup_force_flagX s r� c C s� | j d�}| j d�}t| j dd��}| j d�}| j dg �}| j dg �}| j dd �}| j d
d �}t|� tjj|�}tjd |� | s�|j� �r*|j� r�d ||f }tjd
||� tjd|� t |�\} }
} tjd|| |
� | |ko�|
|k�r tjd|� |�stjd|� dS tj
d|� n tjd|� n�|�r�t|�j � d0k�r�|} tjd|� d}
|j � dk�rhd }
t ||||
|d�\}}tjd| |� |�r�tjd� dS | �r�|�r�|�r�tjd|� |�stjd | � dS n6| �s�t|�j � d k�r tjd |� ntjd � dS tjd |||� |�s4tj
d!|� dS |�p<|�sPt
d"j|d#���d }|�r�| d |||d$� }d}|�r�tj
d%|� |�r2tj
d&|� n�tjd'| �}|�s�tjd(| �}|�s�tj
d)||� dS |g}|�r�|jd*|g� |�s t|�d+k�rt|�}|�r|j|� |�r(|j|� |j|� tjd,||� tjd-t|�� ytj||d.� W n6 tk
�r� } zt
d/||f �|�W Y dd}~X nX dS )1a�
Create a file system on the device.
label: defines the label to use on the device
fs_cfg: defines how the filesystem is to look
The following values are required generally:
device: which device or cloud defined default_device
filesystem: which file system type
overwrite: indiscriminately create the file system
partition: when device does not define a partition,
setting this to a number will mean
device + partition. When set to 'auto', the
first free device or the first device which
matches both label and type will be used.
'any' means the first filesystem that matches
on the device.
When 'cmd' is provided then no other parameter is required.
rD r r4 �any�
filesystem�cmdZ
extra_optsr^ Fr� z#Checking %s against default devicesz%s%sz%Manual request of partition %s for %szChecking device %sz0Device '%s' has check_label='%s' check_fstype=%sz Existing file system found at %sz"Device %s has required file systemNzDestroying filesystem on %sz"Device %s is cleared for formating�autoz,Identifying device to create %s filesytem onT)rY rD r] r^ z(Automatic device for %s identified as %sz+Found filesystem match, skipping formating.z*Replacing file system on %s as instructed.zENo device available that matches request. Skipping fs creation for %sZnonez.Using the raw device to place filesystem %s onz(Error in device identification handling.z;File system type '%s' with label '%s' will be created on %szDevice is not known: %szENo way to create filesystem '{label}'. fs_type or fs_cmd must be set.)rD )rD r� r z8fs_setup:overwrite ignored because cmd was specified: %sz9fs_setup:extra_opts ignored because cmd was specified: %szmkfs.%szmk%sz.Cannot create fstype '%s'. No mkfs.%s commandz-Lr, z Creating file system %s on %sz Using cmd: %s)�shellzFailed to exec of '%s':
%s)r� r� )r r# r� r� r� r� r! r" ro rZ r% rJ r_ rG r� r �which�extendrP r� rF r' )Zfs_cfgrD r r4 rY Zfs_cmdZfs_optsZ
fs_replacer� Z check_labelr` r[ Zodevicer] Zreuser� Zmkfs_cmdZ
force_flagr. r r r r+ n s�
r+ )N)F)F)NNNTN)?�__doc__Zloggingr� r; �textwrapr Z cloudinitr r Zcloudinit.cloudr Zcloudinit.configr Zcloudinit.config.schemar r Zcloudinit.distrosr Zcloudinit.settingsr
r� rl ry rE rW rb r� r� rz Z getLogger�__name__r! ZMODULE_DESCRIPTION�metar# r) r/ r r* r@ rO rP rR rZ r\ r_ ra rd ru r r� r� r� r� r� r� rk r� r� r� r& r� r+ r r r r �<module> sz
%5&
2
2
').9G