File: //usr/local/sysak/.sysak_components/tools/plugin_cache.pyc
�
�]ic @ s{ d d l Z d d l Z d d l Z d d l Td d l Z e d d � e d d � e � d e f d � � Y� � � Z d S( i����N( t *t namet cachet taskProct meminfot CachePluginc B se e Z d Z d � Z d � Z e d � Z d � Z d � Z d
d � Z
d � Z d � Z d � Z
RS( s# Cache 插件，用于扫描系统 Cache 的使用情况
启用条件：
- 参数带 -f
输出：
1. filecache: 文件 Cache 使用情况
节点：
[
["/tmp/test", {"cached": 1024, "size": 1024, "task": ["systemd(1)"]}],
...
]
pod：
{
"container1": [
["/tmp/test", {"cached": 1024, "size": 1024, "task": ["systemd(1)"]}],
...
],
"container2": [
["/tmp/test", {"cached": 1024, "size": 1024, "task": ["systemd(1)"]}],
...
]]
}
2. filecache_shm: 共享内存文件 Cache 使用情况
节点：
[
["/dev/shm/test", {"cached": 1024, "size": 1024, "task": ["systemd(1)"], "shmem_kind": "Shmem"}],
...
]
Pod:
{
"container1": [
["/dev/shm/test", {"cached": 1024, "size": 1024, "task": ["systemd(1)"], "shmem_kind": "Shmem"}],
...
],
"container2": [
["/dev/shm/test", {"cached": 1024, "size": 1024, "task": ["systemd(1)"], "shmem_kind": "Shmem"}],
}
c C sh g } | | k r | SxK | | D]? } | | k r9 q! n | | d } | j d j | | � � q! W| S( Nt Names {}({})( t appendt format( t selft
filePidMapt taskInfot filenamet rest pidR ( ( s8 /usr/local/sysak/.sysak_components/tools/plugin_cache.pyt get_cache_file_pid6 s c C s? t � } | j j d d � } i d } } xF | j � D]8 } t j | � } | | 7} | d k r8 | | | <q8 q8 Wt | j � d d � d t �} g }
d } xz | D]r \ } }
|
j i | d 6t
|
� d d 6t
|
� d d
6| j | | | � d 6� | d 7} | | k r� Pq� q� Wi |
d
6}
| j d |
� | S( s< 通过 mincore 系统调用来计算文件的缓存大小。t topNi i t keyc S s | d | d f S( Ni i ( ( t kv( ( s8 /usr/local/sysak/.sysak_components/tools/plugin_cache.pyt <lambda>L s t reverset filei t cachedt sizet taski t filecachet host(
t PluginCollectInfot paramst gett keyst mem_utilst cal_cache_sizet sortedt itemst TrueR t intR t store( R R
R t collect_infoR t
filecache_mapt total_cacheR R t sorted_filecacheR t countt kt v( ( s8 /usr/local/sysak/.sysak_components/tools/plugin_cache.pyt get_file_cache_by_mincoreA s4
c ! s� d � � � � � f d � � � f d � } t � } t j d d
� } | s� t j j t j j t � � } t j j | d � } n d } t t
j d � d � �� } | j � } xx | D]p }
|
j
� j � } t | � d k r� q� n | d j
� d
} t | d j
� � }
| d k r� |
} Pq� q� WWd
QXt j � �}} d } d } | d+ k rbd } n | d. k rwd } n � r�| j d � j d � } | d | d d d | j d | g } n! | d | d d d | j d g } t
j | d d �\ } } } | d k rt d | � � n t | j d � ��} t j | j � � } � s�| j d � } | j d! � } | j d" � } | j d# i � j d$ i � d } | | | | � \ } } | j d% i | d& 6| d' 6� | S| j d# i � j | i � } x� | D]� } i } i } | j d( d � } | j d � } | j | � rs| j | � j d! i � } | j | � j d" i � } n | | | | � \ } } | j | i | d& 6| d' 6� q�WWd
QX| SWd
QXd
S(/ s< 通过 podmem 工具扫描系统的文件 Cache 使用情况c S sL | j d � } | d k rH | j d | � } | d k rH | | d Sn | S( s�
Pod 模式下，尝试还原容器的真实路径
/ostree/deploy/LifseaOS/deploy/9401830260e19a6e0c84aad1f3ee493dc634ca208b669dff63159b41e366ce28.0/var/ostree/deploy/LifseaOS/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/29/fs/var/log/ntgh_data.log
↓
/var/log/ntgh_data.log
Args:
item (_type_): _description_
sE /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/i����s /fs/i ( t find( R t overly_indext fs_index( ( s8 /usr/local/sysak/.sysak_components/tools/plugin_cache.pyt extract_real_file_pathe s c s� | j d d � } � r' � | � } n t | j d d � � } | j d d � j � } | d k r� i | d 6| j d d � d 6| j d d � d 6� j | | | � d 6| d 6Si | d 6| j d d � d 6| j d d � d 6� j | | | � d 6Sd S(
NR t t shmemi����t
shmem_kindi R R R ( R R$ t stripR ( t itemR
R R R3 R4 ( t
enable_podR1 R ( s8 /usr/local/sysak/.sysak_components/tools/plugin_cache.pyt convert_podmem_file_cache_itemw s" c
s7 g } g } d | k r� d | k r� | d } x' | D] } | j � | | | � � q5 W| d } x� | D] } | j � | | | � � qi Wn� d | k r-| d } x� t | � D]t \ } } t | j d d � � } | d k r� | j � | | | � � n | d k r� | j � | | | � � q� q� Wn | | f S( Nt filecache_filet
shmem_filet sort_fileR3 i����i i ( R t enumerateR$ R (
t
filecache_insR
R R t
filecache_shmR R6 R3 t system_file_cache_topt idx( R8 ( s8 /usr/local/sysak/.sysak_components/tools/plugin_cache.pyt get_one_file_cache� s,
t PODMEM_PATHt podmemi s
/proc/meminfot ri i����i t MemTotalNR2 t 100i i t 500i@ t 1000t podDetectt pod_names -rs -tt 30s -js -ps -st timeouti� s Failed to run podmem %sR R R
t datat systemR R R> t cnamei � i I i i I ( R t ost getenvt Nonet patht dirnamet abspatht __file__t joint openR t
get_real_patht readlinesR5 t splitt lenR$ t tempfilet NamedTemporaryFilet get_collect_infot loadR t run_cmdt Exceptiont jsont loadst readR R% t exists(! R t contextR7 RA R& t podmem_patht
script_dirt mem_totalt ft linest lineR" R R t tmpfileRJ t scan_ratet cmdt codet resultt errorRM t task_proc_collect_infoR R
t system_filecachet
filecache_rest filecache_shm_rest
pod_filecachet cons_filecacheRO t
con_filecachet cons_filecache_shm( ( R8 R7 R1 R s8 /usr/local/sysak/.sysak_components/tools/plugin_cache.pyt get_file_cache_by_podmemb s�
"
c C s) d } y�| j d � } | d k r( t S| r%| j d � j d � } | j | � sn t j rj d | GHn t S| j | � } | j d d � } | j d d � } | j d d � } | d k r� t j r� d
| GHn t St | | � | }
t j rd | | | | |
d | d f GHn |
| k S| j d
� } | j d � } | j d � } | j d � }
| j d � } |
| } | d k r�t j r�d GHn t S| | } t | | � | }
t j r�d | | | |
d | d f GHn |
| k SWn2 t
k
r$} t j | � t j r d GHn t SXd S( sA 判断是否需要扫描缓存，只有当 filecache + shmem 占用总使用内存的 30% 以上才扫描
Args:
context: 插件上下文
enable_pod: 是否为 pod 模式
Returns:
bool: True 表示需要扫描，False 表示不需要扫描
g333333�?R RI RJ s1 [Cache] Pod %s: no meminfo found, will scan cacheR i R3 t usages( [Cache] Pod %s: usage=0, skip cache scansL [Cache] Pod %s: cache=%s, shmem=%s, usage=%s, ratio=%.2f%%, threshold=%.2f%%id t Cachedt Bufferst ShmemRE t MemFrees) [Cache] Host: mem_used=0, skip cache scansW [Cache] Host: filecache=%sKB, shmem=%sKB, mem_used=%sKB, ratio=%.2f%%, threshold=%.2f%%s> [Cache] Error checking cache ratio, will scan cache by defaultN( R_ RR R# R` Rf R t is_debugR t Falset floatRb t log_exception_in_debug_mode( R Rg R7 t CACHE_THRESHOLDt meminfo_collectRJ t pod_meminfoR R3 R} t cache_ratioR t buffersRj t mem_freet mem_usedR t e( ( s8 /usr/local/sysak/.sysak_components/tools/plugin_cache.pyt should_scan_cache s^
c C s� | j j d t � } t � } | j | | � sE t j rA d GHn | Sy | j d | d | � } Wni t k
r� } t j | � | r� | S| j
d � } | j d � } | j d � } | j | | � } n X| S( NR7 s@ [Cache] Cache ratio is below 30% threshold, skipping podmem scanRg R R R
(
R R R� R R� R R� R| Rb R� R_ R` R- ( R Rg R7 R
R� Rt R R
( ( s8 /usr/local/sysak/.sysak_components/tools/plugin_cache.pyt collectm s"
c C so d | } | r! d | | } n d d d g } g } d d d g } d } xB | D]: }
| j |
d t j |
d
� d j |
d � g � qR W| t j | | | | d
t �7} d | } | r� d | | } n d d d d g } d d d d g } g } xI | D]A }
| j |
d t j |
d
� |
d d j |
d � g � qW| t j | | | | d
t �7} | S( Nu 文件 Cache Top %su
容器 %s s 文件名s 缓存大小s 关联任务iP i R2 R R t ,R t showu 共享内存文件 Cache Top %su 容器 %ss 共享内存类型R4 ( R R t format_unitRW t print_tableR� ( R R R> R RO t titlet headersRM t max_col_lengthst res_strt value( ( s8 /usr/local/sysak/.sysak_components/tools/plugin_cache.pyt analysis_one_table� s:
c C s� t � } d | _ d } xu | j � D]g \ } } | j d g � } | j d g � } i | d 6| d 6| j | <| | j | | | | � 7} q% W| | _ | S( Nt CacheR2 R R> ( t PluginAnalysisInfoR� R" R Rr R� t render_text( R t
cache_infoR t
analysis_infoR� RO R� R R> ( ( s8 /usr/local/sysak/.sysak_components/tools/plugin_cache.pyt analysis_pod� s c C sr t � } d | _ | j d g � } | j d g � } | j | | | � } | | j d <| | j d <| | _ | S( NR� R R> ( R� R� R R� Rr R� ( R R� R R� R R> R� ( ( s8 /usr/local/sysak/.sysak_components/tools/plugin_cache.pyt
analysis_host� s
c C s� | j j d d � } | j j d t � } | j d � } | rq | j s^ t � } d | _ | S| j | j | � S| j d � } | d k r� t � } d | _ | S| j
| | � Sd S( NR i R7 R R� R ( R R R� R_ RM R� R� R� R` RR R� ( R Rg R R7 t cache_collect_infoR� t host_data( ( s8 /usr/local/sysak/.sysak_components/tools/plugin_cache.pyt analysis� s N( t __name__t
__module__t __doc__R R- R� R| R� R� RR R� R� R� R� ( ( ( s8 /usr/local/sysak/.sysak_components/tools/plugin_cache.pyR
s * ! � W + (
RP Rc R] t
plugin_systemR t plugint plugin_requirest plugin_pod_supportt
PluginBaseR ( ( ( s8 /usr/local/sysak/.sysak_components/tools/plugin_cache.pyt <module> s